← Back to the principles list

Hindsight Bias in ecommerce

Next principle →

Knowing the result makes it feel like you knew all along.

When a test ends, someone often says they saw it coming, and soon the whole team half believes it. Write your guess down before the test starts, so you can check what you really expected.

Hindsight bias
Made-up example: a written guess ends the “I knew it” argument in minutes.

Hindsight bias, also called the knew-it-all-along effect, is the tendency to see a past result as more predictable than it really was.

It can show up after any test or launch, and it can drag out arguments and hide the lesson. Before you start, write one line saying what you expect and why. When the result comes in, read that line first.

Hindsight bias example: two shirt pages and a guess nobody wrote down

A store team tests two versions of a shirt page. Page A has the Add to cart bar under the photo and earns $400 a day. Page B pins that bar to the bottom of the phone screen and earns $460 a day. Nobody wrote a guess at the start, so a teammate can say, “B won. I knew it all along!” The team argues for 2 weeks, then puts B live, and the store makes $400 a day until then. Now picture the same test with “A wins” written at the start. The teammate says, “B won. We guessed A.” The team agrees in 15 minutes, and B brings in $460 a day from today. This story is made up. A written guess leaves much less to argue about.

How hindsight bias works with other principles: 4 store examples

Hindsight bias can change what you remember expecting. These four pairs show two ideas it gets mixed up with and two cases where it hides a lesson. Every store below is made up to show the idea.

Hindsight bias vs confirmation bias

Often confused with: Hindsight bias changes what you remember expecting before the result came in, while confirmation bias picks out the evidence that fits what you believe now.

Illustrative store example: A coffee store owner looks back at a new subscription page and says she always knew it would work, and that is hindsight bias. Her habit of quoting the three emails that praise the page and skipping the three that complain is confirmation bias.

Confirmation bias: why we notice only the evidence we like →

Hindsight bias vs the curse of knowledge

Often confused with: Both come from being unable to set aside what you know: hindsight bias misjudges what could have been predicted, and the curse of knowledge misjudges what other people understand.

Illustrative store example: Now that a guitar store owner knows beginners struggle with heavy strings, she is sure she could have predicted it, which is hindsight bias. Expecting new shoppers to see why light strings suit them without being told is the curse of knowledge, so add one plain line to the page.

Curse of knowledge: why experts forget what beginners don't know →

Hindsight bias and self-serving bias

Works with: After a good launch, hindsight bias can make the result feel obvious and self-serving bias can make it feel like your doing, so nobody checks what really caused it.

Illustrative store example: A seed store has a strong spring, and the owner says she saw it coming and that her new homepage did it. Warm weather came early that year too, so write your guess before the season and list other possible causes after it.

Self-serving bias: why we take credit and pass on blame →

Hindsight bias and the second-order effect

Works with: A side effect nobody predicted can look obvious once it happens, so the team stops asking what other ripples it might be missing.

Illustrative store example: A dog treat store adds free shipping on every order, and small orders pile up until packing falls behind. Everyone now says they saw that coming, so check what they wrote down beforehand and add a line to the next plan for what else a change could affect.

Second-order effect: how one change ripples into later problems →

Put it into practice

What should I write down before a test or launch?

Keep it to a few lines: what you're changing, which version you think will win and why, the one number that decides it, and how long you'll run it. Add the date and put it where the whole team can see it. Ten minutes is plenty. If it takes longer, you're writing a report nobody will read.

Can a written guess make things worse?

It can if people start treating a wrong guess as a personal failure. They'll play it safe, or quietly stop writing anything down. Make being wrong normal by reading guesses out loud and moving on. Remember too that a guess only records what you expected. It can't tell you why a page won, so you still need to look for the reason.

Is hindsight bias the same as outcome bias?

No, though they're close and often get mixed up. Hindsight bias says the result was predictable. Outcome bias judges a decision by how it turned out, so a risky launch gets called smart just because it worked. Both lean on knowing the ending, and the same habit helps with both: judge each choice by what you knew when you made it.

What shows that writing guesses is helping my team?

Keep a simple log with the guess, the result and how long the team took to decide. After several tests, look for shorter debates and fewer “I knew it” moments. Then check how often the guesses were right. If they slowly get better, or you stop retrying ideas that already failed, the habit is paying off.

Why keep notes on tests where our guess was wrong?

Wrong guesses show you what you misjudged about your shoppers, and hindsight bias can wipe that lesson away. Once a result feels obvious, it seems there's nothing to learn. A short note fixes this: what you expected, what happened and what you'd try next. Read those notes before you plan the next test, because memory will tidy the story for you.