Hawthorne Effect in ecommerce
Next principle →People often act differently when they know they are being watched.
A shopper in a watched test often takes care, tries harder and finishes the task. Alone on a phone, the same shopper may give up at the first tiny button.
The Hawthorne effect, also called the observer effect, is when people change how they behave because they know someone is watching them.
The name comes from a factory, and people still argue about how strong the effect was there, so think of it as a warning with no fixed size. When someone watches your test shoppers, check the result against visits where nobody was watching.
Hawthorne effect example: watched testers hide a problem with tiny size buttons
Say a made-up sneaker store tests two pages for a $90 sneaker, one with tiny size buttons and one with big ones, on 10 shoppers per page. With someone watching, everybody takes care and 10 of 10 buy on both pages. Seeing no difference, the team keeps the tiny buttons. Only 3 of 10 real shoppers then buy, which is $270 from 10 shoppers. When the test runs with nobody watching, tiny buttons get 3 of 10 and big buttons get 5 of 10. The team switches to big buttons, which brings in $450 from 10 shoppers. Check anything you learn from a watched test against visits nobody is watching.
How the Hawthorne effect works with other principles: 4 store examples
Being watched can change behavior, and other things can get mistaken for it. These pairs separate two lookalikes and show two more ways a test can mislead you. Every store below is made up to show the idea.
Hawthorne effect vs observer-expectancy effect
Often confused with: In the Hawthorne effect people change because they know they are watched, while in the observer-expectancy effect the hopes of the person running the test steer what people do.
Illustrative store example: A backpack store owner sits with testers trying a new size chart, and the Hawthorne effect makes them read it more carefully than they would at home. Saying “You'll love this chart” and nodding at good answers adds observer expectancy, so read from a script.
Observer-expectancy effect: how a tester's hopes steer the results →Hawthorne effect vs spotlight effect
Often confused with: The Hawthorne effect is behavior changing under real watching, while the spotlight effect is believing people notice you far more than they do.
Illustrative store example: A lingerie store shopper worries the neighbors will notice the package, though nobody is looking, so that worry is the spotlight effect and plain packaging helps. Asked to browse while the owner watches on a video call, the same shopper really is being watched and may shop differently, which is the Hawthorne effect.
Spotlight effect: why people feel more watched than they are →Hawthorne effect and survey bias
Works with: A watched test user and a surveyed shopper are both aware of being judged, so each can act or answer the way they think they should.
Illustrative store example: A cleaning-products store asks shoppers if they care about recycled packaging and everyone says yes, then a watched tester praises the packaging too. Both may be saying what sounds right, so also check which products people actually buy.
Survey bias: why polite answers can mislead you →Hawthorne effect and empathy gap
Works with: A calm, watched tester at a desk shows little about a rushed shopper on a phone, and the empathy gap makes it hard for the team to picture that difference.
Illustrative store example: A flower-delivery store tests checkout with calm helpers at a desk while the owner watches, and everyone finishes. A real shopper is ordering anniversary flowers on a phone at the last minute, so also run the test on a phone with a two-minute limit.
Empathy gap: why calm owners misjudge stressed shoppers →Put it into practice
How can I make a watched test feel less like a test?
Let people use their own phone at home, on their own time. Recording them alone, with their permission, feels less like an exam than having you next to them. Give a real task, such as “find running shoes in your size under $100.” Tell them you are checking the store and not them. Pay the same whether they finish or not, so nobody feels they owe you praise.
Does the Hawthorne effect make watched tests useless?
No. Watched tests are great for finding out why something confuses people, and a shopper thinking aloud can show you a problem no chart will. They are poor at telling you how many real shoppers would finish or buy. Use them to find problems and live numbers to measure them. Being watched doesn't always change behavior much either, and the effect can fade as people settle into a task.
Is it fair to watch shoppers without telling them?
Only if you are open about it. Watching visits in bulk is normal, and tools that record visits or count clicks need a clear mention in your privacy policy. Some places require consent first. Mask card numbers and anything typed into forms. Look at patterns across many visits instead of singling out one person. Your aim is to leave shoppers undisturbed while being honest about what you collect.
How is the Hawthorne effect different from the novelty effect?
The Hawthorne effect comes from being watched. The novelty effect comes from something being new: regular shoppers may click a redesigned button because it looks different, and that interest fades. Both can make a result look better than it will stay. So run a test past the first few days, and check whether the gap holds for returning shoppers.
How do I compare a watched test with real shoppers?
Take the page that won your watched test and show it to only some of your live shoppers, while the rest see the current page. This is an A/B test, and nobody feels watched. Compare how many buy, over far more visits than the 10 shoppers in our example, which only keep the numbers simple. If the two results disagree, as with the tiny size buttons, trust what unwatched shoppers did and go back to your recordings to find out why.