[grigri907]

I'm tempted to try this experiment, but I get stuck at "what if my list of 100 things is subconsciously skewed by ads I'm already seeing?"

It's the ads I'm seeing, but aren't registering on a conscious level that concern me. I think we see far more ads than we are aware of.

[gruez]

That's where the "randomly choose 50" comes in. Even if your initial list of 100 topics was biased, randomly picking from that list allows you to do a comparison between topics that got randomly picked vs ones that didn't. In other words, a randomized controlled trial, the gold standard in experiments. If the group that got randomly picked got a 40% hit rate (ie. that corresponded to ads), but group that didn't randomly get picked got 41%, then you can probably conclude they're not listening to you, even if a 40% hit rate seems spooky. On the other hand if there were significant differences (eg. 40% vs 20%), then there might be other stuff going on worth investigating.

[giancarlostoro]

I both love and am horrified by this answer. What if the airport grade tar was there all along?

The other wild thing is I was severely out of town, the ad was very local but way too uncomfortably relevant.