[hgsgm]

You can't simply ignore the base rate, even if you don't know it.

In a purely random world, 5% of experiments are false positives, at p=0.05. None are true positives.

In a well ordered world with brilliant hypotheses, there are no false positives.

If more than 5% of experiments show positive results at p=0.05, some of them are probably true, so you can try to replicate them with lower p.

p=0.05 is a filter for "worth trying to replicate" (but even that is modulated by cost of replication vs value of result).

The crisis in science is largely that people confuse "publishable" with "probably true". Anything "probably better then random guessing" is publishable to help other researchers, but that doesn't mean it's probably true.

[lisper]

> p=0.05 is a filter for "worth trying to replicate"

Yes, I think that is an excellent way to put it.

> The crisis in science is largely that people confuse "publishable" with "probably true".

I would put it slightly differently: people conflate "published in a top-tier peer-reviewed journal" with "true beyond reasonable dispute". They also conflate "not published in a top-tier peer-reviewed journal" with "almost certainly false."

But I think we're in substantial agreement here.