top | item 39610067

(no title)

EricMausler | 2 years ago

Once you start discarding runs, doesn't that count as lying about the data/experiment? As in effectively equivalent to flipping the coin and only reporting it if it lands on heads?

In any case he does mention it won't fix all the problems but that it can clean up some of them and make addressing other problems a bit more straightforward. With p-values, sometimes even honest experiments end up with misleading conclusions. It takes a decent amount of complex understanding to perform properly, while something like gathering data and doing a conditional probability calculation has presumably less risk of honest mistakes

discuss

order

CrazyStat|2 years ago

Well, I'm discarding whole experiments. Each experiment is reported in full, I just don't publish all of them. No scientist ever does.

His claim about likelihoods being immune to p-hacking is far too strong.

> With p-values, sometimes even honest experiments end up with misleading conclusions. It takes a decent amount of complex understanding to perform properly, while something like gathering data and doing a conditional probability calculation has presumably less risk of honest mistakes

Maybe. I've seen an experienced Bayesian statistician who had published a paper about Lindley's paradox fall prey to Lindley's paradox and publish a misleading conclusion as a result. Bayesian analysis also has some pitfalls. Name withheld to protect the innocent.