top | item 20740036

(no title)

BookmarkSaver | 6 years ago

Kinda looks like p-hacking. They ran calculated the p-value for a bunch of different criteria until they found one that was "statistically significant" (in this case gender, specifically just one gender). The issue is that you'll find 5% outliers 5% of the time, so if you just keep looking you'll eventually find "significance".

This would demand reproduction before action.

discuss

order

xref|6 years ago

Why is someone raising the question of p-hacking and suggesting a replicated study is needed being downvoted?? The paper is described as “barely statistically significant” so p-hacking, which on HN we see is rampant, seems like it should be discussed

Smithalicious|6 years ago

It only looks like p-hacking if you go in with the belief that the results can not possibly be correct, and even then Hanlon's Razor would make me reluctant to accuse them of p-hacking.

dtech|6 years ago

If an effect only occurs in a subgroup which we have no explanation for, it's a smell that different subgroups were tried until a "statistically significant" one was found by chance.