Are you sure about that, tho? Can you prove that it's actually a p-hack? You can level the accusation, but how do you go about substantiating it? I'd like to see the numbers and the data, not bombast.
It's actually up to the paper to prove that it's not a p-hack. That's done by declaring a null hypothesis and the intended stats analysis to be done before doing the experiment and collecting the data, because otherwise you can just compare arbitrary things until you find that some of them are correlated in the set you have. There's no numerical post-hoc analysis that would be able to prove that.
Exactly - a p-hack is a form of misreporting, so isn't easy to detect at the level of individual studies.
However, p-hacking isn't the only threat to replicability: in this case the authors have reported a large set of tests, so we can ask why they didn't control family-wise error via Bonferroni or friends (in which case the reported statistical significance almost certainly disappears). Also, I suspect if you fit Bayesian models (either separately or a hierarchical model) using reasonably narrow priors based on what we know about human sensitivity to magnetic fields, how much other senses are affected by sex differences, hunger, etc., and not starting each test assuming a complete state of ignorance of the world, then the data would be compatible with no effect.
Is it really tho? Up to the paper to prove it's not something you say it is? I don't think so. The paper is what it is. If you claim it's fake, you need to make evidence for that. Not just launch lazy accusations. Seriously, what is what you said, if not just a lazy dismissal (oh it must be a p-hack)? And you're content with that?
That's not science. They went to the trouble. If you're going to just dismiss it, and claim that's valid, you should go to the trouble to do science on that hypothesis you have. Thanks, friend! :P ;) xx
thatcat|5 years ago
bcdarwin|5 years ago
However, p-hacking isn't the only threat to replicability: in this case the authors have reported a large set of tests, so we can ask why they didn't control family-wise error via Bonferroni or friends (in which case the reported statistical significance almost certainly disappears). Also, I suspect if you fit Bayesian models (either separately or a hierarchical model) using reasonably narrow priors based on what we know about human sensitivity to magnetic fields, how much other senses are affected by sex differences, hunger, etc., and not starting each test assuming a complete state of ignorance of the world, then the data would be compatible with no effect.
jolincost|5 years ago
That's not science. They went to the trouble. If you're going to just dismiss it, and claim that's valid, you should go to the trouble to do science on that hypothesis you have. Thanks, friend! :P ;) xx