p values are meant to relate whether or not 'some' hypothesis is statistically relevant versus a control study. For something like biology, human nature, dieting, weight loss, in a world where you can't really control how stressed someone is, how far they walk to work, among 100 other things... if you're trying to find out how 'the potato' works, safe citizen science (a Riff trial) may be far more effective than 'traditional' proofs to get to the bottom of something that works, and then someone can do a controlled study.
bumby|2 years ago
I don't know that I agree. Because the world has an infinite number of variants to "the potato" but an individual has a finite amount of time to try them. So while I agree that there are a lot of confounding factors in biology, we still need some way to triage our efforts. Analyzing the results of controlled studies seems like the best way to do so, even if it's not guaranteed to work for a specific individual.
What I suspect will happen is that people will end up wasting too much time on endless "potato" variants that don't work, and they would have had better results on focusing on those controlled studies that show promise.* For every person that finds a diamond in the 'riff', there's probably countless people that found nothing (or worse, detrimental effects). We also have other tools, like meta-analysis, to help. That seems more effective that "throw everything at the wall and see what sticks".
* you can see this in the supplement space. There are people who are constantly spending money searching for the secret new supplement hack when they'd be better off focusing on the things that science has shown to consistently work. They end up majoring in the minors, as it were
beanjuice|2 years ago