(no title)
guzey | 4 years ago
>Do you believe in power-posing? In ego depletion? In hungry judges and brain training? [1]
>If the answer is no, then your priors for our knowledge about sleep should be weak because “sleep science” is mostly just rebranded cognitive psychology, with the vast majority of it being small-n, not pre-registered, p-hacked experiments.
>I have been able to find exactly one pre-registered experiment of the impact of prolonged sleep deprivation on cognition. It was published by economists from Harvard and MIT in 2020 and its pre-registered analysis found null effects of sleep on all variables of interest [2] (the authors changed analysis post-hoc and fished out some significant effects. Notably, they put the post-hoc results into the abstract but decided not to mention the null-preregistered results there or anywhere else in the paper explicitly).
>So why has sleep research not been facing a severe replication crisis, similar to psychology?
>First, compared to psychology, where you just have people fill out questionnaires, sleep research is slow, relatively expensive, and requires specialized equipment (e.g. EEG, actigraphs). So skeptical outsiders go for easier targets (like social psychology) while the insiders keep doing the same shoddy experiments because they need to keep their careers going somehow.
>Second, imagine if sleep researchers had conclusively shown that sleep is not important for memory, health, etc. – would they get any funding? No. Their jobs are literally predicated on convincing the NIH and other grantmakers that sleep is important. As Patrick McKenzie notes [3], “If you want a problem solved make it someone’s project. If you want it managed make it someone’s job.”
>Even in medicine, without pre-registered RCTs truth is extremely difficult to come by, with more than one half [4] of high-impact cancer papers failing to be replicated, and with one half of RCTs without pre-registration of positive outcomes being spun [5] by researchers as providing benefit when there’s none. And this is in medicine, which is infinitely more consequential and rigorous than psychology.
And here's my critique of Why We Sleep, which the author of the comment above decided to omit for some reason:
>Here are just a few of biggest issues (there were many more) with the book.
>1. Walker wrote: “Routinely sleeping less than six or seven hours a night demolishes your immune system, more than doubling your risk of cancer”, despite there being no evidence that cancer in general and sleep are related. There are obviously no RCTs on this, and, in fact, there’s not even a correlation between general cancer risk and sleep duration. [6]
>2. Walker falsified a graph from an academic study in the book. [7]
>3. Walker outright fakes data to support his “sleep epidemic” argument. The data on sleep duration Walker presents on the graph below simply does not exist [8]
[1] https://www.gleech.org/psych
[2] https://economics.mit.edu/files/16994
[3] https://twitter.com/patio11/status/1223695673742151680
[4] https://www.science.org/content/article/more-half-high-impac...
[5] https://journals.plos.org/plosone/article?id=10.1371/journal...
[6] https://guzey.com/books/why-we-sleep/#also-no----sleeping-le...
[7] https://guzey.com/books/why-we-sleep/#appendix-what-do-you-d...
[8] https://guzey.com/books/why-we-sleep/#ok-even-if-the-who-nev...
No comments yet.