[timtadh]

In response to several threads here: it is important to distinguish when scientists are self critical vs. when non-scientists are critical of the scientific method. For instance, there is a long history of scientists criticizing how the scientific process is currently conducted for the purposes of improving the scientific endeavor. That work is sometimes used by non-scientists who question the overall scientific method. However, such use is invalid as the scientific self-criticism

1. assumes the validity of the scientific method

2. relies on the scientific method as its critical lens

Whereas those who critique science as a whole:

1. assume that the scientific method does not work and does not arrive at "truth"

2. then use scientists being self critical to prove #1.

Such a "proof" does not work as there is its uses the assumption "the scientific method arrives at truth" to derive the contradiction "the scientific method does not arrive at truth". See for instance comment: https://news.ycombinator.com/item?id=16859200

In reality, work on reproducibility is about improving the practice of science overall. It does not in itself show that science is inherently untrustworthy. What it does show is that scientific discovery is difficult and it takes a lot of effort and new findings should be treated critically. What does critically mean in this context? It means with in the boundaries of science analyzing the theoretical basis, hypothesis, method, and experimental results for potential flaws. It does not mean to be skeptical as a default because science "doesn't work."

[323454]

I agree with your overall point, but technically speaking it is logically valid to prove a hypothesis false by first assuming it and then deriving a contradiction, even when the contradiction is the negation of the original hypothesis (as it is in your example).

What you should have said is that some critics start with the premise "the scientific method does not arrive at truth", and then use other people's arguments that depend on the premise "the scientific method arrives at truth" to support their claim, which is indeed logically invalid.

[buvanshak]

>scientists criticizing how the scientific process is currently conducted for the purposes of improving the scientific endeavor.

I think what happening here is a bit more serious. They are showing a widespread crisis. It is not just some minor feedback to improve the process.

>It does not in itself show that science is inherently untrustworthy.

I think when statistics is involved, the results are inherently untrustworthy. This is not really surprising because there is a whole bunch of ways these studies that involve statistics could go wrong. And we are still finding new ways on how this could go wrong.

Then there are things like publication bias, that takes this to a whole new level. Things like that means that a biased body of journals can project any consensus that it favors just by selecting studies that fit its narrative. The inherent issues with statistics means that you can find studies that shows any possible outcome.

[TangoTrotFox]

>"I think when statistics is involved, the results are inherently untrustworthy. This is not really surprising because there is a whole bunch of ways these studies that involve statistics could go wrong. And we are still finding new ways on how this could go wrong."

Another very real issue here is that malicious use of statistics can be used to show nearly anything in ways that can be extremely difficult to detect, even when the maliciousness is hidden in plain sight. And then going a step beyond that there's plain old number fudging which is almost impossible to prove since variance works as sufficient plausible deniability. And finally there is of course plain old ineptitude. Like you mention even when trying to do things completely by the book, statistics are incredibly difficult to get right.

Something that comes to mind here is the recent MIT study stating that Uber drivers earned $3.37/hour. That study was completely broken. [1] It's debatable whether the cause was maliciousness or ineptitude, but the point is that these problems arise, with a disturbing regularity, even when the most reputable of names are attached to them.

[1] - https://qz.com/1222744/mits-uber-study-couldnt-possibly-have...

[haZard_OS]

>I think when statistics is involved, the results are inherently untrustworthy.

Ummm...are you kidding? Statistically vetted results are inherently UNCERTAIN, but how could they possibly be inherently untrustworthy?

Even if a mechanistic effect is observed, its relationship to a particular cause or influence is only established statistically. In fact, the very observation is often performed u Dee the umbrella of statistical calibration of appropriate instruments.

  As Pearson said, "Statistics is the grammar of science".