[noslenwerdna]

He is implying that the scientists involved haven't thought of those questions, when in reality this field is one of the strictest in terms of statistical procedures like pre registeration, blinding, multiple hypothesis testing etc

Also he makes many factual claims that are just incorrect.

Just seems like an extremely arrogant guy who hasn't done his homework

[ttpphd]

A computer scientist/electrical engineer who is arrogant? I dunno, I need to see the statistical test to believe that's possible.

[eightysixfour]

Computers are a “complete” system where everything they do is inspectable and, eventually, explainable, and I have observed that people who work with computers (myself included) over estimate their ability to interrogate and explain complex, emergent systems - economics, physics, etc. - which are not literally built on formal logic.

[BeetleB]

> when in reality this field is one of the strictest in terms of statistical procedures like pre registeration, blinding, multiple hypothesis testing etc

I'm not in HEP, but my graduate work had overlap with condensed matter physics. I worked with physics professors/students in a top 10 physics school (which had Nobel laureates, although I didn't work with them).

Things may have changed since then, but the majority of them had no idea what pre-registration meant, and none had taken a course on statistics. In most US universities, statistics is not required for a physics degree (although it is for an engineering one). When I probed them, the response was "Why should we take a whole course on it? We study what we need in quantum mechanics courses."

No, my friend. You studied probability. Not statistics.

Whatever you can say about reproducibility in the social sciences, a typical professor in those fields knew and understood an order of magnitude more statistics than physicists.

[noslenwerdna]

As an ex-HEP, I can confirm that yes, we had blinding and did correct for multiple hypothesis testing explicitly. As Kyle Cranmer points out, we called it the "look elsewhere effect." Blinding is enforced by the physics group. You are not allowed to look at a signal region until you have basically finished your analysis.

For pre-registration, this might be debatable, but what I meant was that we have teams of people looking for specific signals (SUSY, etc). Each of those teams would have generated monte carlo simulations of their signals and compared those with backgrounds. Generally speaking, analysis teams were looking for something specific in the data.

However, there are sometimes more general "bump hunts", which you could argue didn't have preregistration. But on the other hand, they are generally looking for bumps with a specific signature (say, two leptons).

So yes, people in HEP generally are knowledgeable about stats... and yes, this field is extremely strict compared to psychology for example.