[btheshoe]

On the whole, all these scandals in manipulated research have deeply shaken my trust in many of our scientific institutions. It's clear by now this isn't the case of a few bad apples - our scientific institutions are systemically broken in ways that promote spreading fraudulent results as established scientific truth.

[Veserv]

As a aside, the phrase "a few bad apples" is actually originally "a few bad apples spoil the barrel" referencing the fact that a bad/overripe apple causes nearby apples to quickly ripen and go bad which is now known to be due to ripe apples producing ethylene gas which accelerates the ripening of other nearby apples. The phrase originally meant that one bad thing corrupts and destroys all associated. The discovery of a bad apple actually means everything is already irrevocably destroyed and thus reason for not tolerating even a single bad apple.

A modern metaphor with a somewhat similar meaning to the original is: "A fish rots from the head down." Pointing out that organizational failures are usually the result of bad leadership. A rotten leadership will quickly result in a rotten organization. Therefore, it is important to make sure the leadership is not rotten in a organization. It also points out that low-level failures indicate there are deeper high-level failures. If the line-level is screwed up, the leadership is almost certainly just as screwed up. The fix being replacing the rotten leadership with a new one as lower-level fixes will not fix the rotten head.

Another, more direct equivalent metaphor is a Chinese saying translated as: "One piece of rat poop spoils the pot of soup." That is hopefully self-explanatory. We should probably use it instead of "a few bad apples" as nobody will reverse the meaning of that one.

[capableweb]

As an aside to your aside, it's also the case that phrases/words change meaning over time, as usage in one grows above the usage in a different way.

In this case, the "a few bad apples are not representative of a group" meaning have grown above the "One bad apple spoils the barrel" meaning, and so the phrase as changed, for better or worse.

Maybe it would be best if everyone used the long version instead of the short one. When you say/write "A few bad apples", the meaning is ambiguous, but if you use the long version, it's not. Problem solved :)

[dataflow]

I don't think the phrase has changed meaning?

> The phrase originally meant that one bad thing corrupts and destroys all associated.

It's saying if you don't remove the bad apple, you will get a lot of bad apples in the future. The presence of a bad apple doesn't imply all of them are already spoiled right now. If you're seeing other apples that still haven't spoiled, it suggests you still have time to do damage control. That seems consistent with how the metaphor is used nowadays.

[stubybubs]

I think there's a transition phase between the two that people miss out on. I recall hearing "let's not let a few bad apples spoil the bunch" which is an acknowledgement of the original phrase, reworked to implore listeners not to throw it all away. You could say "let's not throw the baby out with the bathwater" but I guess some people are apple enthusiasts?

[blackbear_]

I would like to point out that "scientific truth" does not really exist, or at least is far from straightforward to define and establish. Basically, you should see each piece of research as evidence for a certain hypothesis, and the more evidence is available, the more that hypothesis is believable.

But the larger issue here is that all public institutions are, by that definition, broken. For example, businesses also won't hesitate to spread falsehoods to sell their stuff, governments will try to convince their people that they are needed through propaganda and policing, and so on.

How do we solve these problems? We have laws to regulate what businesses can't do (nevermind lobbying), and we split governments' responsibility so that no single branch becomes too powerful. In general, we have several independent institutions that keep an eye on each other.

In case of science, we trust other scientists to replicate and confirm previous findings. It is a self-correcting mechanism, whereby sloppy or fraudulent research is eventually singled-out, as it happened in this and many other cases.

So I guess the gist of what I want to say is that you're right in not trusting a piece of research just because it was made by a reputable institute, but look for solid results that were replicated by independent researchers (and the gold standard here is replication, not peer review)

[mike_hearn]

> businesses also won't hesitate to spread falsehoods to sell their stuff

They do hesitate. It's quite hard to catch businesses openly lying about their own products because, as you observe, there are so many systems and institutions out there trying to get them. Regulators but also lawyers (class action + ambulance-chasers), politicians, journalists, activists, consumer research people. Also you can criticize companies all day and not get banned from social media.

A good example of what happens when someone forgets this is Elizabeth Holmes. Exposed by a journalist, prosecuted, jailed.

Public institutions are quite well insulated in comparison. Journalists virtually never investigate them, preferring to take their word as gospel. There are few crimes on the book that can jail them regardless of what they say or do, they are often allowed to investigate themselves, criticism is often branded misinformation and then banned, and many people automatically discard any accusation of malfeasance on the assumption that as the institutions claim to be non-profit, corruption is nearly impossible.

> It is a self-correcting mechanism, whereby sloppy or fraudulent research is eventually singled-out, as it happened in this and many other cases.

It's not self correcting sadly, far from it. If it were self-correcting then the Stanford President's fraud would have been exposed by other scientists years ago, it wouldn't be so easy to find examples of it and we wouldn't see editors of famous journals estimate that half or more of their research is bad. In practice cases where there are consequences are the exception rather than the norm, it's usually found by highly patient outsiders and it almost always takes years of effort by them to get anywhere. Even then the default expected outcome is nothing. Bear in mind that there had been many attempts to flag fraud at the MTL labs before and he had simply ignored them without consequence.

[matthewdgreen]

Alternatively there is a baseline of fraudulent behavior in any human organization of 1-5% and since there are tens of thousands of high-profile researchers this sort of thing is inevitable. The question you should be asking is whether the field is able to correct and address its mistakes. Ironically cases like this one are the success stories: we don’t have enough data to know how many cases we’re missing.

[vintermann]

I don't think the baseline is the same. The more competition, the more temptation to cheat. When the margins to win are small enough, cheaters are disproportionately rewarded.

Think of Tour de France. Famously doping-riddled. There are a lot of clean cyclists, but they are much less likely to be able to compete in the tour.

You can fight cheating with policing: doping controls, etc. But as the competition gets more extreme, the more resources you need to spend on policing. There's a breaking point, where what you need to spend on policing exceeds what you get from competition.

This is why almost no municipalities have a free-for-all policy for taxis. There are too many people technically able to drive people for money. All that competition drives prices lower, sure, but asymptotically. You get less and less lower prices the more competition you pile on - but the incentives for taxi drivers to cheat (by evading taxes, doing money laundering as a side gig etc.) keep growing. London did an interesting thing - with their gruelling geography knowledge exam, they tried to use all that competitive energy to buy something other than marginally lower prices. Still incentive to cheat, of course, but catching cheaters on an exam is probably cheaper and easier than catching cheaters in the economy.

(Municipalities that auction taxi permits get to keep most of the bad incentives, without the advantage of competition on price.)

[hayd]

It's only a story because he's president, if he were only a researcher/professor this would not even be a story. This is NOT a success story, it shows that this fraudulent behavior is endemic and an effective strategy for climbing the academic ladder.

A success story would be this is exposed at large... we work out some kind of effective peer-reproduced tests... and the hundreds/thousands of cheating professors are fired.

[panarky]

The very fact that the fraud is discovered, that reporters amplify it, and that it can bring down the president of the university, is evidence to me that the system still works.

[michael1999]

No. This level of scrutiny and diligence is rare, and was selectively applied based on the targets profile. The "field" did nothing about this over 20 years. A computer science freshman did this as a hobby, not as a participant in neuroscience.

Perhaps "nothing" is too harsh. Various people in the field raised concerns on several occasions. But the journals did nothing. The "field" still honoured him. And _Stanford_ did nothing (except enable him and pay him well) until public embarrassment revealed the ugliness.

[karaterobot]

The problem is that we don't know what the baseline really is. We know that between a third and a half of results from peer reviewed papers in many domains cannot be replicated. Looking closer, we see what look like irregularities in some of them, but it's harder to say which of them are fraud, which are honest mistakes, and which of them just can't be replicated due to some other factors. But because so many of these studies just don't pay off for one reason or another, I would agree that it is getting really hard to rely on a process which is, if nothing else, supposed to result in reliable and trustworthy information.

[whydoyoucare]

If a field takes two decades to "correct" its mistakes, then there are several things wrong with it. And if we have top positions held by unethical people, who have got away with it, and possibly climbed to the top because of it, then I do not know what to feel or say about this.

[jlawson]

Any human organization?

I don't expect 1-5% fraud in airline pilots, bank tellers, grocery store clerks, judges, structural engineers, restaurant chefs, or even cops (they can be assholes but you don't have to bribe them in functional countries).

I think academics can do better than 1-5% fraudulent.

[huijzer]

I’ve come to believe that science is mostly about popularity and not about truth-finding. As long as peers like what you write, then you will get through the reviews and get cited. Feynman called this Cargo Cult Science. I think much of science is like this, see also Why Most Published Scientific Findings are False. Not much has changed since the publication of that paper. A few Open Science checks are not gonna solve the fundamental misalignment of incentives.

[asynchronous]

Wholeheartedly agree, really a shame to see what it’s become. Wish I could still see research the way I dreamed it was as a child.

[strikelaserclaw]

it is impossible for most scientists to understand / critically think about all the research coming out from so many institutions, so most of these academics mainly focus on research coming from someone they respect / institutions they respect, so yes it is kind of like a popularity contest but i would argue that most things in life are due to the limited nature of the human brain we cannot think independently about everything for ourselves and rely on external judgements to what is important / true etc...

[arvinsim]

"It is difficult to get a man to understand something when his salary depends on his not understanding it." -- Upton Sinclair

[dadrian]

The peer review system is not designed to catch fraud, it's designed to catch scientific or experimental errors.

Giving up on science is such a vast overgeneralization. You could take your statement and replace "manipulated research", "scientific institutions" and "established scientific truth" with just about any negative article in any domain. You could just as easily make this statement about startups (Theranos, Juicero), or government, or religion, or suburbs, or cities...

[btheshoe]

> The peer review system is not designed to catch fraud, it's designed to catch scientific or experimental errors.

Yes.

> Giving up on science is such a vast overgeneralization. You could take your statement and replace "manipulated research", "scientific institutions" and "established scientific truth" with just about any negative article in any domain. You could just as easily make this statement about startups (Theranos, Juicero), or government, or religion, or suburbs, or cities...

Institutions go through similar cycles of breaking and systemic reform. Not surprised that you can see patterns in other domains.

[strangattractor]

It often does neither:( The only real protection from fraud, mistakes and poor science is replication. If results can't be replicated by others it is not science.

[zitterbewegung]

If you implement a strategy such as publish or perish exceedingly smart people will game the system to win. Any metric gets gamed.

Look at papers that have real impact they get cited. Look at ones that don’t …

[thelittlenag]

And not just that, but rewarding outsized effect sizes so that you reward folks who create the biggest lies with fraudulent stats.

[akhayam]

And you have some of the smartest brains gaming it too... Such a sad use of good neurons :(

[Der_Einzige]

Even, and especially fields that should be reproducable, like Machine Learning, are FULL of garbage:

https://www.technologyreview.com/2020/11/12/1011944/artifici...

https://reproducible.cs.princeton.edu/

https://ui.adsabs.harvard.edu/abs/2018Sci...359..725H/abstra...

[NeuroCoder]

I can't speak for other fields but in Neuro there's plenty of this but often one learns how to catch it before using it in your own research, even if it never becomes a matter of public scrutiny. Unfortunately, I can't reassure you that bad research gets caught all the time. However, there's usually at least a couple of experts in a given sub field of Neuro that quickly call BS before something goes too far.

[rgmerk]

This is an excellent point. A lot of crappy research goes on, and nobody pays it any attention (except, occasionally, when cranks outside the field want to prove that "peer-reviewed research proves the Earth is flat).

It's frankly not worth the effort to debunk a shitty piece of researchin a low-profile journal that's never been cited in a decade.

[jeremyjh]

> in Neuro there's plenty of this but often one learns how to catch it before using it in your own research, even if it never becomes a matter of public scrutiny.

And what happens when it is caught, it is just quietly ignored by the field, right? How often are there retractions?

[tptacek]

A vast amount of "science" is being done at all times. You can likely count the scandals cognitively available to you on one hand; even if it took dozens of hands, you'd still be talking about an infinitesimal sliver of science on the whole. What's actually happening here is an availability bias: you remember scandals, because they're scandalous and thus memorable. You don't know anything about the overwhelming majority of scientific work that is being done, so you have no way of weighting it against the impression those scandals create in your mind.

[kansface]

Via HN yesterday [1]- an editor of _Anaesthesia_ did a meta study of the papers he handled that conducted RCTs. He had data from 150 of them and concluded:

> ...26% of the papers had problems that were so widespread that the trial was impossible to trust, he judged — either because the authors were incompetent, or because they had faked the data.

This is not a one off.

[1] https://www.nature.com/articles/d41586-023-02299-w

[carpet_wheel]

No one is shocked by the concept of misconduct occurring, the issue here is that it is no longer surprising when those committing the misconduct end up running the organization. You can pretend that the conversation is about whether scientific misconduct is endemic, but that conversation being had is about the failure of these hierarchies to actually succeed in promoting the best from among their ranks.

Of course misconduct is unavoidable, that doesn't mean you should become president. The politics aren't working.

[cycomanic]

Is that the correct conclusion to draw? I mean there are definitely big problems on how we conduct and fund scientific research (which might also contribute to fraud), but the number of research scandals is a tiny fraction to the amount of research being done.

Considering that we get fraud every time we have humans and prestige money, I would really like to see some statistics against other things human activities. I suspect science still has some of the lowest fraud rates and the strongest mechanisms to detect and deal with it.

[P_I_Staker]

The problem is a tiny percentage gets any attention whatsoever. It's the same with police and doctoral abuse. These things are hugely prevalent with 30-80% of professionals engaging in some form of abuse... same with fraud.

People know about police abuse. We don't talk about doctor abuse. I'm honestly not confident that there's any police/doctors that don't engage in abusive practices (or it's a tiny percentage of the population).

[downWidOutaFite]

It's the same everywhere not just science. The fake-it-till-you-make-it type-A charismatic bullshitters rise up the ranks in all organizations.

[akhayam]

I feel this trend taking root in academia is still a new-ish thing. The boundaries of academia and research, especially for computer science, really started blending 15-20 years ago as Big Tech took over Oil for the best paying job / grant.

The decay has been super fast though. Maybe some academics will find the courage to do a longitudinal study of this decay. Now that'll be an interesting paper to read.

[iancmceachern]

Especially that the folks that are committing the fraud are raising to high places. It goes to show that we have systemic problems. This isn't a failure of a few individuals but a failure of our institutions. Clearly our incentive structure is messed up if people like this are in positions like this. Clearly we need to not only address this individuals actions, but the systemic issues that led to his ability to do what he did and still rise to the position he did.

[janalsncm]

Scientific institutions aren’t perfect. They’re made up of people like anywhere else. And where there are people there will be politics and gamesmanship. That doesn’t mean science isn’t our best shot at figuring out how the world works.

The fact that a Stanford president can be pushed out for bad research conducted before he was even there? It tells me there’s still some integrity left.

[kansface]

The article from Nature yesterday came up with 26% of the peer reviewed published papers they examined (all RCT) were untrustworthy based on close examination of their data. They could only invalidate 2% without data.

I personally believe this is an underestimate.

[wesleywt]

The problem is not with science, it is your science illiteracy. You never accept scientific research as "true" until it has been verify and repeated by independent sources. It has become the culture to glorify unexpected and "interesting" results in the media and society at large. But you should find these "interesting" but not necessary believe it to be true.

We do get unbelievable findings such as CRISPR, but beware these will be very few and far between.

[0xDEF]

>our scientific institutions are systemically broken in ways that promote spreading fraudulent results as established scientific truth

Scientific consensus is still very reliable and if 95% of accredited scientists in a field say something is true it is in society's best interest to consider that to be the truth.

[Apofis]

I truly hope they toss every single paper and citations to them that ever crossed this assholes desk. This misconduct literally should be treated the same as a dirty detectives cases being reviewed and tossed out since they are no longer trustworthy.

[robocat]

I hope you are forced to live in an authoritarian situation: so you may truly learn what it is like to be punished for the mistakes of others.

The point here is to save the good apples - not throw out the whole barrel for zero gain.

[edgyquant]

Yep. After years of pushing back against claims that researchers skewed scientific results to fit their agenda this is a huge, demoralizing blow. Even if it isn’t widespread, how can you honestly blame anyone for being skeptical anymore.

[strangattractor]

Wide spread? PI's are required to publish. It is impossible to maintain quality of papers via peer review at scale so bad papers usually get through simply because of the volume. Throw in a profit motive and people get creative about hiding it.

See this recently published article https://www.nature.com/articles/d41586-023-02299-w.

One would think that clinical trials would be documented and scrutinized out the yin-yang but they are not.

[mistermann]

But it was caught, demonstrating that what we're constantly assured is true is actually true: science may not be perfect, but it catches all of its mistakes, therefore we should trust it above all(!) other disciplines.

[throwaway4837]

How? Peer-review, re-review, journalism, and reproduction of results are the systems the scientific community is built upon. The system does its job of finding the bad apples, as it did here.

Bad things are gonna happen in every single institution ever created. A better measure is how long those things persist.

Science is about getting closer to "the truth". Sometimes science goes further away from the truth, sometimes it gets closer. Sometimes bad actors get us further away from the truth. It gets reconciled eventually.

[alfalfasprout]

A combination of "publish or perish" and papers not accepting "negative results" (which results in a ton of repeated research) has led to this.

[jonhohle]

You mean we can’t just Trust the Science™?!

I completely agree. Seeing slack chats and emails regarding Proximal Origin and how researchers were disagreeing with it all the way until they published a paper that served what purpose are really disheartening. Instead of guiding future research toward preventing similar outcomes, countless scientists spent untold years of combined effort on a theory the authors didn’t even believe.

[callalex]

It should be noted that the volume of corruption coming out of state-run schools is much smaller than that from private institutions.

[piuantiderp]

I'd wager it is just better covered up

[stjohnswarts]

It strengthens mine. Science is self correcting in a way the religion and politics never can be, they keep making the same faith based mistakes over and over, while science continues to progress. Evidence of that is everywhere you look, whereas politics and religion have barely made any progress in hundreds of years.

[rmbyrro]

To the contrary, it increases my trust.

Just the fact that Stanford managed to conduct an independent investigation against its OWN PRESIDENT, tells very positive things about the University.

After this episode, I might trust Stanford research even a bit more than any University that never caught fraudsters.

[lynx23]

Agreed. The system is flawed. And as a result, many scientific "findings" simply can't be trusted. And there is no solution in sight.

[chasing]

"A database of retractions shows that only four in every 10,000 papers are retracted."

Every time a plane crashes it's international news. But just because you regularly hear about plane crashes doesn't mean flying is unsafe.

[btheshoe]

do me a favor and look up all the papers in thinking fast and slow that failed to replicate

[partomniscient]

One has to ask what there is left to trust at all?

[stillbourne]

But they got caught, they retracted, the system works. It's not a perfect system, in a perfect system people wouldn't be incentivized to publish publish publish or be damned to the back waters. The institution is broken, but the safety nets work.