top | item 20626500

Hospital checklists are meant to save lives, so why do they often fail?

158 points| areoform | 6 years ago |nature.com | reply

148 comments

order
[+] OliverJones|6 years ago|reply
I wonder. Does the Hawthorne Effect (https://en.wikipedia.org/wiki/Hawthorne_effect) contribute to the success of these checklist regimes?

When surgical or emergency-department teams believe they're participating in an experiment to learn whether certain practices improve outcomes, do the outcomes improve?

Airplane pilots use checklists. It's illegal to start the engines unless the approved checklist book is within reach of the pilot in command. In training it's drilled into us that the response to panic is: get out the checklist and follow it. It works. duh. It's all too easy under pressure to forget to lower the wheels or turn on the carburetor heat, or whatever. The drilling in training is what makes the difference.

(Aviation gasoline cools as it vaporizes in carburetors. Under certain conditions that can make the carburetor fill up with ice. If that happens the airplane will land soon. Not good. So carburetors have heaters. )

Checklists have far less success in medicine. Heck, almost two centuries ago Dr. Ignatz Semmelweis (https://en.wikipedia.org/wiki/Ignaz_Semmelweis) found that patients lived a lot longer when doctors and nurses washed their hands.

But, even in the 21st century, it's a struggle to get compliance with hand-washing rules in hospitals. Maybe something about medical training makes people resistant to fixed procedures. Maybe it's the age-old practice of eminence-based medicine rather than evidence-based medicine. It's a serious problem.

[+] redis_mlc|6 years ago|reply
> Airplane pilots use checklists. It's illegal to start the engines unless the approved checklist book is within reach of the pilot in command. In training it's drilled into us that the response to panic is: get out the checklist and follow it. It works.

Just some clarifications:

- It's not illegal to fly without a checklist under Part 91 (non air-carriers.) When I fly with military pilots, they use a flow for starting small airplanes:

http://fsims.faa.gov/PICDetail.aspx?docId=8900.1,Vol.3,Ch32,...

- the airplane manual is a book, and must be on the airplane. A checklist is a list, not a manual. The checklist can be electronic, as in the HondaJet.

- it's a good idea to follow the emergency procedures either from memory, or a checklist if there's time. Typically recovering from a stall or spin, or feathering a bad engine is done from memory, for example.

Source: commercially-rated airplane pilot.

[+] perl4ever|6 years ago|reply
Without any specific relevant experience in a hospital, isn't the most likely reason procedures fail that people are required to ignore/bypass parts of them constantly, rather than having them updated to match reality?

It seems so very obvious to me that any time you start asking, from a 40,000 foot level, why people can't follow simple instructions, the answer is going to be, not that they are all idiots, but that they are trained and required to ignore them selectively based on complex ever shifting criteria. That is basically what makes any job require a human and not a computer in the first place.

The more rigid you want people to be, the better the process has to be.

[+] mbesto|6 years ago|reply
> But, even in the 21st century, it's a struggle to get compliance with hand-washing rules in hospitals.

I think it's more likely that there is an immediate feedback loop to disregarding the checklist (i.e. the plane crashes and the pilot dies), whereas not washing your hands has a lot more delayed consequences in which the causation isn't readily known. I think this is the general problem with Outcomes Based Health - we have the stats to back up its effectiveness, but don't have the immediate feedback loops to promote it.

[+] AmericanChopper|6 years ago|reply
> In training it's drilled into us that the response to panic is: get out the checklist and follow it

I do this, and always teach others to do this, during incident response. The reason I think it works is it reduces your burden for making decisions during a stressful time. No matter who you are, how good you are at managing stress, or how experienced you are, you’ll almost always end up making worse decisions under pressure, so removing the need to make decisions as much as possible seems to generally lead to better outcomes.

[+] bsder|6 years ago|reply
> But, even in the 21st century, it's a struggle to get compliance with hand-washing rules in hospitals.

Wash your hands 12 times a day for a week and you'll understand why people might not want to comply.

I want my doctor to wash his hands, but I can at least sympathize as to why he might not want to.

[+] mschuster91|6 years ago|reply
> But, even in the 21st century, it's a struggle to get compliance with hand-washing rules in hospitals.

Ever talked with a nurse or a doctor? The reason is simple: way, way not enough staff at hand combined with maximum allowed/planned time for a certain procedure (e.g. 10 minutes for a full assisted shower) which only works in perfect conditions but falls apart under the slightest change (e.g. patient wet themselves), and combined with ruthless MBA-style C-level execs actually firing people for not keeping up with the numbers.

Medicine, especially highly critical / urgent care, needs to be socialized and shielded off from all negative effects of modern-day capitalism if this is to change.

[+] sjg007|6 years ago|reply
Gloves are another one.. blows my mind.
[+] heymijo|6 years ago|reply
An ophthalmologist I know is an attending physician, which means she teachers new resident physicians.

She described a few surgeries to me and her role in the room as the resident performs the surgery. Her brain is the checklist for the surgical procedure. The culture seemed to be, you should be good enough to memorize these complex procedures.

Meanwhile I'm thinking, why not just have someone else in the room reading and confirming the next step to the surgeon?

I suspect the culture of medicine, and especially the culture of surgeons does not jive well with checklist culture.

[+] joshgel|6 years ago|reply
This [1] is a great article that I think gets at this topic a bit. Basically, germ theory and anesthesia (ether) were both discovered around the same time. Yet anesthesia spread around the world in a matter of years, while germ theory is still a struggle for doctors and hospitals. I presume the reasons checklists are underutilized are similar, lack of immediate consequence, low probably of consequence, and added difficulty instead of simplifying things. We must do better.

[1] https://www.newyorker.com/magazine/2013/07/29/slow-ideas

[+] iguy|6 years ago|reply
I suspect that what made the culture of pilots change is that their own lives are on the line. That, and the fact that the guys flying the coolest fastest planes had commanding officers who could simply ground them if they didn't like it.
[+] bluGill|6 years ago|reply
Part of the problem is humans are all slightly different. Thus doctors need to understand the system because there is always something in the details that is unique and so the checklist needs to be deviated from. Once you are in the habit of this is different you quit using the checklist...

Checklists are very useful anyway because they let you focus on and remember the parts that are actually different. However when someone is highly trained it is easy to think you already know everything and not use the checklist.

[+] derekp7|6 years ago|reply
Complexity probably has something to do with it. Every surgery is slightly different. So a reasonable size checklist would be missing enough items to kill the patient, and a complete one, the patient dies before you are 1% through it.
[+] pmiller2|6 years ago|reply
I want my surgeon to know the procedure cold, but I would definitely not mind having a backup in the form of a checklist.
[+] therealx|6 years ago|reply
On the flip side, I've seen with my two eyes, doctors watching a youtube version of the surgery before doing the real thing.

I wasn't in a position to verbally ask but it seemed common.

[+] blondin|6 years ago|reply
well, i heard in a podcast that surgeons -- not to pick on them but that was the example given -- don't have their own lives on the line and would likely ignore checklists. compared to say pilots.

yet, pilots make mistake too. examples were given when they rush and ignore the checklist or the second in command for example. rushing is usually the biggest mistake. and ignoring a lower ranking pilot is another big one.

[+] conorh|6 years ago|reply
They fail because of severe checklist fatigue according to my wife (a surgeon). Checklists are great - so let's do them for everything, ALL the time. People start to tune them out because they do the same checklist hundreds and hundreds of times and they often have many irrelevant things on them. The people using the checklists have little ability to change them or to improve them and so they start to suffer from checklist fatigue. As they say in the article:

> “There's no point in having an item that says, 'Have the antibiotics been given?' if there are no antibiotics in the hospital,” says Dixon-Woods.

The checklists that my wife uses are apparently filled with these types of irrelevant checks.

Not to say that they aren't useful, they are, but as this article points out the implementation of checklists often leaves a lot to be desired.

[+] extrapickles|6 years ago|reply
I'm currently in the process of introducing checklists into a process, and the hardest part of the entire endeavor is giving the people who are using the checklist the power to modify them, and keeping other people from forcing the system to have chiseled into stone type checklists.

So far, the biggest contributor to 'checklist fatigue' among people using the system is from them not being able to modify the checklist to suit the task at hand. Checklists really need to be living documents, so errors and usability improvements can be made. The article does mention that the more proscribed from on-high the checklist was, the more the performance of the checklist was degraded.

[+] noneeeed|6 years ago|reply
> "The people using the checklists have little ability to change them or to improve them"

I was listing to an interview with the author of The Checklist Manifesto, Atul Gawande, and he cited this as one of the main reasons why attempts to use them fail in some organisations. He seems a big proponent of the people using checklists being the ones who should write and imporve them or at least be involved, and that they be kept short and focused. They shouldn't be stuffed with needless detail, they are not instruction manuals.

Checklists imposed from above without input from those who use them is just another version of beauracratic organisations imposing processes on people.

[+] privateSFacct|6 years ago|reply
Also did some work ages ago with a safety critical element. Checklist fatigue is real.

Especially in govt compliance areas - a lot of folks have attitude of parent, get over the fatigue because this is "important". I once went to a presentation where they showed someone in handcuffs because they didn't follow the procedure. There are 100s of irrelevant items (yes - 8 track data standards compliance in some docs still) on some of these lists and people know it is total BS.

The authors of the article make a great point, local adaption and implementation. The checklists that work are carefully curated and honed. I currently work in a space with lots of checklists, and I take that approach, do we still need this item? Things change. If you make the checklist useful MUCH higher compliance. That means updating and refining it so it's the reference guide for what should be happening not an endless list of all possible things one could do.

[+] JoeAltmaier|6 years ago|reply
There are solutions for that (e.g. point-and-say). Surgeons need to stop making excuses.
[+] y0ghur7_xxx|6 years ago|reply
> They fail because of severe checklist fatigue

> checklists often leaves a lot to be desired.

You say yourself that checklists are great, and it is proven that they save lives. So if there is fatigue in using them over and over, and they are not perfect, well... get over it? Sure i can understand that it's boring to go through the same checklist 5 times a day, but come on, there are lives at stake here. If one of your wife patients gets an infection and dies because she forgot some important, simple, step because of "checklist fatigue" how would she feel?

[+] monocasa|6 years ago|reply
I like what they do in the Japanese train system. You have to point at whatever you're doing and day aloud what it is before you check it off. That heavily reduces the amount of mindless checking via scanning the sheet that leads to items being checked off without being fully validated.

https://en.wikipedia.org/wiki/Pointing_and_calling

[+] ChicagoBoy11|6 years ago|reply
I fly private planes as a hobby and during my training that practice was kinda grilled in to me. After flying for a bit, you end up realizing how easy it is to "see" something incorrectly, or to skip an item, etc. Even when there's no one else in the plane with me, I say things out let, touch nobs to confirm their position even though I checked it earlier and didn't touch the nob at all at any portion in the flight. (Good) Pilot training will make you aware of just how much trouble your mind can get you into.
[+] jfries|6 years ago|reply
Yes, I think it's frustrating that this is not widely adopted. The upsides are really obvious and the only downside (I guess?) is that it feels a bit awkward to do it before society has gotten used to it.

Another thing with obvious benefits I wish we would adopt from east Asia is wearing a face mask when having a cold to prevent infection those around you.

[+] sopooneo|6 years ago|reply
I've never worked in a hospital, but I have plenty of experience with massive bureaucracies. And that gives me guesses, perhaps fanciful, about things that could go awry with surgical checklists. Anyone with relevant experience, please weigh in if these seem feasible.

1. Every prominent error that occurs prompts administration to decree a new checklist item be added to provent it in the future. But these added items will not be root-cause fixes so will proliferate to the point that there are now several dozen of negative checks to prevent these one-in-a-million mistakes. And so the lists will grow and grow, but no one will ever dare to take an item off. Because then if that mistake ever does reoccur, the admin that authorized its removal is up for a hearing in front of the firing squad.

2. Some administrator somewhere, noticing redundancy within a given checklist will reinvent control structures. You'll have lists with loops and conditionals. And since the people creating these lists don't know what is "considered harmful" they will almost certainly add GOTO's. In fact, I'd be very surprised if GOTO's don't already exist in surgical checklists somewhere.

But then this addition of control structures could increase to the point that a surgeon could, potentially, be put into an infinite loop! So committees will be formed to ensure every path through a checklist actually can be completed. And someone, somewhere in a hospital conference room, when asked to PROVE the lists all terminate, will take a breath, sit back, and rediscover the halting problem.

[+] therealx|6 years ago|reply
Some patient record programs already allow control structures in the digital forms. Basic stuff like if(X) then show options Y or page Y - you get the point. I think I saw a goto type item last I checked, which was a little bit ago.

As offices go digital, some are doing it right and recreating the forms in the signing application. Other, lazier firms, have you fill out a scan (gag me with a spoon.)

The idea of a loop is a funny one, I wonder if anything checks for it?

[+] scarejunba|6 years ago|reply
Well, they only have to discover that their program halts, not all arbitrary programs so that's not quite the Halting Problem.
[+] adolph|6 years ago|reply
Key point is the last paragraph:

Fortunately, Fixsen says, the lessons of implementation science are “completely generalizable”, and all programmes could benefit by noting the importance of engaged leadership, local adaptation and user buy-in. “It doesn't matter how good the innovation is, it doesn't matter how much has been invested,” says Fixsen. “If we don't have the implementation savvy, we're going to get the crummy outcomes that we have seen decade after decade.”

[+] rscho|6 years ago|reply
Official role for checklists in healthcare: save lives!

True role of checklists in healthcare: save management in case of legal action.

Consequence: the checklist becomes longer and longer, and is done more and more frequently. To the point of being quite impractical.

Goal: in case of legal action, take refuge in checklists. At least one of the items is bound to be missing so you can divert responsibility to someone else.

Source: I do checklists. That's my job.

[+] tonyarkles|6 years ago|reply
I replied elsewhere in the thread with a bit more detail, but my UAV checklists contain 100% material learned from experience only. They've grown a little (especially the packing list) as I've gotten out to the field and realized I didn't have a tool that would be useful to have, but they've also shrunk as I've eliminated equipment I no longer need (e.g. a programming cable for an ESC that I no longer use).

It's pretty disappointing to hear that that's how checklists work in your org. I feel like it's doubly negative: not only does it pretty much guarantee that they won't be followed because they're not particularly useful in that form; it also helps to ensure that the folks that have to use them will push back against checklists in general at their next job!

[I suppose that's what happened with me and JIRA too... it might be a decent product, but every org I've used it in has done a very bad job with it]

[+] lonelappde|6 years ago|reply
This is like Agile vs micromanagement in software dev.

The workaround for that is for the practicioners to have their own good checklists in addition.

[+] hestipod|6 years ago|reply
They fail because of egos and arrogance. This is the root cause of a lot of life's problems. People think they are above it and too smart etc. They hate being "told what to do". An example, I have a family member who will complain about texting and driving from some position of superiority but who does it constantly himself. When challenged with that behavior he refuses to acknowledge it. He believes he is above it, better than those who cause accidents, and only those "lesser" people need to follow rules. It's an incredibly common behavior and doctors are no different...and in some cultures more likely to assert their status.
[+] stronglikedan|6 years ago|reply
> He believes he is above it

Honestly, he probably is. We need to have laws because most people do it carelessly. Not necessarily because all people do it carelessly. Naturally, those who know they do it safely aren't going to just lump themselves in with the majority that don't.

[+] srikz|6 years ago|reply
I was just thinking about the comparison between the medical industry and airline industry. In medicine you can (or have to) pay a premium to get the best doctors and expertise [1]. In the airline industry, the safety aspect is standardised across the board.

It doesn’t matter if you are flying economy or 1st class you get the best in class safety, which is collectively shared across the industry. We pay a premium for the service and not to have better chances of surviving.

In an alternate universe, we could have had cheaper airline tickets which had not so great safety record and the good ones only in reach of the super wealthy.

[1]: I’m aware it may not be the case in some countries with completely free healthcare, but this applies to the majority of the world. Even in those countries with free healthcare for all, it is not uncommon to see huge wait times for some important procedures and being able to skip it by going through a private hospital.

[+] L_Rahman|6 years ago|reply
There are some interesting experiments with hospital operation models in India that try to make it more like flight. Every patient gets the same surgeon and treatment, but people who pay more get nicer facilities (private rooms, less crowding, separate waiting area for family).

It's not an equal model, but it is more just than what we have in America today.

https://www.bloomberg.com/news/features/2019-03-26/the-world...

[+] lonelappde|6 years ago|reply
That's only true for commercial airlines. Private planes have lower safety records because wealthy clients tell pilots to skip safety checks to save time. And personal general aviators don't have copilot's checking their work.
[+] GhostVII|6 years ago|reply
I think the big difference is that with airlines, it is realistic to have a zero-tolerance approach to deaths, since the default expectation is that a plane will not crash. With hospitals, deaths are innevitable, so you can't just have a uniform set of regulations to ensure no one dies.

Also, I think you can definitely pay more to go on a safer plane - I would imagine more expensive airlines have newer planes and better maintenance - but the risk of death is so small it isn't worth the extra cost.

[+] wisty|6 years ago|reply
A lot of people seem to be hung up on the fact that there must be something special about doctors. They're too skilled, they operate in a unique environment, or that they're awful egotists with god complexes.

Pick almost any profession other than pilot, and checklists are something that most people will ignore, often for the same excuses doctors have.

It seems like it should be comparable, as both medical errors and aircraft crashes are taken seriously, but there's a huge difference in magnitude.

[+] WalterBright|6 years ago|reply
Checklists have been incredibly effective in improving aviation safety. But just having checklists doesn't help much - there has to be a culture of following them and a ritual to following them.
[+] mjevans|6 years ago|reply
When followed correctly it is my believe that checklists can raise the floor of the worst case outcomes. Said another way, actually following the checklist eliminates the low-hanging fruit of preventable mistakes.
[+] dctoedt|6 years ago|reply
Atul Gawande's book, The Checklist Manifesto, said that WHO's surveys found significant resistance by surgeons to using checklists — but if they were going to be operated on, something like 93% of the surgeons said, in essence, you're damned right I want the surgeon to use a checklist.
[+] charles_f|6 years ago|reply
Article seems to identify main reasons as either poor implementation, or poor change management. Funny how much this reminds me of capital-A-gile, and Jeffries "we've tried baseball"[0]

- we've implemented some simple changes that make things better!

- everyone, let's all do this thing!

- there! We've tried! It doesn't work

- but you've bastardized the original idea beyond recognition

- doesn't work!

[0] https://ronjeffries.com/xprog/articles/jatbaseball/

edit: formatting

[+] mieseratte|6 years ago|reply
Does anyone here use point-and-call or something similar for more mundane activities than trains and planes? I started doing this while driving a few years ago. I've gotten a good bit of shit over the years for it from skeptical passengers. Hard to point at a clear-cut win when it is about prevention.
[+] skybrian|6 years ago|reply
If something as seemingly straightforward as using checklists is difficult to implement in hospitals, maybe we shouldn't feel too bad about the slow, partial, and controversial adoption of programming methodologies?
[+] dctoedt|6 years ago|reply
In aviation and nuclear-reactor operation, the ethos is that 1) of course people are going to make mistakes, because we're human; 2) the point of systems such as checklists and second-checking is to try to make sure the mistakes get caught before they have any effect; so 3) if you harbor resentment toward checklists, try to give yourself an attitude adjustment — learn to embrace and even enjoy them.
[+] wysifnwyg|6 years ago|reply
Doing UX Design right now and the popular phrase is: "If only people would read instructions everything would be alright."
[+] justinmeiners|6 years ago|reply
As the article states, the issue is they are often poorly implemented or designed. If that's the case, what is surprising?
[+] iamleppert|6 years ago|reply
Sounds like a great opportunity for a digital checklist with a computer vision, audio analysis component and deep learning backend that could quantify, and eventually automate the adherence to such checklists.

I wonder how many people would actually follow the checklist if they knew their compliance was being continually monitored and audited?

[+] cortesoft|6 years ago|reply
I also wonder how much of an effect the prior conditions have... if a hospital is already doing a good job of following procedures without a checklist, it might not make as much a difference as for a place that wasn't doing so well before.