Atul Gawande wrote an article for the New Yorker in 2007 about checklists for inserting central lines in hospital ORs which elaborates on some of the points in this FT piece:
The central line checklist was extremely basic — almost condescendingly so — and some doctors resisted it as a waste of time. Here's the checklist, from the article:
(1) Wash hands with soap
(2) Clean the patient’s skin with chlorhexidine antiseptic
(3) Put sterile drapes over the entire patient
(4) Wear a sterile mask, hat, gown, and gloves
(5) Put a sterile dressing over the catheter site once the line is in
That's it. Very, very basic. And yet one ICU that implemented it saw central line infection rates drop by 66 percent, and that's representative of the drop in infection rates at other hospitals that started using it.
This is something we're trying to apply to our operations. Definitely not an easy task to convince everyone to use them and like the article says, many people think checklists are beneath them so it's helpful when I can point to data in medicine or now like the article, venture capital.
A friend of mine is spearheading trials in some UK hospitals and the early results are ridiculously good.
There doesn't seem to be that much resistance to this idea though. When we talked about it, the only good explanation we came up with is that doctors have heard about this and know how good it can be and so are open to trying it, even begrudgingly.
It seems like entrepreneurs should have this kind of process as well. I've been developing what I'm calling "Idea Sieve" to help filter through ideas more methodically. Basically it's a spreadsheet with a bunch of questions & a scoring function to make a sort order. The impetus to this was a review of my projects back to 2008 & a realization that I was seeing some avoidable repeat problems.
Does anyone else do something like this or have some pointers?
The checklist technique, like anything else can be abused to the point of ineffectiveness. (So what's new?!)
When I was at Intuit, we had a VP (who was a management consultant in an earlier life) develop an "innovation pipeline" with checklists at each stage of the pipe line.
Anyone who had an idea for a new product had to run through this checklist before they could produce a prototype , get some funding, time etc. IIRC there were about a 20 questions in all and to move from stage to stage each set had to be answered in some detail. In theory it should have worked. In practice it was disaster. The answers to these questions had to make sense to the VP (who was very bright but knew nothing about prod dev or the local market, so nothing would get past her "approval sense") and a whole array of managers(and Intuit has a LOT of managers. Oh boy did we have managers everywhere) who had to approve of every item in the checklist before the person who proposed the idea could start moving on it.
Soon enough, anyone foolish enough to submit an innovative proposal would have to send endless powerpoint slides in support of every checklist question, usually to be gathered by a set number of what Intuit called "Follow Me home"s. When one manager was satisfied, the next would ask for more data on some other point. Then the first would have a rethink and ask for more data. People spent months gathering data for something that could have been bulit and deployed over a weekend and got real data from real customers (and people wonder why Intuit is stagnant!).
What I took away from observing this "pipeline" in action, is that anyone using a "checklist approach" to creative work should be well aware of its limitations. I guess this is good advice for any tool. Use it as a tool, not a crutch - a substitute for thinking or deciding.
At times in the past, I've done something similar: rated various ideas 1-10 on things like:
- ease of build
- viral potential
- level of maintenance required after launch
- revenue options
- etc
Then weighted each if necessary (e.g., maintenance might not be a huge factor if you have someone willing to help with that) and ranked ideas on their overall merits.
Of course, I could've probably got part-way through building one of the simpler ideas in the time it took me to rank the rest...
This is a really nice article. Venture capital industry reminds me of primitive medicine - people used to do things because it intuitively felt right, but had no rigorous means of telling good ideas from bad ideas (and hence, made mistakes about half the time). Statistics revolutionized medicine, and some day will revolutionize early stage investment.
It seems like even the most fundamental questions aren't answered yet - how do VCs do compared to how they would have done if they picked the companies that pitch to them at random? How do they do compared to each other (if you manage to control for the fact that top-tier VCs get to pick the best deals later in the game my guess is there will be no difference)? How do they do over long term compared to investing into treasuries or other safe instruments (it's comparing apples to oranges, but there is valuable information there)?
Considering that even these simple questions aren't answered, any rigorous study of the VC industry is like a breath of fresh air. At least we know now that checklists work.
- Airlines and venture backed startups have vastly different goals. Flights are supposed to, above all things, be predictable. Crashing is not acceptable. If startups were predictable, there would be no reason for venture capital to exist.
- This is talking about an average over the whole of an industry, venture capital, in which the average, and indeed most of the above average firms are dysfunctional -- they don't make money. Not just, you know, 51%, like > 90%. So being better than average can still mean that you're completely economically ineffective. So the right question isn't, "What makes you better than average?" but "What makes you one of the few that is even functional?"
I think the approach the article is making is that it pays to be methodical about deciding to invest in something, it's not about saying that startups will be predictable, but about reducing risk. Planes just happen to be a case where risk can be reduced to a pretty minimal level with a good enough checklist.
The whole VC pitch process is basically a test that VCs run to detect future success. Obviously, it's badly prone to false negatives and false positives. So I think the thesis of the article is that checklists help VCs reduce both those error rates.
I don't think, however, that Gawande's necessarily saying that checklists improve a VC's winner detection algorithm; just that checklists help the VC make sure to run the algorithm thoroughly.
"Those with other styles were not failures by any stretch – experience does count for something. But those who added checklists to experience proved substantially more successful."
And Buffett might argue with you that investing (VC or otherwise) should try to be predictable too!
[+] [-] karzeem|16 years ago|reply
http://www.newyorker.com/reporting/2007/12/10/071210fa_fact_...
The central line checklist was extremely basic — almost condescendingly so — and some doctors resisted it as a waste of time. Here's the checklist, from the article:
That's it. Very, very basic. And yet one ICU that implemented it saw central line infection rates drop by 66 percent, and that's representative of the drop in infection rates at other hospitals that started using it.[+] [-] cwan|16 years ago|reply
This is something we're trying to apply to our operations. Definitely not an easy task to convince everyone to use them and like the article says, many people think checklists are beneath them so it's helpful when I can point to data in medicine or now like the article, venture capital.
[+] [-] pierrefar|16 years ago|reply
There doesn't seem to be that much resistance to this idea though. When we talked about it, the only good explanation we came up with is that doctors have heard about this and know how good it can be and so are open to trying it, even begrudgingly.
[+] [-] jeffcoat|16 years ago|reply
1) No ? in 'svn status'
2) Checkstyle accepts all changed files
3) 'svn update'
4) Unit tests pass
5) 'svn commit'
As you say -- condescendingly simple. And astonishingly helpful.
[+] [-] hedgehog|16 years ago|reply
Does anyone else do something like this or have some pointers?
[+] [-] plinkplonk|16 years ago|reply
When I was at Intuit, we had a VP (who was a management consultant in an earlier life) develop an "innovation pipeline" with checklists at each stage of the pipe line. Anyone who had an idea for a new product had to run through this checklist before they could produce a prototype , get some funding, time etc. IIRC there were about a 20 questions in all and to move from stage to stage each set had to be answered in some detail. In theory it should have worked. In practice it was disaster. The answers to these questions had to make sense to the VP (who was very bright but knew nothing about prod dev or the local market, so nothing would get past her "approval sense") and a whole array of managers(and Intuit has a LOT of managers. Oh boy did we have managers everywhere) who had to approve of every item in the checklist before the person who proposed the idea could start moving on it.
Soon enough, anyone foolish enough to submit an innovative proposal would have to send endless powerpoint slides in support of every checklist question, usually to be gathered by a set number of what Intuit called "Follow Me home"s. When one manager was satisfied, the next would ask for more data on some other point. Then the first would have a rethink and ask for more data. People spent months gathering data for something that could have been bulit and deployed over a weekend and got real data from real customers (and people wonder why Intuit is stagnant!).
What I took away from observing this "pipeline" in action, is that anyone using a "checklist approach" to creative work should be well aware of its limitations. I guess this is good advice for any tool. Use it as a tool, not a crutch - a substitute for thinking or deciding.
[+] [-] prawn|16 years ago|reply
Of course, I could've probably got part-way through building one of the simpler ideas in the time it took me to rank the rest...
[+] [-] pmikal|16 years ago|reply
[+] [-] blrgeek|16 years ago|reply
[+] [-] coffeemug|16 years ago|reply
It seems like even the most fundamental questions aren't answered yet - how do VCs do compared to how they would have done if they picked the companies that pitch to them at random? How do they do compared to each other (if you manage to control for the fact that top-tier VCs get to pick the best deals later in the game my guess is there will be no difference)? How do they do over long term compared to investing into treasuries or other safe instruments (it's comparing apples to oranges, but there is valuable information there)?
Considering that even these simple questions aren't answered, any rigorous study of the VC industry is like a breath of fresh air. At least we know now that checklists work.
[+] [-] RiderOfGiraffes|16 years ago|reply
http://news.ycombinator.com/item?id=798009
[+] [-] wheels|16 years ago|reply
- Airlines and venture backed startups have vastly different goals. Flights are supposed to, above all things, be predictable. Crashing is not acceptable. If startups were predictable, there would be no reason for venture capital to exist.
- This is talking about an average over the whole of an industry, venture capital, in which the average, and indeed most of the above average firms are dysfunctional -- they don't make money. Not just, you know, 51%, like > 90%. So being better than average can still mean that you're completely economically ineffective. So the right question isn't, "What makes you better than average?" but "What makes you one of the few that is even functional?"
[+] [-] redcap|16 years ago|reply
[+] [-] karzeem|16 years ago|reply
I don't think, however, that Gawande's necessarily saying that checklists improve a VC's winner detection algorithm; just that checklists help the VC make sure to run the algorithm thoroughly.
[+] [-] blrgeek|16 years ago|reply
"Those with other styles were not failures by any stretch – experience does count for something. But those who added checklists to experience proved substantially more successful."
And Buffett might argue with you that investing (VC or otherwise) should try to be predictable too!
[+] [-] Slashed|16 years ago|reply
Can someone share this know-how with us, please?
If you were to invest in Google in its early days, would consider a yet another search engine a good idea?
So basically, a good idea means that you're better than your competitors?
Or if you created something unique, how do you know it's going to work on the market?