When business approach me asking for automating a business process, I usually ask them to do the process in manual steps and create check lists for it. The check lists force them to think it through and are the agreed upon steps to perform the process. After they do it by hand for a while and find out where the pain points are, we can then automate. Since the check lists are already in place, automation is really easy.
Checklists are criminally underused in most industries. It's a testament to how great they work when looking at highly-antifragile contexts: aviation, military, etc., where a mistake can (and often does) cost lives.
The problem is that to use them broadly, you also need to implement operational-level changes. So if I made a startup that helped build checklists, the problem would not just be selling the software (which everyone has to do), but also convincing executives that checklists (even moreso than to-do lists) are worth investing in. The friction a checklist brings, at least in non-ultra-risk-averse verticals, might not be worth it.
Heck, they're underused in general. Checklists to pack for vacation. Checklists for "what did I mean to do this weekend". Checklists for "here's what we need to run the volunteer dinner this organization I belong to does weekly". Checklists for "things to check before my presentation".
I'm nowhere near as obsessed with them as that may make me sound; by temperament I'm definitely a go-with-the-flow sort of guy... which is perhaps exactly why I make sure to use them in certain important places. Go-with-the-flow has its strengths but also its weaknesses. Selective augmentation with checklists means I can often end up with the best of both worlds between that and being "detail oriented" for important tasks.
I have been called "detail oriented" a few times in my professional career, and I sort of laugh to myself because it is absolutely and utterly untrue. I'm not detail oriented in the slightest. It's all offloaded to the computer, with my unit tests, checklists, and such. I just know I need that augmentation and am quite aggressive about getting it precisely because I know I need it.
We use them at our business for travel deployments/installs. They must be printed, each line item must be signed off + dated + timestamped in the presence of another employee on the same travel team (who attests the signature and the work), and they can never be electronic in nature except for the creation + printing of them.
Typically we have a non-traveling employee or manager do the final once-over and attestation, though they don't review the work necessarily. (Usually they just spot check the gear being shipped/sent to the airport, Pelican cases, Airtag trackers, etc.) This has been helpful for organizational purposes.
We don't even computerize them in any way outside of taking a cell phone picture of the final checklist and uploading it to a shared Slack channel for archival purposes.
It has prevented so many problems. It also fails to prevent some... which get added to the next checklist.
I've proposed using checklists at multiple jobs and people never want them. They always claim they're not necessary and add a ton of work. Then we all keep making the same mistakes every time.
I've quit proposing them for this exact reason. Sometimes I just make my own.
You probably don't want a lot of process reliability in anti-fragile contexts, like the startups you mentioned. You insist on reliability in fragile contexts.
The travel checklist goes with me on the trip and serves to answer three more questions: (1) Am I bringing back everything I took, (2) what did I not use on the trip, and (3) what should I have brought (i.e. add it to the checklist after the trip has started)?
I take the list for a completed trip and use it to plan the next trip of a like nature. It has been very helpful.
What's the difference between a checklist and a todo list? Is it that a todo list is the things you need to do and a checklist is how todo one of those things?
One thing that might not be obvious about checklists is how they're used.
I used to think checklists were used by reading the item, then doing the thing. I literally thought of them as a recipe that you would follow. Complete a step, check the box, repeat... This is typically referred to as a "read, do" checklist. In aviation this style of checklist is typically reserved for non-normal operations—procedures that you wouldn't use often enough to commit to memory.
The other style of checklist is "do, confirm". In this style you complete a procedure by memory, and read through a checklist to ensure you didn't miss anything (no box ticking, you just read the items and confirm to yourself that they're complete). In aviation this is the style used for normal operations, and for the initial action-items in an emergency (which although not commonly used, must be committed to memory because they are time-critical).
Because you're expecting that the procedure is completed by memory, a "do, confirm" checklist can be extremely brief. You do not need to write in detail what each step involves, you just need a word or two to name the step. Additionally, they're an extremely low operational burden; it takes a couple of seconds to read through a "do, confirm" checklist but the upside of catching common errors is significant.
Many people in this discussion seem to confuse checklists with to do lists.
A checklist is an established, recurring procedure that is ideally memorised, but despite that one goes through each item on the checklist to make sure nothing is forgotten or performed out of sequence or twice.
A to do list is an ad hoc collection of tasks that need to be performed. The list is built for each occasion rather than covering a standard procedure.
Tim Harford mentions in his podcast "Cautionary Tales – A Fascination with Failure / Death on the Dance Floor" that checklists might have prevented the 1981 skywalk collapse in Kansas City Hyatt Regency Hotel (killing 114 people).
It's really a good podcast (pretty much all of his podcasts are)
I've been fascinated for checklists for a long time, and I've been thinking about creating this "checklists on steroids" app, where checklists can be shared, created from templates, executed (and information about each executing collected), etc.
I finally had some time to spend on it, and here's the result: https://checkoff.ai/
Which is why programming, reality, and even recipes are so hard.
Ever go try to cook something new and you read "Pre-heat the pan on low-medium...", and your programmer brain just can't take it? What kind of pan, what's low-medium on this burner, how much pre-heating are you talking about? These can't be all the instructions.
And perhaps like programming, it takes a few recipes and a few burnt steaks for you "not to worry" about that, you know what's good enough eventually. These lists (and algorithms) are never completely thorough.
Good recipes do explain this though. They say heat the oil until it shimmers, or until it smokes, or until beads of water in the pan sizzle. Or they give an exact temperature, which you can read (imperfectly) with an infrared thermometer.
None of these descriptions is perfect, but each is less likely to result in a burnt steak.
"Pre-heat" is done until your apparatus reaches the stable cooking temperature. The recipe writer doesn't know your pan size, room temperature, stove power, or anything like that, so they can't tell you the details.
"Low-medium" is just bad instructions. The recipe should be more detailed.
Anyway, what you are complaining about on your example is just jargon ignorance. You need to learn some stuff before you understand recipes. That's not really what makes programming hard. But it does make learning new things hard.
Not everything needs to be rationalised back to programming.
It's not programming experience that makes you ask those questions but because they are basic cooking questions you would ask.
Checklists are either checklists or a set of instructions to follow.
Good instructions would specify some of those attributes, other times experience helps to understand what general pan you might use or what medium-low heat looks like. Exactly to your point, it takes a few burnt steaks to figure it out.
Checklist is for people who have a fair amount of experience doing the task. It's just a reminder to perform the steps. An experienced chef reading "pre-heat the pan on low-medium.." would know what those terms mean.
That's one of the reasons I loved Ansible from the moment I saw it. As the OP points out, traditionally machines accumulated ad-hoc changes over a long period of time. Describing the "known good" state and running this "checklist" to make sure it is in that state both documents the checklist and evaluates it.
Same reason we haven't typed "cc" on the command line to call the C compiler on individual files for about 30 years or more.
The last time I typed (well, pasted) "cc" on the command line to call the C compiler on an individual file was 26 hours ago. I wanted to recompile a single-file program I'd just written with debugging information (-g) and it seemed easier to copy, paste, and edit the command line rather than to manually delete the file and reinvoke make with different CFLAGS.
I mean, I've surely compiled orders of magnitude more C files without typing "cc" on the command line over the last week. But it's actually pretty common for me to tweak options like -mcpu, -m32, -pg, -Os, or -std=c11 -pedantic (not to mention diet cc) by running a "cc" command line directly.
Similarly, I often run Python or JS code in the REPL or in Jupyter rather than putting it in a file. The rapid feedback sometimes helps me learn things faster. (Other times it's an attractive nuisance.)
But I may be a bit of an odd duck. I've designed my own CPU, on paper. I write assembly code for fun. I've implemented several different programming languages for fun. I like to know what's underneath, behind the surface appearances of things. And that requires experimenting with it.
A checklist can be gradually automated, step-by-step.
Start by automating the low hanging fruit.
After a few iterations you have a somewhat automated process and some steps that are harder to automate. Those hard-to-automate steps can be annotated with detailed instructions that expose opportunities to partly automate them. You can break down each step more over time. As you run the checklist, you’ll learn & iterate. Then with the newly broken down steps, you can automate what’s become automatable. Repeat forever!
Checklists are extremely hard. Writing them requires the skill of the programmer and direct response copywriter, and few people have both. Ensuring that people actually use the checklists is brutally difficult. And creating systems to improve the checklists as they are used is even harder. I suspect checklists being so hard is the reason they're so rarely used.
One of the great thing about that book was how he described checklist failure, when best practices checklists from one institution didn't generate the same results because the people in the receiving institution weren't bought into all the items within the checklist and it was just something to click through and get done.
Indeed. It’s actually one of the books that influenced me the most. It’s squarely in the middle of a Venn diagram of psychology, systems thinking, project management, neuroscience, healthcare, technology, productivity, and problem solving.
I have be wondering about do-not-check lists to go with checklists.
I am mostly thinking about packing list, as it is common to overpack when travelling, so it can be good to have a list of things you think will be useful but are not. For example, if you are travelling in hotels (as opposed to hitchhiking ;)), you probably don't need a towel, because they are usually provided, so, to the do-not-pack list.
The problem with checklist, especially the ones that change over time is that they sometimes go out of control with useless items, a do-not-check list means "I removed this from the checklist, there is a good reason, don't put it back"
Checklists are one of the most powerful universal tool I know for avoiding/minimizing disasters/failures. In the context of software, I try to create user interfaces as masked checklists, because it forces making critical steps/decisions visible and clear (common domain specific understanding for both developer and end user), and guides user to goal. It's almost like old-school "wizard" UIs, but professionals in mind instead of beginners.
Also, critical business processes are great expressed as checklists.
I learned about checklists (versus to do lists) from Dr. Atul Gawande. He has done a great deal of work on checklists in medicine because of the complexity in the field and as a way to force/facilitate communication.
He wrote The Checklist Manifesto - How to Get Things Right, has been interviewed a gazillion times about the need for the surgical checklist, and talks extensively about the challenges associated with getting hospitals and medical practices to adopt the surgical checklist.
NPR's Atul Gawande's Checklist For Surgery Success [1] is a short version of the problem, challenges, inspiration, and solution.
And more recently, look forward to Captain Steeeve [2] talking about checklists pilots use.
I use checklist as a pilot, but trying to introduce the same habit to software engineers are hard.
The cost of reversing a decision is cheaper in software, than to say flying or doing hardware. Plus the spec requirements for flying and hardware design rarely changes as often as software.
Still swear by checklists and SOPs but have to be at peace that others don't see it the same way.
The checklist doesn't have to be perfect. Just continually improve it.
I keep several checklists - some I use several times a week, others every few months or so. If I notice something needs to be added to the checklist or removed, I do so.
It's always better to start with an imperfect checklist vs not having any checklist at all. With no checklist, you start from scratch every single time. Not starting from scratch allows you to focus on marginal improvements with each use.
I made a checklist to migrate storage from one host to another recently. Like TFA mentions, it ended up being incomplete with some key details missing. But I filled in the gaps as I went and saved the checklist for next time.
[+] [-] ww520|8 months ago|reply
[+] [-] dvt|8 months ago|reply
The problem is that to use them broadly, you also need to implement operational-level changes. So if I made a startup that helped build checklists, the problem would not just be selling the software (which everyone has to do), but also convincing executives that checklists (even moreso than to-do lists) are worth investing in. The friction a checklist brings, at least in non-ultra-risk-averse verticals, might not be worth it.
[+] [-] jerf|8 months ago|reply
I'm nowhere near as obsessed with them as that may make me sound; by temperament I'm definitely a go-with-the-flow sort of guy... which is perhaps exactly why I make sure to use them in certain important places. Go-with-the-flow has its strengths but also its weaknesses. Selective augmentation with checklists means I can often end up with the best of both worlds between that and being "detail oriented" for important tasks.
I have been called "detail oriented" a few times in my professional career, and I sort of laugh to myself because it is absolutely and utterly untrue. I'm not detail oriented in the slightest. It's all offloaded to the computer, with my unit tests, checklists, and such. I just know I need that augmentation and am quite aggressive about getting it precisely because I know I need it.
[+] [-] icelancer|8 months ago|reply
Typically we have a non-traveling employee or manager do the final once-over and attestation, though they don't review the work necessarily. (Usually they just spot check the gear being shipped/sent to the airport, Pelican cases, Airtag trackers, etc.) This has been helpful for organizational purposes.
We don't even computerize them in any way outside of taking a cell phone picture of the final checklist and uploading it to a shared Slack channel for archival purposes.
It has prevented so many problems. It also fails to prevent some... which get added to the next checklist.
[+] [-] sien|8 months ago|reply
Gawande is a crazy smart surgeon and writer. The book is pleasantly short and makes its points really well.
https://en.wikipedia.org/wiki/The_Checklist_Manifesto
[+] [-] wickedsight|8 months ago|reply
I've quit proposing them for this exact reason. Sometimes I just make my own.
[+] [-] marcosdumay|8 months ago|reply
You probably don't want a lot of process reliability in anti-fragile contexts, like the startups you mentioned. You insist on reliability in fragile contexts.
[+] [-] EvanAnderson|8 months ago|reply
I take the list for a completed trip and use it to plan the next trip of a like nature. It has been very helpful.
[+] [-] chromiummmm|8 months ago|reply
[+] [-] ozim|8 months ago|reply
Having dozen of checklists that work for specific scenarios would be worth it.
[+] [-] jiehong|8 months ago|reply
Run the list manually, then automated each check point by point.
Voilà!
[+] [-] unknown|8 months ago|reply
[deleted]
[+] [-] blakehaswell|8 months ago|reply
I used to think checklists were used by reading the item, then doing the thing. I literally thought of them as a recipe that you would follow. Complete a step, check the box, repeat... This is typically referred to as a "read, do" checklist. In aviation this style of checklist is typically reserved for non-normal operations—procedures that you wouldn't use often enough to commit to memory.
The other style of checklist is "do, confirm". In this style you complete a procedure by memory, and read through a checklist to ensure you didn't miss anything (no box ticking, you just read the items and confirm to yourself that they're complete). In aviation this is the style used for normal operations, and for the initial action-items in an emergency (which although not commonly used, must be committed to memory because they are time-critical).
Because you're expecting that the procedure is completed by memory, a "do, confirm" checklist can be extremely brief. You do not need to write in detail what each step involves, you just need a word or two to name the step. Additionally, they're an extremely low operational burden; it takes a couple of seconds to read through a "do, confirm" checklist but the upside of catching common errors is significant.
[+] [-] kqr|8 months ago|reply
A checklist is an established, recurring procedure that is ideally memorised, but despite that one goes through each item on the checklist to make sure nothing is forgotten or performed out of sequence or twice.
A to do list is an ad hoc collection of tasks that need to be performed. The list is built for each occasion rather than covering a standard procedure.
[+] [-] vorgol|8 months ago|reply
It's really a good podcast (pretty much all of his podcasts are)
https://timharford.com/2023/07/cautionary-tales-a-fascinatio...
[+] [-] regnull|8 months ago|reply
I finally had some time to spend on it, and here's the result: https://checkoff.ai/
Would be grateful for any feedback!
[+] [-] rapfaria|8 months ago|reply
Ever go try to cook something new and you read "Pre-heat the pan on low-medium...", and your programmer brain just can't take it? What kind of pan, what's low-medium on this burner, how much pre-heating are you talking about? These can't be all the instructions.
And perhaps like programming, it takes a few recipes and a few burnt steaks for you "not to worry" about that, you know what's good enough eventually. These lists (and algorithms) are never completely thorough.
[+] [-] massysett|8 months ago|reply
None of these descriptions is perfect, but each is less likely to result in a burnt steak.
[+] [-] marcosdumay|8 months ago|reply
"Low-medium" is just bad instructions. The recipe should be more detailed.
Anyway, what you are complaining about on your example is just jargon ignorance. You need to learn some stuff before you understand recipes. That's not really what makes programming hard. But it does make learning new things hard.
[+] [-] NoPicklez|8 months ago|reply
It's not programming experience that makes you ask those questions but because they are basic cooking questions you would ask.
Checklists are either checklists or a set of instructions to follow.
Good instructions would specify some of those attributes, other times experience helps to understand what general pan you might use or what medium-low heat looks like. Exactly to your point, it takes a few burnt steaks to figure it out.
[+] [-] ww520|8 months ago|reply
[+] [-] squeedles|8 months ago|reply
Same reason we haven't typed "cc" on the command line to call the C compiler on individual files for about 30 years or more.
[+] [-] kragen|8 months ago|reply
I mean, I've surely compiled orders of magnitude more C files without typing "cc" on the command line over the last week. But it's actually pretty common for me to tweak options like -mcpu, -m32, -pg, -Os, or -std=c11 -pedantic (not to mention diet cc) by running a "cc" command line directly.
Similarly, I often run Python or JS code in the REPL or in Jupyter rather than putting it in a file. The rapid feedback sometimes helps me learn things faster. (Other times it's an attractive nuisance.)
But I may be a bit of an odd duck. I've designed my own CPU, on paper. I write assembly code for fun. I've implemented several different programming languages for fun. I like to know what's underneath, behind the surface appearances of things. And that requires experimenting with it.
[+] [-] cadamsdotcom|8 months ago|reply
Start by automating the low hanging fruit.
After a few iterations you have a somewhat automated process and some steps that are harder to automate. Those hard-to-automate steps can be annotated with detailed instructions that expose opportunities to partly automate them. You can break down each step more over time. As you run the checklist, you’ll learn & iterate. Then with the newly broken down steps, you can automate what’s become automatable. Repeat forever!
[+] [-] scubbo|8 months ago|reply
[0] https://blog.danslimmon.com/2019/07/15/do-nothing-scripting-...
[+] [-] gleb|8 months ago|reply
[+] [-] hermitcrab|8 months ago|reply
But any checklist needs to be a living document that has is easily updated and under version control.
Personally, I used MyLifeOrganized documents stored in Subversion.
[+] [-] naiquevin|8 months ago|reply
I write them in (emacs) org mode in a way that I can “execute” them repeatedly. Have written about it: https://www.naiquev.in/recurring-checklists-using-org-mode-i...
[+] [-] azmarks|8 months ago|reply
[+] [-] adolph|8 months ago|reply
[+] [-] jtrn|8 months ago|reply
[+] [-] GuB-42|8 months ago|reply
I am mostly thinking about packing list, as it is common to overpack when travelling, so it can be good to have a list of things you think will be useful but are not. For example, if you are travelling in hotels (as opposed to hitchhiking ;)), you probably don't need a towel, because they are usually provided, so, to the do-not-pack list.
The problem with checklist, especially the ones that change over time is that they sometimes go out of control with useless items, a do-not-check list means "I removed this from the checklist, there is a good reason, don't put it back"
[+] [-] 0xCE0|8 months ago|reply
Also, critical business processes are great expressed as checklists.
[+] [-] susiecambria|8 months ago|reply
He wrote The Checklist Manifesto - How to Get Things Right, has been interviewed a gazillion times about the need for the surgical checklist, and talks extensively about the challenges associated with getting hospitals and medical practices to adopt the surgical checklist.
NPR's Atul Gawande's Checklist For Surgery Success [1] is a short version of the problem, challenges, inspiration, and solution.
And more recently, look forward to Captain Steeeve [2] talking about checklists pilots use.
[1] https://www.npr.org/2010/01/05/122226184/atul-gawandes-check... [2] https://www.youtube.com/@CaptainSteeeve
[+] [-] zhaohan_dong|8 months ago|reply
The cost of reversing a decision is cheaper in software, than to say flying or doing hardware. Plus the spec requirements for flying and hardware design rarely changes as often as software.
Still swear by checklists and SOPs but have to be at peace that others don't see it the same way.
[+] [-] shooker435|8 months ago|reply
[+] [-] ChrisMarshallNY|8 months ago|reply
Of course, these checklists manifested as Excel spreadsheets.
[+] [-] danparsonson|8 months ago|reply
[+] [-] devenson|8 months ago|reply
[+] [-] massysett|8 months ago|reply
I keep several checklists - some I use several times a week, others every few months or so. If I notice something needs to be added to the checklist or removed, I do so.
It's always better to start with an imperfect checklist vs not having any checklist at all. With no checklist, you start from scratch every single time. Not starting from scratch allows you to focus on marginal improvements with each use.
[+] [-] ethan_smith|8 months ago|reply
[+] [-] vorgol|8 months ago|reply
[+] [-] NewEntryHN|8 months ago|reply
[+] [-] kragen|8 months ago|reply
[+] [-] accrual|8 months ago|reply