I don't see anyone here commenting on this, but there's an interesting software / UI angle in this story: the captain preferred the fancier, animated-map-style weather reports from a 3rd-party company (B.V.S. reports) to the more terse text-only reports from the National Hurricane Center (sat-C reports).
Turns out, the B.V.S reports were using raw data that was 10 to 12 hours old - and they didn't explicitly mention that. In the case of a rapidly evolving hurricane, it mattered a lot.
> The B.V.S. map included a time stamp that showed when the processing had been completed, but gave no indication of the age of the raw data on which the forecast was based. Davidson knew that all the forecasts were uncertain, and that they sometimes disagreed. But how aware was he that when he looked at the B.V.S. maps he was looking into the past?
[...]
> Davidson dismissed the plan with a thank-you and did not come to the bridge. Evidence suggests that he was still showing a preference for the animated B.V.S. graphics, which indicated the storm progressing more slowly.
> He went down to his stateroom after his conversation with Schultz, and when he returned to the bridge he said, “All right, I just sent up the latest weather. Let us clear everything off the chart table with the exception of the charts.” Schultz opened the B.V.S. program. As it happened, according to the N.T.S.B. report, because of a software glitch, the map that appeared was the very same map that had come in with the previous download, six hours earlier. The raw data on which it was based was at least 12 hours old.
It sounds like it wouldn't actually have helped if the captain had the per-hour update, because this undetailed software glitch meant the device wasn't showing the updated maps anyway.
The apparent moral of that part of the story is that the B.V.S. charts are past useless and into actively harmful territory, since they may not just be old but instead actively incorrect.
Yes, I saw the same thing. Even when I've had people reviewing logs, I make the date prominent and train them to check it first. BVS's data is far more critical.
(I learned from a burnt hand: We engaged a non-technical but reliable user to check the daily backup log for errors and report any to us. One day we needed the backup and discovered it hadn't run at all for many weeks. Ouch. I asked the user; she said she indeed checked the logs daily and they were fine. She was right: She was seeing the log from the last backup, unchanged every day. My fault entirely, not hers: We should have anticipated the date problem, and we should have utilized someone technically literate enough to understand what they were reading - in this case, someone who would recognize an 'obvious' problem such as the numbers of files and bytes not changing. And we should have tested our backups more often, but that old lesson almost isn't worth mentioning.)
Also note that B.V.S did appear to have a 'Hourly Update' feature that did not appear to be subscribed to.
It's also important to realize that the captain was confident with the plan- except the storm in question was abnormal. That's an important fact to consider, as it throws the mariners heuristics off deciding how much more attention to pay.
If I see a timestamp on rapidly changing data I sure as hell would like to see if there's a delayed effective stamp (most stock market tickers indicate the "effective date" of their data or say clearly there's a XX min delay).
Reminded me of this aircraft accident where the time delay of radar satellite datalink weather information was critical. The time delay there is minutes not hours.
An aviation accident case study pinpointed a similar issue with in-cockpit weather displays. In this case the delay was less than 10 minutes, but it was enough to be fatal: https://youtu.be/83uvKWJS2os
> She called Davidson on the house phone and told him that the hurricane was now a Category 3. He knew that already. She proposed the escape route to the south and a smooth sail on to San Juan. He rejected her suggestion.
I wonder if in the maritime world there is the equivalent of CRM (Cockpit Resource Management), as in aviation.
Too many lives were lost due to 1st officer never taking over command when the error in the commander's thinking had already become apparent or the entire aircraft was being flown, effectively, by a single infallible captain and not a team.
Lessons were learned decades ago and at least in a 1st world outfit I would like to believe that no 1st officer would hesitate to utter the words "I have control" when in such stress.
There is: bridge resource management. It actually grew out of CRM. Section 2.7 of the NTSB report (linked elsewhere in this thread) gives an overview of it and discusses how it played a role in this incident. It's an interesting read and seems to be relatively in-depth.
One of my takeaways from the article is that the captain's confidence and willingness to do the right thing had been shaken after he'd been burned the last time he'd refused unsafe orders. I'm wondering if the 1st officer similarly faced bad consequences for doing the right thing and taking command -- mutiny charges are still a thing, aren't they?
Also from the world of aviation, seems the captain had developed a classic case of get-there-itis. The symptoms are there, the prognosis is never good.
My big takeaway from all of this is that, given the choice between driving your boat into a hurricane and doing something else, something else should be the default.
Another conclusion was that a tiered subscription model that withholds life-saving information is a business I never want to be in. El Faro subscribed to the Bon Voyage System for weather updates, but they didn't subscribe to the routing guidance and always received six hour-old information. The stale data is a critical part of this story.
That's a pretty bad take away - no one wanted to drive trough the hurricane. The captain thought he was further away from the storm than he was, perhaps because of his use of an outdated weather feed.
A better take away might be, for important decisions, you should deeply understand the data(and importantly, it's limitations) you are using to make the decision.
1. FYI the article is new but the disaster described happened on October 1, 2015.
2. I was astonished at this:
> "It has been reported that a major merchant ship goes down somewhere in the world every two or three days; most are ships sailing under flags of convenience, with underpaid crews and poor safety records."
It still boggles my mind how three different GPS position formats can be used. And then not used correctly by search crews, which then missed location by 23nm.
The article mentions that there are major merchant vessels sinking every two/three days! That seems huge. How many deaths per year does that amount to? We're lucky to be in tech I guess.
Fermi estimate, about 120-180 sinkings a year * 30 crewmembers/ship * 0.75 chance of dying (I'm guessing some sink in port or are evacuated?) = 2700 deaths / year.
A recent (paywalled) article in the New Scientist stated that the fatal accident rate for ships' crews is twenty times that of the average British worker.
For anyone interested in stories like these, I highly recommend "Normal Accidents" by Charles Perrow - there's an entire chapter on maritime accidents.
Wonder why shippers don't have a 24/7 Operations Center employing meteorologists and tracking weather systems and ships and in close contact with ship's crew. It struck me that there was a lot of friction for the crew to just plot up-to-date weather/forecasts and the ship's course. Crew was constantly fighting for current data and good/correct visualization.
What amazed me reading the report was just how vulnerable the ship was to sinking if it had a list during a storm. While an 18 degree list is extreme it's not unforeseeable in a serious emergency. The problem was this put ventilation openings at a height where they were directly subject to green water infiltration. To make it worse only the intakes were 'watertight' the exhausts were only 'weathertight' meaning that this would have resulted in flooding that would have made the problem worse.
>Fisker-Andersen wrote, “Captain Mike, diversion request heads up through Old Bahama Channel understood and authorized. Thank you for the heads up. Kind regards."
I'm not quite following that sentence. The article makes a lot of the "authorized", saying that meant the captain was being ordered to go directly to San Juan and not use the Old Bahama Channel. However, it seems to say that the diversion into the channel was authorized.
What’s really interesting about this is the sort of horror film aspect to these situations where everyone gets a bad feeling of impending doom but they push forward anyway.
People should trust that instinct. Takes a lot of effort to pull away from the brink, but you should do it. 100% of the time.
The article pointed out that couple years earlier the captain did exactly as you suggest and pulled away from the brink because of safety concerns. His company punished him immediately with a big demotion.
It seems clear that the culture in the industry is set up to weed out those who want to err on the side of safety, and reward those who err on the side of maintaining their shipping schedules.
I suspect that human driven errors like this will lead the shipping industry to transition to autonomy. Through this whole story the autopilot was the only one steering the ship, the humans in the loop were responsible for plotting and course setting. Obviously this is a bad idea. It's not hard to imagine an expert software system that could have done a better job with this situation, especially in processing the text based weather data coming over the C-sat.
Apparently some Norweigans agree with me and are doing exactly this. I suspect this will become the future norm once the bugs are ironed out.
Anyone else get a weird pop-up after opening this article? I'm using Firefox for Android, and after I open the article and begin scrolling, a little box pops up that lists the names of 2 local financial institutions and the word "remove." As I scroll down, the box moves, too. If I tap on one of the names or the word "remove" the box disappears. I only get this on Firefox for Android.
The only reason it's rather worrying is because I happen to bank at both of the financial institutions on the list. They are both small, local institutions.
Tried opening the link on a couple other devices but was unable to recreate this. The pop-up appears every time I open the link in Firefox on my phone, though.
[+] [-] gregschlom|8 years ago|reply
Turns out, the B.V.S reports were using raw data that was 10 to 12 hours old - and they didn't explicitly mention that. In the case of a rapidly evolving hurricane, it mattered a lot.
> The B.V.S. map included a time stamp that showed when the processing had been completed, but gave no indication of the age of the raw data on which the forecast was based. Davidson knew that all the forecasts were uncertain, and that they sometimes disagreed. But how aware was he that when he looked at the B.V.S. maps he was looking into the past?
[...]
> Davidson dismissed the plan with a thank-you and did not come to the bridge. Evidence suggests that he was still showing a preference for the animated B.V.S. graphics, which indicated the storm progressing more slowly.
[+] [-] crooked-v|8 years ago|reply
> He went down to his stateroom after his conversation with Schultz, and when he returned to the bridge he said, “All right, I just sent up the latest weather. Let us clear everything off the chart table with the exception of the charts.” Schultz opened the B.V.S. program. As it happened, according to the N.T.S.B. report, because of a software glitch, the map that appeared was the very same map that had come in with the previous download, six hours earlier. The raw data on which it was based was at least 12 hours old.
It sounds like it wouldn't actually have helped if the captain had the per-hour update, because this undetailed software glitch meant the device wasn't showing the updated maps anyway.
The apparent moral of that part of the story is that the B.V.S. charts are past useless and into actively harmful territory, since they may not just be old but instead actively incorrect.
[+] [-] forapurpose|8 years ago|reply
(I learned from a burnt hand: We engaged a non-technical but reliable user to check the daily backup log for errors and report any to us. One day we needed the backup and discovered it hadn't run at all for many weeks. Ouch. I asked the user; she said she indeed checked the logs daily and they were fine. She was right: She was seeing the log from the last backup, unchanged every day. My fault entirely, not hers: We should have anticipated the date problem, and we should have utilized someone technically literate enough to understand what they were reading - in this case, someone who would recognize an 'obvious' problem such as the numbers of files and bytes not changing. And we should have tested our backups more often, but that old lesson almost isn't worth mentioning.)
[+] [-] badloginagain|8 years ago|reply
It's also important to realize that the captain was confident with the plan- except the storm in question was abnormal. That's an important fact to consider, as it throws the mariners heuristics off deciding how much more attention to pay.
[+] [-] r00fus|8 years ago|reply
If I see a timestamp on rapidly changing data I sure as hell would like to see if there's a delayed effective stamp (most stock market tickers indicate the "effective date" of their data or say clearly there's a XX min delay).
[+] [-] quadyeast|8 years ago|reply
https://www.youtube.com/watch?v=83uvKWJS2os
[+] [-] organman91|8 years ago|reply
[+] [-] YeGoblynQueenne|8 years ago|reply
Voyage Data Recorder
https://dms.ntsb.gov/pubdms/search/document.cfm?docID=447547...
Errata to Voyage Data Recorder
https://dms.ntsb.gov/pubdms/search/document.cfm?docID=455021...
Addendum to Voyage Data Recorder
https://dms.ntsb.gov/pubdms/search/document.cfm?docID=455022...
Attachment 1 to Addendum to Voyage Data Recorder
https://dms.ntsb.gov/pubdms/search/document.cfm?docID=455023...
[+] [-] dmix|8 years ago|reply
[+] [-] clon|8 years ago|reply
I wonder if in the maritime world there is the equivalent of CRM (Cockpit Resource Management), as in aviation.
Too many lives were lost due to 1st officer never taking over command when the error in the commander's thinking had already become apparent or the entire aircraft was being flown, effectively, by a single infallible captain and not a team.
Lessons were learned decades ago and at least in a 1st world outfit I would like to believe that no 1st officer would hesitate to utter the words "I have control" when in such stress.
[+] [-] aglionby|8 years ago|reply
[+] [-] korethr|8 years ago|reply
[+] [-] clon|8 years ago|reply
[+] [-] Luc|8 years ago|reply
All his articles and books seem to be movie-worthy. I'm not entirely convinced reality is that exciting, though.
[+] [-] hyperpallium|8 years ago|reply
The reality in this case was far, far more exciting than you ever want. If anything, the captain had a preternaturally professional calm.
> ship’s anemometer was in disrepair and had been for weeks
Can't measure wind; the co. didn't subscribe to hourly updates; a "software glitch" gave old reports.
[+] [-] SeanBoocock|8 years ago|reply
[+] [-] jonathanberger|8 years ago|reply
He writes in a gripping manner and normally seems to have done pretty thorough background research, as well as personal experience.
[+] [-] nradov|8 years ago|reply
http://gcaptain.com/tag/el-faro-investigation/
[+] [-] sdhgaiojfsa|8 years ago|reply
[+] [-] bklyn11201|8 years ago|reply
http://www.stormgeo.com/solutions/shipping/on-board-services...
[+] [-] oh_sigh|8 years ago|reply
A better take away might be, for important decisions, you should deeply understand the data(and importantly, it's limitations) you are using to make the decision.
[+] [-] wglb|8 years ago|reply
[+] [-] chrisweekly|8 years ago|reply
2. I was astonished at this: > "It has been reported that a major merchant ship goes down somewhere in the world every two or three days; most are ships sailing under flags of convenience, with underpaid crews and poor safety records."
[+] [-] Someone|8 years ago|reply
[+] [-] dredmorbius|8 years ago|reply
It's the key reason I don't find commercial marine nuclear propulsion viable.
[+] [-] cesarb|8 years ago|reply
[+] [-] dz0ny|8 years ago|reply
https://i.imgur.com/nbwkCPY.png
[+] [-] jtwaleson|8 years ago|reply
[+] [-] lainga|8 years ago|reply
[+] [-] matte_black|8 years ago|reply
[+] [-] blub|8 years ago|reply
[+] [-] hinkley|8 years ago|reply
[+] [-] mannykannot|8 years ago|reply
https://www.newscientist.com/article/mg23731623-300-ghost-sh...
[+] [-] doughj3|8 years ago|reply
[+] [-] BuffaloBagel|8 years ago|reply
[+] [-] leeter|8 years ago|reply
[+] [-] dwighttk|8 years ago|reply
I'm not quite following that sentence. The article makes a lot of the "authorized", saying that meant the captain was being ordered to go directly to San Juan and not use the Old Bahama Channel. However, it seems to say that the diversion into the channel was authorized.
[+] [-] Nrbelex|8 years ago|reply
[+] [-] foobarbazetc|8 years ago|reply
People should trust that instinct. Takes a lot of effort to pull away from the brink, but you should do it. 100% of the time.
[+] [-] boomka|8 years ago|reply
It seems clear that the culture in the industry is set up to weed out those who want to err on the side of safety, and reward those who err on the side of maintaining their shipping schedules.
[+] [-] oldmariner|8 years ago|reply
Some don't pull back because of cowardice.
The rest keep on going because of the sense of adventure. It makes you feel alive.
[+] [-] AnatMl2|8 years ago|reply
[+] [-] brightball|8 years ago|reply
[+] [-] Baeocystin|8 years ago|reply
[+] [-] 089723645897236|8 years ago|reply
Apparently some Norweigans agree with me and are doing exactly this. I suspect this will become the future norm once the bugs are ironed out.
https://www.km.kongsberg.com/ks/web/nokbg0238.nsf/AllWeb/0CB...
[+] [-] deadwing0|8 years ago|reply
The only reason it's rather worrying is because I happen to bank at both of the financial institutions on the list. They are both small, local institutions.
Tried opening the link on a couple other devices but was unable to recreate this. The pop-up appears every time I open the link in Firefox on my phone, though.
[+] [-] NKosmatos|8 years ago|reply