> "[A]lgorithms are made by people and the results ... were reviewed multiple times by people,"
You got that right. The way they talk about "an algorithm did it" makes it seem as as if they think that somehow explains it, like there was only one algorithm possible handed down from god or something.
We'll be seeing more and more of this of course. "We can't be blamed, it was an algorithm! We can't be blamed for trusting in the algorithm, because everyone knows algorithms are objective, right?"
I'm not sure "laypeople" realize that, especially in this particular case, "algorithm" is just a fancy word for "formula". Right, you MADE the formula, and it was wrong.
I think most people are misunderstanding the likely root cause.
> An algorithm was used to assign its first allotment of the vaccine. The algorithm prioritized health care workers at highest risk for COVID infections, along with factors like age and the location or unit where they work in the hospital. Residents did not have an assigned location, and along with their typically young age, they were dropped low on the priority list.
Whoever coded up the algorithm for personal COVID-risk probably forgot to take care of null-states for the location input. Residents had their location set to null, and were therefore prioritized lower than they should've been.
It's possible that the admins purposefully created this null-state bug so as to have a reasonable fallback story in the case they got caught, but per Occam's Razor, I think it's much more likely it was a dumb, honest mistake.
In general, blaming the algorithm is an indirect way of blaming software engineers.
From Facebook to the VW emissions scandal to Stanford’s vaccine allocation, people love narratives that blame engineers. I suspect this trend will only get worse as the general public realizes that engineers are now a highly compensated professional, similar to how lawyers are the butt of so many jokes.
I'm not even sure this formula is WRONG. It's just not great for morale.
Residents are generally under 34 and in good heath. In the USA, only ~2400 people 34 have died from COVID--and that is mostly people with comorbidites.
In comparison about ~250,000 people over 55 have died.
You'd obviously have do account for life-years lost, risk of exposure [1], etc. I wouldn't be shocked if giving the vaccine to a 25 year old resident is a sub-optimal choice.
[1]Based on my MiL's experience (an ER doc), the risk of infection at a hospital is a lot lower than it used to be because they have PPE now. She's more worried about catching it from her son who works in an office. That said, they need to take into account risk of exposure and risk that the residents spread it to others.
“Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.”
The most charitable interpretation is that they're admitting they failed, but saying where they failed is in basic due diligence of looking at the output of running the algorithm. And that they're claiming that, if they had done so, they wouldn't have been OK with it either.
It's sort of like saying, "That's definitely a bug. It was unacceptable for me to put my code into production without ever testing it, and for that I take the blame. But I assure you that I did not intentionally write code designed to delete everyone's data."
Obviously this may or may not be what happened. It may even be what they're pretending happened.
I do think people understand that. Blaming the algorithm is a way of saying that we agreed on the rules by which vaccines would be assigned and that they were fair. And that we can’t just go back and change them because we don’t like the results when the rules are applied. At that point why even have a formal process for assigning vaccines when all we’re doing is coming a post-hoc justification for a particular outcome. If we already know the outcome we want then just use that!
Fwiw, the administration said:
“””
We are writing to acknowledge the significant concerns expressed by our community regarding the development and execution of our vaccine distribution plan. We take complete responsibility and profusely apologize to all of you. We fully recognize we should have acted more swiftly to address the errors that resulted in an outcome we did not anticipate
“””
The chief resident sent an email explaining the nature of the error. The chief resident is, per my limited understanding, a resident from the previous year who stays on in a leadership role. Not obvious to me that someone in that position explaining the root cause is problematic.
Standing on the shoulders of Giants, I would like to suggest the term 'Distributed Bureaucracy' to reflect when an algorithm you didn't even know existed determines your allocation of resources.
I'm more curious why there was an algorithm involved at all. Based upon the number of doses they had, they should have immediately been given to as many staff on first responder units, like ER, ICU, etc as would take it. My wife works at a hospital as a nurse and this is what they have done for the first phase of vaccination. After that I can see a need for an algorithm to determine who is most at need but there is no excuse for the administrators for what happened at Stanford.
Sounds very similar to “don’t blame the decision maker, because they were just following a moral/legal principle.” Except that, bizarrely, “principles” are usually regarded in high esteem even by people who vehemently disagree with the principles and the decisions excused by them. How often do you hear things (especially in US politics) like “well at least he/she is a principled politician, that’s so rare these days.”
My pet peeve is when people present averages with no care. If the average is is the result they want to see they assume everything is fine under the hood.
There's so many lessons why that is terrible, like the fighter cockpit design history, and yet it's super common for people to stop all inquisition when the average shows them what they hope is true across all subgroups.
With you right until the end where formulas are made by people. In many NN setups, the “formula” is pretty much a self trained black box so in those cases people can only be blamed for the input data that went into training the system.
The labor of medical residents is something hospital systems exploit during normal times, but that exploitation has severely deepened during the pandemic. At the hospital my partner works at, respiratory therapists and nurses got a $10k bonus for working during COVID; the residents got nothing despite working insane hours in ICU, routinely working more than the legally mandated 90 hours per week. Just because doctors earn more later in their careers does not excuse the level of labor exploitation they are subject to during residency.
Stanford is not the only hospital system to restrict access to the vaccine from frontline residents. I can name 3 other local hospital systems in my city that have vaccinated administrative & C-suite/VP level staff before doctors, nurses, and other frontline employees. If vaccine allocation is getting messed up this early on within these closed systems, I can't help but think the next 2-3 phases will go awry as well--what checks are in place to ensure these vaccines get distributed to grocery store workers before people who are willing to pay more to get it early?
> I can name 3 other local hospital systems in my city that have vaccinated administrative & C-suite/VP level staff
I don't understand why society is putting up with this. Right now if you're not in a daily COVID-facing role (i.e. an actual front line medical worker) or in a nursing home you should not be getting the shot. This makes my blood boil. There should have been laws passed regarding ordering of the distribution with criminal penalties for line jumpers like this.
There's an article in our local paper with a happy picture of one of our state's congressional representatives (a healthy 34-year-old!) getting the shot. Like WTF? There's doctors and nurses who are treating covid patients who can't get it yet. Why the heck does Congress get priority over them?
Not only do these people have no shame, half of them even have the nerve to brag about it to the rest of us plebes who will have to wait months or more to get it.
My wife is one of these trainees left out by this "algorithm." She's an intensive care fellow and was caught up in this mess yesterday. It's really hard to witness considering how much I worry about her everyday she's on service.
My understanding is the training is capped at 80 hours/week, but they average that over the month. They definitely work more than that some weeks, but it's usually because it's not scheduled hours; it's shifts that run long because of emergencies or codes or whatever. When she was a resident and had to write patient notes, that definitely took extra hours each night after her shifts. It's brutal.
And for what it's worth, not all doctors make great money after training. In pediatrics, salaries are generally half what adult doctors make. For a lot of subspecialties, that's on par with average software engineering salaries, even after about 14 years of education and training.
I can name 3 other local hospital systems in my city that have vaccinated administrative & C-suite/VP level staff before doctors, nurses, and other frontline employees.
Not surprising, unfortunately. Our "call room" in one hospital during residency was a utility closet. There are mandates of what to do, but hospitals routinely ignore them. Who will report problems? No resident wants to have their program lose accreditation.
Our program director (in psychiatry) always pointed out that psychiatry had one of the highest rates of people going over the 80 hour-per-week limit in the health system. It was much more uncommon for us than for most of our colleagues, but we tended to report it on the rare occasion it happened. My surgical colleagues who went over the limit were asked to meet with their program director when it happened, and it was much easier just to "round down."
Glad I did a residency, but mine was relatively easy. I probably wouldn't have made it through some programs.
The great thing about an algorithm is that when it gives you results that benefit you, you can accept them and then pass the buck on to it when people get outraged:
"According to an email sent by a chief resident to other residents, Stanford's leaders explained that an algorithm was used to assign its first allotment of the vaccine. The algorithm was said to have prioritized those health care workers at highest risk for COVID infections, along with factors like age and the location or unit where they work in the hospital. Residents apparently did not have an assigned location, and along with their typically young age, they were dropped low on the priority list."
The whole mess has demonstrated why ethical AI research is important. It's so easy for people to disregard why and how biases exist in their systems when it doesn't affect them personally.
Exactly. There can be a cottage industry using evolutionary algorithms in which the fitness to optimize is the benefit of the interested part. "This new algorithm indicates that the best way to improve the economy is to increase 3X the compensation of Senators".
Another messed up thing I've heard is that many hospitals are prioritizing vaccines for WFH Administrators over staff that are actually working at hospitals..
Its a weird system, I asked someone I know who is doing her residency how many times she's been COVID tested this year... just once. Apparently as long as you're not working on the COVID floor there isn't really a requirement at her hospital.
Since this algorithm is obviously flawed this way, is it likely to be flawed with respect to the timing of the second dose?
Every report I’ve seen from experiment subjects (who may have received the placebo) indicates that the second dose sucked for 2-3 days, and was much worse than the first.
If you give all your, for example, ICU staff the second dose at the same time, and then they can’t work for 2 days, how do you staff that?
Did you plan for this, or will it be a surprise that “The Algorithm” missed?
This is going to be how it goes broadly, at a national and international level.
I guarantee you that you'll see access for the wealthy much sooner than you'll see access for the folks at highest risk. I don't know how to help with this, but I'd much rather see farm workers, grocery store clerks, the homeless, bus drivers, etc. get access after we take care of the medical staff (who are exposed to patients) and the elderly (and others in extremely risky environments.)
Yet I'm sure that's not how this will go.
How long before testing and vaccination becomes an employment perk at a FAANG company?
Residency requirements are nothing more than a form of hazing.
The doctors are half-asleep while making important decisions. How can this be not be outrageously unacceptable?
Oh, wait, no more need for false PCR tests, we have vaccines now. Forget testing, they’re wrong anyway, everyone get in line for a dose of nothing.
The good news appearing in media recently are the high percentage of people all over the world against being vaccinated, it means the human reason algorithm is still top notch.
[+] [-] jrochkind1|5 years ago|reply
You got that right. The way they talk about "an algorithm did it" makes it seem as as if they think that somehow explains it, like there was only one algorithm possible handed down from god or something.
We'll be seeing more and more of this of course. "We can't be blamed, it was an algorithm! We can't be blamed for trusting in the algorithm, because everyone knows algorithms are objective, right?"
I'm not sure "laypeople" realize that, especially in this particular case, "algorithm" is just a fancy word for "formula". Right, you MADE the formula, and it was wrong.
[+] [-] smnrchrds|5 years ago|reply
https://twitter.com/pinboard/status/744595961217835008
[+] [-] fairity|5 years ago|reply
> An algorithm was used to assign its first allotment of the vaccine. The algorithm prioritized health care workers at highest risk for COVID infections, along with factors like age and the location or unit where they work in the hospital. Residents did not have an assigned location, and along with their typically young age, they were dropped low on the priority list.
Whoever coded up the algorithm for personal COVID-risk probably forgot to take care of null-states for the location input. Residents had their location set to null, and were therefore prioritized lower than they should've been.
It's possible that the admins purposefully created this null-state bug so as to have a reasonable fallback story in the case they got caught, but per Occam's Razor, I think it's much more likely it was a dumb, honest mistake.
[+] [-] PragmaticPulp|5 years ago|reply
From Facebook to the VW emissions scandal to Stanford’s vaccine allocation, people love narratives that blame engineers. I suspect this trend will only get worse as the general public realizes that engineers are now a highly compensated professional, similar to how lawyers are the butt of so many jokes.
[+] [-] rhino369|5 years ago|reply
Residents are generally under 34 and in good heath. In the USA, only ~2400 people 34 have died from COVID--and that is mostly people with comorbidites.
In comparison about ~250,000 people over 55 have died.
You'd obviously have do account for life-years lost, risk of exposure [1], etc. I wouldn't be shocked if giving the vaccine to a 25 year old resident is a sub-optimal choice.
[1]Based on my MiL's experience (an ER doc), the risk of infection at a hospital is a lot lower than it used to be because they have PPE now. She's more worried about catching it from her son who works in an office. That said, they need to take into account risk of exposure and risk that the residents spread it to others.
[+] [-] cbanek|5 years ago|reply
- Dune
[+] [-] beamatronic|5 years ago|reply
[+] [-] adrianmonk|5 years ago|reply
It's sort of like saying, "That's definitely a bug. It was unacceptable for me to put my code into production without ever testing it, and for that I take the blame. But I assure you that I did not intentionally write code designed to delete everyone's data."
Obviously this may or may not be what happened. It may even be what they're pretending happened.
[+] [-] Spivak|5 years ago|reply
[+] [-] glaugh|5 years ago|reply
The chief resident sent an email explaining the nature of the error. The chief resident is, per my limited understanding, a resident from the previous year who stays on in a leadership role. Not obvious to me that someone in that position explaining the root cause is problematic.
[+] [-] to11mtm|5 years ago|reply
[+] [-] moocow01|5 years ago|reply
[+] [-] tshaddox|5 years ago|reply
[+] [-] icedistilled|5 years ago|reply
There's so many lessons why that is terrible, like the fighter cockpit design history, and yet it's super common for people to stop all inquisition when the average shows them what they hope is true across all subgroups.
[+] [-] JoshTko|5 years ago|reply
[+] [-] tomcam|5 years ago|reply
[+] [-] trianglem|5 years ago|reply
[+] [-] amb23|5 years ago|reply
Stanford is not the only hospital system to restrict access to the vaccine from frontline residents. I can name 3 other local hospital systems in my city that have vaccinated administrative & C-suite/VP level staff before doctors, nurses, and other frontline employees. If vaccine allocation is getting messed up this early on within these closed systems, I can't help but think the next 2-3 phases will go awry as well--what checks are in place to ensure these vaccines get distributed to grocery store workers before people who are willing to pay more to get it early?
[+] [-] davidf560|5 years ago|reply
I don't understand why society is putting up with this. Right now if you're not in a daily COVID-facing role (i.e. an actual front line medical worker) or in a nursing home you should not be getting the shot. This makes my blood boil. There should have been laws passed regarding ordering of the distribution with criminal penalties for line jumpers like this.
There's an article in our local paper with a happy picture of one of our state's congressional representatives (a healthy 34-year-old!) getting the shot. Like WTF? There's doctors and nurses who are treating covid patients who can't get it yet. Why the heck does Congress get priority over them?
Not only do these people have no shame, half of them even have the nerve to brag about it to the rest of us plebes who will have to wait months or more to get it.
[+] [-] pdxandi|5 years ago|reply
My understanding is the training is capped at 80 hours/week, but they average that over the month. They definitely work more than that some weeks, but it's usually because it's not scheduled hours; it's shifts that run long because of emergencies or codes or whatever. When she was a resident and had to write patient notes, that definitely took extra hours each night after her shifts. It's brutal.
And for what it's worth, not all doctors make great money after training. In pediatrics, salaries are generally half what adult doctors make. For a lot of subspecialties, that's on par with average software engineering salaries, even after about 14 years of education and training.
[+] [-] epmaybe|5 years ago|reply
I thought about a top level comment to link this, but we are naming and shaming programs that exploit residents during this time.
https://docs.google.com/spreadsheets/d/1ZgEKvTr1lvTLHsREeill...
[+] [-] reaperducer|5 years ago|reply
What's stopping you?
[+] [-] stainforth|5 years ago|reply
[+] [-] chromatin|5 years ago|reply
Our institution built a new billion dollar hospital and did not include call rooms.
Great job.
[+] [-] neilv|5 years ago|reply
[+] [-] jseliger|5 years ago|reply
[+] [-] euthymiclabs|5 years ago|reply
Our program director (in psychiatry) always pointed out that psychiatry had one of the highest rates of people going over the 80 hour-per-week limit in the health system. It was much more uncommon for us than for most of our colleagues, but we tended to report it on the rare occasion it happened. My surgical colleagues who went over the limit were asked to meet with their program director when it happened, and it was much easier just to "round down."
Glad I did a residency, but mine was relatively easy. I probably wouldn't have made it through some programs.
[+] [-] mmastrac|5 years ago|reply
"According to an email sent by a chief resident to other residents, Stanford's leaders explained that an algorithm was used to assign its first allotment of the vaccine. The algorithm was said to have prioritized those health care workers at highest risk for COVID infections, along with factors like age and the location or unit where they work in the hospital. Residents apparently did not have an assigned location, and along with their typically young age, they were dropped low on the priority list."
[+] [-] 1-6|5 years ago|reply
[+] [-] tmotwu|5 years ago|reply
[+] [-] cambalache|5 years ago|reply
[+] [-] ssivark|5 years ago|reply
[+] [-] craftinator|5 years ago|reply
[+] [-] at_a_remove|5 years ago|reply
[+] [-] flowersjeff|5 years ago|reply
So we're to believe that a rogue computer code screwed up?.. i.e.: nobody in charge mess'ed up?
Why again do those with the largest paychecks, given the largest slack?
[+] [-] Rebelgecko|5 years ago|reply
Its a weird system, I asked someone I know who is doing her residency how many times she's been COVID tested this year... just once. Apparently as long as you're not working on the COVID floor there isn't really a requirement at her hospital.
[+] [-] craz8|5 years ago|reply
Every report I’ve seen from experiment subjects (who may have received the placebo) indicates that the second dose sucked for 2-3 days, and was much worse than the first.
If you give all your, for example, ICU staff the second dose at the same time, and then they can’t work for 2 days, how do you staff that?
Did you plan for this, or will it be a surprise that “The Algorithm” missed?
[+] [-] emcq|5 years ago|reply
[+] [-] cultus|5 years ago|reply
[+] [-] 1024core|5 years ago|reply
I bet the "outcome" they're talking about is the _protest_, not the allocation.
[+] [-] throwaway9870|5 years ago|reply
[+] [-] Pfhreak|5 years ago|reply
I guarantee you that you'll see access for the wealthy much sooner than you'll see access for the folks at highest risk. I don't know how to help with this, but I'd much rather see farm workers, grocery store clerks, the homeless, bus drivers, etc. get access after we take care of the medical staff (who are exposed to patients) and the elderly (and others in extremely risky environments.)
Yet I'm sure that's not how this will go.
How long before testing and vaccination becomes an employment perk at a FAANG company?
[+] [-] unknown|5 years ago|reply
[deleted]
[+] [-] ArtDev|5 years ago|reply
[+] [-] JohnGB|5 years ago|reply
[+] [-] curiousllama|5 years ago|reply
[+] [-] almost_usual|5 years ago|reply
Inequality in the US is getting more and more extreme. This shit will not be tolerated much longer, riots will get worse, people will get angrier.
[+] [-] WalterBright|5 years ago|reply
Under a socialist system, resources are allocated to those with the most political power.
Under an anarchist system, resources are allocated to those with the most firepower.
The vaccine is being distributed under a socialist system. Nobody should be surprised at the results.
[+] [-] knolan|5 years ago|reply
[+] [-] raadore|5 years ago|reply
Oh, wait, no more need for false PCR tests, we have vaccines now. Forget testing, they’re wrong anyway, everyone get in line for a dose of nothing.
The good news appearing in media recently are the high percentage of people all over the world against being vaccinated, it means the human reason algorithm is still top notch.