top | item 38145062

GM's Cruise alleged to rely on human operators to achieve "autonomous" driving

172 points| midnightdiesel | 2 years ago |nytimes.com

184 comments

order

kvogt|2 years ago

Cruise CEO here. Some relevant context follows.

Cruise AVs are being remotely assisted (RA) 2-4% of the time on average, in complex urban environments. This is low enough already that there isn’t a huge cost benefit to optimizing much further, especially given how useful it is to have humans review things in certain situations.

The stat quoted by nyt is how frequently the AVs initiate an RA session. Of those, many are resolved by the AV itself before the human even looks at things, since we often have the AV initiate proactively and before it is certain it will need help. Many sessions are quick confirmation requests (it is ok to proceed?) that are resolved in seconds. There are some that take longer and involve guiding the AV through tricky situations. Again, in aggregate this is 2-4% of time in driverless mode.

In terms of staffing, we are intentionally over staffed given our small fleet size in order to handle localized bursts of RA demand. With a larger fleet we expect to handle bursts with a smaller ratio of RA operators to AVs. Lastly, I believe the staffing numbers quoted by nyt include several other functions involved in operating fleets of AVs beyond remote assistance (people who clean, charge, maintain, etc.) which are also something that improve significantly with scale and over time.

sroussey|2 years ago

The relevant staffing section:

> Those vehicles were supported by a vast operations staff, with 1.5 workers per vehicle. The workers intervened to assist the company’s vehicles every 2.5 to five miles

The NYT is definitely implying 1.5 workers per vehicle intervene to assist driving at first read. Only after reading the above comment do I notice that they shoved the statements together using different meanings for “workers” as they didn’t have the actual statistic on hand.

vowelless|2 years ago

Where are these remote assistant human drivers located (country), and how have they been screened for temporarily “driving” vehicles in those cities (do these humans have American licenses? ). How is all this regulated?

Basically, I am curious if these remote assistant drivers are located in foreign nations without American licenses. And if so, how did you get them cleared to be able to “drive” cars on Americas roads?

Thanks

Ps: I took a cruise once in Austin and it needed remote assistance.

pj_mukh|2 years ago

"The stat quoted by nyt is how frequently the AVs initiate an RA session. Of those, many are resolved by the AV itself before the human even looks at things, since we often have the AV initiate proactively and before it is certain it will need help."

Hoo boy, sure wish the NYT had clarified that. That changes things significantly.

patrick451|2 years ago

It's telling that you declined a request for an interview, yet still feel the need to clarify on HN. You'd be doing a lot better with transparency and public trust by just taking the interview.

simonebrunozzi|2 years ago

Hi Kyle (hello again), thanks for being so transparent with this.

I suspect this is a moment where news are looking for a scapegoat / villain from the AV sector, and your team is an easy target given what has happened recently.

I believe that transparency is the right way to address issues and concerns. Please keep doing that.

phkahler|2 years ago

>> Cruise AVs are being remotely assisted (RA) 2-4% of the time on average, in complex urban environments. This is low enough already that there isn’t a huge cost benefit to optimizing much further, especially given how useful it is to have humans review things in certain situations.

Funny, since I thought full autonomy was the goal of the company. 2 percent human intervention isn't scalable.

throwaway1104|2 years ago

Keep it up, Kyle! All new tech will have hiccups and opposition. Really enjoyed my ride experience when I visited SF.

ra7|2 years ago

Thanks for the clarifying! This makes a lot of sense. I think NYT did a really poor job of explaining the remote assistance bit.

momento_uno|2 years ago

2-4% is very high. It essentially means none of your trips can be completed without remote intervention.

This puts your cars and the safety of the whole city at the mercy of reliability of mobile networks. This is a fundamental architectural change in the design of the city. Do the telecom operators take liability if they can't meet their designed SLOs for availability? What are the worse case scenarios that you have considered?

vlovich123|2 years ago

Did NYT not reach out to you for comment? Curious why your reply is here instead of within the article itself.

gctwnl|2 years ago

I would consider this realistic service design, just as Meta’s Cicero (plays blitz Diplomacy) is smart design. It might work as a service.

What the answer glances over is that even with just 3% of the time requiring human assistance (2 minutes out of every hour) the term ‘autonomous vehicle’ is not really applicable anymore in the sense everybody is using/understanding that term. The idea behind that term assumed ‘full’ autonomy. Self driving cars. And there is no reason to assume that this is still in sight. The answer puts that ‘self-driving car’ on the shelf.

PS. Human assistent seems to me a difficult job, given the constant speed and concentration requirements.

MagicMoonlight|2 years ago

Your plan is to make someone in india remote control my car? What if the signal goes down? What if it lags and they accidentally give the wrong instructions?

This is the stupidest idea I’ve ever heard

southerntofu|2 years ago

Hello Cruise CEO, there's a huge market for durable and profitable "dumb" cars. Why don't you get on that market? In a time when electronics represents over 30% of car costs and ~50% of car failures, people like me would be happy to buy a car that doesn't suck (low-tech) and can be maintained for decades for a reasonable price. In the meantime, i'll keep buying old Renault/Peugeot cars from the fifties/sixties i guess :(

cheeselip420|2 years ago

remote operation of vehicles often makes a lot of sense economically, since you can effectively decouple drivers from vehicles/riders. As you pointed out, this means you can shift to deal with peak loads and all of that - great.

Given everything you know now, was it wise to push for expansion over improvements to safety and reliability of the vehicles? On one hand, there is certainly value in expanding a bit to uncover edge-cases sooner. On the other hand, I'm not convinced it was worth expanding before getting the business sorted out.

My guess is that given the relatively large fixed costs involve in operating an AV fleet, that it makes some sense to expand at least up to that sort of 'break even' point. Do we know what that point is? Put differently, is there some natural "stopping point" of expansion where Cruise could hit break-even on its fixed costs and then shift focus towards reliability?

flandish|2 years ago

So when low wage mechanical turk costs turn out cheaper than engineering to improve driverless vehs… this will just be another exploitative gig job for folks in remote locations?

I don’t trust proper attention will be given to improvements in tech once profit and roi is considered compared to human labor costs especially in lower wage nations.

jdjdjdhhd|2 years ago

Can spying be disabled on your cars?

monero-xmr|2 years ago

Huge cojones on the CEO to risk public statements given the enormous legal and regulatory pressure being applied. I certainly wouldn’t recommend this tactic!

cheeselip420|2 years ago

Cruise is leveraging human-in-the-loop to expand faster than they otherwise would, with the hope that they will solve autonomy later to bring this down.

I don't think this is a viable strategy though given the enormous costs and challenges involved.

There doesn't exist a short-term timeline where Cruise makes money, and the window is rapidly closing. They needed to expand to show big revenues, even if they had to throw 1.5 bodies per car at the problem.

Prediction: GM will offload cruise, a buyer will replace leadership and layoff 40% of the company. The tech may live to see another day, but given the challenges that GM has generally (strikes, EVs, etc), they can no longer endlessly subsidize Cruise.

Retric|2 years ago

Human in the loop can be vastly cheaper than you might think.

If this lets them have the only level 5 system on the market they could double that and millions would happily pay. Suppose your a trucking company would you rather pay 50k / year or 5k/year? That’s a stupidly easy choice.

Americans drive roughly 500 hours per year. If they can replace 98% with automation and the other 2% with someone making 20$/hour that only costs them ~200$/year, which then drops as the system improves.

chaostheory|2 years ago

GM actually spun off Cruise in 2018. Honda now has shares in Cruise. SoftBank used to own some as well, but GM bought out their share last year

jakobson14|2 years ago

you don't need autonomy to be profitable.

Imagine a car rental service where someone in an office building drives the empty car to you, then drives it back when you're done with it. No taking public transportation just to get back to the garage to pick up the next drop-off. Imagine simply swapping the driver controlling a ling haul truck remotely when it's time for a shift change. With good handoff the truck can be driving 24/7 without ever slowing down.

Really the only autonomy you need in that situation is enough to pull the truck/car/whatever over and park it if the connection is lost.

dontblink|2 years ago

So Waymo/Google winning here in your opinion?

wolverine876|2 years ago

> Two months ago, Kyle Vogt, the chief executive of Cruise, choked up as he recounted how a driver had killed a 4-year-old girl in a stroller at a San Francisco intersection. “It barely made the news,” he said, pausing to collect himself. “Sorry. I get emotional.”

...

> Cruise’s board has hired the law firm Quinn Emanuel to investigate the company’s response to the incident, including its interactions with regulators, law enforcement and the media. / The board plans to evaluate the findings and any recommended changes. Exponent, a consulting firm that evaluates complex software systems, is conducting a separate review of the crash, said two people who attended a companywide meeting at Cruise on Monday.

After the first [edit: the first performative charade, about little girl in a stroller], why should we trust the second isn't also a performative charade? What independence or credibility does some hired law firm have, that the company itself does not? How about using an independent third party?

pj_mukh|2 years ago

Hmm? I saw it exactly the opposite. A lot of people in the autonomous driving industry are driven by exactly what Vogt describes (little girl in the stroller etc.). See also Chris Urmson of Waymo fame's TED talk, he talks about a similar motivation[1].

Its a fallacy everyone conveniently ignores. The woman the Cruise car ran over was actually first hit by a human driver who is still at-large, not a peep about him. The press kinda just accepts this as the "cost of doing business".

The way I see it, Vogt sincerely believes autonomous cars will make things safer from the #2 killer of Children under 19 (outside of guns) by a wide margin [2] and therefore accelerated the rollout past what was safe. I see no evidence otherwise.

[1] https://www.ted.com/talks/chris_urmson_how_a_driverless_car_...

[2] https://www.nejm.org/doi/full/10.1056/nejmc2201761

wmf|2 years ago

Independent third parties don't work for free and if you pay them (by your logic) they're no longer independent. The best you can probably hope for is a government investigation.

pests|2 years ago

> performative charade

How is it performative?

Is it not sad that a 4-year-old girl in a stroller got killed by a car? That it barely made the news?

Or is that just not sad and is normal these days?

icedistilled|2 years ago

People like to say self driving cars are safer than human drivers - but the human drivers that tend to do the most unsafe antics seem to be the humans that are least likely to make use of self driving cars.

ciabattabread|2 years ago

Cruise is trying to save itself from getting shut down by GM. I guess it would look slightly better for optics if the GM board hired them instead of Cruise’s board. But it’s the same money, and it’s GM’s decision at the end.

raldi|2 years ago

Can you explain what you mean by “after the first”?

batmansmk|2 years ago

Having to be remotely operated every 2.5 to 5 miles seem to defeat most of the economics of self driving cars.

Back of the napkin math, cars drive at an average of 18mph in cities, so every 10-20min. Let’s assume it takes over for 1min, and that you need remote drivers not too far for ping purposes, so at the same hourly rate. To guarantee you’ll be able to take over all demands immediately, due to the birthday paradox, you end up needing like 30 drivers for 100 vehicles? It’s not that incredible of a tech…

joshjob42|2 years ago

Where do you get 30 drivers per 100 vehicles from?

Let's model it as every takeover is 1 min, and vehicles need help 5% of minutes. Then you'd model the # of vehicles needing help in any given minute as a binomial distribution with p=0.05 and N=100, and you find you get 99.99% of the time you need less than 15 drivers per 100 vehicles. By 20 drivers, you cover all but 2e-8 of the time (or once per century).

But that's a bit misleading. It's a small-size effect. With 10k cars, you get cover all but 4e-6 of all minute periods with just 600 drivers (0.06 per car).

By 100k cars, you have 44 9s of reliability with 0.06 drivers per car.

There's some more complicated things that arise since probably the distribution of how long vehicles need help will be Poisson distributed (with an average of 1min) most likley, etc. But the point will stand, for large fleets you only need a modest margin over the average rate to get effectively complete coverage under normal conditions. It would only be in really extreme situations, like a hurricane messing up badly a lot of the Eastern seaboard or something, that you'd maybe run into issues (which, admittedly, is a real potential problem).

cheriot|2 years ago

Wages can fall off a cliff within modest distances. To use unemployment rate as a proxy for driver pay, Bakersfield, CA 7.5% and San Francisco, CA 3.5%. Go a little farther to Los Vegas 5.7% and one can avoid California's minimum wage.

pj_mukh|2 years ago

Just FYI, Most autonomous car companies have backup drivers.

Its the disengagement rate that drives the number of operators you need per driver and therefore the economics. Theoretically, this rate should be improving steadily at all these companies.

Cruise seems to have a bad disengagement rate right now (<5miles seems really low), but methinks nytimes might be partaking in some obfuscation here.

Waymo's should be much better already. Curious by how much though.

SkyPuncher|2 years ago

> you end up needing like 30 drivers for 100 vehicles?

What? That's literally insane compared to the current standard of 100 drivers for 100 vehicles. They're literally reducing 70% of the labor cost compared to uber/lyft/etc.

It's pretty reasonable to expect that this will improve over time as well. This is exactly how you want a startup to roll out a new technology.

* Build a pretty good base implementation

* Do things that don't scale for the edge cases

* Reduce the things that don't scale over time

Even if they can only improve this to 10 for 100, that's still a massive improvement.

In my area, a small, rural city, this would literally be a game changer. Right now, there's a single Uber within 15 minutes - if I'm lucky. Meanwhile, cruise could drop a handful of car in town, let them idle (at no cost), then pay a driver for a few minutes of intervention every now and then.

This also enables intercity transit. Most of that is highway miles. Outside of the start and end, those are easy and predictable. You could have dozens/hundreds of miles where Cruise can compete with the cost of privately owned vehicles.

Lastly, this makes it feasible for Cruise to reposition cars between cities without huge costs. Currently, that's basically impossible. Any human driven car needs to offer the driver a ride in the opposite direction.

spondylosaurus|2 years ago

Yeah, the driver-to-passenger ratio is still way less efficient than a train or even a bus.

AmigoCharlie|2 years ago

Look, isn't remotely assisted driving something unbelievably stupid? Why should I rely, when I am on my "driverless" car, rely on someone else who is remote, need to be updated at all times about the situation (when things can go wrong in a matter of tenth of seconds, while driving), and needs to react, and it's not as much motivated as me (as I am risking my life, while he is sitting somewhere without having as much skin in the game as me)? It makes a lot more sense, then, to have just an assisted driving car, or a semi-autonomous car where the "assistant" to the AI it's me and not someone else.

genewitch|2 years ago

In my estimation, the 1/10th of a second thing isn't what the remote drivers are for. The car should just stop and avoid getting rear-ended, or dodge, or whatever. charitably, the remote drivers are aiding where a human would need assistance too, like "what hand signal is that cop making? do they mean go, or are they telling me to stop?" or "this light appears to be broken, is it safe to treat it as a 4 way stop?"

my subaru can avoid accidents, it can even avoid things that 100% would not be an accident, even on black ice; so i don't think this is what the remote drivers are doing.

wolverine876|2 years ago

If humans need to remotely intervene for a car in motion, that implies it could impact safety.

If that's correct, then the remote signaling of a problem and the human's response and control must have flawless availability and low latency. How does Cruise achieve that?

Cellular isn't that reliable. Maybe I misunderstand something.

mwint|2 years ago

Appears Cruise isn’t giving these remote drivers a steering wheel and gas; rather they make strategic decisions: Go around this, follow this path, pull over, etc. The car is able to follow a path on its own. Determining the correct path is where it gets hard.

bookofjoe|2 years ago

I've said this here before and I will repeat it:

An overwhelming majority of Americans will choose 45,000 deaths in car crashes annually (last year's number) in human-driven cars over 450 deaths/year with all self-driving cars.

In the American (and probably ALL) mind(s), human agency trumps all.

ooterness|2 years ago

This was a plot point in Captain Laserhawk: All the self-driving cars and flying drones were actually being remotely piloted by prisoners in a massive VR facility.

https://en.wikipedia.org/wiki/Captain_Laserhawk:_A_Blood_Dra...

tibbydudeza|2 years ago

There is a movie where the border wall has been built and they pay Mexicans to remote operate drones and other equipment north of the border.

sroussey|2 years ago

Fully remote driven cars is another company (can’t recall their name).

neilv|2 years ago

> Company insiders are putting the blame for what went wrong on a tech industry culture — led by the 38-year-old Mr. Vogt — that put a priority on the speed of the program over safety. [...] He named Louise Zhang, vice president of safety, as the company’s interim chief safety officer [...]

I hope Chief Safety Officer isn't just a sacrificial lamb job, like CISO tends to be.

Is the "interim" part hinting at insufficient faith, and maybe future blame will be put on how the VP Safety performed previously (discovered after the non-interim person is hired)?

> [...] and said she would report directly to him.

Is the CSO nominally responsible for safety?

Does the CSO have any leverage to push back when their recommendations aren't taken, other than resigning?

lacker|2 years ago

It sounds like there are a lot of people working at GM who don't like Cruise and are willing to complain to the NYT about it. One of those frustrating "we're a startup inside a large company" things.

Cruise employees worry that there is no easy way to fix the company’s problems.

Company insiders are putting the blame for what went wrong on a tech industry culture.

What, because car companies with car company culture are doing such a great job building self-driving cars?

I'm rooting for both Cruise and Waymo here. Self-driving cars would be great for humanity. Good luck to the teams working hard to make them happen.

dreamcompiler|2 years ago

Here we go again with a CEO who proclaims "autonomous cars are safer than human-driven cars." And their definition of "safer" conveniently ignores that autonomous cars create new failure modes which do not exist in manually-driven cars.

It may be true that statistically fewer fatalities per mile happen with autonomous cars than with human-driven cars. But that's irrelevant. If the car kills one person because it did something utterly stupid like driving under a semi crossing the highway or dragging a pedestrian along the ground, the public will not accept it.

This is another example of the uncanny valley problem: Most "smart" devices are merely dumb in new ways. If your "smart" gizmo is only smart in how it collects private information from people (e.g. smart TVs), or it's merely smarter than a toggle switch, that's not what the public considers smart. It has to be smarter than a reasonably competent human along almost all dimensions; otherwise you're just using "smart" as a euphemism for "idiot savant." Self-driving cars are a particularly difficult "smart" problem because lives are at stake, and the number of edge cases is astronomical.

DelightOne|2 years ago

> Having to be remotely operated every 2.5 to 5 miles

Regarding Cruises' suspension, how likely is it that the backup driver restarted the car to drive again after the car stopped with the pedestrian below?

evbogue|2 years ago

This is the same whacky theory I've been spreading about Tesla self-driving for a year or so. "Imagine Tesla self-driving is like some dude driving your car via videogame on the other side of the world."

Most people are pretty sure my theory is wrong. I have absolutely no evidence this is true, it's just some crazy idea that popped into my head one day.

cheeselip420|2 years ago

Like some sort of fucked up Ender's Game situation.

xyst|2 years ago

I get a feeling Cruise is going to get sold off within the next 5 yrs. Waymo will likely be the leading provider for “autonomous vehicle” software/hardware.

Government Motors can only sustain such a loss on their books for a short time. This is probably why Vogt has been pushing so hard for market dominance.

haltist|2 years ago

It's not an allegation. It's the same as using human feedback for tuning large language models. There are no autonomous cars currently regardless of what is written on the marketing brochures. In various "emergency" situations the cars phone home and ask a human operator to take over the controls.

ra7|2 years ago

This is completely incorrect. Remote operators cannot “take over controls” at all and hence cannot help in any “emergency” i.e. safety critical situation (e.g. preventing a crash). All they can do is assist the vehicle with things drawing a path to get around a parked vehicle, instructing it to do a multi-point turn when it’s stuck and so on.

What the article says is that Cruise vehicles need some sort of assistance every 2.5 to 5 miles (I highly doubt this number is accurate). Not that they’re getting into emergency situations that frequently.

l33t7332273|2 years ago

> It's the same as using human feedback for tuning large language models

It isn’t remotely the same. This would be like if human operators typed some of chatGPTs answers

hartator|2 years ago

There is an emergency every 2-5 miles?

throwaway5959|2 years ago

The latency on that has to be massive.

causality0|2 years ago

I thought being a social media moderator and being constantly exposed to violence, racism, and child pornography was bad. Having your whole day being a series of "quick, don't let these people die!" moments seems like the worst tech job on earth.

tempsy|2 years ago

I wonder if there are rails to prevent a bad actor Cruise worker from remote driving erratically…

KennyBlanken|2 years ago

@dang title not the same as original

mlinhares|2 years ago

The title isn't news at all as every single trustworthy autonomous driving solution MUST HAVE human operators somewhere to take over but the actual article is a good summary of Cruise's current situation and I'd guess the competition as well.

dventimi|2 years ago

Where is the article does it even support the title? I'll reread it but I didn't see anything about human operators.

donsupreme|2 years ago

> staff intervened to assist Cruise's vehicles every 2.5 to five miles

ProAm|2 years ago

Cruise came out of YC if I recall?

chrisgd|2 years ago

Can we just admit that this likely isn’t possible in our lifetime and put more money into early childhood education, better healthcare and geriatric care?