[zelphirkalt]

Since lidar has distance information and cameras do not, it was always a ridiculous idea by a certain company to use cameras only. Lidar using cars are going to replace at least the ones that don't make use of this obvious answer to obstacle detection challenges.

[runjake]

Karpathy provided additional context on the removal of LiDAR during his Lex Fridman Podcast appearance. This article condenses what he said:

https://archive.is/PPiVG

And here's one of Elon's mentions (he also has talked about it quite a bit in various spots).

https://xcancel.com/elonmusk/status/1959831831668228450?s=20

Edit: My personal view is that LiDAR and other sensors are extremely useful, but I worked on aircraft, not cars.

[willio58]

Based on that list it boils down to 2 things it seems:

- cost (no longer a problem)

- too much code needed and it bloats the data pipelines. Does anyone have any actual evidence of this being the case? Like yes, code would be needed, but why is that innately a bad thing? Bloated data pipelines feels like another hand-wave when I think if you do it right it’s fine. As proven by Waymo.

Really curious if any Tesla engineers feel like this is still the best way forward or if it’s just a matter of having to listen to the big guy musk.

I’ve always felt that relying on vision only would be a detriment because even humans with good vision get into circumstances where they get hurt because of temporary vision hindrances. Think heavy snow, heavy rain, heavy fog, even just when you crest a hill at a certain time of day and the sun flashes you

[AnotherGoodName]

The points linked repeatedly focus on cost and complexity as justification, even explicitly stating musks desire to minimise components in Kaparthy’s list.

They don’t focus on safety or effectiveness except to say that vision should be ‘sufficient’. Which is damning with faint praise imho.

If that link was to try and argue that the removal of sensors makes perfect sense i have to point out that anyone that reads that would likely have their negative viewpoint hardened. It was done to reduce cost (back when the sensors were 1000’s) and out of a ridiculous desire by Musk for minimalism. It’s the same desire that removed the indicator stalk i might add.

[kappi]

Instead of betting on RADAR and LIDAR HW getting better and cost going down, they went with vision only approach. Everybody in this field knows the strengths and weakness of each system. Multi-modal sensor fusion is the way to go for L4 autonomy. There is no other way to reduce the risk. Vision only will never be able to achieve L4 in all the weather conditions. Tesla may try to demonstrate L4 in limited geography and in good weather conditions but it won't scale.

[utopcell]

From the article:

Karpathy’s main points: Extra sensors add cost to the system, and more importantly complexity. They make the software task harder, and increase the cost of all the data pipelines. They add risk and complexity to the supply chain and manufacturing. Elon Musk pushes a philosophy of “the best part is no part” which can be seen throughout the car in things like doing everything through the touchscreen. This is an expression of this philosophy. Vision is necessary to the task (which almost all agree on) and it should also be sufficient as well. If it is sufficient, the cost of extra sensors and tools outweighs their benefit. Sensors change as parts change or become available and unavailable. They must be maintained and software adapted to these changes. They must also be calibrated to make fusion work properly. Having a fleet gathering more data is more important than having more sensors. Having to process LIDAR and radar produces a lot of bloat in the code and data pipelines. He predicts other companies will also drop these sensors in time. Mapping the world and keeping it up to date is much too expensive. You won’t change the world with this limitation, you need to focus on vision which is the most important. The roads are designed to be interpreted with vision.

[galangalalgol]

The reasoning is cynical but sound. If the system uses only the sensing modes people have, it will make the mistakes people do. If a jury thinks "well I could have done that either!" You win. It doesn't matter if your system has fewer accidents if some of the failure modes are different than human ones, because the jury will think "how could it not figure that out?"

[estearum]

I don't think that's the reasoning.

The reasoning was simply that LIDAR was (and incorrectly predicted to always be) significantly more expensive than cameras, and hypothetically that should be fine because, well, humans drive with only two eyes.

Musk miscalculated on 1) cost reduction in LIDAR and 2) how incredible the human brain is compared to computers.

Having similar sensors certainly doesn't guarantee your accidents look the same, so I don't think your logic is even internally sound.

[bluGill]

Until a lawyer points out other cars see that. My car already has various sensors and in manual driving sounds alarms if there is a danger I seem not to have noticed. (There are false alarms - but most of the type I did notice and probably should have left more safety margin even though I wouldn't hit it)

also regulators gather srastics and if cars with something do better they will mandate it.

[small_model]

Very recent issue with Waymo https://dmnews.co.uk/waymo-robotaxi-spotted-unable-to-cross-.... This is 17 years after they bet the farm on LIDAR, with no signs its ever going to be cost effective or that it's better than multiple cameras, with millisecond reaction 360 degrees, that never gets tired, drunk, distracted, and also has other cheaper sensors and NN trained on Billions or real world data.

[JumpCrisscross]

> If a jury thinks "well I could have done that either!" You win

“A federal judge” recently “rejected Tesla's request to overturn a $243 million jury verdict over the 2019 crash of an Autopilot-equipped Model S” [1]. If a human supervising still incurs liability, human-like errors, particularly if Waymo and BYD aren’t making them, is a poor defense.

[1] https://www.reuters.com/world/us-judge-upholds-243-million-v...

[georgeecollins]

It is sound to think that cameras plus an accelerometer, plus data about about the car and environment (that you get from your ears) ought to be able to mimic and improve on human driving. However humans general purpose spatial awareness and ability to integrate all kinds of general information is probably really hard to replicate. A human would realize that an orange fluid spilling across the road might be slippery, guess the way a person might travel from the way their eyes are pointing...

It may just be faster to make lidar cheap. And lidar can do things humans can't.

[bko]

Most accidents happen because people are human, aren't paying attention, are inebriated, not experienced enough drivers, or reckless.

It's not fair to say that vision based models will "make the same mistakes people do" as >99% of the mistakes people make are avoidable if these issues were addressed. And a computer can easily address all those issues

[lesuorac]

IIUC, the cameras in a Tesla have worse vision (resolution) at far distances than a human. So while in the abstract your argument sounds fine; it'll crumble in court when a lawyer points out a similar driver would've needed corrective lens.

[xnx]

This is a new and flawed rationale that I haven't heard before. Tesla cameras are worse (lower resolution, sensitivity, and dynamic range) than human eyes and don't have "ears" (microphones).

[lazide]

Pretty hard to do if your whole selling point is ‘better and safer than human’ however?

[Someone]

> Since lidar has distance information and cameras do not, it was always a ridiculous idea by a certain company to use cameras only

Human eyes do not have distance information, either, but derive it well enough from spatial (by ‘comparing’ inputs from 2 eyes) or temporal parallax (by ‘comparing’ inputs from one eye at different points in time) to drive cars.

One can also argue that detecting absolute distance isn’t necessary to drive a car. Time to-contact may be more useful. Even only detecting “change in bearing” can be sufficient to avoid collision (https://eoceanic.com/sailing/tips/27/179/how_to_tell_if_you_...)

Having said that, LiDAR works better than vision in mild fog, and if it’s possible to add a decent absolute distance sensor for little extra cost, why wouldn’t you?

[tsimionescu]

Human/animal vision uses way more than parallax to judge distances and bearings - it uses a world model that evolved over millions of years to model the environment. That's why we can get excellent 3D images from a 2D screen, and also why our depth perception can be easily tricked with objects of unexpected size. Put a human or animal in an abstract environment with no shadows and no familiar objects, and you'll see that depth perception based solely on parallax is actually very bad.

[larsnystrom]

Human eyes are much better than cameras at dealing with dynamic range. They’re also attached to a super-computer which has been continuously trained for many years to determine distances and classify objects.

[dymk]

> Human eyes do not have distance information

Single human eyes do resolve depth perception. Not as good as binocular vision, but you don't loose all depth perception of you lose an eye.

https://en.wikipedia.org/wiki/Monocular_vision

[dumbfounder]

I don’t like the comparison between humans and humans. Humans don’t travel around at 100mph in packs of other humans. Why not use every sensor type at our disposal if it gives us more info to make decisions? Yes I understand it’s more complicated, but we figure stuff out.

[idiotsecant]

Let me know when you have a camera package with human eye equivalency.

[nlitened]

As I understand, lidars don't work well in rain/snow/fog. So in the real world, where you have limited resources (research and production investment, people talent, AI training time and dataset breadth, power consumption) that you could redistribute between two systems (vision and lidar), but one of the systems would contradict the other in dangerous driving conditions — it's smarter to just max out vision and ignore lidar altogether.

[RobotToaster]

> lidars don't work well in rain/snow/fog.

Neither do cameras, or eyeballs.

[Zigurd]

No, it isn't "smarter." Camera-only driving is the product of a stubborn dogmatic boss who can't admit a fundamental error. "Just make it work" is a terrible approach to engineering.

[zozbot234]

Why does this matter? You have to slow down in rain/snow/fog anyway, so only having cameras available doesn't hurt you all that much. But then in clear weather lidar can only help.

[lazide]

Evidence clearly shows otherwise.

Also, military sensor use shows the best answer is to have as many different types of sensors as possible and then do sensor fusion. So machine vision, lidar, radar, etc.

That way you pick up things that are missed by one or more sensor types, catches problems and errors from any of them, and end up with the most accurate ‘view’ of the world - even better than a normal human would.

It’s what Waymo is doing, and they also unsurprisingly, have the best self driving right now.

[brk]

Nothing works perfectly in all conditions and scenarios. Sensor fusion has been the most logical approach now, and into the foreseeable future.

Computer vision does not work exactly like human vision, closely equating the two has tended to work out poorly in extreme circumstances.

High performance fully automated driving that relies solely on vision is a losing bet.

[zemvpferreira]

Limited resources? Billions per year are being thrown at the base technology. We have the capital deployed to exhaust every path ten times over.

[philistine]

Why does that strategy absolutely require the lidar to be absent from the car? When was less technology the solution to a software problem?

[heisenbit]

The Swiss cheese model would like to disagree.

[Yossarrian22]

When you have sensor ambiguity sounds like the perfect time to fail safely and slow to a halt unless the human takes over.

[idiotsecant]

This is silly. Cameras are cheap. Have both. Sensors that sense differently in different conditions is not an exotic new problem. The kalman filter has existed for about a billion years and machine learning filters do an even better job.

[theappsecguy]

Do cameras work well in those conditions? Nope. Also cameras don't work well with certain answer of glare, so as a consumer I'd rather have something over-engineered for my safety to cover all edge cases...

[foooorsyth]

I wouldn’t take too much issue with the “cameras are enough” claim if cameras actually performed like eyes. Human eyes have high dynamic range and continuous autofocus performance that no camera can match. They also have lids with eyelashes that can dynamically block light and assist with aperture adjustment.

The appeal to human biology and argument against fusion between disparate sensors kinda falls flat when you’re building a world model by fusing feeds from cameras all around the car. Humans don’t have 8 eyes in a 360 array around their head. What they do have is two eyes (super cameras) on ~180 degree swiveling and ~180 degree tilting gimbal. With mics attached that help sense other vehicles via road noise. And equilibrioception, vibration detection, and more all in the same system, all fused. If someone were actually building this system to drive the car, the argument based on “how did you drive here today?” gets a lot stronger. One time I had some water blocking my ear and I drove myself to the hospital to get it fixed. That was a shockingly scary drive — your hearing is doing a lot of sensing while driving that you don’t value until it’s gone.

[spyder]

Yea, even in the case they could match human level stereo depth perception with AI, why would they say "no" to superhuman lidar capabilities. Cost could be a somewhat acceptable answer if there wouldn't be problems with the camera only approach but there are still examples of silly failures of it. And if I remember correctly they also removed their other superhuman radar in their newer models, the one which in certain conditions was capable of sensing multiple cars ahead by bouncing the signal below other cars.

[peterfirefly]

Because they don't have superhuman LIDAR. They never did. Nobody ever did. LIDAR input is not completely reliable so what do you do then?

[radial_symmetry]

I'm not an expert on ML vision, but I do have a Tesla and it seems to be able to tell how far away things are just fine. I'm not sure what would be wrong with the vision system that lidar needs to fix.

[tw04]

The phantom braking issue with auto pilot tells me it can’t. A shadow from a tree doesn’t trigger your brakes locking up at 70+ mph when there’s a lidar sensor to tell you it’s not a physical object.

“Just buy FSD” isn’t a reasonable answer to a problem literally no other automaker suffers from.

[thinkcontext]

How do you explain the reports of Robotaxis running into fixed objects? If what you are saying is true that shouldn't be able to happen.

https://electrek.co/2026/02/17/tesla-robotaxi-adds-5-more-cr...

[DustinBrett]

Luckily everyone else in the comments is an expert. And also doesn't recognize that Tesla's already drive themselves and did not need Lidar. They also mischaracterize the reasoning.

[xpe]

> I'm not sure what would be wrong with the vision system that lidar needs to fix.

This conversational disconnect is as old as the hills:

1. Person 1 asks "what's wrong" (if it ain't broke don't fix it)

2. Person 2 wants to make something better

My meta-goal here on HN (and many places where people converse) is for people to step back and recognize the conversational context and not fall into the predictable patterns that prevent us from making sense of the world as best as we can.

[Mawr]

> I'm not an expert on ML vision, but I do have a Tesla

Well, you did get a chuckle out of me, so that's something!

[Phil_Latio]

Yeah it's BS. Tesla uses lidar where it makes sense: They have a small lidar fleet to collect ground truth depth data for better vision estimation. This part is long solved.

[wasmainiac]

Just say Tesla, why censor yourself.

[zelphirkalt]

I have a suspicion here on HN. When criticizing big tech, especially Google and FB, at a certain time of the day a specific cohort comes online and downvotes. Suspiciously, that is a time when one could conclude, that now people in the US start working or come online. Either fanboys, employees or an organized group of users trying to silence big tech criticism.

I have no proof of course and it might be coincidence, or just difference of mindset between US citizens and Europe citizens. It happened a few times already and to me looks sus.

But if they actually read and not just ctrl+f <company name>, then of course not writing the company name, but hinting at it in an obvious way is no more helpful either.

[uyzstvqs]

It's not that simple. Cameras don't report 3D depth, but these AI models can and do pick up on pictorial depth cues. LiDAR is incredibly valuable for collecting training and validation data, but may also make only an insignificant difference in production inference.

[pwarner]

Stereo cameras? My 2015 Subaru has them to detect obstacles and it works great.

[mgoetzke]

considering cameras can create reliable enough distance measurements AND also handle all the color reception needed for legally driving roads it was always a ridiculous idea by a certain set of people that lidar is necessary.

[tsimionescu]

No, cameras cannot create reliable distance measurements in real-world conditions. Parallax is not a great way to measure distance for fast, unpredictably moving objects (such as cars on the road). And dirt or misalignment can significantly reduce accuracy compared to lab conditions.

Note that humans do not rely strictly on our eyes as cameras to measure distances. There is a huge amount of inference about the world based on our internal world models that goes into vision. For example, if you put is in a false-perspective or otherwise highly artifical environment, our visual acuity goes down significantly; conversely, people with a single eye (so no parallax-based measurement ability) still have quite decent depth perception compared to what you'd naively expect. Not to mention, our eyes are kept very clean, and maintain their alignment to a very high degree of precision.

[numpad0]

Stereo cameras are useless against repeating patterns. They easily match neighboring copies. And there are lots of repeating or repeating-like patterns that computers aren't smart enough to handle.

You can solve this by adding an emitter next to the camera that does something useful, be it just beaconing lights or noise patterns or phase synced laser pulses. And those "active cameras" are what everyone call LIDARs.

[ImPostingOnHN]

'cameras can see in color, therefore lidar is unnecessary for self driving' is unconvincing

[throwa356262]

There are tons of evidence showing that cameras are alone are not safe enough and even Tesla has realized that removing lidar to save cost was a mistake.

[xpe]

> ridiculous idea by a certain set of people that lidar is necessary.

"Necessary"? Seems like a straw man, don't you think? I strive to argue against the strongest reasonable claim someone is making.

Lots of reasonable people suggest LIDAR is helpful to fill in gaps when vision is compromised, degraded, or less capable.

People running businesses, of course, will make economic trade-offs. That's fine. But don't confuse, say, Elon's economic tradeoff with the full explanation of reality which must include an awareness that different sensors have different strengths in different contexts.

So, when one thinks about what sensor mix is best for a given application, one would be wise to ask (and answer) such questions as:

- What is the quality bar?

- What sensors are available?

- Wow well do various combinations of sensors work across the range of conditions that matter for the quality bar?

- WRT "quality bar": who gets to decide "what matters"? The company making the cars? The people that drive them? regulators that care about public safety. The answer: it is a complex combination.

It is time to dismiss any claim (or implication) that "technology good, regulation bad". That might be the dumbest excuse for a philosophy I've ever heard. It is the modern-day analogue of "Brawndo's got what plants crave." Smart people won't make this argument outright, but unfortunately, their claims sometimes reduce to this level of absurdity. Neither innovation nor regulation are inherently good nor bad. There are deeper principles in play.

Yes, some individuals would use their self-proclaimed freedom to e.g. drive without seatbelts at 100 mph at night with headlights off. An extreme example, but it is the logical extension of pure individualism run amok. Regulators and anyone who cares about public safety will draw a line somewhere and say "No. Individual stupidity has a limit." Even those same people would eventually come to their senses after they kill someone, but by then it is too late.

[nova22033]

It's not complicated. LIDAR hardware was in short supply during COVID. Elon obviously couldn't slow down production and sink the inflated stock price.

[Phil_Latio]

April 2019: https://www.youtube.com/live/Ucp0TTmvqOE?t=9220s

There are probably even earlier statements from him against lidar...

[jollyllama]

WTF was their calculus on the break-even liability point? The "if we do this, we save X amount of money, but stand to lose Y in lawsuits for cases where the usage of LIDAR could have otherwise prevented it."

[rustystump]

There are more practical difficulties than just cost. If you have lidar it must be calibrated relative to all other sensors. Bumps in the road, weather, thermals, this all causes drift which is non trivial. Waymos are constantly brought in and recalibrated. The advantage of camera only is you have less moving parts which is not insignificant.

But cost isnt the issue as much.

[SecretDreams]

I'll preface by saying lidar should be used with autonomous vehicles.

Individual cameras don't have distance information, but you can easily calibrate a system of cameras to give you distance information. Your eyes do this already, albeit not quantitatively. The quantitative part comes from math our brains aren't setup to do in real time.

[dzhiurgis]

Certain company has 300k subscribers that rely on that ridiculous service.

My father lost vision in 1 eye and 50% in other one something like 20 years ago. He struggles in parking but otherwise doing ok without lidar. Turns out motion vision is more accurate after 10-20 meters than stereoscopic vision.

[leptons]

One camera can't really produce depth/distance information, but two cameras sure can. The eyes in your head don't capture distance information individually, but with two eyes you can infer distance.

[moogly]

You're forgetting the nervous system and the brain connected to those eyes (and vestibular system).

[DonsDiscountGas]

Humans don't have explicit distance sensors either. When LIDAR sensors were $20k+ I think it made a lot of sense to avoid them.

[FrustratedMonky]

It was cost wasn't it?

If this lowers Lidar costs, and Tesla has spent all this time refining the camara technology. Now have both.

Use both.

[pbreit]

All of driving is designed for visual.

[turtlesdown11]

TIL roads don't have rumble strips

[DoesntMatter22]

It was a great decision to drop LiDAR. The cars are running excellently without it

[MetaWhirledPeas]

I find it comical that people continue to go back to this rage well against "a certain company" for their vision-only approach when the truth is they have the best automatic driving system an individual can buy, rivaling Waymo and beating the Chinese brands.

Why are the commenters not pissed at the dozens of other car companies who have done absolutely nothing in this space? Answer: because it's not nearly as fun to be pissed at Kia or Mercedes or whoever. Clearly they are just enjoying the shared anger, regardless of whether it is justified.

[array_key_first]

1. Tesla is not competitive with Waymo, they're not even in the same class. Waymo is 10 years ahead at least. I understand you can't buy a Waymo, but still.

2. Other car companies are properly valued, Tesla is overinflated.

3. Other cars, even basic Hondas, have the same level of self driving as Teslas.

4. Other car companies don't lie to their customers about their capabilities or what they're buying.

[TulliusCicero]

Because other car companies don't have CEOs who've been super confident about predicting actual full self driving either "this year" or "next year" for the past decade. If Ford had been swearing up and down they'd have full self driving cracked any day now for ten years, and been charging people for the hardware along the way, everyone would be pissed at them too.

Surely you already know this, so why pretend otherwise?

[epolanski]

You're way off if you think that Waymo and FSD are anywhere close.

[superxpro12]

There is certainly some truth that "some company" overpromised and underdelivered. They advertise "full self driving" but then hide in the fine-print that "oh jk, not really, but its still full self driving if anyone asks ;) ;) ;)"

I think the frustration stems from the obvious falsehoods in the advertising, and the doubling-down on the tech, despite the well-documented weaknesses of the implementation.

[bko]

Why make things more complicated than they need to be? Humans don't have lidar and we are the only intelligence that can reliably drive. Lidar just seems like feature engineering, which has proven to be a dead end in most other AI applications (bitter lesson).

https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson...

[thunky]

> Why make things more complicated than they need to be? Humans don't have lidar and we are the only intelligence that can reliably drive.

Because we want self driving cars to be safer than human driven cars.

If humans had built in lidar we would use it when driving.

[afavour]

Self driving cars are not equipped with human brains so this doesn’t really make sense.

“We should achieve self driving cars via replicating the human brain” strikes me as an incredibly inefficient and difficult way to solve the problem.

[Analemma_]

This knee-jerk reply is old and tired, and the counterarguments are well-trod at this point. Even if cameras-only can build a car that’s as good as humans, why should we settle for “as good as“ humans, who cause 40,000 fatalities a year in the US? If we can do better than humans with more advanced sensors, we are practically morally obligated to do that.

[a_better_world]

> we are the only intelligence that can reliably drive.

Science would like to point out that rats also can learn to drive

https://theconversation.com/im-a-neuroscientist-who-taught-r...

[jeltz]

Humans can drive with eyes only, but we are better drivers when we can also use other senses like hearing. If humans has lidar we would use it when driving.

[Ajedi32]

The bitter lesson I think is a great way of explaining the logic behind Tesla's strategy. People aren't getting it.

Whether or not it'll actually work remains to be seen, but it's a perfectly reasonable strategy. One counterargument would be that the bitter lesson can be applied to LIDAR too; you don't have to use that data for feature engineering just because it seems well suited for it.

[elicash]

Don't cars already use a ton of sensors that don't reproduce human senses and ways of doing things?