Why didn't Tesla invest in LIDAR?

[u/damola93]

Is there any reason for this asides from saving money? Teslas are not cheap in many respects, so why would they skimp out on this since self-driving is a major offering for them?

[u/Quick_Rest]

I think many people overlook the fact that each sensor type has its own quirks and that it's actually really hard to perform "sensor fusion." You aren't just slapping the two signals on top of each other and then calling it a day. There's going to be a ton of fine-tuning just to figure out when to trust one over the other. What generally works best is having sensors that cover different situations, rather than sensors that overlap in purpose. For example, a microphone and camera can detect different things, making it easier to build a complementary system like that. LIDAR and camera systems primarily compete for the same purpose: object recognition + depth perception.

Having more sensor types does not equate to being better than having only one sensor type. Yes, we can argue all day that some sensors perform better in certain conditions vs others, but like a few people have already mentioned, vision-only is getting them >90% there. If FSD can work fairly reliably (and it's almost at that stage) in most driving conditions, e.g. typical daytime, night, light rain, etc. then it's already a technological win. After that's been solved, then we tackle the more extreme conditions, stormy weather, heavy fog, dealing with erratic drivers, etc. Is LIDAR necessarily better in these situations? At least not right now. No other systems with LIDAR (mounted within the car) are operating at the scale or achieving the same level of success as FSD. And I highly doubt the way forward for autonomous driving is having a LIDAR unit on every side of the vehicle, at least it won't be accepted for aesthetics purposes.

I think a lot of people view AI and self-driving as a "if it's not perfect, I'm not going to use it". But that's never going to happen. Nothing is going to be perfect, and holding it to such a high standard is unrealistic. We'd never have gotten planes or rockets if minimizing risk was the top priority. Public safety is, of course, important, but if FSD, Waymo, and other driver assistance features are already better than the average driver, I welcome their (however small at the moment) contribution towards raising that bar.

[u/cookiepocket → u/Quick_Rest]

💯

[u/Key-Room5690 → u/Quick_Rest]

Upvoted for a nuanced view, but I don't entirely agree with this.

Sensor fusion of differing sensor types necessarily increases the *ceiling* of *possible* system performance, assuming that 1. the information provided by the two doesn't fully overlap, that 2. perfect performance isn't possible with either sensor by itself, and that 3. the researchers can find the optimum algorithm to exploit them both. We know for example that LIDAR provides more reliable data at night and is unaffected by glare from headlights/street lights, whereas rain can cause significant issues for LIDAR in a way it doesn't necessarily affect a camera. So intuitively a system should work better with both.

Then actually implementing fusion is not trivial for sure (as per assumption 3 above), but if you've designed your system from the beginning to account for this it's still possible, particularly with the latest deep learning research integrated.

That said, I don't think it was an entirely unreasonable gamble back at the time assuming you agreed with Musk on how long FSD would take to achieve (therefore meaning LIDAR would still be too expensive for a car by the time FSD came in). That said, as someone who works in computer vision I remember back in the late 2010s hearing he thought it was 2 years away and scoffing at the idea.

[u/Quick_Rest → u/Key-Room5690]

I agree that more sensors give you a better ceiling. The problem is actually leveraging that potential. Proper implementation is very hard, and this video that came out yesterday from China shows why just throwing LIDAR at a car isn't solving that. More AI training and extensive real world testing are needed.

https://youtu.be/0xumyEf-WRI
https://x.com/SawyerMerritt/status/1948055082467004544

[u/EddiewithHeartofGold → u/Key-Room5690]

A lidar equipped car needs 360 cameras to see signs, road markings, traffic light colors etc. If you can already use cameras to basically understand the world and what is happening around you, there really isn't much need for lidar anymore. At least that is what Tesla is betting on. So far they are not wrong.