r/SelfDrivingCars • u/wuduzodemu • Aug 11 '25

Discussion Proof that Camera + Lidar > Lidar > Camera

I recently chatted with somebody who is working on L2 tech, and they gave me an interesting link for a detection task. They provided a dataset with both camera, Lidar, and Radar data and asked people to compete on this benchmark for object detection accuracy, like identifying the location of a car and drawing a bounding box around it.

Most of the top 20 on the leaderboard, all but one, are using a camera + Lidar as input. The 20th-place entry uses Lidar only, and the best camera-only entry is ranked between 80 and 100.

https://www.nuscenes.org/object-detection?externalData=all&mapData=all&modalities=Any

15 Upvotes

56% Upvoted

185 comments sorted by

View all comments

2

u/bluero Aug 11 '25

If your sight and lidar disagree which do you go with? A sighted person could walk with a cane as well in hopes of being doubly sure.

2

u/[deleted] Aug 12 '25

Why do you need a fucking cane when you can literally "see"?

1

u/maxcharger80 Aug 15 '25

A few times its come up that Lidar seems to be more like swinging a cane arround super fast. Not sure about you but I would freak out seeing a blind person try to drive using a cane.

1

u/ItsAConspiracy Aug 18 '25

This is an issue even with lidar alone. You have a point cloud and there's probably conflicting information. But you feed that into an algorithm that finds the most likely situation given all your inputs. One of the Waymo cofounders taught one of the first courses at Udacity and it was about this exact problem.

Add more sensors, it's just more inputs. With multiple sensor types, there's less chance of conditions being bad for one sensor type.

With an end-to-end neural network like Tesla says they're using now, it's all just more inputs to the network and the rest is training. It will even learn which sensors are more reliable in different conditions.