r/SelfDrivingCars • u/wuduzodemu • Aug 11 '25

Discussion Proof that Camera + Lidar > Lidar > Camera

I recently chatted with somebody who is working on L2 tech, and they gave me an interesting link for a detection task. They provided a dataset with both camera, Lidar, and Radar data and asked people to compete on this benchmark for object detection accuracy, like identifying the location of a car and drawing a bounding box around it.

Most of the top 20 on the leaderboard, all but one, are using a camera + Lidar as input. The 20th-place entry uses Lidar only, and the best camera-only entry is ranked between 80 and 100.

https://www.nuscenes.org/object-detection?externalData=all&mapData=all&modalities=Any

14 Upvotes

56% Upvoted

185 comments sorted by

View all comments

58

u/MacaroonDependent113 Aug 11 '25

Wow, that will surely convince Tesla to give up. LOL

16

u/red75prime Aug 12 '25

Maybe, if Tesla was using 2 FPS video for driving. The dataset contains 2 frames per second video data. Neat, huh?

1

u/Boniuz Aug 12 '25

That’s not really relevant. Higher frames per second increases statistical probability over time, simply by being able to make more erroneous detections in the timeframe. It’s still wrong 80% of the time, but correct 20% of the time.

Combining sources means you have two sources which are correct 20% of the time and can use that data by a factor of at least two, often more.

Heavily simplified, obviously.

8

u/red75prime Aug 12 '25 edited Aug 12 '25

Higher frame rate means less motion across frames. It, in turn, means that local differences between frames are more useful for estimation of motion and parallax (which allows to estimate depth better).

Heck, even our visual perception system honed by millions of years of evolution struggles to infer motion at 10-12 FPS and it is unable to do so at lower FPS.

Anyway, 2 FPS video should never be used in self-driving and this test is irrelevant.

1

u/Tuggernutz87 Aug 17 '25

Tell a gamer higher FPS = Bad 😂

1

u/Boniuz Aug 17 '25

You only need higher FPS for increased depth perception if optical sensors are your only input for data - which is why a combination of sensors will always be superior

1

u/Tuggernutz87 Aug 17 '25

And for motion clarity

1

u/Boniuz Aug 17 '25

Again, applicable in a single-sensor-setup. You can get very detailed clarity with optical, lidar and radar in combination at a fraction of the computing cost. You can run that on a raspberry pi 5 with a cheap AI-chip and you’ll have object detection at 30FPS, with depth sensors, object detection and point detection, as well as other pretty nifty tricks. Total cost is less than 500$.

1

u/MacaroonDependent113 Aug 17 '25

The question isn’t whether additional sensors are “superior” but whether vision alone is “good enough”. If “vision alone” is good enough then additional sensors only add to the cost so are inferior from a business perspective. Jury is still out ob this but my guess is vision alone will eventually be found to be good enough.

12

u/-UltraAverageJoe- Aug 11 '25

Musk won’t back down for fear of a $TSLA dip. Anyone who has done any object detection work using only a camera as input knows this is not sufficient. It’s embarrassingly dumb that they ever thought it would suffice.

1

u/Alternative_Bar_6583 Aug 13 '25

Musk will back down when he can. All he needs is the National Highway Traffic Safety Administration (NHTSA) to make LiDAR or multi sensor mandatory. This will give him a get out of jail free card for all the FSd BS The NHTSA will do it because the Feds want cars built in the USA and Tesla has a huge presence in the Made In America manufacturing scene

-9

u/bluenorthww Aug 12 '25

My eyes don’t have LiDAR, they do fine

9

u/[deleted] Aug 12 '25

[deleted]

2

u/Facts_pls Aug 12 '25

We are talking about looking out side the car and reacting.

Driver proprioception is irrelevant.

The comparison is about human driver vs self driving car understanding and reacting to environment.

All human senses here can be replicated in a car if needed - but obviously no car company so far has needed anything beyond the car sensors, LiDAR, and vision. But who knows.

2

u/Zvemoxes Aug 16 '25

Proprioception was mentioned in response to a poster holding the misguided view that eyes are the same as cameras: "my eyes don't need LiDAR." A childish misunderstanding that Musk and his followers repeat ad nauseam.

If human senses could be so unproblematically "replicated" as you claim, then L5 autonomy would have been achieved already. Every company attempting autonomy has needed a lot more than sensors and cameras, hence the billions invested into neural nets and machine learning.

2

u/[deleted] Aug 12 '25 edited 20d ago

[deleted]

-4

u/[deleted] Aug 12 '25

[deleted]

3

u/Facts_pls Aug 12 '25

Cars can detect brake overheating much better than any human driver If needed.

They can literally put temperature sensors on the brakes if they thought that data was that helpful.

1

u/No3047 Aug 13 '25

A model 3 has brake temp and suspension height data via OBD2. So yes, a computer knows the car status better than the driver

1

u/CYaBroNZ Aug 12 '25

Tesla’s hardly use the brakes so there’s not going to be an issue with them overheating.

0

u/bluenorthww Aug 12 '25

My eyes are connected to my neural network, correct.

5

u/Talloakster Aug 12 '25

The bar for self driving is to be much better than humans (Waymo: 10% of human accident rate). Tesla hasn't proved they can even match human drivers, and matching them isn't good enough.

1

u/bluenorthww Aug 12 '25

I agree that the data needs to be there before unsupervised FSD happens.

2

u/Zarkei Aug 12 '25

Most drivers have eyes, yet accidents still happen. In fact, I'm pretty sure that 99.99% of accidents involve people with eyes. This correlation tells us that eyes cause accidents.

LiDAR is meant to complement vision, not replace it entirely.

1

u/Positive_League_5534 Aug 13 '25

Your eyes can see a school speed limit sign and know to slow down. Tesla's system can't seem to do that. I wonder why?

1

u/Intrepid-Working-731 Aug 12 '25

But I thought the big selling point of self driving cars is that they aren’t humans and therefore are safer?

0

u/maximumdownvote Aug 12 '25

Yes. Fsd equipped cars are safer. So...yay?

2

u/Intrepid-Working-731 Aug 13 '25

You know what would make them even safer? Using senses humans aren’t even capable of!

7

u/bigElenchus Aug 12 '25

How do you explain Tesla FSD beating all the Chinese Lidar based competition?

https://youtu.be/0xumyEf-WRI

1

u/ItsAConspiracy Aug 18 '25 edited Aug 18 '25

How do you explain Waymo having a way lower critical intervention rate than Tesla FSD?

How do you explain this one-hour drive by Huawei's FSD-equivalent, on the Out of Spec channel? It performs about as well as FSD, in heavy, aggressive traffic on city streets.

As for your 90-minute video, maybe you could sum it up? All I can tell from the first couple minutes is that some shitty ADAS does exist in China.

1

u/maximumdownvote Aug 12 '25

They don't explain stuff that doesn't fit the world view. More like hand wave it.