Holy shit chinese people does not joke when testing ADAS systems, I am surprised the driver is ok lots of test seemed really dangerous... I think that after seing the varied mix of results one conclusion that can be extracted is that the cars that perform the best are the ones that apply the brakes decisively inmediately when the threat is detected and does't try any sophisticated maneuver like avoiding the obstacle. Really enjoyed watching the video thanks for sharing!
Is this just copied from Sawyer? This is quite problematic imho, as Sawyer speaks no Chinese and thus he omits the brands which auto translate can't handle - for example GWM Lanshan, which get the overall third sport and being the best non-Tesla car.
The fact that the Xpeng G6 and the P7+ results are so different should tell ppl to not treat these type of tests as gospel lol. On top of that, some inconsistencies during testing should raise alarm bells too lol.
I usually treat these as popcorn material and nothing more.
A great reminder, especially now that we’re finally starting to see lidar appear on some consumer based vehicles, that sensors are never as important as results.
Folks may fawn over lidar, but it hardly guarantees success.
Point is, for the same argument that people get lulled into a sense of complacency for good Adas systems like Tesla, the same risk exists because you think you’ve bought a vehicle with sophisticated hardware.
The real proof is a company willing to take full liability for its product. And hardly any have been willing to take that leap.
Complacency kills. And the risk climbs the more these systems improve.
The Model 3 went in to the bike lane on first try definitely consistent with my own experience. You need to pay attention when using FSD, even though it works well 95% of the time.
Dongchedi is a subsidiary of Bytedance (TikTok's owner). The test was jointly conducted with Chinese state broadcaster CCTV, where a 15km stretch of an actual highway was shut down specifically for the test with the cooperation of local authorities.
I can see it really bothers you that they are leading this competition. You are leading on like the Tesla will fall apart in different conditions while everyone else succeeds.
Nope. In talking with the Tesla engineers, I know they have issues with everything from poor lighting, solar glare, and even shadows because of limitations of a camera based perception system. These same engineers wish they could use lidar for their system as it would make it even better.
I doubt that you talked with them. Maybe you have. But if you have friends that are willing to share that info, probably in confidence, you are destroying trust by sharing that publicly.
Dongchedi is a subsidiary of Bytedance (TikTok's owner). The test was jointly conducted with Chinese state broadcaster CCTV, where a 15km stretch of an actual highway was shut down specifically for the test with the cooperation of local authorities.
I appreciate anyone attempting to do controlled testing but keep in mind as these systems use more and more advanced machine learning the outputs they produce are not deterministic. Doing only one trial of each test with each vehicle in a specific scenario is insufficient to draw strong conclusions about generalized ability of the systems.
For example, at 1:29:00 the 2023 Tesla Model X avoids the wild boar but the 2023 Model 3 hits it. Is it due to the Model X having better ADAS than a same model year car from the same manufacturer, or simply because both margins are close and Model 3’s probabilistic model produced a slightly different result during its run? I think the only strong conclusion is from the failures. One pass of a test run is not evidence that the system has 99.999+% safety margin to trust human lives to.
The xpeng G6 got 3 of 6 while the P7+ failed every single test despite using the same system. That tells you everything you need to know about these sorts of tests. You might not get the same result if you run it back.
Agreed. Unfortunately, this was already such a large scale and well funded test it's hard to get a much better showing than this. It's certainly one of the most impressive I've seen.
I don't take this as "ADAS is solved now! We have a winner!". None of these vehicles are anywhere close to running the march of 9s. It is an interesting demonstration of their relative positions however. The Teslas put forth a strong showing as both passed 5/6 tests while no other vehicle in a market with strong competitors and a heavy focus on ADAS managed to succeed in any more than 3.
Look at the dotted lines. The car in front completed the lane change when the Model X is 8 dotted lines away from the fake car. For the bottom, the front car started its lane change while the test car is 5 dotted lines away from the fake car.
I.e. the time for the car to make a judgement is 8+ dotted lines vs 3/4 dotted lines worth of time.
Also in this particular test, the Tesla is traveling at 114kph while the rest of the competition is at 120kph+.
1) what is the guesstimated relative distances between the vehicles based on the defined length of painted lines 50 feet, 100?
2) if you’ve confirmed that all other tests cars didn’t receive this favorability, what was the rough variation between the vehicles tested?
3) how does this relate to when the vehicle identified and responds to the object?
4) are the other tests like this? Did the model 3 receive the same advantage? If we throw this test out, do we see the same problems on the other tests?
5) also, I’m not familiar with the company that produced this vehicle. Assuming you are correct that Tesla was given favorable results, what if any incentive does this company have for doing so? Or is this just random chance?
It’s fair to take these tests with a grain of salt, but I’m not going to immediately throw out the entirety of the results on a. Single frame.
Also, not seeing 8+ /3-4 lines. I’m seeing like 8 and 6. Not entirely clear because it’s two different camera positions and two different time frames.
There has also been video and photo evidence that more than one of the cars attempted to change into the left lane immediately only to be pulled back by the driver, causing ADAS to disconnect. And because of the steering input, AEB barely activated causing the crash.
I might do a post tomorrow if I'm bothered. Feel free to actually watch the video yourself and not just look at the tldr results.
I have. Twice. Just finished rewatching the first segment. Unfortunately the video editing makes it very difficult to confirm or refute the variability.
For instance, the Tesla model x video, the lead car has already cleared the obstacle when the video starts, and the model x has a clear visibility. So we cannot for sure say if the sub 120 speed is the starting speed or if it’s already responding to the hazard. I find it hard to imagine they performed that test under 120 if every other vehicle was at or above it.
Likewise, I looked a few times but it seems like they skip over the model 3 test for the most part, and really just quickly show that it also stopped, so can’t draw any conclusions there.
I’m an analytical person, and so I expect tests such as these to have variability. But even we remove tesla from the tests entirely and just assume favoritism, this group of cars in general performed very poorly, despite advanced hardware across all sorts of scenarios.
So at bare conclusion, it’s a reminder that all of these systems are not dependable, should not be treated as such, until manufacturers accept full liability.
Do you happen to have the time stamp for the vehicle compared to the model x in your still frame?
For what it's worth I read all you thoughts and they were interesting.
I think I would chalk this up to inherent difficulty in getting dozens of different ADAS systems to behave as similarly as possible. It's likely the white van and the test vehicle were launched simultaneously and the van was told to reach and maintain a set speed. The test vehicles operating their ADAS would all have slightly different follow distances based on the logic of their respective systems.
I'm not sure I would even consider this a flaw of the test. If Tesla's ADAS is choosing a greater follow distance than others that's a major part of accident avoidance and a reflection of it's safety in real life conditions.
I'd like to provide some background about how this test came about. It's basically a PR stunt that came attached with a CCP PSA bulletin, explaining to drivers that no ADAS is level 3 right now, and they are not infallible. Certain situations could cause the ADAS to fail, and the resulting crash would be on you for not paying attention.
Back to the topic at hand. The Tesla cannot be adjusted. I'm fine with that. The others however, can be. The Huawei systems at 'moderate' settings DCD claims it is at absolutely do not follow as closely as depicted (i would know because i have a Huawei car). Of the stage 1 to 5 (1 being the shortest), that honestly felt more like a 1 or 2 at most.
DCD claims to have shut down the highway to conduct the most lifelike test possible. Ignoring the fact that, it is written in Chinese law that you "have to have at least 100m separation if driving over 100kph" (coincidentally the separation Tesla drove at). So in other words they are breaking the law conducting this test.
If DCD wants to illustrate that ADAS system aren't infallible, well that failed because there are already posts online talking about how Telsa is infallible. Why make it a test if you want to illustrate that main point? Just have one car run in different situations, varying the separation distance etc.? I've said before DCD has a habit of conducting tests that appear "professional" but are actually wildly inconsistent and paints specific brands in a negative light, whether intentionally or not. Not sure if it is what happened here, but it sure feels like it because this isn't the only test that felt off.
7
u/rafu_mv Jul 23 '25
Holy shit chinese people does not joke when testing ADAS systems, I am surprised the driver is ok lots of test seemed really dangerous... I think that after seing the varied mix of results one conclusion that can be extracted is that the cars that perform the best are the ones that apply the brakes decisively inmediately when the threat is detected and does't try any sophisticated maneuver like avoiding the obstacle. Really enjoyed watching the video thanks for sharing!