Tesla Model S Autopilot testing with IIHS [CREDIT: IIHS]
Self-driving systems aren't ready for prime time, says a new report by IIHS.
The Institute tested five cars: two Teslas, a BMW, a Mercedes-Benz, and a Volvo, all with systems that earned top marks in its tests of advanced driver safety aids.
In new tests on its track and on public roads, all of the systems made mistakes that could be fatal without driver intervention.
Many of those mistakes included drifting over lane lines even when active lane control (or Autopilot, in Tesla's terminology) was activated.
"Self-driving" seems to mean different things to drivers and to safety experts. SAE International and the NHTSA have laid out guidelines for five different levels of self driving cars. The systems that IIHS tested are considered Level 2 of the five levels.
Joe Young, an IIHS spokesman who works at the facility's test center and has driven these cars on nearby roads says, "I had a hard time with these systems. I did not feel at all comfortable with how these systems were driving going over crests and around curves, especially with oncoming traffic on a right curve," he told Green Car Reports in an interview.
Tesla Model 3 dashboard in Autopilot testing with IIHS [CREDIT: IIHS]
In addition to automatic emergency braking and adaptive cruise-control tests on its test track, the IIHS tested each cars' active lane control systems six times going around three curves and over the tops of three hills—the types of situations where the systems have shown the most difficulty tracking lane markings—on public roads.
It tested two Teslas—a 2018 Model 3 running Autopilot version 8.1 software, and a 2016 Model S running Autopilot version 7.1 software—a 2017 BMW 5-Series, a 2017 Mercedes-Benz E-Class, and a 2018 Volvo S90, each with its automaker's Level 2 self-driving feature.
The tests were part of an ongoing effort at the IIHS to develop ratings for the systems that would allow car buyers to see which systems are better than others.
"We're not ready to say yet which company has the safest implementation of Level 2 driver assistance" says IIHS chief research officer David Zuby. "None of these vehicles is capable of driving safely on its own."
It is possible to draw a few anecdotal conclusions from the tests, though.
Active lane control not always active
The Tesla Model 3 with Autopilot version 8.1 stood out as the only car that consistently kept the car within its lane through all 18 curves, though it touched the center line once when going over a hill. The older Tesla Model S, with Autopilot version 7.1, crossed or touched a line 13 times going over hills and once on a curve. The Teslas were the only cars that never disengaged the self-driving systems in the six laps.
IIHS test results of lane-keeping assist systems [CREDIT: IIHS]
The BMW 5-Series was by far the most likely to disengage the system. When it couldn't track lane markings well enough going over a hill, it simply shut down the system and forced the driver to intervene. This happened almost half the time. Notably, in the other half the time, the BMW was at least as likely to touch or cross a lane marking as not.
The other cars, including the Model S, negotiated the hills and curves successfully between half and two-thirds of the time.
All the cars showed a tendency at low speeds to follow a leading car out of the lane if the lead car changed lanes or turned off, the report notes.
On its track, the IIHS tested automatic emergency braking by itself and under adaptive cruise control by driving the cars toward a fixed obstacle at 31 mph, and also by following another car in front that braked smoothly, braked abruptly, and moved out of the way just before a stationary obstacle, to see whether the following car could then "see" and brake in time for the obstacle.