A couple of weeks ago, I wrote about Tesla’s claim that its Autopilot driver-assistance software is safer than a human driver.
After a fatal Autopilot crash last May, the company said the death was the first in 130 million miles of Autopilot driving—and noted that, “among all vehicles, in the U.S., there is a fatality every 94 million miles.”
The clear implication: Autopiloted Teslas are safer than human-piloted cars, and lives would be saved if every car had Autopilot.
But Tesla’s statistics are questionable at best. The small sample size—one crash—makes any calculation of Autopilot fatality rate almost meaningless.
Furthermore, Tesla compared its Autopilot crash rate to the overall U.S. traffic fatality rate—which includes bicyclists, pedestrians, buses, and 18-wheelers. This is not just apples-to-oranges. This is apples-to-aardvarks.
One statistician called Tesla’s comparison “ludicrous on the face of it.”
Tesla Autopilot Test
A better yardstick for comparison is the fatality rate for U.S. drivers of cars and light trucks compiled by the Insurance Institute for Highway Safety.
By that yardstick, the Tesla Autopilot driver fatality rate is almost four times higher than typical passenger vehicles.
But even that comparison is suspect because it measures the Autopilot Tesla against vehicles as disparate as the hulking Ford F-150 and the tiny Smart ForTwo.
In the end, I concluded, “…perhaps the only valid comparative benchmark for the Autopilot Tesla is the non-Autopilot Tesla.”
So let’s do that comparison.
Seven Model S fatalities
“Bubslug,” a writer for the stock market website Seeking Alpha, recently compiled a list of fatal Model S crashes in the U.S.
According to his list, there have been six in all, including the May Autopilot crash in Florida.
Tesla Autopilot suite of features - with version 7.0 update
Just last week, however, a seventh fatal crash occurred in Indianapolis. To date, there’s no suggestion that Autopilot was engaged before that crash.
So, in the U.S., that’s a total of one Autopilot and six human-pilot fatal crashes.
To arrive at comparative fatal crash rates, simply divide by the total miles driven by each kind of pilot.
Conveniently, Tesla itself has provided just those numbers.
On October 7, Elon Musk tweeted “Cumulative Tesla Autopilot miles now at 222 million.”
Earlier this year, Tesla announced that total Tesla miles driven had surpassed the two-billion mile mark, and published a chart projecting future total mileage.
According to that chart projection, total Tesla miles driven through September 2016 were predicted to be right at three billion miles.
We’ll accept that figure, with the caveat that it presumably includes European and Chinese cars. (We’re counting only U.S. incidents, remember.)
But we believe this discrepancy applies to both Autopilot and human-pilot cars in roughly equal measure, and should thus cancel each other out.
So, using round numbers, let’s do the math.
Human-piloted Teslas: six crashes, 2.8 billion miles. That’s one every 470 million miles.
Autopiloted Teslas: one crash, 220 million miles. One every 220 million miles.
And there it is, in hard numerical data.
Conclusion: Autopiloted Teslas have twice the fatal-crash rate of human-piloted Teslas.
Tesla Enhanced Autopilot
Still not statistically significant?
Clearly, the sample-size ogre raises its ugly head in this calculation: any statistic with a sample size of one will have a huge margin for error.
But these numbers, while not definitive, are far more meaningful than Tesla’s original comparison.
That’s because at least the comparative yardstick—Autopilot Teslas vs human-pilot Teslas—is reasonably valid.
If Tesla continues to throw around Autopilot crash statistics, it should at least get the comparative yardstick right.
Based on this more accurate comparison, Autopilot safety appears to be a far murkier and more troubling subject than Tesla has been letting on.