Waymo released a new study on their incident data which attempts to factor in the reality that different types of incidents happen at different rates in different towns, on different roads and at different times of day. That lets them compare the rates of safety incidents for their vehicles and compare them with human incident rates in a more “apples to apples” comparison. In general, companies have liked to follow the vastly easier path of comparing general crash rates against nationwide data. This is not entirely invalid, but controlling for all the various factors that will affect crash rate gives more meaningful comparisons.
We want to answer the question of whether robocars going out on the road are improving overall road safety, and understand by how much they are doing that. Regulators should look at these sorts of numbers, because the job of regulators is not to directly prevent individual incidents, but to favor things which reduce the risk on our roads, and which show to potential to offer even greater reduction. This is what robocars promise to do, and Waymo offers data to say they are doing so. Now they offer more fine grained data to make a better evaluation of that claim.
Notoriously, Tesla has sometimes done the exact opposite, and offered comparisons they have to know are not apples-to-apples comparisons. For example, Tesla has regularly published statistics comparing airbag deployment crashes in Teslas with general police reported crashes for all cars. They have used that comparison to claim that Teslas with Autopilot, and later FSD supervised, had 10x fewer crashes than all cars. That claim is grossly incorrect. (Tesla releases other data which allows the more apples-to-apples comparison of Teslas with FSD and those without, suggesting the real number is 1.5x fewer, which is not nothing, but is hardly 10x.)
Statistics like these are very important when trying to evaluate, or regulate, the progress of robocars. Human beings, especially the public, have a natural desire to focus on single incidents, particularly ones that trigger emotions because they are scary, unusual or in the worst cases, tragic. Human driving every day involves many thousands of crashes and 100 fatalities, a level so overwhelming we have become numb to it. On the other hand, in press and social media we also see stories of robocars doing something stupid every day. Those stories usually cause somebody to react by saying, “these things aren’t ready for the roads if they will make that sort of mistake.”
The bad news for those folks is that since Waymo is doing 500,000 rides/week, they are going to make mistakes constantly. And this is never going to stop. In fact, it’s going to continue to grow, in absolute numbers, even as the rate of mistakes gets better. Mistakes will be made forever., because perfection isn’t on the table as an option.
Instead, we want to gather good data on the incidents and mistakes. Unfortunately, companies, even Waymo, have a natural instinct to keep things close to the vest. We want more transparency, but they fear it, in part because they know about the way most people will focus on particular scary incidents and not look at the big picture. The key factors to look at for mistakes and incidents are:
- Is the rate improving? Is each mistake a learning that means it won’t happen again?
- Are there regressions–things which used to work but now are broken?
- Are they avoiding “unacceptable risk?” (Defining this is tough, but we know what we’ve come to accept from human drivers and it’s quite a lot!)
- Do they offer the potential, if allowed to learn, of doing even better at reducing risk on the roads?
These are the questions the regulators, and more informed members of the public, should be asking, and they should be demanding the data they need to evaluate these questions. The unacceptable risk question is complex. Traditional safety zealot Phil Koopman points out that the public would not accept a system that cuts fatalities by 80% but the remaining ones are all children. (Fortunately that’s a very unlikely outcome.)
One other factor, as a matter of policy, is that it seems we should be much more tolerant of “mostly harmless” mistakes which only cause traffic problems or minor property damage that can quickly be remedied with money from the operators. We don’t like those things, of course, but they are worth trading for anything with a decent promise of reducing the real carnage on the roads, namely serious injuries. As frustrating as it may be to be stuck in congestion, I will take it (if temporary) to help build something that will reduce injuries. So should you.
Cities have been frustrated at the lack of transparency from teams on minor incidents and traffic blockages, and it would be nice to have that, possibly by reducing the reasons the companies have to fear release and thus fight it.
Notably not on the list are things like ISO-26262 (A widely used functional safety standard) compliance or a good safety case. These are useful tools to use to improve safety and meet the criteria above, and should be looked at when a team is building the system to make sure they have learned from all the techniques developed by others. But regulation should be about making sure players are motivated to, and do meet the public interest goals of safety and good traffic flow, not about telling them how to do that while the technology is still being invented.
