GM/Cruise leaks show them way, way behind Waymo. It's time for better metrics from everybody

Topic: 
Tags: 
Cruise car with sensors all around.

GM's "Cruise" robocar unit is often cited as #2 behind Waymo. Some recent leaks of their internal metrics for progress paint a dim picture; that they aren't nearly as far along as they hoped, which does not bode well for the planned 2019 launch. In fact, they show as an order of magnitude behind where Google/Waymo was back in 2015.

The numbers that get published due to legal requirements tell us almost nothing. The methodology used by Waymo and cruse, the distance between simulated contacts, is a good start, and more teams should use it. And we should also dream up better metrics. (The simulated contact metric means every time the human safety drivers have to intervene, you build the situation in simulator, and you figure out if the car would have hit anything or not.)

In the new article linked below, I outline what this means for Cruise and what better metrics we might use. Measuring the safety of Robocars is now the biggest problem they face. We aren't that good at it yet, and even once you've done it, you have to prove that you've done it.

See GM/Cruise leaks show them ridiculously behind Waymo

Comments

GM pulled it's best people off this effort once Softbank made its investment last year. I used to be working on the project and now I'm not along with many others.

Cruise thought they could do a better job running this but apparently not. I work at GM but own Google stock, go figure

These metrics strike me as such a naive way to judge progress in creating a self-driving vehicle. Like judging two five-year-olds on how far along they are in becoming a Jeopardy champion by seeing how many Jeopardy questions they get right.

And each of them is tested on a different set of questions, each from only a single category, with 99% of the questions being the exact same questions repeated over and over again.

A set of standard metrics would be awesome, and you'd think Waymo would want to define the standard instead of letting someone else do it. I can see Cruise being so enthusiastic, if they are really as far behind as Amir Efrati says. Then again, he has plenty bad to say about Waymo, too, and they're also way behind schedule.

Add new comment