Did Uber really botch the switch to one driver and a timeline of the failure
Yesterday I examined some of the details released by the NTSB about the Uber fatality. Now I want to dig deeper with speculation as to the why. Of course, speculation is risky, though I can claim a pretty good track record. When I outlined various possible causes of the incident just after it, I put 4 at the top. I figured that only one might be true, but it turned out that two were (Misclassification as a bicycle, and the car wanting to stop but being unable to actuate the brakes) though I did not suspect Uber deliberately blocked the car from doing hard stops. So I'll try my luck at speculating again.
A probable timeline of error
- Coming in: Uber is in right lane, planning to enter right turn lane when it opens up
- 6 seconds out: Victim perceived with LIDAR and radar. Victim is walking across the two left turn lanes. (Unknown: Were LIDAR and radar targets properly recognized as the same thing, ie. fused?)
- Victim is classified as an unknown obstacle with no forward velocity, in the left turn lanes. No action needed, things are often stopped in the left turn lanes.
- Unknown time: Classification of victim changes from unknown to vehicle (ie. car.) Cars are not expected to move sideways. "Car" is either in left turn lane or may be by this point in left driving lane. Mistake made in measuring "car s" vector of movement. System decides not to slow because "car" it sees is stopped in the left lane, approaching a light, and the Uber is in the right lane and heading to the right turn lane. It does not, it seems, identify the left-to-right trajectory.
- Unknown time: Victim is reclassified as a cyclist. If classified as cyclist in left driving lane, vehicle should slow, though typical cyclists in the left lane are heading for the left turn lane, which could explain a decision not to slow. Uber car must not have had correct velocity vector still, since that vector would show high risk of incursion into the Uber's planned path.
- 1.3 seconds out: Vehicle identifies victim as cyclist in the Uber's lane (right lane,) with likely impact. Vehicle finally makes a correct decision that emergency brake is needed. Due to problems with false positives in the past, system is programmed not to engage brakes in this situation.
- 1.3 seconds out: Poor system design means alert about need for emergency brake is not conveyed to driver. If there are warnings on the screen (which the driver is looking at) she does not immediately comprehend them.
- 0 seconds: Impact
- Shortly after: Driver applies hard brakes, too late.
It is not yet clear from the report exactly when the vehicle switched from thinking the victim was a car to thinking she was a cyclist. At 1.3 seconds out she would have been in the right lane, but only partly. A typical fit person crossing a street will travel about 7 feet in 1.3 seconds according to research, while lanes are typically 10-12 feet wide. She was hit about 8-9 feet into the lane, from the look of it, on the right side of the Volvo.
There are explanations as to why the vehicle might not consider a car in the left lane (which is primarily how it identified her) as no reason to slow from its plan to enter the right turn lane. I say explanations because there were also anomalies which it could have tried to understand, so the explanations and not excuses. Why does the "car" appear to be moving slowly sideways to the right -- which cars don't do? Radar would have told it the obstacle was not moving forward. Cars are often seen stopped at lights, even well ahead of lights, but we humans know this is only when other cars are in front of them if well ahead of the light. (It is also possible the system did not realize the radar target and LIDAR target were the same obstacle. All systems try to do that, but they are not perfect at it.)
If the classification switched to bicycle shortly before 1.3 seconds, then that failure did not play a role, since at that time it decided on emergency stop. If it switched to bicycle well before the 1.3 seconds, again there is an explanation, but a not acceptable one, for not slowing. One does see bicycles in the left lane at this point in this type of intersection. Usually they are heading for the left turn lane. However, the past movement of the "bicycle/car/unknown" should have told the system this is not what was happening. A stopped bicycle in the left lane should trigger caution and slowing down, even in the right lane. You pass a bicycle on the right only with caution.
It is also possible that the system did not connect the unknown, car and bicycle as the same object. Robocars see the world as a series of frames, like a movie. They must depend on their software to say, "This obstacle I see at location X is the same thing I saw 1/10th of a second ago at location X+delta, and the same one I saw 3/10ths of a second ago at X+3*delta. I didn't identify anything 2/10ths of a second ago but that was probably just a transient mistake." For humans this is trivial, but not for software, and such systems do make mistakes. Normally with only a little bit of time, they build a good model of what they see and how it's moving. Perhaps the Uber system did not.
It is also strange that the NTSB report does not contain in any way the report leaked from Uber much earlier by the news site "The Information." That report said the system classified the pedestrian as a "false positive." Ie. "we see something, but we are confident it's one of the things we should ignore, like sensor ghosts, blowing debris, birds or nearby radar targets." This would be a very important fact, and should not be left out of even a preliminary report if true. It would, however, offer an explanation for why Uber's system ignored until too late what should have been a fairly obvious trajectory of crossing the road, putting whatever the obstacle was on a collision course with the Uber's planned path.
Still to learn
Some important things still to learn:
- How well did the system "fuse" the radar and LIDAR, ie. did it identify that the radar pings and LIDAR points were from the same thing? Radar accurately tells you how fast the object is moving relative to you, LIDAR knows where it is. Both know how far it is.
- What did the visual component of the perception system do, and when, and did it get properly fused with the other systems? Results from the LIDAR can tell the vision systems where to look for objects of interest if all is working well.
- Given that it had radar, and knew the victim was not moving up or down the road, why did not that help classify the victim? Were there other non-moving radar pings in the same general direction?
- What was on the display screen? In videos of Uber cars, they show a typical visualization of the perception system, where it shows on the screen where it sees cars, bicycles and pedestrians. Did they not have this, and if so, why didn't the safety driver see that?
An unplanned and badly executed switch from 2 safety drivers to 1
Almost everybody agrees that Uber should not have had only one safety operator in the vehicle. It is speculated that they made that switch because they were in a rush for a demo, and this was the quickest way to rack up more miles quickly. For many teams, you could not do this -- your limiting factor is how many cars you have, not how many drivers you have. It is not something a well funded team would do to economize. But I am guessing Uber was not using all vehicles in full shifts and made this change so they could go to full testing of every car 24/7. (Given they were testing late Sunday night, one presumes that means they were 24/7.)
When you use two people, one has the wheel with primary duty of watching the road. The other has primary duty of monitoring the software, looking at the road from time to time. On the screen, you can see if the system is classifying obstacles correctly. Your human perception is superior to the robot's, so even looking at the screen you can see when it's making mistakes. You can tell the other safety driver to watch out because the system keeps flipping its mind about whether it sees a pedestrian or a bicycle or a car. You are also looking at diagnostics coming from the system and interpreting what they mean.
When Uber collapsed the two roles, I wonder if they did it without thinking. In particular, somebody without proper prudence, just had the solo safety driver perform both roles. This is a dumb idea, but it might happen in a rushed change.
Likewise, the lack of audible alerts to serious events could be a result of this change. With 2 operators, the software operator, looking at the screen, is a slower, but smarter, audible alert. If that operator shouts out or says something is wrong, that will put the driver with the wheel into alert. They switched to one driver without replacing that human based alert system.
It's pretty clear, of course, that you should not switch to one driver when your car is not yet very good. But if, for some reason, you decided you wanted to make that unwise choice, you would do it better with deliberation -- changing emergency brake policy, changing audio alert systems and making sure the solo drivers know to stop looking at the diagnostic console -- even training them not to do that.
I will admit that when I have been manually driving a self-driving car, I have made glances at the software diagnostic console. I know enough not to stare at it -- I am manually driving after all -- but it is seductive. You can't help but be curious as to how the system is perceiving the world. (You can usually drive these vehicles in manual but still leave the perception system running and displaying its view of the world.)
You would need to actively discourage it, or simply turn it off. But Uber was rushed and didn't think to do that.
How could it not emergency brake?
Perhaps the most common question I have seen is how they could possibly have programmed the system to deliberately not brake once it decided emergency braking was necessary.
The answer, implied in statements and leaks, is that Uber's system often decides to emergency brake for no valid reason. If this happens often enough, there is no choice but to not let it actually apply the brakes. Or at least that is the simple solution. You can't put a car on the road that is regularly hard braking for ghosts. It's not safe or pleasant.
Uber decided that since it could not do that, it would rely on safety drivers for emergency braking. After all, there are hundreds of millions of cars out there all relying on human drivers for their emergency braking. It's a system that generally works. The problem with that, of course, was the fact that the safety driver was looking at the console.
"Use the human" is generally acceptable, but of course Uber could have done better. There are two things they should have done: * The system should send an audible alert, and even a short brake jab alert. (Short brake jab is very attention getting.) * The system should have levels of confidence in its emergency braking decisions, and there should be some level above which there are almost no false emergency braking events, and at that level it should still be able to actuate the brakes.
The second one can be hard in a prototype car. In this case, we have a car that has suddenly realized it is confident it has a bicycle in front of it and is going to hit it. Yet it had no cognizance of that before, because it does not associate this bicycle ahead of it with the car in the left lane it thought was there. That has many of the hallmarks of a false positive, and you don't have too much time to get more sensor readings and be sure. So it might not qualify.