Archives

Date

Federal government involvement

NHTSA, the federal car safety agency has been talking about getting into the robocar game for a while, and now declares it wants more involvement with two important details:

  • Unlike California, they are keen on making sure full robocars (able to run unmanned) are part of the regulations, and
  • Their regulations might supersede those of states like California.

In the next six months, the DoT will work with states and others on a unified policy. There are some other details here.

(California, by the way will have hearings in the next couple of weeks on their regulations. I will be out of the state, unfortunately.)

On top of this there is a $4 billion (over 10 years) proposal in the new Obama budget to support and accelerate robocars and (sadly) connected cars.

Perhaps most heartening is a plan to offer reduced regulation for up to 2,500 early deployment vehicles — a way to get companies out there in the field without shackling them first. Public attitudes on robocars have pushed regulators to a rather radical approach to regulation, namely attempting to define regulations before a product is actually on the market, with California even thinking of banning unmanned cars before they arrive. In the normal history of car safety regulation, technologies are built and deployed by vendors and are usually on the road for decades before they get regulated, but people are so afraid of robots that this normal approach may not happen here.

GM Delays super-cruise again

There was a fair bit of excitement when Cadillac announced “super-cruise,” a product similar to what you see in the Tesla autopilot, for the 2014 model year, or so we thought. It was the first effort from a big car company at some level of self-driving, even if minimal. Since then, they’ve kept delaying it, while Mercedes, Tesla and others have released such products. Now they have said it won’t show until at least 2017. GM is quickly dropping in the ranks of active Robocar companies, leaving the U.S. mantle to Tesla and Ford. Chrysler has never announced anything an even ran anti-self-driving-car ads in the Superbowl a few years ago.

Tesla releases “summon” and hints at more

The latest Tesla firmware release offers a “summon” function, so you can train your car to park and come back to you (with a range of 39 feet.) Primary use is to have your car go park itself in the garage, or at a robotic charging station. This didn’t stop Elon Musk from promising we are not very far away from being able to summon the car from very far away.

They have also detailed that those sorts of functions, and other autonomy, will require more sensors than they put in the model S, and that this sensor suite is a few years away, perhaps in time for the model 3

But wait, there’s more…

The pace of news is getting fast. Even I’m having trouble keeping up with everything even though it’s part of my job. This blog will continue to be a place not for all the news, but the news that actually makes a difference, with analysis.

Here are some other items you might find of interest:

Google releases detailed intervention rates -- and the real unsolved problem of robocars

Hot on the heels of my CES Report is the release of the latest article from Chris Urmson on The View from the Front Seat of the Google Car. Chris heads engineering on the project (and until recently led the entire project.)

Chris reports two interesting statistics. The first is “simulated contacts” — times when a safety driver intervened, and the vehicle would have hit something without the intervention:

There were 13 [Simulated Contact] incidents in the DMV reporting period (though 2 involved traffic cones and 3 were caused by another driver’s reckless behavior). What we find encouraging is that 8 of these incidents took place in ~53,000 miles in ~3 months of 2014, but only 5 of them took place in ~370,000 miles in 11 months of 2015. (There were 69 safety disengages, of which 13 were determined to be likely to cause a “contact.”)

The second is detected system anomalies:

There were 272 instances in which the software detected an anomaly somewhere in the system that could have had possible safety implications; in these cases it immediately handed control of the vehicle to our test driver. We’ve recently been driving ~5300 autonomous miles between these events, which is a nearly 7-fold improvement since the start of the reporting period, when we logged only ~785 autonomous miles between them. We’re pleased.

Let’s look at these and why they are different and how they compare to humans.

The “simulated contacts” are events which would have been accidents in an unsupervised or unmanned vehicle, which is serious. Google is now having one once every 74,000 miles, though Urmson suggests this rate may not keep going down as they test the vehicle in new and more challenging environments. It’s also noted that a few were not the fault of the system. Indeed, for the full set of 69 safety disengagements, the rate of those is actually going up, with 29 of them in the last 5 months reported.

How does that number compare with humans? Well, regular people in the USA have about 6 million accidents per year reported to the police, which means about once every 500,000 miles. But for some time, insurance companies have said the number is twice that, or once every 250,000 miles. Google’s own new research suggests even more accidents are taking place that go entirely unreported by anybody. For example, how often have you struck a curb, or even had a minor touch in a parking lot that nobody else knew about? Many people would admit to that, and altogether there are suggestions the human number for a “contact” could be as bad as one per 100,000 miles.

Which would put the Google cars at close to that level, though this is from driving in simple environments with no snow and easy California driving situations. In other words, there is still some distance to go, but at least one possible goal seems in striking distance. Google even reports going 230,000 miles from April to November of last year without a simulated contact, a (cherry-picked) stretch that nonetheless matches human levels.

For the past while, when people have asked me, “What is the biggest obstacle to robocar deployment, is it technology or regulation?” I have given an unexpected answer — that it’s testing. I’ve said we have to figure out just how to test these vehicles so we can know when a safety goal has been met. We also have to figure out what the safety goal is.

Various suggestions have come out for the goal: Having a safety record to match humans. Matching good humans. Getting twice or even 10 times or even 100 times as good as humans. Those higher, stretch goals will become good targets one day, but for now the first question is how to get to the level of humans.

One problem is that the way humans have accidents is quite different from how robots probably will. Human accidents sometimes have a single cause (such as falling asleep at the wheel) but many arise because 2 or more things went wrong. Almost everybody I talk to will agree a time has come when they were looking away from the road to adjust the radio or even play with their phone, and they looked up to see traffic slowing ahead of them, and quickly hit the brakes just in time, narrowly avoiding an accident. Accidents often happen when luck like this runs out. Robotic accidents will probably mostly come from one single flaw or error. Robots doing anything unsafe, even for a moment, will be cause for alarm and the source of the error will be fixed as quickly as possible.

Safety anomalies

This leads us to look at the other number — the safety anomalies. At first, this sounds more frightening. They range from 39 hardware issues and anomalies to 80 “software discrepancies” which may include rarer full-on “blue screen” style crashes (if the cars ran Windows, which they don’t). People often wonder how we can trust robocars when they know computers can be so unreliable. (The most common detected fault is a perception discrepancy, with 119. It is not said, but I will presume these will include strange sensor data or serious disagreement between different sensors.)

It’s important to note the hidden message. These “safety anomaly” interventions did not generally cause simulated contacts. With human beings, the fact that you zone out, take your eyes off the road, text or even in many cases even briefly fall asleep does not always result in a crash for humans, and nor will similar events for robocars. In the event of a detected anomaly, one presumes that independent (less capable) backup systems will immediately take over. Because they are less capable, they might cause an error, but that should be quite rare.

As such, the 5300 miles between anomalies, while clearly in need of improvement, may also not be a bad number. Certainly many humans have such an “anomaly” that often (that’s about every 6 months of human driving.) It depends how often such anomalies might lead to a crash, and what severity of crash it would be.

The report does not describe something more frightening — a problem with the system that it does not detect. This is the sort of issue that could lead to a dangerous “careen into oncoming traffic” style event in the worst case scenario. The “unexpected motion” anomalies may be of this class. (As such would be a contact incident, we can conclude it’s very rare if it happens at all in the modern car.) (While I worked on Google’s car a few years ago, I have no inside data on the performance of the current generations of cars.)

I have particular concern with the new wave of projects hoping to drive with trained machine learning and neural networks. Unlike Google’s car and most others, the programmers of those vehicles have only a limited idea how the neural networks are operating. It’s harder to tell if they’re having an “anomaly,” though the usual things like hardware errors, processor faults and memory overflows are of course just as visible.

The other vendors

Google didn’t publish total disengagements, judging most of them to be inconsequential. Safety drivers are regularly disengaging for lots of reasons:

  • Taking a break, swapping drivers or returning to base
  • Moving to a road the car doesn’t handle or isn’t being tested on
  • Any suspicion of a risky situation

The latter is the most interesting. Drivers are told to take the wheel if anything dangerous is happening on the road, not just with the vehicle. This is the right approach — you don’t want to use the public as test subjects, you don’t want to say, “let’s leave the car auto-driving and see what it does with that crazy driver trying to hassle the car or that group of schoolchildren jaywalking.” Instead the approach is to play out the scenario in simulator and see if the car did the right thing.

Delphi reports 405 disengagements in 16,600 miles — but their breakdown suggests only a few were system problems. Delphi is testing on highway where disengagement rates are expected to be much lower.

Nissan reports 106 disengagements in 1485 miles, most in their early stages. For Oct-Nov their rate was 36 for 866 miles. They seem to be reporting the more serious ones, like Google.

Tesla reports zero disengagements, presumably because they would define what their vehicle does as not a truly autonomous mode.

VW’s report is a bit harder to read, but it suggests 5500 total miles and 85 disengagements.

Google’s lead continues to be overwhelming. That shows up very clearly in the nice charts that the Washington Post made from these numbers.

How safe do we have to be?

If the number is the 100,000 mile or 250,000 mile number we estimate for humans, that’s still pretty hard to test. You can’t just take every new software build and drive it for a million miles (about 25,000 hours) to see if it has fewer than 4 or even 10 accidents. You can and will test the car over billions of miles in simulator, encountering every strange situation ever seen or imagined. Before the car has a first accident it will be unlike a human. It will probably perform flawlessly. if it doesn’t, that will be immediate cause for alarm back at HQ, and correction of the problem.

Makers of robocars will need to convince themselves, their lawyers and safety officers, their boards, the public and eventually even the government that they have met some reasonable safety goal.

Over time we will hopefully see even more detailed numbers on this. That is how we’ll answer this question.

This does turn out to be one advantage of the supervised autopilots, such as what Tesla has released. Because it can count on all the Tesla owners to be the fail-safe (or if you prefer, guinea-pig) for their autopilot system, Tesla is able to quickly gather a lot of data about the safety record of its system over a lot of miles. Far more than can be gathered if you have to run the testing operation with paid drivers or even your own unmanned cars. This ability to test could help the supervised autopilots get to good confidence numbers faster than expected. Indeed, though I have often written that I don’t feel there is a good evolutionary path from supervised robocars to unmanned ones, this approach could make my prediction be in error. For if Tesla or some other car maker with lots of cars on the road is able to make an autopilot, and then observe that it never fails in several million miles, then they might have a legitimate claim on having something safe enough to run unmanned, at least on the classes of roads and situations which the customers tested it on. Though a car that does 10 million perfect highway miles is still not ready to bring itself to you door to door on urban streets, as Elon Musk claimed would happen soon with the Tesla yesterday.

CES 2016 Robocar News

I’m back from CES 2016 with a raft of news, starting with robocars. Some news was reported before the show but almost everybody had something to say — even if it was only to have something to say!

I have many more photos with coverage in my CES 2016 Photo Gallery.

Ford makes strong commitment

Ford’s CEO talks like he gets it. Ford did not have too much to show — they announced they will be moving to Velodyne’s new lower cost 32-laser puck-sized LIDAR for their research, and boosting their research fleet to 30 vehicles. They plan for full-auto operation in limited regions fairly soon.

Ford is also making its own efforts into one-way car share (similar to Daimler Car2Go and BMW DriveNow) called GoDrive, which pushes Ford more firmly into the idea of selling rides rather than cars. The car companies are clearly believing this sooner than I expected, and the reason is very clearly the success of Uber. (As I have said, it’s a mistake to think of Uber as competition for the taxi companies. Uber is competition for the car companies.)

Ford is also doing an interesting “car swap” product. While details are scant, it seems what the service will do is let you swap your Ford for somebody else’s different Ford. For example, if somebody has an F-150 or Transit Van that they know they won’t use the cargo features on some day or weekend, you drive over with your ordinary sedan and swap temporarily for their truck — presumably with a small amount of money flowing to the more popular vehicle. Useful idea.

The big announcement that didn’t happen was the much-rumoured alliance between Ford and Google. Ford did not overtly refute it but suggested they had enough partners at present. The alliance would be a good idea, but either the rumours were wrong, or they are waiting for another event (such as the upcoming Detroit Auto Show) to talk about it.

Faraday Future, where art thou?

The big disappointment of the event was the silly concept racecar shown by Faraday Future. Oh, sure, it’s a cool electric racecar, but it has absolutely nothing to do with everything we’ve heard about this company, namely that they are building a consumer electric car-on-demand service with autonomous delivery. Everybody wondered if they had booked the space and did not have their real demo ready on time. It stays secret for a while, it seems. Recent hires, such as Jan Becker, the former head of the autonomous lab for Bosch, suggest they are definitely going autonomous.

Mapping heats up

Google’s car drives by having super-detailed maps of all the roads, and that’s the correct approach. Google is unlikely to hand out its maps, so both Here/Navteq (now owned by a consortium of auto companies in Germany) and TomTom have efforts to produce similar maps to licence to non-Google robocar teams. They are taking fairly different approaches, which will be the subject of a future article.

One interesting edge is that these companies plan to partner with big automakers and not just give them map data but expect data in return. That means that each company will have a giant fleet of cars constantly scanning the road, and immediately reporting any differences between the map and the territory. With proper scale, they should get reports on changes to the road literally within minutes of them happening. The first car to encounter a change will still need to be able to handle it, possibly by pulling over and/or asking the human passenger to help, but this will be a very rare event.

MobilEye has announced a similar plan, and they are already the camera in a large fraction of advanced cars on the road today. MobilEye has a primary focus on vision, rather than Lidar, but will have lots of sources of data. Tesla has also been uploading data from their cars, though it does not (as far as I know) make as extensive use of detailed maps, though it does rely on general maps.  read more »

Lyft and GM, Sidecar, the nature of competition and CES

Lyft announced a $500M investment from GM with $500M more, pushing them to a $5.4B valuation, which is both huge and just a tenth of Uber. This was combined with talk of a push to robocars. (GM will provide a car rental service to Lyft drivers to start, but the speculation is that whatever robocar GM gets involved in will show up at Lyft.)

With no details, Lyft’s announcement doesn’t really add anything to the robocar world that Uber doesn’t already add. It is GM’s participation that is more interesting, because it’s another car company showing they are not just giving lip service to the idea of selling rides rather than cars. (Mercedes and BWM have also started saying real things in this area.)

My initial expectations for the big car companies were much more bleak for them. I felt that their century long histories of doing nothing but selling cars would impede them from switching models until it was too late. That might still happen, and will happen for some companies, but more might survive than expected. The story also contains some more pure PR comments about OnStar in the new Lyft rental cars. Lyft drivers are all linked in real time with their smartphones; OnStar is obsolete technology, named only to make it seem GM is adding something. GM is not a great robocar leader. They have been very slow even with their highway “super cruise” efforts and the best they have done is partner with Rajkumar at CMU only to find Uber more successful at working with CMU folks.

Sidecar and where are you going?

Also frightening is the news last week of the death of Sidecar. Sidecar was the 3rd place smartphone-hail company after Uber and Lyft, but so distant a third that it decided to shut down. Where Lyft can raise another billion, Sidecar could not get a dime. The CEO is a friend of mine and I’ve been impressed that Sidecar was willing to innovate, even building a successful delivery business on top of the fact that you had to tell Sidecar where you were going. I think it’s important that users say where they are going. It allows much better planning of the use of robocar resources. If customers say where they are going, you can not only do some of the things Sidecar did (deliveries in the trunk the passenger doesn’t even know about, pricing set by drivers, directional goals set by drivers etc.) you can do more:

  • Send short-range cars (electric cars) for short trips
  • Send small (one or two person) cars when there is just one rider
  • Send cars not even capable of the highway if the trip doesn’t involve the highway
  • Pool riders far more efficiently, sometimes in vehicles designed for pooling which have 2-12 private “cabins.”

All of this is important to making transportation vastly more efficient, and in allowing a wide variety of vehicle designs, and a wide variety of power trains. It is only by knowing the destination that many of these benefits can be seen.

Uber lets you enter the destination but does not require it, and people do like having less to do when summoning a vehicle. (I always enter the destination when in places they don’t speak English, it’s a handy way to communicate with the driver.) The driver is not shown the destination until after they pick you up. This stops drivers from refusing rides going places they don’t want to go, which has its merits. It also has serious downsides for drivers, who sometimes at the end of their shift pick up a rider who wants to go 40 miles in the opposite direction of their home.

Even more frightening is what Sidecar’s death says about how much room there is for competitors in the robotaxi space. There are dozens of car makers competing for a new car customer, but San Francisco, the birthplace of Uber, Lyft and Sidecar, could not support 3 players in one of the world’s hottest investment spaces. Two unicorns, but nobody else.

When it comes to competition, the ride business is a strange one. For scheduled rides (which was most of the black car business before Uber) there are minimal economies of scale. A one-car limo “fleet” is still a viable business today, picking up customers for scheduled rides. They provide the same service as a 100 car limo-fleet, though they sometimes have to turn you down or redirect you to a partner.

For on-demand rides, there is a big economy of scale. I want a car now, so you have to have a lot of cars to be sure to have one near me. I will go with the service that can get to me soonest. While price and vehicle quality matter, they can be trumped by pickup time, within reason. Sidecar, being small, often failed in this area, including my attempt to use it on its last day on my way home from the airport.

Robocars offer up a middle ground. Because there is no driver who minds waiting, it will be common to summon a robocar longer in advance of when you want it. Once you know that “I’m leaving in around 20 minutes” you can summon, and the car can find somewhere to wait except in the most congested zones. Waiting time for a robotaxi can be very cheap, well under a dollar/hour, though during peak times, robotaxi owners will raise the price a little to avoid lost opportunity costs. (Finance costs will be under 20 cents/hour at 5% interest, and waiting space will range from free to probably 30 cents/hour in a competitive parking “spot market.”)

The more willing customers are to summon in advance, the more competitive a small player can be. They can offer you instant service when you actually are ready to leave, and that way they can compete on factors other than wait time. Small players can be your first choice, and they can subcontract your business to another company who has a car close by when you forget to summon in advance.)

CES in Las Vegas

I’m off to CES Wednesday. This show, as before promises to have quite a lot of car announcements. Rumours suggest the potential Ford/Google announcement could happen there, along with updates from most major companies. There will also be too many “connected” car announcements because companies need to announce something, and it’s easy to come up with something in that space that sounds cool without the actual need that it be useful.

This morning already sees an announcement from Volvo and Ericsson about streaming video in cars. This is a strange one, a mix of something real — as cars become more like living rooms and offices they are going to want more and better bandwidth, including bandwidth reliable enough for video conferencing — but also something silly, in that watching movies and TV shows is, with a bit of buffering, a high-bandwidth application that’s easy to get right on an unreliable network. Though in truth, because wireless bandwidth on the highway is always going to be more expensive than wifi in the parking space, it really makes more sense to pre-load your likely video choices to win both ways on cost and quality. I have been fascinated watching the shift between semi-planned watching (DVD rental, Netflix DVD queue, DVR, prepaid series subscriptions, watchlists and old-school live TV) and totally ad-hoc streaming on demand. While I understand the attraction of ad-hoc streaming (even for what you planned far ahead to watch) it surprises me that people do it even at the expense of cost and quality. Of course, there are parallels to how we might summon cars!