The future of computer-driven cars and deliverbots
Robocar news is fast and furious these days. I certainly don't cover it all, but will point to stories that have some significance. Plus, to tease you, here's a clip from my 4K video of the new Apple car that you'll find at the end of this post.
In a major milestone for robocars, Waymo has announced they will deploy in Phoenix with no human safety drivers behind the wheel. Until now, almost all robocars out there have only gone out on public streets with a trained human driver behind the wheel, ready to take over at any sign of trouble.
Once robocars got public attention, a certain faction promoted the view that we should be giving much more attention to the idea of the "connected car." The connected car was coming sooner, would have a big effect, and some said that it was silly to talk about robocars at all without first thinking of them as connected cars. Many even pushed for the vocabulary around robocars to always include connectivity, pushing names like "connected autonomous vehicle" as a primary term for the technology.
Robocars will be connected, but not nearly as much as people in the "connected car" world imagine. And the connection won't be essential. Some cars will work with only a connection when they are parked, or with intermittent connectivity during the day. But most of all, they won't connect out to the world. The robocar probably will connect only to servers at its HQ -- the company that made it or which runs the fleet it's in. It won't talk directly to infrastructure and other cars, it may not even talk two-way with the rider's phone.
Fortunately, the efforts to require vehicle-to-vehicle and vehicle-to-infrastructure connectivity in cars are rumoured to have suffered a setback in the USA.
Rumours are swirling that the US Federal government will drop the proposed mandate that all new cars include a DSRC radio to do vehicle to vehicle communications. Regular readers will know that I have been quite critical of this mandate and submitted commentary on it. Whether they listened to my commentary, or this is just a Trump administration deregulation, it's the right step.
A few years ago, Eran Shir (who was one of my students at Singularity University and who today has an interesting startup using mobile phones to solve ADAS and self driving problems) suggested that rather than delivery robots, the future might see roving stores. These would be self-driving trucks filled with the most popular items for their region which come to you. You would open them, shop, and automatically be charged for items. From time to time they would travel to a depot for restocking.
This blog, and many other sites, paint a very positive picture of the robocar future. And it is positive, but far from perfect. One problem I worry about in the short term is the way robocars are going to make traffic worse before they get a chance to make it better.
The goal of all robocars is to make car travel more pleasant and convenient, and eventually cheaper. You can't make something better and cheaper without increasing demand for it, and that means more traffic.
Recently Madrona Ventures, in partnership with Craig Mundie (former Microsoft CTO) released a white paper proposing an autonomous vehicle corridor between Seattle and Vancouver on I-5 and BC Highway 99. While there are some useful ideas in it, the basic concept contains some misconceptions about both traffic management, infrastructure planning, and robocars.
The proposal starts with a call for allowing robocars in the carpool lanes, and then moving to having a robocar only lane. Eventually it moves to more lanes being robocar only, and finally the whole highway. Generally I have (mostly) avoided too much talk of the all-robocar road because there are so many barriers to this that it remains very far in the future. This proposal wants to make it happen sooner, which is not necessarily bad, but it sure is difficult.
Carpool lanes are poorly understood, even by some transportation planners. For optimum traffic flow, you want to keep every lane at near capacity, but not over it. If you have a carpool lane at half-capacity, you have a serious waste of resources, because the vast majority (around 90%) of the carpools are "natural carpools" that would exist regardless of the lane perk. They are things like couples or parents with children. A half-empty carpool lane makes traffic worse for everybody but the carpoolers, for whom the trip does improve.
That's why carpool lanes will often let in electric cars, and why "high occupancy toll" lanes let in solo drivers willing to pay a price. In particular with the HOT lane, you can set the price so you get just enough cars in the carpool lane to make it efficient, but no more.
(It is not, of course, this simple, as sometimes carpool lanes jam up because people are scared of driving next to slow moving regular lanes, and merging is problematic. Putting a barrier in helps sometimes but can also hurt. An all-robocar lane would avoid these problems, and that is a big plus.)
Letting robocars into the carpool lane can be a good idea, if you have room. If you have to push electric cars out, that may not be the best public goal, but it is a decision a highway authority could make. (If the robocars are electric, which many will be, it's OK.)
The transition, however, from "robocars allowed" to "robocars only" for the lane is very difficult. Because you do indeed have a decent number of carpools (even if only 10% are induced) you have to kick them out at some point to grow robocar capacity. You can't have a switch day without causing more traffic congestion for some time after it. If you are willing to build a whole new lane (as is normal for carpool creation) you can do it, but only by wasting a lot of the new lane at first.
Many are attracted to the idea that robocars can follow more closely behind another vehicle if they have faster reaction times. They also have the dream that the cars will be talking to one another, so they can form platoons that follow even more closely.) The inter car communication (V2V) creates too much computer security risk to be likely, though some still dream of a magic solution which will make it safe to have 1500kg robots exchanging complex messages with every car they randomly encounter on the road. Slightly closer following is still possible without it.
General Motors announced this week that they would "take full responsibility" if a crash takes place during an autonomous driving trip. This follows a pledge to do the same made some time ago by Daimler, Google and Volvo and possibly others.
What's interesting is that they don't add the caveat "if the system is at fault." Of course, if the system is not at fault, they can get payment from the other driver, and so it's still OK to tell the passenger or owner that GM takes responsibility.
The robocar revolution has the potential to assist China in dominating vehicle manufacturing. That's the bad news -- unless you are a Chinese manufacturer. The better news is that manufacturing is only part of the car industry, and it's getting smaller.
Today, Chinese brands are not sold in any numbers in the USA, or almost anywhere outside of China, but China is already the largest car manufacturing country in the world. Chinese brands have no cachet (even in China, it seems) and western and Korean/Japanese brands are strong. How might that change?
Car brand is very important for people buying a car to own. In fact, the nameplate is the top source of value in a modern car sale. The difference is that we will be moving from people buying cars to own towards people buying rides.
When you order "Uber Select" (Uber's nicer-car offering) you don't care if what shows up is a Lexus, BMW or Mercedes. You don't even car if its a Hyundai Genesis, their brand-new attempt at making a luxury marquee. You are only going to ride in it for 15 minutes. It has to be comfortable, smooth and look nice, but rarely does the logo on the outside matter.
It's the Uber brand that matters (though not as much, as most people would find no difference between an UberSelect and a Lyft Premiere as far as the vehicles are concerned. And you might not even care if it's a Great Wall Wey (a Chinese luxury car you've never heard of) that picks you up if it looks nice and gives a reliable ride.
Of course, today the top makers like Mercedes, BMW, Lexus, Audi, Acura, Infiniti and others are known not just for luxury, but for quality. They make well engineered, reliable cars in a way the Chinese are not quite ready to do.
But do they have to? If your expensive BMW breaks down, you have to get it towed, arrange its repair and get a rental car. You're pretty angry at BMW when it does, and you paid a lot for that car to avoid that experience, and usually you do. If a car in a robotaxi fleet breaks down, you're very unlikely to even know it happened. Very rarely, a car like that might break down when you are riding in it. It would pull to the side of the road and have already summoned a replacement car. Within 2-3 minutes a new vehicle will pull up and take you on your way while the company sends a tow truck to deal with the broken car.
Of course, if it broke down while on its way to you, might might not even know it. But even the breakdown while driving will be barely worth mentioning to friends, it just didn't inconvenience you very much at all.
While the BMW will surely break down less than the Great Wall (at least for now) it also costs a great deal more. That might be worth it to avoid that owner's breakdown scenario, but it's not for a fleet breakdown. For a fleet manager, it's just a question of whether vehicle downtime cost is more or less than the extra cost of more robust engineering, with a small factor for customer inconvenience.
To top things off, I predict robocars will have fewer breakdowns. They will always been monitoring themselves, and will come loaded with sensors. They will always get proper maintenance, taking themselves to maintenance depots when it is needed. They will test all systems like brakes, steering, tires, engines and more every day or every hour when running vacant. They will never let anything get too hot or vibrate too much. Both the BMW and the cheap car will do that.
NHTSA released their latest draft robocar regulations just a week after the U.S. House passed a new regulatory regime and the senate started working on its own. The proposed regulations preempt state regulation of vehicle design, and allow companies to apply for high volume exemptions from the standards that exist for human-driven cars.
It's clear that the new approach will be quite different from the Obama-era one, much more hands-off. There are not a lot of things to like about the Trump administration but this could be one of them. The prior regulations reached 116 pages with much detail, though they were mostly listed as "voluntary." I wrote a long critique of the regulations in a 4 part series which can be found in my NHTSA tag. They seem to have paid attention to that commentary and the similar commentary of others.
At 26 pages, the new report is much more modest, and actually says very little. Indeed, I could sum it up as follows:
The document does a better job at understanding the difference between goals -- public goods that it is the government's role to promote -- and approaches to those goals, which should be entirely the province of industry.
The new document is much more explicit that the 12 "safety design elements" are voluntary. I continue to believe that there is a risk they may not be truly voluntary, as there will be great pressure to conform with them, and possible increased liability for those who don't, but the new document tries to avoid that, and its requests are much milder.
The document understands the important realization that developers in this space will be creating new paths to safety and establishing new and different concepts of best practices. Existing standards have value, but they can at best encode conventional wisdom. Robocars will not be created using conventional wisdom. The new document takes the approach of more likely recommending that the existing standards be considered, which is a reasonable plan.
My own analysis is guided by a lightweight regulatory approach which has been the norm until now. The government's role is to determine important public goals and interests, and to use regulations and enforcement when, and only when, it becomes clear that industry can't be trusted to meet these goals on its own.
In particular, the government should very rarely regulate how something should be done, and focus instead on what needs to happen as the end result, and why. In the past, all automotive safety technologies were developed by vendors and deployed, sometimes for decades, before they were regulated. When they were regulated, it was more along the lines of "All cars should now have anti-lock brakes." Only with the more mature technologies have the regulations had to go into detail on how to build them.
Worthwhile public goals include safety, of course, and the promotion of innovation. We want to encourage both competition and cooperation in the right places. We want to protect consumer rights and privacy. (The prior regulations proposed a mandatory sharing of incident data which is watered down greatly in these new regulations.)
The NTSB (National Transportation Safety Board) has released a preliminary report on the fatal Tesla crash with the full report expected later this week. The report is much less favourable to autopilots than their earlier evaluation.
How will robocars fare in a disaster, like Harvey in Houston, Irma, or the tsunamis in Japan or Indonesia, or a big Earthquake, or a fire, or 9/11, or a war?
These are very complex questions, and certainly most teams developing cars have not spent a lot of time on solutions to them at present. Indeed, I expect that these will not be solved issues until after the first significant pilot projects are deployed, because as long as robocars are a small fraction of the car population, they will not have that much effect on how things go. Some people who have given up car ownership for robocars -- not that many in the early days -- will possibly find themselves hunting for transportation the way other people who don't own cars do today.
It's a different story when, perhaps a decade from now, we get significant numbers of people who don't own cars and rely on robocar transportation. That means people who don't have any cars, not the larger number of people who have dropped from 2 cars to 1 thanks to robocar services.
I addressed a few of these questions before regarding Tsunamis and Earthquakes.
A few key questions should be addressed:
Most of these issues revolve around fleets. Privately owned robocars will tend to have steering wheels and be usable as regular cars, and so only improve the situation. If they encounter unsafe roads, they will ask their passengers for guidance, or full driving. (However, in a few decades, their passengers may no longer be very capable at driving but the car will handle the hard parts and leave them just to provide video-game style directions.)
An immediately positive thing is the potential ability for private robocars to, once they have taken their owners to safety, drive back into the evacuation zone as temporary fleet cars, and fetch other people, starting with those selected by the car's owner, but also members of the public needing assistance. This should dramatically increase the ability of the car fleet to get people moved.
Nonetheless, it is often noted that in a robocar taxi world, there don't need to be nearly as many cars in a city as we have today. With ideal efficiency, there would be exactly enough seats to handle the annual peak, but few more. We might drop to just 1/4 of the cars, and we might also have many of them be only 1 or 2 seater cars. There will be far fewer SUVs, pickup trucks, minivans and other large cars, because we don't really need nearly as many as we have today.
For those in Silicon Valley, I will be giving a talk at the monthly autonomous vehicle enthusiast meetup. Some time ago I did my general talk, but this one will get into the meat on some of the big myths and issues. With luck we'll get some good debate going.
Almost all robocars use maps to drive. Not the basic maps you find in your phone navigation app, but more detailed maps that help them understand where they are on the road, and where they should go. These maps will include full details of all lane geometries, positions and meaning of all road signs and traffic signals, and also details like the texture of the road or the 3-D shape of objects around it. They may also include potholes, parking spaces and more.
The maps perform two functions. By holding a representation of the road texture or surrounding 3D objects, they let the car figure out exactly where it is on the map without much use of GPS. A car scans the world around it, and looks in the maps to find a location that matches that scan. GPS and other tools help it not have to search the whole world, making this quick and easy.
Google, for example, uses a 2D map of the texture of the road as seen by LIDAR. (The use of LIDAR means the image is the same night and day.) In this map you see the location of things like curbs and lane markers but also all the defects in those lane markers and the road surface itself. Every crack and repair is visible. Just as you, a human being, will know where you are by recognizing things around you, a robocar does the same thing.
Some providers measure things about the 3D world around them. By noting where poles, signs, trees, curbs, buildings and more are, you can also figure out where you are. Road texture is very accurate but fails if the road is covered with fresh snow. (3D objects also change shape in heavy snow.)
Once you find out where you are (the problem called "localization") you want a map to tell you where the lanes are so you can drive them. That's a more traditional computer map, though much more detailed than the typical navigation app map.
Some teams hope to get a car to drive without a map. That is possible for simpler tasks like following a road edge or a lane. There you just look for a generic idea of what lane markings or road edges should look like, find them and figure out what the lanes look like and how to stay in the one you want to drive in. This is a way to get a car up and running fast. It is what humans do, most of the time.
Most teams try to do more than driving without a map because software good enough to do that is also software good enough to make a map. To drive without a map you must understand the geometry of the road and where you are on it. You must understand even more, like what to do at intersections or off-ramps.
Creating maps is effectively the act of saying, "I will remember what previous cars to drive on this road learned about it, and make use of that the next time a car drives it."
Put this way it seems crazy not to build and use maps, even with the challenges listed below. Perhaps some day the technology will be so good that it can't be helped by remembering, but that is not this day.
There are many strong advantages of having the map:
One intermediate goal might be to have cars that can drive with only a navigation map, but use more detailed maps in "problem" areas. This is pretty similar, except in database size, with automatic map generation with human input only on the problem areas. If your non-map driving is trustworthy, such that it knows not to try problem areas, you could follow the lower cost approach of "don't map it until somebody's car pulled over because it could not handle an area."
There are two or three components of the maps people are building, in order to perform the functions above. At the most basic level is something not too far above the navigation maps found in phones. That's a vector map, except with lane level detail. Such maps know how many lanes there are, and usually what lanes connect to what lanes. For example, they will indicate that to turn right, you can use either of the right two lanes at some intersections.
Earlier I noted that Nidi Kalra of Rand spoke at the AVS about Rand's research suggesting that purely road testing robocars is an almost impossible task, because it would take hundreds of millions to a billion miles of driving to prove that a robocar is 10% better than human drivers.
(If the car is 10x better than humans, it doesn't take that long, but that's not where the first cars will be.)
This study has often been cited as saying that it's next to impossible to test robocars. The authors don't say that -- their claim is that road testing will not be enough, and will take too long to really work -- but commenters and press have taken it further to the belief that we'll never be able to test.
The mistake is that while it could take a billion miles to prove a vehicle is 10% safer than human drivers, that is not the goal. Rather, the goal is to decide that it's unlikely it is much worse than that number. It may seem like "better than X" and "not worse than X" are the same thing, but they are not. The difference is where you give the benefit of the doubt.
Consider how we deal with new drivers. We give them a very basic test and hand them a licence. We presume, because they are human teens, that they will have a safety record similar to other human teens. Such a record is worse than the level for experienced drivers, and in fact one could argue it's not at all safe enough, but we know of no way to turn people into experienced drivers without going through the risky phase.
If a human driver starts showing evidence of poor skills or judgments -- lots of tickets, and in particular multiple accidents, we pull their licence. It actually takes a really bad record for that to happen. By my calculations the average human takes around 20 years to have an accident that gets reported to insurance, and 40-50 years to have one that gets reported to police. (Most people never have an injury accident, and a large fraction never have any reported or claimed accident.)
Today's news is preliminary, but a U.S. house committee panel passed some new federal regulations which suggest sweeping change in the US regulatory approach to robocars.
I have recently managed to dig up some old documents from the earliest days of car regulation. Here is a report from NHTSA on the state of affairs near the turn of the 20th century.
In recent years, we've seen much excitement about the idea of carriages and coaches with the addition of "motors" which can propel the carriage without relying entirely on the normal use of horses or other beasts of burden. These "Horseless carriages," sometimes also known as "auto mobile" are generating major excitement, and prototypes have been generated by men such as Karl Benz and Armand Peugeot, along with the Duryea brothers, Ransom Olds and others in the the USA. The potential for these carriages has resulted in many safety questions and many have asked if and how NHTSA will regulate safety of these carriages when they are common.
Previously, NHTSA released a set of 4, and later 5 levels to classify and lay out the future progression of this technology.
Level zero is just the existing rider on horseback.
Level one is the traditional horse drawn carriage or coach, as has been used for many years.
A level 2 carriage has a motor to assist the horses. The motor may do the work where the horses trot
along side, but at any time the horses may need to take over on short notice.
In a level 3 carriage, sometimes the horses will provide the power, but it is allowed to switch over entirely to the "motor," with the
horses stepping onto a platform or otherwise being raised to avoid working them. If the carriage approaches an area it can't handle, or the motor has problems,
the horses should be ready, with about 10-20 seconds notice, to step back on the ground and start pulling. In some systems the horse(s) can be in a hoist which can raise or lower them from the trail.
A Level 4 carriage is one which can be pulled entirely by a motor in certain types of terrain or types of weather -- an operating domain --
but may need a horse at other times. There is no need for a sudden switch to the horses, which should be pulled in a trailer so
they can be hitched up for travel outside the operating domain.
The recently added fifth level is much further in the future, and involves a "horseless" carriage that can be auto mobile in all situations, with no need for any horse at all. (It should carry a horse for off-road use or to handle breakdowns, but this is voluntary.)
In San Francisco, I'm just back from the annual Automated Vehicle Symposium, co-hosted by the AUVSI (a commercial unmanned vehicle organization) and the Transportation Research Board, a government/academic research organization. It's an odd mix of business and research, but also the oldest self-driving car conference. I've been at every one, from the tiny one with perhaps 100-200 people to this one with 1,400 that fills a large ballroom.
Tuesday morning did not offer too many surprises. The first was an announcement by Toyota Research Institute of a $100M venture fund. Toyota committed $1B to this group a couple of years ago, but surprisingly Gil Pratt (who ran the DARPA Robotics Challenge for humanoid-like robots) has been somewhat a man of mixed views, with less optimistic forecasts.
Different about this VC fund will be the use of DARPA like "calls." The fund will declare, "Toyota would really like to see startups solving problem X" and then startups will apply, and a couple will be funded. It will be interesting to see how that pans out.
At CES, Nissan showed off their plan to have a remote control room to help robocars get out of sticky situations they can't understand like unusual construction zones or police directing traffic. Here, they showed it as further along and suggested it will go into operation soon.
This idea has been around for a while (Nissan based it on some NASA research) and at Starship, it has always been our plan for our delivery robots. Others are building such centers as well. The key question is how often robocars need to use the human assistance, and how you make sure that unmanned vehicles stay in regions where they can get a data connection through which to get help. As long as interventions are rare, the cost is quite reasonable for a larger fleet.
This answers the question that Rod Brooks (of Rethink Robotics and iRobot) recently asked, pondering how robocars will handle his street in Cambridge, where strange things like trucks blocking the road to do deliveries, are frequently found.
It's a pretty good bet that almost all our urban spaces will have data connectivity in the 2020s. If any street doesn't have solid data, and has frequent bizarre problems of any type, yet is really important for traversal by unmanned vehicles -- an unlikely trifecta -- it's quite reasonable for vehicle operators to install local connectivity (with wifi, for example) on that street if they can't wait for the mobile data companies to do it. Otherwise, don't go down such streets in empty cars unless you are doing a pickup/drop-off on the street.
Karl Iagenemma of nuTonomy told the story of moving their cars from Singapore, where driving is very regulated and done on the left, to Boston where it is chaotic and done on the right.
I've written a few times that perhaps the biggest unsolved problem in robocars is how to know we have made them safe enough. While most people think of that in terms of government certification, the truth is that the teams building the cars are very focused on this, and know more about it than any regulator, but they still don't know enough. The challenge is going to be convincing your board of directors that the car is safe enough to release, for if it is not, it could ruin the company that releases it, at least if it's a big company with a reputation.
While very few details have come out, Reuters reports that new proposed congressional bills on self-driving cars will reverse many of the provisions I critiqued in the NHTSA regulations last year.