Tesla’s spat with MobilEye reached a new pitch this week, and Tesla announced a new release of their autopilot and new plans. As reported here earlier, MobilEye announced during the summer that they would not be supplying the new and better versions of their EyeQ system to Tesla. Since that system was and is central to the operation of the Telsa autopilot, they may have been surprised that MBLY stock took a big hit after that announcement (though it recovered for a while and is now back down) and TSLA did not.
Tesla’s own efforts represent a threat to MobilEye from the growing revolution in neural network pattern matchers. Computer vision is going through a big revolution. MobilEye is a big player in that revolution, because their ASICs do both standard machine vision functions and can do neural networks. An ASIC will beat a general purpose processor when it comes to cost, speed and power, but only if the ASIC’s abilities were designed to solve those particular problems. Since it takes years to bring an ASIC to production, you have to aim right. MobilEye aimed pretty well, but at the same time lots of research out there is trying to aim even better, or do things with more general purpose chips like GPUs. Soon we will see ASICs aimed directly at neural network computations. To solve the problem with neural networks, you need the computing horsepower, and you need well designed deep network architectures, and you need the right training data and lots of it. Tesla and ME both are gaining lots of training data. Many companies, including Nvidia, Intel and others are working on the hardware for neural networks. Most people would point to Google as the company with the best skills in architecting the networks, though there are many doing interesting work there. (Google’s DeepMind built the tools that beat humans at the seemingly impossible game of Go, for example.) It’s definitely a competitive race.
While Tesla works on their vision systems, they also announced a plan to make much more use of radar. That’s an interesting plan. Radar has been the poor 3rd-class sensor of the robocar, after LIDAR and vision. Everybody uses it — you would be crazy not to unless you need to be very low cost. Radar sees further than the other systems, and it tells you immediately how fast any radar target you see is moving relative to you. It sees through fog and other weather, and it can even see under and around big cars in front of you as it bounces off the road and other objects. It’s really good at licence plates as well.
What radar doesn’t have is high resolution. Today’s automotive radars have gotten good enough to tell you what lane an object like another car is in, but they are not designed to have any vertical resolution — you will get radar returns from a stalled car ahead of you on the road and a sign above that lane, and not be sure of the difference. You need your car to avoid a stalled car in your lane, but you can’t have a car that hits the brakes every time it sees a road sign or bridge!
Real world radar is messy. Your antennas send out and receive from a very broad cone with potential signals from other directions and from side lobes. Reflections are coming from vehicles and road users but also from the ground, hills, trees, fences, signs, bushes and bridges. It’s work to get reliable information from it. Early automotive radars found the best solution was to use the doppler speed information, and discard all returns from anything that wasn’t moving towards or away from you — including stalled cars and cross traffic.
One thing that can help (imperfectly) is a map. You can know where the bridges and signs are so you don’t brake for them. Now you can brake for the stalled cars and the cross traffic the Tesla failed to see. You still have an issue with a stalled car under a bridge or sign, but you’re doing a lot better.
There’s a lot of room for improvement in radar, and I will presume — Tesla has not said — that Tesla plans to work on this. The automotive radars everybody buys (from companies like Bosch) were made for the ADAS market — adaptive cruise control, emergency braking etc. It is possible to design new radars with more resolution (particularly in the vertical) and other approaches. You can also try for more resolution, particularly by splitting the transmitter and receiver to produce a synthetic larger aperture. You can go into different bands and get more bandwidth and get more resolution in general. You can play more software tricks, and most particularly, you can learn by examining not just single radar returns, but rather the pattern of returns over time. (After all, humans don’t navigate from still frames, we depend on our visual system’s deep evolved ability to use motion and other clues to understand the world.)
The neural networks are making strides here. For example, while pedestrians produce basic radar returns, it turns out that their walking stride has a particular pattern of changes that can be identified by neural networks. People are doing research now on how examining the moving and dynamic pattern of radar returns can help you get more resolution and also identify shapes and motion patterns of objects and figure out what they are.
I will also speculate that it might be possible to return to a successor of the “sweeped” radars of old, the ones we are used to seeing in old war movies. Modern car radars don’t scan like that, but I have to wonder if with new techniques, like phased arrays to steer virtual beams (already the norm in military radar) and modern high speed electronics, that we might produce radars that get a better sense of where their target is. We’re also getting better at sensor fusion — identifying a radar target in an image or LIDAR return to help learn more about it.
The one best way to improve radar resolution would be to use more bandwidth. There have been experiments in using ultrawideband signals in the very high frequencies which may offer promise. As the name suggests, UWB uses a very wide band, and it distributes its energy over that very wide band, which means it doesn’t put too much energy into any one band, and has less chance of interfering in those bands. It’s also possible that the FCC, seeing the tremendous public value that reliable robocars offer, might consider opening up more spectrum for use in radar applications using modern techniques, and thus increase the resolution.
In other words, Tesla is wise to work on getting more from radar. With the loss of all MobilEye’s vision tools, they will have to work hard to duplicate and surpass that. For now, Tesla is committed to using parts that are for sale for existing production cars, costing hundreds of dollars. That has taken LIDAR “off their radar” even though almost all research teams depend on LIDAR and expect LIDAR to be cheap in a couple of years. (Including the LIDAR from Quanergy, a company I advise.)
To do this, they are working with only some specific car models, namely some Honda vehicles that already have advanced ADAS in them. Using the car’s internal bus, they can talk to the sensors in these cars (in particular the radar, since the Comma One has a camera) and also send control signals to actuate the steering, brakes and throttle. Then their neural networks can take the sensor information, and output the steering and speed commands to keep you in the lane. (Details are scant so I don’t know if the Comma One box uses its own camera or depends on access to the car’s.)
When I rode in Comma’s prototype it certainly wasn’t up to the level of the Tesla autopilot or some others, but it has been several months so I can’t judge it now. Like the Tesla autopilot, the Comma will not be safe enough to drive the car on its own, and you will need to supervise and be ready to intervene at any time. If you get complacent, as some Tesla drivers have, you could get injured or killed. I have yet to learn what measures Comma will take to make sure people keep their eyes on the road.
Generally, I feel that autopilots are not very exciting products when you have to watch them all the time — as you do — and also that bolt-on products are also not particularly exciting. Cruise’s initial plan (after they abandoned valet parking) was a bolt-on autopilot, but they soon switched to trying to build a real vehicle, and that got them the huge $700M sale to General Motors.
But for Comma, there is a worthwhile angle. Users of this bolt-on box will be helping to provide training data to improve their systems. In fact they will be paying for the privilege of testing the system and training it. Something that companies like Google did the old fashioned way, paying a staff of professionals to drive the cars and gather data. For a tiny, young startup it’s a worthwhile approach.
The vision of many of us for robocars is a world of less private car ownership and more use of robotaxis — on demand ride service in a robocar. That’s what companies like Uber clearly are pushing for, and probably Google, but several of the big car companies including Mercedes, Ford and BMW among others have also said they want to get there — in the case of Ford, without first making private robocars for their traditional customers.
In this world, what does it cost to operate these cars? How much might competitive services charge for rides? How much money will they make? What factors, including price, will they compete on, and how will that alter the landscape?
Here are some basic models of cost. I compare a low-cost 1-2 person robotaxi, a higher-end 1-2 person robotaxi, a 4-person traditional sedan robotaxi and the costs of ownership for a private car, the Toyota Prius 2, as calculated by Edmunds. An important difference is that the taxis are forecast to drive 50,000 miles/year (as taxis do) and wear out fully in 5 years. The private car is forecast to drive 15,000 miles/year (higher than the average for new cars, which is 12,000) and to have many years and miles of life left in it. As such the taxis are fully depreciated in this 5 year timeline, and the private car only partly.
Some numbers are speculative. I am predicting that the robotaxis will have an insurance cost well below today’s cars, which cost about 6 cents/mile for liability insurance. The taxis will actually be self-insured, meaning this is the expected cost of any incidents. In the early days, this will not be true — the taxis will be safer, but the incidents will cost more until things settle down. As such the insurance prices are for the future. This is a model of an early maturing market where the volume of robotaxis is fairly high (they are made in the low millions) and the safety record is well established. It’s a world where battery prices and reliability have improved. It’s a world where there is still a parking glut, before most surplus parking is converted to other purposes.
Fuel is electric for the taxis, gasoline/hybrid for the Prius. The light vehicle is very efficient.
Maintenance is also speculative. Today’s cars spend about 6 cents/mile, including 1 cent/mile for the tires. Electric cars are expected to have lower maintenance costs, but the totals here are higher because the car is going 250,000 miles not 75,000 miles like the Prius. With this high level of maintenance and such smooth driving, I forecast low repair cost.
Parking is cheaper for the taxis for several reasons. First, they can freely move around looking for the cheapest place to wait, which will often be free city parking, or the cheapest advertised parking on the auction “spot” market. They do not need to park right where the passenger is going, as the private car does. They will park valet style, and so the small cars will use less space and pay less too. Parking may actually be much cheaper than this, even free in many cases. Of course, many private car owners do not pay for parking overtly, so this varies a lot from city to city.
The Prius has one of the lowest costs of ownership of any regular car (take out the parking and it’s only 38 cents/mile) but its price is massively undercut by the electric robotaxi, especially my estimates for the half-width electric city car. (I have not even included the tax credits that apply to electric cars today.) For the taxis I add 15% vacant miles to come up with the final cost.
The price of the Prius is the retail cost (on which you must also pay tax) but a taxi fleet operator would pay a wholesale, or even manufacturer’s cost. Of course, they now have the costs of running a fleet of self-driving cars. That includes all the virtual stuff (software, maps and apps) with web sites and all the other staff of a big service company ranging from lawyers to marketing departments. This is hard to estimate because if the company gets big, this cost will not be based on miles, and even so, it will not add many cents per mile. The costs of the Prius for fuel, repair, maintenance and the rest are also all retail. The taxi operator wants a margin, and a big margin at first, though with competition this margin would settle to that of other service businesses. read more »
We’re on the cusp of a new wave of virtual reality and augmented reality technology. The most exciting is probably the Magic Leap. I have yet to look through it, but friends who have describe it as hard to tell from actual physical objects in your environment. The Hololens (which I have looked through) is not that good, and has a very limited field of view, but it already shows good potential.
It’s becoming easier and easier to create VR versions of both fictional and real environments. Every historical documentary show seems to include a nice model reconstructing what something used to look like, and this is going to get better and better with time.
This will be an interesting solution for many of the world’s museums and historical sites. A few years from now, every visit to a ruin or historical building won’t just include a boring and slow audioguide, but some AR glasses to allow you to see a model of what the building was really like in its glory. Not just a building — it should be possible to walk around ancient Rome or other towns and do this as well.
Now with VR you’ll be able to do that in your own home if you like, but you won’t be able to walk very far in that space. (There are tricks that let you fool people into thinking they walked further but they are just not the same as walking in the real space with the real geometry.) They will also be able to populate the space with recordings or animations of people in period costumes doing period things.
This is good news for historical museums. Many of them have very few actual interesting artifacts to see, so they end up just being placards and photos and videos and other multimedia presentations. Things I could easily see on the museum web site; their only virtue is that I am reading the text and looking at the picture in the greatly changed remains of where it happened. These days, I tend to skip museums that have become little more than multimedia. But going to see the virtual recreation will be a different story, I predict.
Soon will be the time for museum and tourist organizations to start considering what spaces will be good for this. You don’t need to restore or rebuild that old castle, as long as it’s safe to walk around. You just need to instrument it with tracking sensors for the AR gear and build and refine those models. Over time, the resolution of the AR glasses will approach that of the eyes, and the reality of the models will improve too. In time, many will feel like they got an experience very close to going back and time and seeing it as it was.
Well, not quite as it was. It will be full of tourists from the future, including yourself. AR keeps them present, which is good because you don’t want to bump into them. A more advanced system will cover the tourists in period clothing, or even replace their faces. You would probably light the space somewhat dimly to assure the AR can cover up what it needs to cover up, while still keeping enough good vision of the floor so you don’t trip.
Of course, if you cover everything up with the AR, you could just do this in a warehouse, and that will happen too. You would need to reproduce the staircases of the recreated building but could possibly get away with producing very little else. As long as the other visitors don’t walk through walls the walls don’t have to be there. This might be popular (since it needs no travel) but many of us still do have an attraction to the idea that we’re standing in the actual old place, not in our hometown. And the museums would also have rooms with real world artifacts to examine, if they have them.
The past period has seen some very big robocar news. Real news, not the constant “X is partnering with Y” press releases that fill the airwaves some times.
Uber has made a deal to purchase Otto, a self-driving truck company I wrote about earlier founded by several friends of mine from Google. The rumoured terms of the deal as astronomical — possibly 1% of Uber’s highly valued stock (which means almost $700M) and other performance rewards. I have no other information yet on the terms, but it’s safe to say Otto was just getting started with ambitious goals and would not have sold for less than an impressive amount. For a company only 6 months old, the rumoured terms surpass even the amazing valuation stories of Cruise and Zoox.
While Otto has been working on self-driving technology for trucks, any such technology can also move into cars. Uber already has an active lab in Pittsburgh, but up to now has not been involved in long haul trucking. (It does do local deliveries in some places.) There are many startups out there calling themselves the “Uber for Trucks” and Otto has revealed it was also working on shipping management platform tools, so this will strike some fear into those startups. Because of my friendship with Otto’s team, I will do more commentary when more details become public.
In other Uber news, Uber has announced it will sell randomly assigned Uber rides in their self-driving vehicles in Pittsburgh. If your ride request is picked at random (and because it’s in the right place) Uber will send one of their own cars to drive you on your ride, and will make the ride free, to boot. Of course, there will be an Uber safety driver in the vehicle monitoring it and ready to take over in any problem or complex situation. So the rides are a gimmick to some extent, but if they were not free, it would be a sign of another way to get customers to pay for the cost of testing and verifying self-driving cars. The free rides, however, will probably actually cause more people to take Uber rides hoping they will win the lottery and get not simply the free ride but the self-driving ride.
GM announced a similar program for Lyft — but not until next year.
Ford also goes all-in, but with a later date
Ford has announced it wants to commit to making unmanned capable taxi vehicles, the same thing Uber, Google, Cruise/GM, Zoox and most non-car companies want to make. For many years I have outlined the difference between the usual car company approaches, which are evolutionary and involve taking cars and improving their computers and the approaches of the non-car companies which bypass all legacy thinking (mostly around ADAS) to go directly to the final target. I call that “taking a computer and putting wheels on it.” It’s a big and bold move for Ford to switch to the other camp, and a good sign for them. They have said they will have a fleet of such vehicles as soon as 2021. read more »
At the recent AUVSI/TRB conference in San Francisco, there was much talk of upcoming regulation, particularly from NHTSA. Secretary of Transportation Foxx and his NHTSA staff spoke with just vague hints about what might come in the proposals due this fall. Generally, they said good things, namely that they are wary of slowing down the development of the technology. But they said things that suggest other directions.
Secretary Foxx began by agreeing that the past history of automotive driving systems was quite different. Regulations have typically been written years or decades after technologies have been deployed. And the written regulations have tended to involve standards which the vendors self-certify their compliance with. What this means is that there is not a government test center which confirms a car complies with the rules in the safety standards. Instead, the vendor certifies they are following the rules. If they certify falsely, that can get them in trouble later with regulators and more importantly in lawsuits. It’s by far the best approach unless the vendors have shown that they can’t be trusted in spite of the fear of these actions.
But Foxx said that they were going to go against that history and consider “pre-market regulation.” Regular readers will know I think that’s an unwise idea, and so do many regulators, who admit that we don’t know enough about the final form of the technology to regulate yet.
Fortunately it was also suggested that NHTSA’s new documents would be more in the form of “guidance” for states. Many states ask NHTSA to help them write self-driving car regulations. Which gets us to a statement that was echoed by several speakers to justify federal regulation, “Nobody wants 50 different regulations” on these cars.
At first, that seems obvious. I mean, who would want it to be that complex? Clearly it’s simpler to have to deal with only one set of regulations. But while that’s true, it doesn’t mean it’s the best idea. They are overestimating the work involved in dealing with different regulations, and underestimating the value of having the ability for states to experiment with new ideas in regulation, and the value of having states compete on who can write the best regulations.
If regulations differed so much between states as to require different hardware, that makes a stronger case. But most probably we are talking about rules that affect the software. That can be annoying, but it’s just annoying. A car can switch what rules it follows in software when it crosses a border with no trouble. It already has to, just because of the different rules of the road found in every state, and indeed every city and even every street! Having a few different policies state by state is no big addition.
Jurisdictional competition is a good thing though, particularly with emerging technologies. Let some states do it wrong, and others do it better, at least at the start. Le them compete to bring the technology first to their region, and invent new ideas on how to regulate something the world has never seen. Over time these regulations can be normalized. By the time people are making 10s of millions of robocars, that normalization will make more sense. But most vendors only plan to deploy in just a few states to begin, anyway. If a state feels its regulations are making it harder for the cars to spread to its cities, it can copy the rules of the other state it likes best.
The competition assures any mistake is localized — and probably eventually fixed. If California follows through with banning unmanned operation, as they have proposed, Texas has said it won’t.
I noted that if the hardware has to change, that’s more of an issue. It’s still not that much of an issue, because cars that operate as taxi services will probably never leave their base state. Most of them will have limited operational zones, and except in cities that straddle state borders, they won’t even leave town, let alone leave the state. Some day, the cars might do interstate trips, but even then you can solve this by having one car drive you to the border and then transfer to a car for the other state. Annoying, but only slight, and not a deal-breaker on the service. A car you own and take on road trips is a different story.
The one way having different state regulations would be a burden would be if there were 50 different complex certification processes to go through. Today, the federal government regulates how cars are made and the safety standards for that. The states regulate how cars operate on the roads. Robocars do blur that line, because how they are made controls how they drive.
For now, I still believe the tort system — even though it differs in all 50 states — is the best approach to regulation. It already has all developers highly paranoid about safety. When the day comes for certification, a unified process could make sense, but that day is still very far away. But for the regulations of just how these cars will operate, it might make sense to keep that with the states, even though it’s now part of the design of the car rather than the intentions of a human driver.
In time, unified regulations will indeed be desired by all, once we’ve had the time to figure out what the right regulations should be. But today? It’s too soon. Innovation requires variety.
Today, Robin Chase wrote an article wondering if robocars will improve or ruin our cities and asked for my comment on it. It’s a long article, and I have lots of comment, since I have been considering these issues for a while. On this site, I spend most of my time on the potential positive future, though I have written various articles on downsides and there are yet more to write about.
Robin’s question has been a popular one of late, in part a reaction by urban planners who are finally starting to think more deeply on the topic, and reacting to the utopian visions sometimes presented. I am guilty of such visions, though not as guilty as some. We are all seduced in part by excitement of what’s possible in a world where most or all cars are robocars — a world that is not coming for several decades, if in our lifetimes at all. It’s very fair to look at the topic from both sides, and no technology does nothing but good.
When I first met Robin, she was, like most people, a robocar skeptic. She’s done pioneering work in new transportation ideas, but the pace of improvement has surprised even the optimists. I agree with many of the potential negatives directions that she and others paint; in fact I’ve said them myself. Nonetheless my core position is that we can and probably will get tremendous good out of this. While I want city planners to understand these trends, I think it’s too early for them to actually attempt to guide them. Even the developers of the technology don’t quite know the final form it will take when it starts taking over the transport world in the 2020s. Long term planning is simply impossible at this stage — it must be done not with the knowledge of 2016 but with the knowledge of 2023. That approach — the norm in the high tech world, where we expect the world to constantly change underneath us — is anathema to governments and planners. When Marc A. said that software was eating the world, he was telling the world that it will need to start learning the rules of innovation that come from the high tech, internet and computer worlds.
Instead, today’s knowledge can at least guide planners in what not to do. Not to put big investments in things likely to become obsolete. Not to be too clever in thinking they understand the “smart city” of 2025. They need to be like the builders of the internet, who made the infrastructure as simple and stupid as they could, moving innovation away from the infrastructure and into the edges where it could flourish in a way that astounded humanity.
We will get more congestion in the start. Not because of empty vehicles cruising around — most research suggests that will be around 15% of miles, and then only after everybody switches. We’ll get more congestion from two factors:
The early cars, especially the big car company offerings, will make traffic jams more tolerable. As such, people will not work as hard to avoid them.
Car travel will be come much better and much cheaper; far more people will be able to use it, so they’ll travel more miles in cars than they do today.
For some, longer commutes will be more tolerable so they will live further from work. That won’t increase congestion in the central areas (they would still have driven those roads if they lived closer to work) but will increase it in the more remote places.
The tolerance for longer commutes may increase “sprawl.”
The good news is that the era of the ubiquitous smartphone brings us the potential for a traffic “miracle” — the ability to entirely eliminate traffic congestion. I first made that remarkable claim in 2008 in my article on congestion. I have a new article in the works which expands on this and makes it easier to understand. The plan is a rare one for me, because the city is heavily involved, but mostly in virtual infrastructure rather than physical. Virtual infrastructure needs to be the new buzzword of the city planner, because only virtual infrastructure is flexible enough to adapt to a changing world.
While this, and other plans to eliminate congestion won’t actually arise very quickly, the reason is not technological, it’s political. So the rise in congestion for the reasons cited above has a silver lining — it will push the public to be more accepting of entirely new ways of managing traffic.
The other way we can attack congestion is through the potential to make vastly superior group transit. Today’s transit sucks. It uses more energy than cars, provides slow and limited service from station to station (not door to door) in limited areas. When it does work efficiently, at rush hour, people travel standing, packed like sardines. People hate it so much that they spend over $8,000/year on vastly more expensive car ownership, the 2nd largest expense in most households. Robocars offer the potential for very appealing group transit which takes people efficiently from door to door in luxury vans on their schedule and along fast routes. Truly appealing transit might greatly increase ridership at congested times.
Robin suggests her Prius could drive around for $1.50/hour rather than park and that will make things worse. Perhaps if people make the same mistake it could, but when you look at it, you realize it costs closer to $20/hour to have a car drive around, and the fuel is just part of that. (Most auto web sites rate the Prius as costing 50 cents/mile, and at 25mph that’s only $12.50 per hour but in reality urban miles tend to cost more than highway miles so I like hourly rates. The Prius is rare though in that it uses less fuel in city miles.) Certainly no rational actor would do this. In addition, as more cars are shared, parking will become plentiful, particularly since a car no longer needs to park right where it dropped you off, but can instead request price bids on the “spot market” and find space going spare not too far away, which will certainly be available for well under $1.50/hour.
Fewer people will drive for a living. At the same time there are more bank tellers today than in 1970. They just don’t cash your cheques and give out withdrawls much any more. This topic deserves a great deal more verbiage, of course, but the kicker is this: These professional drivers are killing several thousand Americans every year while doing their jobs. Only doctors kill more. While the economic disruption is not an illusion, there is no way you can justify artificially preserving a job that is killing so many people. It’s a bit like arguing everybody should smoke so that tobacco farmers don’t lose their jobs.
Shared Cars & Parking
This will be huge, at least the part about sharing rides. Sharing cars for solo rides does not reduce miles driven or the number of cars made, but it does vastly reduce the amount of parking needed. Sharing rides reduces everything. I go much further in my vision to bring ride sharing to the level of dynamically allocated self-driving vans which replace today’s mass transit with something much more desired by the public and much more efficient at the same time.
I do hope the city parking lots are turned into parks mostly. The privately owned lots will get other uses, though downtown multi-floor lots are a bit harder to change.
It’s true that a major move to electric cars might require more electric capacity. Though they will charge mostly at night when power is cheap (though not solar.) One thing that many people don’t realize we won’t need is charging infrastructure. The great thing about robocars is they go where the energy is. The robocar will drive to the transformer substation which is packed with charging points — you don’t need to put charging stations in parking lots or houses.
However, at least today, electric cars are not cheaper than gasoline ones. The electricity is dirt cheap — under 3 cents/mile. The problem is at today’s battery prices, the battery depreciation is 20 to 40 cents per mile, much more than gasoline. Fortunately, there are optimistic signs about cheaper batteries and longer lasting batteries which could fix this.
But as robocars shrink — especially to one person vehicles for solo riders — they will become much cheaper than today’s cars, and also much more efficient. More efficient than the cars, but also all US transit systems. At a cost of around 30 cents/mile, car transportation will be available to billions more than can afford it today, and certainly to almost all Americans. That has its congestion downsides.
What Should Cities Do?
As noted above, it’s more about what they should not do. I am rebuilding my recommendations here, but my current list includes this:
Avoid regulation until you know what players can’t be trusted to do, and then fix only that
No more light rail or other single-use right-of-way. Stick to plain, bare pavement which can handle everything.
Create “transfer points” for carpools, robotaxi and robovan services to quickly — really quickly — transfer passengers between vehicles. These are useful for robocars, smartphone carpooling and even today’s transit.
Don’t require new buildings to put in tons of parking if they don’t want to
Make as much of your infrastructure virtual as you can. Encourage lots of data networks in the town, with the newest (5G and later) protocols in 2020.
If installing dedicated ROW for transit, make sure it can be converted to use by robocars in the future so the capacity isn’t wasted most of the time. If making tunnels, make sure stations are “offline” so that other vehicles can pass stopped vehicles, and make ramps for access by approved vehicles from the street.
At the recent AUVSI/TRB symposium, a popular research topic was platooning for robocars and trucks. Platooning is perhaps the oldest practical proposal when it comes to car automation because you can have the lead vehicle driven by a human, even a specially trained one, and thus resolve all the problems that come from road situations too complex for software to easily handle.
Early experiments indicated fuel savings, though relatively modest ones. At practical distances, you can see about 10% saving for following vehicles and 5% for the lead vehicle. Unfortunately, a few big negatives showed up. It’s hard to arrange platoons, errors can become catastrophic multi-car pile-ups, other drivers keep inserting themselves into the gap unless it’s dangerously small, and the surprising deal-breaker that comes from the stone chips which are thrown up by lead vehicles which destroy the finish — and in some cases the radiator or windshield — of following cars. They can also create a congestion problem and highway exit problem the way existing convoys of trucks sometimes do that.
One local company named Peloton is making progress with a very simple platooning problem. They platoon two (and only two) trucks on rural highways. The trucks find one another over the regular data networks, and when they get close they establish a local radio connection (using the DSRC protocol that many mistakenly hope will be the standard for vehicle to vehicle communications.) Both drivers keep driving, but the rear driver goes feet-off-the-pedals like a cruise control. The system keeps the vehicles a fixed distance to save fuel. The trucks don’t mind the stone chips too much. Some day, the rear driver might be allowed to go in the back and sleep, which would allow cargo to move 22 hours/day at a lower cost, probably similar to the cost of today’s team driving (about 50 cents/mile) but with two loads instead of one.
Trucks are an easy win, but I also saw a lot of proposals for car platoons. Car platoons are meant to save fuel, but also to increase road capacity. But after looking at all the research a stronger realization came to me. If you have robocars, why would you platoon when you can carpool?. To carpool, you need to find two cars who are going to share a long segment of trip together. Once you have found that, however, you get far more savings in fuel and road usage if the cars can quickly pause together and the passengers from one transfer into the other. Then the empty car can go and move other commuters. This presumes, of course, that the cars are like almost all cars out there today, with many empty seats. When the groups of passengers come to where their path diverts, the vehicle would need to stop at a transfer point and some passengers would move into waiting robotaxis to take them the rest of the way.
All of this is not as convenient as platooning, which in theory can happen without slowing down and finding a transfer point. This is one reason that the carpool transfer stations I wrote about last month could be a very useful thing. Such stations would add only 1-2 minutes of delay, and that’s well worth it if you consider that compared to platooning, this carpooling means a vastly greater fuel saving (almost 50%) and a much greater increase in road capacity, with none of the other downsides of platooning.
If you’re thinking ahead, however, you will connect this idea to my proposed plan for the future of group transit. The real win is to have the computers of the transport service providers notice the common routes of passengers early, before they even get into a vehicle, and thus pool them together with minimal need to stop and switch cars.
A number of folks have imagined designing cars that can physically couple, which would produce very efficient platoons and not add a delay. The problem (aside from the difficulties in doing this safely) is that this requires a physical standard, and physical standards are much harder to get working than software ones. It requires you find a platooning partner who has the same hardware you do, rather than software platooning, which can work with any style of car. Automated matching and carpooling makes no requirements on the individual robocars and their design, which gives it the best path to success.
It is possible (though a bit frightening) to imagine a special bus which could dock to robocars to allow transfer of passengers at speed. Some of you may have seen that a Chinese company has actually built the formerly hypothetical straddling bus (really a train) that has cars drive under it. If you were assured a perfectly smooth road one could imagine a docking extension which could surround a car door of a perfectly synced robocar and allow transfer. I suspect that’s all pretty far in the future.
Beyond the carpool
In a robocar world, we should see a move to having vehicles with fewer empty seats. This happens if more people use single person vehicles for their solo trips, and as carpooling and other technologies make sure that the 4 seater vehicles end up with more people. Indeed, if the carpooling works, that happens naturally. At that point one might say, “now’s the time to platoon.” There is merit to that, but it comes later, rather than sooner. At this later date we can be more comfortable with the safety, and have a greater density of vehicles making it more likely to find others vehicles ready to platoon. Of course, we’ll also have more vans and buses on the road who can combine even larger groups, if you find groups with a lot of journey in common. Platooning is practical even for a few miles, while carpooling tends to need a longer amount of shared journey to make it worth the switch.
At that point in the technology, you can do much more serious platoons, with larger groups of cars, and distances which are short enough for even greater benefit, and short enough to strongly discourage people trying to insert themselves in the middle of the platoon.
So platoons will come and give us even more road capacity. Carpooling, though, is already happening, with 50% of Uber requests in San Francisco being done in UberPool mode. It is the more likely early answer.
Today I want to look at some implications of Tesla’s Master Plan Part Deux which caused some buzz this week. (There was other news of course, including the AUVSI/TRB meeting which I attended and will report on shortly, forecast dates from Volvo, BMW and others, hints from Baidu, Faraday Future and Apple, and more.)
In Musk’s blog post he lays out these elements of Tesla’s plan
Integrating generation and storage (with SolarCity and the PowerWall and your car.)
Expand into trucks and minibuses
More autonomy in Tesla cars
Hiring out your Tesla as a robotaxi when not using it
Except for the first one, all of these are ideas I have covered extensively here. It is good to see an automaker start work in these directions. As such while I will mostly agree with what Tesla is saying, there are a few issues to discuss.
Electric (self-driving) minibus and Trucks
In my article earlier this year on the future of transit I laid out why transit should mostly be done with smaller (van sized) vehicles, taking ad-hoc trips on dynamic paths, rather than the big-vehicle, fixed-route, fixed-schedule approach taken today. The automation is what makes this happen (especially when you add the ability of single person robocars to do first and last miles.) Making the bus electric can make it greener, though making it run full almost all the time is far more important for that.
The same is true for trucks, but both trucks and buses have huge power needs which presents problems for having them be electric. Electric’s biggest problem here is the long recharge time, which puts your valuable asset out of service. For trucks, the big win of having a robotruck is that it can drive 24 hours/day, you don’t want to take that away by making it electric. This means you want to look into things like battery swap, or perhaps more simply tractor swap. In that case, a truck would pull in to a charging station and disconnect from its trailer, and another tractor that just recharged would grab on and keep it going. read more »
The cell phone ride hail apps like Uber and Lyft are now reporting great success with actual ride-sharing, under the names UberPool, LyftLines and Lyft Carpool. In addition, a whole new raft of apps to enable semi-planned and planned carpooling are out making changes.
The most remarkable number I have seen has Uber stating that 50% of rides in San Francisco are now UberPool. With UberPool, the system tries to find people with overlapping ride segments and quotes you a flat price for your ride. When you get it, there may already be somebody there, or your car may travel a small bit out of your way to pick up or drop somebody off. It’s particularly good for airports, but is also working in cities. The prices are often extremely good. During a surge it might be a much more affordable alternative.
It’s often been observed that as you watch any road, you see a huge volume of empty seats go down it. Even partial filling all those empty seats would make our roads vastly more efficient and higher capacity, as well as greener. Indeed, the entire volume of most transit systems could probably be easily absorbed, and a great deal more, if those empty seats were filled.
The strongest approach to date has been the hope that carpool lanes would encourage people to carpool. Sadly, this doesn’t happen very much. Estimates suggest that only 10% of the cars in the carpool lane are “induced” carpools — the rest are people like couples who already would have gone together. As such, many carpool lanes actually increase congestion rather than reducing it, because they create few induced carpools and take away road capacity. That’s why many cities are switching to HOT lanes where solo drivers can pay to get access to excess carpool lane capacity, or allowing electric/PHEV vehicles into the carpool lane.
Most carpool apps today have a focus on people who are employees of the same company. Companies have had tools to organize carpools for ages, and this works modestly well, but typically the carpools are semi-permanent — the same group rides in together each day, sometimes trading off who drives. The companies provide incentives like cash and special parking.
The new generation of carpool apps (outside Uber) tend to focus on people at the same company, and as such they mostly work with big companies. There they can add the magic of dynamic carpooling, which means allowing people to be flexible about when they come and go, and matching them up with different cars of other employees. This makes sense as an early business for many reasons:
People can inherently trust their co-workers
Co-workers naturally share the same workplace, so you only have to find one who live within a reasonable distance
Companies will subsidize the carpooling for many reasons, including saving them parking.
The subsidies can often include a very important one, the guaranteed ride back. Some of these apps say that when you want to leave, if they can’t find a carpool going near your house, they will provide alternate transportation, such as transit tickets or a Taxi/Uber style ride. This gives people the confidence to carpool in with one dynamically assigned group, knowing they will never be stuck at the office with no way home. Independent carpool services can also offer such a guarantee by adding a cost to every ride, but it’s easier for a company to do it. In fact, companies will often pay for the cost of the apps that do this, so that all the employees see is the car operating cost being shared among the poolers.
What has not happened much today is the potential of the multi-leg carpool, where you ride in one car for part of the trip, and another car (or another mode) for another part. Of course changing cars or modes is annoying compared to door-to-door transportation, though it’s the norm for transit riders.
Today, must carpool apps will have the driver go slightly off their route — often off the highway — to pick up a rider or return one home. (Normally the morning destination is a commercial building, usually the same building.)
A multi-leg service has some similarities to the concepts of multi-leg robocar transit I outlined previously. In one vision, the actual carpool sticks to highways and arterial roads, and never deviates from the expected route of the driver or any of the poolers. Poolers get to the carpool by using some other means — including a private Uber style ride — and then join it for the highway portion. If they are not going to the same place as other poolers, they can also use such a ride at the other end, though having two transfers reduces the appeal a fair bit.
This “last mile” leg can be something like Uber, or transit, or a bicycle (including one-way bicycle systems) or a “kiss and ride” drop-off by a spouse, or even another carpool. The difference is to make it dynamic, with live tracking of all parties involved, to reduce waits at the transfer points to very short times. (With robocars and vans, the waits will be measured in seconds, but human drivers won’t be that reliable.)
In spite of the inconvenience of having to do a transfer, if the wait is short, it’s better than the downsides of the driver or other poolers having to go far off the highway to handle a fellow pooler, and there can even be financial incentives to make things smooth.
Transfer points on arterials
The main barrier in the way of a truly frictionless transfer is the absence of good and easy places to do the transfer in many locations. This might be something that highway planners should consider in building or modifying future roads. The benefits can happen today, well before robocars, so it can get on the radar of the planners today. When the robocar transit arrives, tremendous benefits are possible.
Today, there is something a bit like this. In many cities, there are bus lines that run on highways. In some cases, bus stops have been built embedded in the highway, allowing the bus to stop without fully leaving the highway. A common example can be found on intersections which have a private on-ramp/off-ramp lane which stops mergers from interfering with primary traffic. Sometimes these are just off to the side of the regular highway, but in all cases the bus pulls off the highway and then into the bus stop. Riders have some safe path from the non-highway world, including bus stops on regular streets and arterials.
In the fast-transfer world, you want something like this, though you don’t necessarily need a path to other roads. A rider brought in an Uber can be dropped off there, and in interchanges with a private collector lane, the car that drops the rider off can easily get back onto the regular road in the opposite direction.
In the map is an intersection that already has all the ingredients needed for carpool transfer points — collector lanes, long ramps and lots of spare space. Most intersections are not as adaptable as this one, but new and reconstructed intersections can be adapted in much less space. In addition, transfer points may be possible in the center median, if there is room, under bridges, through the installation of a staircase from the bridge. (If there is no elevator, the disabled can be brought to the transfer point through a longer route that goes on the highway.) This is a common layout for transit lines which run down the median.
Full cloverleaf is better for the placement of transfer points, though there are other places they can go in other intersection designs. (It’s become popular of late to replace full coverleaf intersections with the parclo design that comes from my home town of Toronto. This change is mostly done to avoid the complex merge and tight turns of a full cloverleaf, though robocars can handle the full clover just fine. You can easily put some transfer points in a parclo, you just have an extra minute or two spent by the stopping carpool.
Transfer points are dirt cheap infrastructure, pretty much identical to bus stops, though ideally they would use angled parking so vehicles can come and go without blocking others. You do want space for a van or even a bus to come when you have found a super-carpool synergy, as will probably be the case at the peak of rush hour. Of course, if the volume of poolers grows very high, it justifies making larger transfer points and more of them. For super peak times, it’s OK to use transfer points that are just off the highways (where parking lots to do this are plentiful) because with high volume, pools are making just one stop to pick up passengers and can handle a small detour.
Transfer with parking
Of course, today the easiest way to do these carpools is with “carpool lots” not too far from the highway — places with spare parking which allow carpool riders to drive to the lot to meet their carpool driver. Indeed, carpoolers should be those who own cars because the first goal is to take a car off the road that otherwise have driven, and the second goal is to fill the empty seat with somebody who would otherwise have been on transit.
It can be difficult to get lots of parking convenient to the highway. One carpool lot I use has room for only about 50 cars. Nice that it’s there, but it takes no more than 50 cars off the road. At scale, one could imagine it being worthwhile to have shuttles from parking lots to on-highway transfer points, though nobody likes having to do 3 or 4 legs for a trip unless it’s zero wait time. If Robocars were not coming, one could imagine designing future highways with transfer points connected to parking lots. The people of the past did not imagine robocars or cell phone coordination of carpooling.
It’s not surprising there is huge debate about the fatal Tesla autopilot crash revealed to us last week. The big surprise to me is actually that Tesla and MobilEye stock seem entirely unaffected. For many years, one of the most common refrains I would hear in discussions about robocars was, “This is all great, but the first fatality and it’s all over.” I never believed it would all be over, but I didn’t think there would barely be a blip.
There’s been lots of blips in the press and online, of course, but most of it has had some pretty wrong assumptions. Tesla’s autopilot is a distant cousin of a real robocar, and that would explain why the fatality is no big deal for the field, but the press shows that people don’t know that.
Tesla’s autopilot is really a fancy cruise control. It combines several key features from the ADAS (Advance Driver Assist) world, such as adaptive cruise control, lane-keeping and forward collision avoidance, among others. All these features have been in cars for years, and they are also combined in similar products in other cars, both commercial offerings and demonstrated prototypes. In fact, Honda promoted such a function over 10 years ago!
Tesla’s autopilot primarily uses the MobilEye EyeQ3 camera, combined with radars and some ultrasonic sensors. It doesn’t have a lidar (the gold standard in robocar sensors) and it doesn’t use a map to help it understand the road and environment.
Most importantly, it is far from complete. There is tons of stuff it’s not able to handle. Some of those things it can’t do are known, some are unknown. Because of this, it is designed to only work under constant supervision by a driver. Tesla drivers get this explained in detail in their manual and when they turn on the autopilot.
ADAS cars are declared not to be self-driving cars in many state laws
This is nothing new — lots of cars have lots of features to help drive (including the components used like cruise controls, each available on their own) which are not good enough to drive the car, and only are supposed to augment an alert driver, not replace one. Because car companies have been selling things like this for years, when the first robocar laws were drafted, they made sure there was a carve-out in the laws so that their systems would not be subject to the robocar regulations companies like Google wanted.
The Florida law, similar to other laws, says:
The term [Autonomous Vehicle] excludes a motor vehicle enabled with active safety systems or driver assistance systems, including, without limitation, a system to provide electronic blind spot
assistance, crash avoidance, emergency braking, parking
assistance, adaptive cruise control, lane keep assistance, lane
departure warning, or traffic jam and queuing assistant, unless
any such system alone or in combination with other systems
enables the vehicle on which the technology is installed to
drive without the active control or monitoring by a human
The Tesla’s failure to see the truck was not surprising
There’s been a lot of writing (and I did some of it) about the particulars of the failure of Tesla’s technology, and what might be done to fix it. That’s an interesting topic, but it misses a very key point. Tesla’s system did not fail. It operated within its design parameters, and according to the way Tesla describes it in its manuals and warnings. The Tesla system, not being a robocar system, has tons of stuff it does not properly detect. A truck crossing the road is just one of those things. It’s also poor on stopped vehicles and many other situations.
Tesla could (and in time, will) fix the system’s problem with cross traffic. (MobilEye itself has that planned for its EyeQ4 chip coming out in 2018, and freely admits that the EyeQ3 Tesla uses does not detect cross traffic well.) But fixing that problem would not change what the system is, and not change the need for constant monitoring that Tesla has always declared it to have. read more »
Today at Starship, we announced our first pilot projects for robotic delivery which will begin operating this summer. We’ll be working with a London food delivery startup Pronto as well as German parcel company Hermes and the Metro Group of retailers, plus Just Eat restaurant food delivery to trial on-your-schedule delivery of packages, groceries and meals to people’s homes.
(It’s a nice break from Tesla news — and besides, our little robots weigh so little and move so slowly that even if something went horribly wrong and they hit you, injury is quite unlikely.)
Hermes, which does traditional package delivery is very interested in what I think is one of the core values of robot delivery — namely delivery on the recipient’s schedule. Today, delivery is done on the schedule of delivery trucks, and you may or may not be home when it arrives. With a personal delivery robot, it will only come when you’re home, reducing the risk of theft and lost packages. Robots don’t mind waiting for you.
The last mile is a huge part of the logistics world. Starship robots will get you packages with less cost, energy, time, traffic, congestion and emissions than you going to the store to get it yourself. They use a combination of autonomous driving with human control centers able to remotely fix any problems the robots can’t figure out. Robots don’t mind pausing if they have a problem and our robots can stop in under 30cm. As we progress, operation will reach near full autonomy and super low cost.
Executive Summary: A rundown of different approaches for validation of self-driving and
driver assist systems, and a recommendation to Tesla and others to have countermeasures
to detect drivers not watching the road, and permanently disable their Autopilot if they
show a pattern of inattention.
The recent fatality for a man who was allowing his car to be driven by the Tesla “autopilot”
system has ignited debate on whether it was appropriate for Tesla to allow their system to
be used as it was.
Tesla’s autopilot is a driver assist system, and Tesla tells customers it must always be
supervised by an alert driver ready to take the controls at any time. The autopilot is not
a working self-driving car system, and it’s not rated for all sorts of driving conditions,
and there are huge numbers of situations that it is not designed to handle and can’t handle. Tesla knows that, but the
public, press and Tesla customers forget that, and there are many Tesla users who are treating
the autopilot like a real self-driving car system, and who are not paying attention to the road —
and Tesla is aware of that as well. Press made this mistake as well, regularly writing
fanciful stories about how Tesla was ahead of Google and other teams.
Brown, the driver killed in the crash, was very likely one of those people, and if so, he paid for
it with his life. In spite of all the warnings Tesla may give about the system, some users
do get a sense of false security. There is debate if that means driver assist systems are
a bad idea.
There have been partial self-driving systems that require supervision since the arrival of the
cruise control. Adaptive cruise control is even better, and other car companies have released
autopilot like systems which combine adaptive cruise control with lane-keeping and forward
collision avoidance, which hits the brakes if you’re about to rear end another car. Mercedes
has sold a “traffic jam assist” like the Telsa autopilot since 2014 that only runs at low speeds
in the USA. You can even go back to a Honda demo in 2005 of an autopilot like system.
With cruise control, you might relax a bit but you know you have to pay attention. You’re steering
and for a long time even the adaptive cruise controls did not slow down for stopped cars.
The problem with Tesla’s autopilot is that it was more comprehensive and better performing than
earlier systems, and even though it had tons of things it could not handle, people started to
trust it with their lives.
Tesla’s plan can be viewed in several ways. One view is that Tesla was using customers as
“beta testers,” as guinea pigs for a primitive self-drive system which is not production ready,
and that this is too much of a risk.
Another is that Tesla built (and tested) a superior driver assist system with known and warned
limitations, and customers should have listened to those warnings.
Neither is quite right. While Tesla has been clear about the latter stance, with the knowledge that
people will over-trust it, we must face the fact that it is not only the daring drivers who
are putting themselves at risk, it’s also others on the road who are put at risk by the
over-trusting drivers — or perhaps by Tesla. What if the errant car had not gone under a truck, but
instead hit another car, or even plowed into a pedestrian when it careened off the road after the crash?
At the same time, Tesla’s early deployment approach is a powerful tool for the development and
quality assurance of self-drive systems. I have written before about how testing is the big
unsolved problem in self-driving cars. Companies like Google have spent many millions to use a
staff of paid drivers to test their cars for 1.6 million miles. This is massively expensive and
time consuming, and even Google’s money can’t easily generate the billions of miles of testing
that some feel might be needed. Human drivers will have about 12 fatalities in a billion miles,
and we want our self-driving cars to do much better. Just how we’ll get enough verification and testing done
to bring this technology to the world is not a solved problem. read more »
A Tesla blog post describes the first fatality involving a self drive system. A Tesla was driving on autopilot down a divided highway. A truck made a left turn and crossed the Tesla’s lanes. A white truck body against a bright sky is not something the MobilEye camera system in the Tesla perceives well, and it is not designed for cross traffic.
The truck trailer was also high, so when the Tesla did not stop, it went “under” it, so that the windshield was the first part of the Tesla to hit the truck body, with fatal consequences for the “driver.” Tesla notes that the autopilot system has driven 130 million miles, while human drivers in the USA have a fatality about every 94 million miles (though it’s a longer interval on the highway.) The Tesla is a “supervised” system where the driver is required to agree they are monitoring the system and will take control in the event of any problem, but this driver, a major Tesla fan named Joshua Brown, did not hit the brakes. As such, the fault for this accident will presumably reside with Brown, or perhaps the Truck driver — the accident report claims the truck did fail to yield to oncoming traffic, but as yet the driver has not been cited for this. (Tesla also notes that had the front of the car hit the truck, the crumple zones and other safety systems would probably have saved the driver — hitting a high target is the worst case situation.)
Any commentary here is preliminary until more facts are established, but here are my initial impressions:
There has been much speculation of whether Tesla was taking too much risk by releasing autopilot so early, and this will be boosted after this.
In particular, a core issue is that the autopilot works too well, and I have seen reports from many Tesla drivers of them trusting it far more than they should. The autopilot is fine if used as Tesla directs, but the better it gets, the more it encourages people to over-trust it.
Both Tesla stock and MobilEye stock were up today, with a bit of downturn after-hours. The market may not have absorbed this. The MobilEye is the vision sensor used by the Tesla to power the autopilot, and the failure to detect the truck in this situation is a not-unexpected result for the sensor.
For years, I have frequently heard it said that “the first fatality with this technology will end it all, or set the industry back many years.” My estimation is that this will not happen.
One report suggests the truck was making a left turn, which is a more expected situation, though if a truck turned with oncoming traffic it would be at fault.
Another report suggests that “friends” claim that the driver often used his laptop while driving, and some sources claim that a Harry Potter movie was playing in the car. (A portable DVD player was found in the wreckage.)
Tesla’s claim of 130M miles is a bit misleading, because most of those miles actually were supervised by humans. So that’s like reporting the record of student drivers with a driving instructor always there to take over. And indeed there are reports of many, many people taking over for the Tesla Autopilot, as Tesla says they should. So at best Tesla can claim that the supervised autopilot has a similar record to human drivers, ie. is no better than the humans on their own. Though one incident does not a driving record make.
Whatever we judge about this, the ability of ordinary users to test systems, if they are well informed and understand what they are doing is a useful one that will advance the field and give us better and safer cars, faster. Just how to do this may require more discussion, but the idea of doing it is worthwhile.
MobilEye issued a statement reminding people their system is not designed to do well on cross traffic at present, but their 2018 product will. It is also worth noting that the camera they use sees only red and gray intensity, it does not see all the colours, making it have an even harder time with the white truck and bright sky. The sun was not a factor, it was up high in the sky.
The Truck Driver claims the Tesla changed lanes before hitting him, an odd thing to happen with the Autopilot, particular if the driver was not paying attention. The lack of braking suggests the driver was not paying attention.
Camera vs. Lidar, and maps.
I have often written about the big question of cameras vs. LIDAR. Elon Musk is famously on record as being against LIDAR, when almost all robocar projects in the world rely on LIDAR. Current LIDARs are too expensive for production automobiles, but many companies, including Quanergy (where I am an advisor) are promising very low cost LIDARs for future generations of vehicles.
Here there is a clear situation where LIDAR would have detected the truck. A white truck against the sky would be no issue at all for a self-driving capable LIDAR, it would see it very well. In fact, a big white target like that would be detected beyond the normal range of a typical LIDAR. That range is an issue here — most LIDARs would only detect other cars about 100m out, but a big white truck would be detected a fair bit further, perhaps even 200m. 100m is not quite far enough to stop in time for an obstacle like this at highway speeds, however, such a car would brake to make the impact vastly less, and a clever car might even have had time to swerve or aim for the wheels of the truck rather than slide underneath the body.
Another sensor that is problematic here is radar. Radar would have seen this truck no problem, but since it was perpendicular to the travel of the car, it would not be moving away from or towards the car, and thus have the doppler speed signature of a stopped object. Radar is great because it tracks the speed of obstacles, but because there are so many stationary objects, most radars have to just disregard such signals — they can’t tell a stalled vehicle from a sign, bridge or berm. To help with that, a map of where all the fixed radar reflection sources are located can help. If you get a sudden bright radar return from a truck or car somewhere that the map says a big object is not known to be, that’s an immediate sign of trouble. (At the same time, it means that you don’t easily detect a stalled vehicle next to a bridge or sign.)
One solution to this is longer range LIDAR or higher resolution radar. Google has said it has developed longer range LIDAR. It is likely in this case that even regular range LIDAR, or radar and a good map, might have noticed the truck.
With Mobility on Demand, you don’t buy a car, you buy rides. That’s certainly Uber’s plan, and is a plan that makes sense for Google, Apple and other no-car companies. But even Daimler, with Car2Go/Car2Come, BMW with DriveNow and GM with Lyft plan to sell you a ride rather than a car, because it’s the more lucrative thing to do.
So what does that car of the future look like? There is no one answer, because in this world, the car that is sent to pick you up is tailored to your trip. The more people traveling, the bigger the car is. If your trip does not involve a highway, it may not be a car capable of the highway. If your trip is up to a mountain cabin, it’s more like an SUV, but you never use an SUV to go get a bottle of milk the way we do today. If it’s for a cruise to the beach on a sunny day, the roof may have been removed at the depot. If it’s for an overnight trip to a country home, it may be just beds.
I outlined many of these changes in this article on design changes in cars but today I will focus on the incredibly cheap and simple design of what should become the most common vehicle made, namely the car designed for a short urban trip by one person. That’s 80% of trips and around 45% of miles, so this should be a large fraction of the fleet. I predict a lot of these cars will be made every year — more than all the cars made today, even though they are used as taxis and shared among many passengers.
What does it look like?
A car for 1-2 people will be small. It will probably be around 1.5m wide, narrow enough that you can fit two in a lane, and have it park very efficiently when it has to wait. If it’s for just one person, it won’t be very long either. For two people, there will be a “face to face” configuration which is longer and an “tandem” configuration which is a bit shorter. The 2 person vehicles aren’t a lot bigger or heavier than the one person, so they might be the most common cars, since you can serve a solo rider fairly efficiently with one, even if not perfectly efficient.
A car that is so narrow can’t corner very fast. A wide stance is much more stable. There are a few solutions to that, including combinations of these:
The wheels bank independently, allowing the vehicle to lean like a motorcycle when in corners. This is the best solution, but it costs some money.
Alternately it’s a two wheeler, which is also able to lean, but has other tricks like the LIT Motors C-1 to stay upright.
It’s electric, and has all the batteries in the floor, giving it a very low center of gravity. (One extreme example of this is the Tango, which uses lead batteries deliberately to give it that stability.)
It never goes on fast roads, so it never needs to corner very fast, and its precision robot driving assures it never corners so fast as to become unstable, and it plans its route accordingly.
Not super aerodynamic
The car already has a big win when it comes to aerodynamic drag by only being half-width. The non-highway version probably gives back a bit of that because you don’t need to worry as much about that if you are not going fast. Energy lost to drag goes up with the square of velocity. So a 30mph car has 1/4 the drag of a 60mph car, and 1/8th the drag of a similar car of full width. The highway car needs to be shaped as close to a “teardrop” as you can, but the city car can get away with being a bit taller for more comfortable seating and entry/exit. read more »
When I give talks on robocars, the most common question, asked almost all the time, is the one known as the “trolley problem” question, “What will the car do if it has to choose between killing one person or another” or other related dilemmas. I have written frequently about how this is a very low priority question in reality, much more interesting to philosophy classes than it is important. It is a super-rare event and there are much more important everyday ethical questions that self-driving car developers have to solve long before they will tackle this one.
In spite of this, the question persists in the public mind. We are fascinated and afraid of the idea of machines making life or death decisions. The tiny number of humans faced with such dilemmas don’t have a detailed ethical debate in their minds; they can only go with their “gut” or very simple and quick reasoning. We are troubled because machines don’t have a difference between instant and carefully pondered reactions. The one time in billions of miles(*) that a machine faces such a question it would presumably make a calculated decision based on its programming. That’s foreign to our nature, and indeed not a task desired by programmers or vendors of robocars.
There have been calls to come up with “ethical calculus” algorithms and put them in the cars. As a programmer, I could imagine coding such an algorithm, but I certainly would not want to, nor would I want to be held accountable for what it does, because by definition, it’s going to do something bad. The programmer’s job is to make driving safer. On their own, I think most builders of robocars would try to punt the decision elsewhere if possible. The simplest way to punt the decision is to program the car to follow the law, which generally means to stay in its right-of-way. Yes, that means running over 3 toddlers who ran into the road instead of veering onto the sidewalk to run over Hitler. Staying in our lane is what the law says to do, and you are not punished for doing it. The law strongly forbids going onto the sidewalk or another lane to deliberately hit something, no matter who you might be saving.
We might not like the law, but we do have the ability to change it.
Thus I propose the following: Driving regulators should create a special panel which can rule on driving ethics questions. If a robocar developer sees a question which requires some sort of ethical calculation whose answer is unclear, they can submit that question to the panel. The panel can deliberate and provide an answer. If the developer conforms to the ruling, they are absolved of responsibility. They did the right thing.
The panel would of course have people with technical skill on it, to make sure rulings are reasonable and can be implemented. Petitioners could also appeal rulings that would impede development, though they would probably suggest answers and describe their difficulty to the panel in any petition.
The panel would not simply be presented with questions like, “How do you choose between hitting 2 adults or one child?” It might make more sense to propose formulae for evaluating multiple different situations. In the end, it would need to be reduced to something you can do with code.
Very important to the rulings would be an understanding of how certain requirements could slow down robocar development or raise costs. For example, a ruling that car must make a decision based on the number of pedestrians it might hit demands it be able to count pedestrians. Today’s robocars may often be unsure whether a blob is 2 or 3 pedestrians, and nobody cares because generally the result is the same — you don’t want to hit any number of pedestrians. Likeways, requirements to know the age of people on the road demands a great deal more of the car’s perception system than anybody would normally develop, particularly if you imagine you will ask it to tell a dwarf adult from a child. read more »
Reports from Tesla suggest they are gathering huge amounts of driving data from logs in their cars — 780 million miles of driving, and as much as 100 million miles in autopilot mode. This contrasts with the 1.6 million miles of test operations at Google. Huge numbers, but what do they mean now, and in the future?
As I’ve written before, testing is one of the biggest remaining challenges in robocar development — how do you prove to yourself and then to others that you’ve reached the desired safety goals? Tons of miles are a very important component to that. If car companies are able to get their customer to do the testing for them, that can be a big advantage. (As I wrote last week, another group which can get others to do testing are companies like Uber and even operators of large commercial and taxi fleets.) Lots of miles mean lots of testing, lots of learning, and lots of data.
Does Tesla’s quick acquisition of so many miles mean they have lapped Google? The short answer is no, but it suggests a significant threat since Google is, for now, limited to testing with its small fleet and team of professional testing drivers.
Tesla is collecting vastly less data from its cars than Google does. Orders of magnitude less. First of all, the Tesla has a lot fewer sensors and no LIDAR, and to the best of my knowledge from various sources I have spoken to, Tesla is only collecting a fraction of what their sensors gather. To collect all that they gather would be a huge data volume, not one you would send over the cell network, and even over the wifi at home it would be very noticeable. Instead, reports suggest Tesla is gathering only data on incidents and road features the car did not expect or did not handle well. However, nothing stops them in the future from logging more, though they might want to get approval from owners to use all that bandwidth.
Tesla wants to make a car for people to buy today. As such, it has no LIDAR, because a car today, and even the autopilot, can be done without LIDAR. Tomorrow’s LIDARs will be cheap but today’s production LIDARs for cars are simple and/or expensive. So while the real production door-to-door self driving car almost certainly uses LIDAR, Tesla is unable and unwilling to test and develop with it. (Of course, they can also argue that in a few years, neural networks will be good enough to eliminate the need for LIDAR. That’s not impossible, but it’s a risky bet. The first cars must be built in a safety-obsessed way, and you’re not going to release the car less safe than you could have made it just to save what will be only a few hundred dollars of cost.)
As noted, Google has being doing their driving with professional safety drivers, who are also recording a lot of data from the human perspective that ordinary drivers never will. That isn’t 100 times better but it’s pretty important.
Tesla is also taking a risk, and this has shown up in a few crashes. Their customers are beta testing a product that’s not yet fully safe. In fact, it was a pretty bold move to do this, and it’s less likely that the big car companies would have turned their customers into beta testers — at least no until forced by Tesla.
If they do, then the big automakers have even more customers than Tesla, and they can rack up even more miles of testing and data gathering.
When it comes to training neural networks, ordinary drivers can provide a lot of useful data. That’s why Commma.ai, who I wrote about earlier is even asking volunteers to put a smartphone on their dash facing out to get them more training data. At present, this app does not do much, but it will not be hard to make one that offers things like forward collision warning and lane departure warning for free, paid for by the data it gathers.
Watch me Sunday night on Dateline NBC: On Assignment
On Sunday, June 5, at 7pm (Eastern and Pacific) the news show Dateline: NBC will do a segment on self driving cars featuring Sebastian Thrun, Jay Leno and myself. I sat down for several hours with Harry Smith, but who knows how much actual airtime that turns into. Here is the promo for the episode and another more specific one.
Executive summary: Can our emotional fear of machines and the call for premature regulation be mollified by a temporary increase in liability which takes the place of specific regulations to keep people safe?
So far, most new automotive technologies, especially ones that control driving such as autopilot, forward collision avoidance, lanekeeping, anti-lock brakes, stability control and adaptive cruise control, have not been covered by specific regulations. They were developed and released by vendors, sold for years or decades, and when (and if) they got specific regulations, those often took the form of “Electronic stability control is so useful, we will now require all cars to have it.” It’s worked reasonably well.
That there are no specific regulations for these things does not mean they are unregulated. There are rafts of general safety regulations on cars, and the biggest deterrent to the deployment of unsafe technology is the liability system, and the huge cost of recalls. As a result, while there are exceptions, most carmakers are safety paranoid to a rather high degree just because of liability. At the same time they are free to experiment and develop new technologies. Specific regulations tend to come into play when it becomes clear that automakers are doing something dangerous, and that they won’t stop doing it because of the liability. In part this is because today, it’s easy to assign blame for accidents to drivers, and often harder to assign it to a manufacturing defect, or to a deliberate design decision.
The exceptions, like GM’s famous ignition switch problem, arise because of the huge cost of doing a recall for a defect that will have rare effects. Companies are afraid of having to replace parts in every car they made when they know they will fail — even fatally — just one time in a million. The one person killed or injured does not feel like one in a million, and our system pushes the car maker (and thus all customers) to bear that cost.
Robocars change some of this equation. First of all, in robocar accidents, the maker of the car (or driving system) is going to be liable by default. Nobody else really makes sense, and indeed some companies, like Volvo, Mercedes and Google, have already accepted that. Some governments are talking about declaring it but frankly it could never be any other way. Making the owner or passenger liable is technically possible, but do you want to ride in an Uber where you have to pay if it crashes for reasons having nothing to do with you?
Due to this, the fear of liability is even stronger for robocar makers.
Robocar failures will almost all be software issues. As such, once fixed, they can be deployed for free. The logistics of the “recall” will cost nothing. GM would have no reason not to send out a software update once they found a problem like the faulty ; they would be crazy not to. Instead, there is the difficult question of what to do between the time a problem is discovered and a fix has been declared safe to deploy. Shutting down the whole fleet is not a workable answer; it would kill deployment of robocars if several times a year they all stopped working.
In spite of all this history and the prospect of it getting even better, a number of people — including government regulators — think they need to start writing robocar safety regulations today, rather than 10-20 years after the cars are on the road as has been traditional. This desire is well-meaning and understandable, but it’s actually dangerous, because it will significantly slow down the deployment of safety technologies which will save many lives by making the world’s 2nd most dangerous consumer product safer. Regulations and standards generally codify existing practice and conventional wisdom. They are very bad ideas with emerging technologies, where developers are coming up with entirely new ways to do things, and entirely new ways to be safe. The last thing you want is to tell vendors they must apply old-world thinking when they can come up with much better thinking.
Sadly, there are groups who love old-world thinking, namely the established players. Big companies start out hating regulation but eventually come to crave it, because it mandates the way they do things and understand into the law. This stops upstarts from figuring out how to do it better, and established players love that.
The fear of machines is strong, so it may be that something else needs to be done to satisfy all desires: The desire of the public to feel the government is working to keep these scary new robots from being unsafe, and the need for unconstrained innovation. I don’t desire to satisfy the need to protect old ways of doing things.
One option would be to propose a temporary rule: For accidents caused by robocar systems, the liability, if the system should be at fault, shall be double that if a similar accident were caused by driver error. (Punitive damages for willful negligence would not be governed by this rule.) We know the cost of accidents caused by humans. We all pay for it with our insurance premiums, at an average rate of about 6 cents/mile. This would double that cost, pushing vendors to make their systems at least twice as safe as the average human in order to match that insurance cost.
Victims of these accidents (including hapless passengers in the vehicles) would now be doubly compensated. Sometimes no compensation is enough, but for better or worse, we have set on values and doubling them is not a bad deal. Creators of systems would have a higher bar to reach, and the public would know it.
While doubling the cost is a high price, I think most system creators would accept this as part of the risk of a bold new venture. You expect those to cost extra as they get started. You invest to make the system sustainable.
Over time, the liability multiplier would reduce, and the rule would go away entirely. I suspect that might take about a decade. The multiplier does present a barrier to entry for small players, and we don’t want something like that around for too long.
Here is the first report of a real Tesla autopilot crash. To be fair to Tesla, their owner warnings specify fairly clearly that the autopilot could crash in just this situation — there is a stalled car partly in the lane, and the car in front of you swerves around it, revealing it with little time for you or the autopilot to react.
The deeper issue is the way that the improving quality of the Tesla Autopilot and systems like it are lulling drivers into a false sense of security. I have heard reports of people who now are trusting the Tesla system enough to work while being driven, and indeed, most people will get away with this. And as people get away with it more and more, we will see more people driving like this driver, not really prepared to react. This is one of the reasons Google decided not to make a system which requires driver takeover ever. As the system gets better, does it get more dangerous?
Some technical notes:
This is one of the things LIDAR is much more reliable at seeing than cameras. Of course, whether you can swerve once the LIDAR sees it is another matter.
On the other hand, this is where radar fails. I mean the stalled car is clear on radar, but it’s stationary, so you can’t tell it from the road or guardrail which are also stationary.
This is one of the classic V2V value propositions, but it’s not a good one. You don’t need 10ms latency to have a stalled car tell you it is stalled. Far better that car report to a server that it’s stalled and for everybody coming down that road to learn it, whether they have line of sight radio to the stall, or V2V at all. Waze already reports this just with human manual reporting and that’s a really primitive way to do it.
Declaration of Amsterdam
Last month, various EU officials gathered in Amsterdam and signed the Declaration of Amsterdam which outlines a plan for normalizing EU laws around self-driving cars. The meeting also included a truck automation demo in the Netherlands and a self-drive transit shuttle demonstration. It’s a fairly bland document, more an expression of the times, and it sadly spends a lot of time on the red herring of “connected” vehicles and V2V/V2I, which governments seem to love, and self-driving car developers care very little about.
Let’s hope the regulatory touch is light. The reality is that even the people building these vehicles can’t make firm pronouncements on their final form or development needs, so governments certainly can’t do that, and we must be careful of attempts to “help” that hinder. We already have a number of examples of that happening in draft and real regulations, and we’ve barely gotten started. For now, government statements should be limited to, “let’s get out of the way until people start figuring out how this will actually work, unless we see somebody doing something demonstrably dangerous that can’t be stopped except through regulations.” Sadly, too many regulators and commentators imagine it should be, “let’s use our limited current knowledge to imagine what might go wrong and write rules to ban it before it happens.”
Speech from the Throne
It was a sign of the times when her Majesty the Queen, giving the speech from the throne in the UK parliament, laid out some elements of self-driving car plans. The Queen drove jeeps during her military days, and so routinely drives herself at her country estates, otherwise she would be among the set of people most used to never driving.
The UK has 4 pilot projects in planning. Milton Keynes is underway, and later this year, a variation of the Ultra PRT pods in use at T5 of Heathrow airport — they run on private tracks to the car park — will go out on the open road in Greenwich. They are already signing up people for rides.
Car companies thinking differently
In deciding which car companies are going to survive the transition to robocars, one thing I look for is willingness to stop thinking like a traditional car company which makes cars and sells them to customers. Most car company CEOs have said they don’t plan to keep thinking that way, but what they do is more important than what they say. read more »
Uber has announced the official start of self-driving tests in Pittsburgh. Uber has been running their lab for over a year, and had various vehicles out there mapping and gathering data, but their new vehicle is sleeker and loaded with sensors - more than on Google’s cars or most of the other research cars I have seen. You can see several lidars on the roof and bumpers, and a seriously big array of cameras and other sensors.
In addition, recently it was announced that the GM-Lyft-Cruise combination will be offering rides in 2017 in a self-driving Chevy Bolt. Of course, there will be a safety driver in the car supervising it so it would be an empty taxi coming to pick you up, but it’s a nice step.
These two announcements bring attention to two of the most important companies in the space, even though their technical efforts are much less mature than Google’s or Daimler’s. That’s because of one key forecast that I have emphasized from the start:
A large fraction of the automotive industry is going to switch to be about selling rides, not selling cars
As we all know, Uber has already become the #1 brand in the world in selling rides in just a few years. It’s a very important position to have. Lyft has #2 but other companies like Didi own China (and just got a $1B investment from Apple.)
As the owner of the ride brand, Uber has a lot of control. The brand of the car that drives you is less important and interchangeable. But that’s not the only advantage these ride companies have:
Ride companies have huge volumes of drivers on the road all day. They can be used as a resource for mapping, testing and verifying self-driving systems. Companies like Google had to pay staff and buy cars to do that.
Ride companies can combine human driven ride service with robotic taxis, to take you from anywhere to anywhere any time. It just costs more if you want to travel where the robots can’t.
Uber and Lyft can fail in their research program and still win. They just have to find somebody else to sell them the cars. Of course that does mean a power trade — it’s very nice to own the magic sauce that makes it all work, but the ride companies are among the few would could have another provider and still have a lot of control.
At the same time, Lyft is now bound to probably work with GM, and Didi possibly with Apple, which leaves Uber with more flexibility among these.
The ride companies are already doing big experiments in real ride-sharing, ie. multiple independent passengers in the same car. Today, using UberPool is popular and can save significant money. A more interesting question arises when robotic taxi service is available for 30 cents/mile. I don’t think people would share their ride to reduce the price from $1.50 to $1. Saving 50 cents does not move the needle for most people of even moderate income levels.
How will ride companies compete?
An important social question asks how many ride companies can compete in a market? Right now Uber has established a lot of dominance. In San Francisco, birthplace of Uber, Lyft and Sidecar, Sidecar shut its doors from difficulty competing with the other two. Is there room for only a few companies? That’s bad, because competition is good for the public.
The first intuition is that fleet size is a big competitive advantage because you can offer faster pickup times and more choices of vehicle. Customers will care a lot about how long they have to wait for a ride. That will vary of course based on random positions of vehicles, and also how good the predictive positioning is in the fleet management system.
At the same time, it is possible to have a successful limo company today with just one limo. You only do scheduled rides (or ad-hoc rides booked via networks like UberBlack) but you have a business. It is not the size of your fleet that fully governs your wait time, but rather the ratio of the size of your fleet to the number of customers you have. Lyft has a smaller fleet but also fewer users, so I find it can often match or beat Uber on wait time, though neither wins all the time. There is a natural balance here — the better your fleet-size/user-base ratio is, the shorter wait times you have, but that brings you more customers until the advantage starts reducing.
Today sees the un-stealthing of a new company called Otto which plans to build self-driving systems for long haul trucks. The company has been formed by a skilled team, including former members of Google’s car team and people I know well. You can see their opening blog post
My entire focus on this blog, and the focus of most people in this space, has been on cars, particularly cars capable of unmanned operation and door-to-door service. Most of those not working on that have had their focus on highway cars and autopilots. The highway is a much simpler environment so much easier to engineer, but it operates at higher speeds so the cost of accidents is worse.
That goes doubly true for trucks that are fast, big and massive. At the same time, 99% of truck driving is actually very straightforward — stay in a highway lane, usually the slow one, with no fancy moving about.
Some companies have done exploration of truck automation. Daimler/Freightliner has been testing trucks in Nevada. Volvo (trucks and cars together) has done truck and platooning experiments, notably the Sartre project some years ago. A recent group of European researchers did a truck demonstration in the Netherlands, leading up to the Declaration of Amsterdam which got government ministers to declare a plan to modify regulations to make self-driving systems legal in Europe. Local company Peloton has gone after the more tractable problem of two-truck platoons with a driver in each truck, aimed primarily at fuel savings and some safety increases.
While trucks are big and thus riskier to automate, they are also risky for humans to drive. Even though truck drivers are professionals who drive all day, there are still around 4,000 killed every year in the USA in truck accidents. More than half of those are truck drivers, but a large number of ordinary road users are also killed. Done well, self-driving trucks will reduce this toll. Just as with cars, companies will not release the systems until they believe they can match and beat the safety record of human drivers.
Self-driving trucks don’t change the way we move, but they will have a big economic effect on trucking. Driver pay accounts for about 25-35% of the cost of truck operation, but in fact early self-driving won’t take away jobs because there is a serious shortage of truck drivers in the market — companies can’t hire enough of them at the wages they currently pay. It is claimed that there are 50,000 job openings unfilled at the present time. Truck driving is grueling work, sometimes mind-numbing, and it takes the long haul driver away from home and family for over a week on every long-haul run. It’s not very exciting work, and it involves long days (11 hours is the legal limit) and a lot of eating and sleeping in truck stops or the cabin of the truck.
Average pay is about 36 cents/mile for a solo trucker on a common route. Alternately, loads that need to move fast are driven by a team of two. They split 50 cents/mile between them, but can drive 22 hours/day — one driver sleeps in the back while the first one takes the wheel. You make less per mile per driver, but you are also paid for the miles you are sleeping or relaxing.
A likely first course is trucks that keep their solo driver who drives up to 11 hours — probably less — and have the software drive the rest. Nonstop team driving speed with just one person. Indeed, that person might be an owner-operator who is paying for the system as a businessperson, rather than a person losing a job to automation. The human would drive the more complex parts of the route (including heavy traffic) while the system can easily handle the long nights and sparse heartland interstate roads.
The economics get interesting when you can do things that are expensive for human drivers and teams. Aside from operating 22 or more hours/day at a lower cost, certain routes will become practical that were not economic with human drivers, opening up new routes and business models.
Computer driven trucks will drive more regularly than humans, effectively driving in “hypermile” style as much as they can. That should save fuel. In addition, while I would not do it at first, the platooning experimented with by Peloton and Sartre does result in fuel savings. Also interesting is the ability to convert trucks to natural gas, which is domestic and burns cleaner (though it still emits CO2.) Automated trucks on fixed routes might be more willing to make this conversion.
There is strong potential to reduce the damage to roads (and thus the cost of maintaining them, which is immense and seriously in arrears) thanks to the robotruck. That’s because heavy trucks and big buses cause almost all the road wear today. A surprising rule of thumb is that road damage goes up with the 4th power of the weight per axle. As such an 80,000lb truck with 34,000lb on two sets of 2 axles and 6,000lb on the front axle does around 2,000 times the road damage of a typical car! read more »