Brad Templeton is Chairman Emeritus of the EFF, Singularity U founding computing faculty, software architect and internet entrepreneur, robotic car strategist, futurist lecturer, photographer and Burning Man artist.

This is an "ideas" blog rather than a "cool thing I saw today" blog. Many of the items are not topical. If you like what you read, I recommend you also browse back in the archives, starting with the best of blog section. It also has various "topic" and "tag" sections (see menu on right) and some are sub blogs like Robocars, photography and Going Green. Try my home page for more info and contact data.

E-mail is more secure than we think, we should use it

E-mail is facing a decline. This is something I lament, and I plan to write more about that general problem, but today I want to point out something that is true, but usually not recognized. Namely that E-mail today is often secure in transit, and we can make better use of that and improve it.

The right way to secure any messaging service is end-to-end. That means that only the endpoints — ie. your mail client — have the keys and encrypt or decrypt the message. It’s impossible, if the crypto works, for anybody along the path, including the operators of the mail servers as well as the pipes, to decode anything but the target address of your message.

We could have built an end-to-end secure E-mail system. I even proposed just how to do it over a decade ago and I still think we should do what I proposed and more. But we didn’t.

Along the way, though, we have mostly secured the individual links an E-mail follows. Most mail servers use encrypted SMTP over TLS when exchanging mail. The major web-mail programs like Gmail use encrypted HTTPS web sessions for reading it. The IMAP and POP servers generally support encrypted connections with clients. My own server supports only IMAPS and never IMAP or POP, and there are others like that.

What this means is that if I send a message to you on Gmail, while my SMTP proxy and Google can read that message, nobody tapping the wire can. Governments and possibly attackers can get into those servers and read that E-mail, but it’s not an easy thing to do. This is not perfect, but it’s actually pretty useful, and could be more useful.  read more »

Don't feed the radical right trolls by counter-protesting them

We’re all shocked at the idea of a growing neo-Nazi movement, at the horrible attack in Virginia and the lack of condemnation by the President. It’s making us forget that the neo-Nazi radical right are trolls with many parallels to online trolls. And the only thing to do is not to feed the trolls, and definitely don’t attack the civil rights that they make use of.

A protest march has 3 main functions:

  1. Get publicity for the cause
  2. Show those of similar mind that they are not alone and foster community
  3. Show the outside world that you have numbers

The first is the primary purpose. They don’t get very far shouting slogans at the people walking their dogs past their march. There are far better ways to get your message out today. The march works because people talk about it, write about it in the press, or even better, if they counter-protest it, vastly multiplying the publicity. Counter-protests are what any small radical group wants, not a quiet and peaceful rally. To do this they will be as outrageous as they can, to goad their opponents. The protest group wants to show they have numbers, but to show it they need publicity. In this case, they don’t have very large numbers.

A perfect example is the Phelps Westboro Baptist “Church.” They protest at funerals with offensive signs saying “God hates fags” and similar. They are not protesting the funerals. They do it only in the hope that people will get riled up and bring them tons of publicity, and it works. (Some even speculate that their goal is to get people to get so upset they assault them, and then they file court actions profitably!)

We have a strong urge not to leave something as pernicious as neo-nazis ignored. We feel we must show “this is not us.” And we want to show our numbers are large while theirs are small. But we know our numbers are large. But the attention given to them grows their numbers. Nobody joins a neo-nazi group imagining the “movement” is not reviled. They join because it is reviled by their ideological opponents. We must resist that urge. We must ignore them, and treat them as the irrelevant last adherents of a philosophy long left in the dustbin of history, which is what they are.

Radical groups have been around forever, and were not always trolls. But today’s online world gives them an alternative and better place to communicate, to promote their ideas, and to find strength in numbers, if they have numbers. As such the “march” is far less important for those things, and much more of its purpose is trolling.

There are those who feel that the approach of not feeding trolls is too simplistic or outdated and that there are other techniques that can work online. In the physical world, which is not a private playground like most online spaces, censorship is not permitted and only non-physical methods can be applied. This has frustated some people…

One alternative

I don’t pretend there can’t be ways to respond to trolls, as long as you’re sure you are not giving them the reaction they goaded you for. One town in Bavaria took pledges so that the more Nazis and the more they marched more money was donated to an anti-Nazi charity. In the USA, rather than a physical counter protest, giving money to Southern Poverty Law Center or others for every neo who shows up could be highly effective.

If you are skilled at it — really this is not for amateurs — mocking them can also work. They want people to be angry, not laughing, and especially not laughing at them. As noted, the comedy must be very good. Mean spirited comedy can get a laugh but is also a reaction of fear or anger.

Don’t attack civil rights

Nazis get people upset, naturally. My grandmother was a Jew out of Vitebsk, where the Einsatzgruppen B set up a base and murdered every Jew in the city, including all her relatives that did not emmigrate as her family fortunately did. I have reason to understand the horror of Nazi thinking. So I get really pissed when people start attacking the civil rights which belong to all, including neo-nazis, forcing me to defend them. I defend the rights, not the assholes, but often to others, and sometimes even to us, it can seem like we have to defend the scum.

If you think you want to call for a reduction in free speech for these scum, think again. It’s what they want. They would like nothing better than to have their marches banned, their web sites shut down. It’s publicity and makes them victims. And yes, they will laugh at those of us who will defend those rights.

Make no mistake. When our rights come under attack, there will be people defending them. It’s just a given, and even if you don’t agree with doing that, it’s the choice of the rights defenders, not your choice. It’s going to happen. Your only choice is whether to force those defenders into action. If you attack the free speech of scum, you are playing into their hands. It also won’t work, and is a bad idea, as I outlined in my article recently on Free Speech Theory.

If there’s a legal case over free speech or other rights of these folks, the ones who attacked those rights are the ones to blame. So don’t say it’s OK to censor or ban them. Don’t say it’s OK to punch them (if they have not been violent.) You might believe it, and in this post I am not arguing with you about that. I’m telling you what your choice will set in motion, and it’s not what you want.

Hard as it may be, as long as they are a small radical fringe, the best course is to ignore them. You won’t fix them by focusing the world on them. If they get to be big enough to be a real threat, you can pay some attention, but don’t help them get there.

Your eclipse guide (with the things not in many eclipse guides)

I will be heading to western Idaho this weekend to watch my sixth total Eclipse. That makes me a mid-grade eclipse chaser, so let me tell you some important things you need to know, which are not in some of the other eclipse guides out there. For good general sites look at places like NASA’s Eclipse Guide which has nice maps or this map.

Totality is everything

The difference between a total solar eclipse and a partial one — even a 98% partial one — is literally night and day. It’s like the difference between sex and holding hands. They are really two different things with a similar sounding name. And a lunar eclipse is again something vastly different. This does not mean a high-partial eclipse is not an interesting thing, but the total eclipse is by far the most spectacular natural phenomenon visible on this planet. Beyond the Grand Canyon, Yosemite, Norway, etc. So if you can get to totality, get there. Do not think you are seeing the eclipse if you don’t get into the zone of totality.

People debate about how total it should be

Many people seek to get close to the centerline of the eclipse. This provides the longest eclipse for your area. You will only lose a modest number of seconds if you are within 15 miles of the centerline, so you don’t have to get exactly there, and in fact it may be too crowded there.

On the other hand there are those who deliberately get close to the edge, giving up 30-40% of their eclipse time in order to see more “edge effects.” Near the edge, the edge effects are longer and a bit more spectacular. In particular the diamond ring will be a fair bit longer, and you may see more prominences and chromosphere for longer. If this is your first eclipse, I am not sure you want to get too close to the edge. But try any of the map web sites that will tell you your duration, and get somewhere that has within 30-40 seconds of the centerline time.

You look at the total eclipse with zero eye protection

You’ve been hearing endless talk about eclipse glasses and how well made they are. Eclipse glasses are only for the boring partial phase. They give you a way to track the progress of the moon while waiting for the main event. Once totality is over, everybody packs up and does not even bother to watch the 2nd half of the partial eclipse, that’s how boring the partial part is.

But don’t be one of those people who, told about the danger of eclipses, does not watch totality with your bare eyes. In fact, use binoculars in addition to your naked eyes, and perhaps a short look through a telescope — but not during the diamond rings or any partial phase.

Update: There is a nice large sunspot group that should still be there on Eclipse day, making the partial phase more interesting to those with good eyesight.

In totality you are looking not at the sun, but its amazing atmosphere — the “corona” — full of streamers, and many times the size of the sun or moon. You may also see jets of fire coming off the sun, and at the start and end of totality you will see the hot red inner atmosphere of the sun, known as the chromosphere.

If you are crazy enough to be outside the total zone but close to it, you still can’t look with your bare eyes at any part of the eclipse.

There are some cool things in a 99% partial eclipse (which you see just before and after totality.)

An eclipse is most glorious in the sky but a lot of other things happen around it. As it gets very close to total you will see the nature of the sunlight change and become quite eerie. Shadows of trees will turn into collections of crescents. About 20-60 seconds before and after totality, if you have a white sheet on the ground, you will see ripples of light waving, like on the bottom of a giant swimming pool. And the shadow. You will see it approach. If you are up on a mountain or in a plane this will be more obvious. It is going at 1,000 to 2,000 miles per hour.  read more »

Many different approaches to Robocar Mapping

Almost all robocars use maps to drive. Not the basic maps you find in your phone navigation app, but more detailed maps that help them understand where they are on the road, and where they should go. These maps will include full details of all lane geometries, positions and meaning of all road signs and traffic signals, and also details like the texture of the road or the 3-D shape of objects around it. They may also include potholes, parking spaces and more.

The maps perform two functions. By holding a representation of the road texture or surrounding 3D objects, they let the car figure out exactly where it is on the map without much use of GPS. A car scans the world around it, and looks in the maps to find a location that matches that scan. GPS and other tools help it not have to search the whole world, making this quick and easy.

Google, for example, uses a 2D map of the texture of the road as seen by LIDAR. (The use of LIDAR means the image is the same night and day.) In this map you see the location of things like curbs and lane markers but also all the defects in those lane markers and the road surface itself. Every crack and repair is visible. Just as you, a human being, will know where you are by recognizing things around you, a robocar does the same thing.

Some providers measure things about the 3D world around them. By noting where poles, signs, trees, curbs, buildings and more are, you can also figure out where you are. Road texture is very accurate but fails if the road is covered with fresh snow. (3D objects also change shape in heavy snow.)

Once you find out where you are (the problem called “localization”) you want a map to tell you where the lanes are so you can drive them. That’s a more traditional computer map, though much more detailed than the typical navigation app map.

Some teams hope to get a car to drive without a map. That is possible for simpler tasks like following a road edge or a lane. There you just look for a generic idea of what lane markings or road edges should look like, find them and figure out what the lanes look like and how to stay in the one you want to drive in. This is a way to get a car up and running fast. It is what humans do, most of the time.

Driving without a map means making a map

Most teams try to do more than driving without a map because software good enough to do that is also software good enough to make a map. To drive without a map you must understand the geometry of the road and where you are on it. You must understand even more, like what to do at intersections or off-ramps.

Creating maps is effectively the act of saying, “I will remember what previous cars to drive on this road learned about it, and make use of that the next time a car drives it.”

Put this way it seems crazy not to build and use maps, even with the challenges listed below. Perhaps some day the technology will be so good that it can’t be helped by remembering, but that is not this day.

The big advantages of the map

There are many strong advantages of having the map:

  • Human beings can review the maps built by software, and correct errors. You don’t need software that understands everything. You can drive a tricky road that software can’t figure out. (You want to keep this to a minimum to control costs and delays, but you don’t want to give it up entirely.)
  • Even if software does all the map building, you can do it using arbitrary amounts of data and computer power in cloud servers. To drive without a map you can must process the data in real time with low computing resources.
  • You can take advantage of multiple scans of the road from different lanes and vantage points. You can spot things that moved.
  • You can make use of data from other sources such as the cities and road authorities themselves.
  • You can cooperate with other players — even competitors — to make everybody’s understanding of the road better.

One intermediate goal might be to have cars that can drive with only a navigation map, but use more detailed maps in “problem” areas. This is pretty similar, except in database size, with automatic map generation with human input only on the problem areas. If your non-map driving is trustworthy, such that it knows not to try problem areas, you could follow the lower cost approach of “don’t map it until somebody’s car pulled over because it could not handle an area.”

Levels of maps

There are two or three components of the maps people are building, in order to perform the functions above. At the most basic level is something not too far above the navigation maps found in phones. That’s a vector map, except with lane level detail. Such maps know how many lanes there are, and usually what lanes connect to what lanes. For example, they will indicate that to turn right, you can use either of the right two lanes at some intersections.  read more »

Project Fi gives extra free Data Sims

Ten years ago, I asked Cell companies to let me have more than one phone on the same number. Recently I noticed the ability to almost do that with Google Project FI cell service.

Project FI is a cell service done (mostly) right. Low $20 basic cost, no phone subsidies or lock-in, fairly priced long distance, pay for data as you use it, roam on multiple networks, but most of all, the same $10/G data price in almost the whole world, at 3G and sometimes 4G speeds. It’s not perfect — it tends to roam with smaller carriers, it has problems sometimes overseas, and $10/G is a high price for those who use more than about 3GB/month of data.

I recently noticed a new feature — you can get up to ten extra data-only SIM cards at no extra cost. You pay for the data you use, as always with this plan. You can get a SIM for a tablet, but I think even more useful is the ability to pick up extra SIMs for your old phones. Now you can use these old phones as backup phones. Keep one in your car, to use if you forget your phone in the morning. Keep one at the office. Loan one when you travel.

The trick that makes these work as phones and not just data devices is project FI links your phone number into hangouts. If anybody calls or texts you, it also comes in via hangouts. So if you have decent data, you can take or make calls and SMS on the phone with the data-only SIM.

It’s not perfect. Sometimes calls over data suck. There is a small cost for the data for the voice — probably about 1 cent/minute. That’s actually a big savings over the one remaining roaming charge they have — 20 cents/minute.

Now my old phones can have a purpose. Old phones are old, though, so you want to strip out the apps you don’t need on a backup phone to keep them zippy. You will want to leave them in airplane-mode-wifi-on when not actively roaming with them so they don’t use your data plan sucking down emails and facebook.

BTW, if you are wondering if $10/GB is a reasonable price compared to plans that give 5 or 6GB in their basic $40-$50 price, I have found it to be quite reasonable at home. That’s because when I am home, I am on wifi in my house and most of the buildings I visit regularly. That keeps my data load down. When I travel, I try to go on wifi in the hotels but I also use the phone a lot more, and I will use 3-4GB in those months — but that’s a much better price than you will get from other carriers. T-mobile’s plan gives unlimited roaming but only at 2G speeds which are very low utility.

(BTW, while I really love the fact that incoming FI calls and texts go to hangouts and thus my desktop and laptop in addition to my extra phones, the Hangouts program is an absolutely terrible phone interface, lacking even redial and call-back and other very basic functions.)

Project FI is not a big network, and it’s other main flaw is that it only officially supports the Google phones like the Pixel and Nexus, though you can use other phones unofficially without quite as much roaming. But it’s also a great competitive push on the other cell phone companies to price in a manner that makes more sense. Cheers to them. (Disclaimer: I own some GOOG stock and am friends with Google management, and consulted there in the past.)

Keeping it in the USA

One serious flaw in FI, though, is that they don’t want people who live outside the USA to get it. One presumes they are not making or even losing money on the international roaming, and making it back inside. I spend 3 or more months a year outside the USA, so I may not be a profitable customer for them.

The problem is that to stop foreign users, they will only activate a SIM card if you are on the US networks. So if your phone fails while you are on a trip, and you have to replace it, or your SIM fails, you may be in a pile of trouble — you won’t be able to get and activate a new phone while on your trip. This hurt me when my Nexus 6 got unreliable on a trip. I had to put up with a crashing phone. It’s possible that had I begged they would have let me activate a new phone, but uncertain. Official policy is not. On the other hand, you are not entirely out of luck. You can get your new phone, put a local (ie. non-USA) SIM in it, and still get your calls and texts via hangouts, or via forwarding at some cost.

Car Rental: Rent me a cooler and lots of other gear for road trips

Something I do from time to time is a road trip in a rental car. And while car rental companies much prefer the business customer who rents a big car at a high price, then just drives it to their meeting and back to the airport, they are not averse to the less profitable road trip business.

So here are some things they could do to make it better for that sort of customer.

Cooler

When I road trip, I want a cooler to keep drinks and food cold. I will pack a fold-up cooler (they make good protection for stuff as well) but they can’t quite cut it in a hot car in a hot place all day.

So it would be nice if they would rent a nice solid cooler, maybe with some freezer packs. It’s easy to make your own freezer packs from used drink bottles, but also useful would be a leakproof bag to put gas station ice in. Those bags of ice you buy at stores always leak, so you need to protect against that. Depending on where you stay you can usually re-freeze packs or get ice from an ice machine.

Now a typical small cooler only costs about $30. And if I know car rental companies, they will charge more to rent you one, since they have to stock it and clean it. The big advantage of an overpriced rental car unit is time. You don’t want to spend time hunting these things down.

What would make sense would be a company which does “road trip provisions” which partners with all the rental car companies, and will either provision your car, or be located near the airport to make it easy for you to pick up and drop off.

Those 12v coolers that run off car electricity don’t do anything. You have to have ice. Though it could be a backup.

Another interesting option would be to partner with stores that do online ordering, so that you could get your cooler pre-loaded with ice and groceries. That’s of value in places you know well — in new countries I actually enjoy visiting a grocery store and picking out local products.

(No, I didn’t rent a Lotus Super Seven on my road trip to the hotel where the Prisoner was filmed, but somebody else drove one there and parked it to show off.)

Other travel supplies

Long ago, you could not depend on your hotel to have a hair dryer and an iron. So they sold travel versions of these that people packed. Nobody packs them any more. Now there are other things starting to be found in hotel rooms, but you still have to pack them because you can’t be sure. Some are things almost everybody wants. Some are specialty items. I would love for them to be on the inventory of a hotel, road trip rental company or car rental company:

For the road

  • A good tripod. I have one of course, but these are bulky things to bring. I can bring the tripod head — those vary too much.
  • Swim fins, masks, beach towels, beach mats and sun umbrellas — if going to a swimming/beach location.
  • Folding chairs for the same reason.
  • Camping gear is avilable for rent in some camping stores. Would be nice to get all this in one place, though.
  • Trekking poles. They make hikes better exercise and lower impact. I have collapsing ones but they are still a pain to fit in luggage and don’t go in carry-ons.
  • Coats. No, they won’t fit very well, but I’ve been on trips where you only needed a coat (or a heavier coat) in one location on the trip, and an ill-fitting coat woudl be better than carrying your own on the whole trip. Sometimes you can rent these — for example on Mt. Etna in Sicily, it’s the only place on the island anybody wants a coat in summer.

For the room

Earlier I wrote on what should be in every hotel room. Every room should just have those things, and the things below, but until then, trip gear rental service should have them too.

  • A universal power strip for the local plug style — because so many rooms don’t have enough plugs.
  • Tools for doing repairs. One of the most frustrating things on the road is breaking stuff and not having your complement of basic tools. Yeah, I pack a multi-tool, but I would also like a hot glue gun and other glues, a soldering iron and electrical repair tools with other useful things. Some Duct Tape, of course. And a tiny voltmeter.
  • For people who did not check a bag, a multi-tool, and a pair of scissors.
  • Video cables (HDMI, VGA, dongles) to let me connect my computer to the TVs in hotel rooms. Now 95% of them are flat-screen HD units, though often just 720p, and often fixed to the wall in a way that’s not useful as a computer monitor. But I don’t know what cable I will need or if I will be able to use it so it’s nice not to have to pack it.
  • For trips where you will stay a long time in one place, a nice large monitor is wonderful if you will be working on the computer. As I pretty much always will be. Recently on extended trip I bought one — just $120 — and sold it to the landlord who will provide it to future guests.
  • Alas, USB charging probably is still best for you to bring or rent. Problem is there are now 5 USB charging systems (Qualcomm QC 2 and 3, USB C, general high-current USB, basic USB and probably more) and everybody needs more than basic 2.5w charging, and to not have this is a complete non-starter.

What else would you add to the road trip rental inventory?

Can't we make overbooking more efficient and less painful with our mobile devices?

I’ve written before about overbooking and how it’s good for passengers as well as for the airlines. If we have a service (airline seats, rental cars, hotel rooms) where the seller knows it’s extremely likely that with 100 available slots, 20 will not show up, we can have two results:

  1. As soon as all 100 are taken, they declare it sold out. People reschedule or abandon trips, or at least take 2nd choices. However, the sold out plane takes off with 20 empty seats. If half of all flights sell out, you would find yourself blocked from taking the flight you want in 1 out of 10 of the flights you book on short notice.
  2. Instead of declaring it sold out, they oversell. When they mispredict, they find ways to compensate people to give up their reservation, or eventually, but ideally rarely or never, force some people to do so. A small fraction of people voluntarily yield, and a very tiny fraction do so involuntarily. (About 1 passenger in 10,000 faces involuntary bump, and it never happens to people who buy more expensive tickets or have status with the airline, unless they are late for the flight.)

I know which travel world I want to live in. To me that’s not the question. The question instead is, in the world where every traveler has a smartphone, why isn’t this done a lot better.

So many of our institutions were designed before the arrival of smartphones — it was just 10 years ago that they started taking over — and they are still the same today. In fact, many things are still designed for a world where people don’t even have email when they travel.

One step taken last week is United Airlines’ new Flex-Schedule program.

In this new program, first you must sign up and be a member of their Mileage Plus FF program. If you do, and you have already booked a seat on a flight that looks like it will be over-over-booked — ie. it will need volunteers — they will email you well in advance asking you to volunteer early. They promise an alternate flight with the same day and airport, and will compensate you you one of their travel vouchers. Those vouchers are not very valuable, of course, but this is what they offer.

When a flight is sure to overbook, the airline still sells tickets, but raises the price up high. This assures that the people buying last minute are very keen to get on that flight. They are premium customers, and unlikely to volunteer. The volunteers have to come from more price conscious customers who booked earlier. And so United realizes that most of those customers have e-mail and phones, and can respond quickly to such an offer.

I think they can go much further than this. Every airline, hotel and rental car should be tracking demand and supply at all times — which they already do. They are making models of just how many people are likely to really show for the flight. They track the history of passengers, to see if they are the sort who always shows or if they often change flights. They know the on-time records of all the feeder flights making connections, and on flight day, they know the actual times for those flights.

What they don’t have is information on the moods and locations of the passengers. Truth is, if I’m not going to use a seat, my phone probably knows that in advance, without me telling it. Or if I’m uncertain, my phone has the ability with notifications to get me to quickly confirm my intentions.

Yes, this means some minor loss of privacy — but actually very little. First of all, this is a very specific yielding of data, which is not nearly the sort of problem that allowing general access to data like your location is. Secondly, the airline doesn’t need to see your data at all. Instead, it can tell your phone, “If he’s not within 20 minutes of the airport by 3:30, ask him if he wants to give up his seat.” The airline does not learn where you are, just that you were not close to the airport. They just give the phone (or a server based tool that knows your location) the parameters of where you need to be if you will make the reservation.

In extreme cases, your phone can know where it is in the airport, and figure out if you have a chance of making the flight or not. But if it’s smart, it buzzed you long ago with the worry that you might not make it, perhaps providing an offer, or another flight if you’re an elite status passenger. You might not even waste the trip to the airport. Likewise, flights might be held slightly (or at least not takeoff early) if important passengers are known to be almost there.

Done well, you could very much love this for they might actually offer you a better deal if you tell them sooner you are going to miss the flight. With most tickets, you pay a penalty if you miss a flight or simply don’t take one, and they could make it smaller, in exchange for their ability to resell that seat sooner.

This is even more important with rental cars and hotel rooms. Unlike airline seats, most of these are reserved without penalty. You can no-show at no cost. Which means more no-shows. In airlines, the number of people who no-show and forfeit their ticket are few. Most of the no-shows are actually people who missed connections. The car rental companies get your flight number to know when you will land but also to know if you won’t make it at all. The hotels could also benefit in how they manage things.

I love the ability to make reservations without any penalty for not showing up. At the same time, that has to come with overselling to make sense, and so I’m willing to see slight reductions in my ability to do this. For example, I don’t think it’s unreasonable for me to have to re-confirm these reservations using my online connectivity. (Since phones and batteries and connections die, this can’t be 100% but it could still be pretty good.

No, you don't need to drive a billion miles to test a robocar

Earlier I noted that Nidi Kalra of Rand spoke at the AVS about Rand’s research suggesting that purely road testing robocars is an almost impossible task, because it would take hundreds of millions to a billion miles of driving to prove that a robocar is 10% better than human drivers.

(If the car is 10x better than humans, it doesn’t take that long, but that’s not where the first cars will be.)

This study has often been cited as saying that it’s next to impossible to test robocars. The authors don’t say that — their claim is that road testing will not be enough, and will take too long to really work — but commenters and press have taken it further to the belief that we’ll never be able to test.

The mistake is that while it could take a billion miles to prove a vehicle is 10% safer than human drivers, that is not the goal. Rather, the goal is to decide that it’s unlikely it is much worse than that number. It may seem like “better than X” and “not worse than X” are the same thing, but they are not. The difference is where you give the benefit of the doubt.

Consider how we deal with new drivers. We give them a very basic test and hand them a licence. We presume, because they are human teens, that they will have a safety record similar to other human teens. Such a record is worse than the level for experienced drivers, and in fact one could argue it’s not at all safe enough, but we know of no way to turn people into experienced drivers without going through the risky phase.

If a human driver starts showing evidence of poor skills or judgments — lots of tickets, and in particular multiple accidents, we pull their licence. It actually takes a really bad record for that to happen. By my calculations the average human takes around 20 years to have an accident that gets reported to insurance, and 40-50 years to have one that gets reported to police. (Most people never have an injury accident, and a large fraction never have any reported or claimed accident.)  read more »

Federal regulations past next hurdle

Today’s news is preliminary, but a U.S. house committee panel passed some new federal regulations which suggest sweeping change in the US regulatory approach to robocars.

Today, all cars sold must comply with the Federal Motor Vehicles Safety Standards (FMVSS.) This is a huge set of standards, and it’s full of things written with human driven cars in mind, and making a radically different vehicle, like the Zoox, or the Waymo Firefly, or a delivery robot, is simply not going to happen under those standards. There is a provision where NHTSA can offer exemptions but it’s in small volumes, for prototype and testing vehicles mostly. The new rules would allow a vendor to get an exemption to make 100,000 vehicles per year, which should be enough for the early years of robocar deployment.

Secondly, these and other new regulations would preempt state regulations. Most players (except some states) have pushed for this. Many states don’t want the burden of regulating robocar design, since they don’t have the resources to do so, and most vendors don’t want what they call a “patchwork” of 50 regulations in the USA. My take is different. I agree the cost of a patchwork is not to be ignored, but the benefits of having jurisdictional competition may be much greater. When California proposed to ban vehicles like the Google Firefly, Texas immediately said, “Come to Texas, we won’t get in your way.” That pushed California to rethink. Having one regulation is good — but it has to be the right regulation, and we’re much too early in the game to know what the right regulation is.

This is just a committee in the house, and there is lots more distance to go, including the Senate and all the other usual hurdles. Whatever people thought about how much regulation there should be, everybody has known that the FMVSS needs a difficult and complex revision to work in the world of robocars, and a temporary exemption can be a solution to that.

Uncovered: NHTSA Levels of 1900 (Satire)

I have recently managed to dig up some old documents from the earliest days of car regulation. Here is a report from NHTSA on the state of affairs near the turn of the 20th century.

National Horse Trail Safety Administration (NHTSA)

Regulation of new Horse-Auto-mobile Vehicles (HAV), sometimes known as “Horseless carriages.”

In recent years, we’ve seen much excitement about the idea of carriages and coaches with the addition of “motors” which can propel the carriage without relying entirely on the normal use of horses or other beasts of burden. These “Horseless carriages,” sometimes also known as “auto mobile” are generating major excitement, and prototypes have been generated by men such as Karl Benz and Armand Peugeot, along with the Duryea brothers, Ransom Olds and others in the the USA. The potential for these carriages has resulted in many safety questions and many have asked if and how NHTSA will regulate safety of these carriages when they are common.

Previously, NHTSA released a set of 4, and later 5 levels to classify and lay out the future progression of this technology.

Levels of Motorized Carriages

Level 0

Level zero is just the existing rider on horseback.

Level 1

Level one is the traditional horse drawn carriage or coach, as has been used for many years.

Level 2

A level 2 carriage has a motor to assist the horses. The motor may do the work where the horses trot along side, but at any time the horses may need to take over on short notice.

Level 3

In a level 3 carriage, sometimes the horses will provide the power, but it is allowed to switch over entirely to the “motor,” with the horses stepping onto a platform to avoid working them. If the carriage approaches an area it can’t handle, or the motor has problems, the horses should be ready, with about 10-20 seconds notice, to step back on the ground and start pulling. In some systems the horse(s) can be in a hoist which can raise or lower them from the trail.

Level 4

A Level 4 carriage is one which can be pulled entirely by a motor in certain types of terrain or types of weather — an operating domain — but may need a horse at other times. There is no need for a sudden switch to the horses, which should be pulled in a trailer so they can be hitched up for travel outside the operating domain.

Level 5

The recently added fifth level is much further in the future, and involves a “horseless” carriage that can be auto mobile in all situations, with no need for any horse at all. (It should carry a horse for off-road use or to handle breakdowns, but this is voluntary.)  read more »

News and commentary from AUVSI/TRB Automated Vehicle Symposium 2017

In San Francisco, I’m just back from the annual Automated Vehicle Symposium, co-hosted by the AUVSI (a commercial unmanned vehicle organization) and the Transportation Research Board, a government/academic research organization. It’s an odd mix of business and research, but also the oldest self-driving car conference. I’ve been at every one, from the tiny one with perhaps 100-200 people to this one with 1,400 that fills a large ballroom.

Toyota Research VC Fund

Tuesday morning did not offer too many surprises. The first was an announcement by Toyota Research Institute of a $100M venture fund. Toyota committed $1B to this group a couple of years ago, but surprisingly Gil Pratt (who ran the DARPA Robotics Challenge for humanoid-like robots) has been somewhat a man of mixed views, with less optimistic forecasts.

Different about this VC fund will be the use of DARPA like “calls.” The fund will declare, “Toyota would really like to see startups solving problem X” and then startups will apply, and a couple will be funded. It will be interesting to see how that pans out.

Nissan’s control room is close to live

At CES, Nissan showed off their plan to have a remote control room to help robocars get out of sticky situations they can’t understand like unusual construction zones or police directing traffic. Here, they showed it as further along and suggested it will go into operation soon.

This idea has been around for a while (Nissan based it on some NASA research) and at Starship, it has always been our plan for our delivery robots. Others are building such centers as well. The key question is how often robocars need to use the human assistance, and how you make sure that unmanned vehicles stay in regions where they can get a data connection through which to get help. As long as interventions are rare, the cost is quite reasonable for a larger fleet.

This answers the question that Rod Brooks (of Rethink Robotics and iRobot) recently asked, pondering how robocars will handle his street in Cambridge, where strange things like trucks blocking the road to do deliveries, are frequently found.

It’s a pretty good bet that almost all our urban spaces will have data connectivity in the 2020s. If any street doesn’t have solid data, and has frequent bizarre problems of any type, yet is really important for traversal by unmanned vehicles — an unlikely trifecta — it’s quite reasonable for vehicle operators to install local connectivity (with wifi, for example) on that street if they can’t wait for the mobile data companies to do it. Otherwise, don’t go down such streets in empty cars unless you are doing a pickup/drop-off on the street.

Switching Cities

Karl Iagenemma of nuTonomy told the story of moving their cars from Singapore, where driving is very regulated and done on the left, to Boston where it is chaotic and done on the right.  read more »

Can we test robocars the way we tested regular cars?

I’ve written a few times that perhaps the biggest unsolved problem in robocars is how to know we have made them safe enough. While most people think of that in terms of government certification, the truth is that the teams building the cars are very focused on this, and know more about it than any regulator, but they still don’t know enough. The challenge is going to be convincing your board of directors that the car is safe enough to release, for if it is not, it could ruin the company that releases it, at least if it’s a big company with a reputation.

We don’t even have a good definition of what “safe enough” is though most people are roughly taking that as “a safety record superior to the average human.” Some think it should be much more, few think it should be less. Tesla, now with the backing of the NTSB, has noted that their autopilot system — combined with a mix of mostly attentive but some inattentive humans, may have a record superior to the average human, for example, even though with the inattentive humans it is worse.

Last week I attended a conference in Stuttgart devoted to robocar safety testing, part of a larger auto show including an auto testing show. It was interesting to see the main auto testing show — scores of expensive and specialized machines and tools that subject cars to wear and tear, slamming doors thousands of times, baking the surfaces, rattling and vibrating everything. And testing the electronics, too.

In Europe, the focus of testing is very strongly on making sure you are compliant with standards and regulations. That’s true in the USA but not quite as much. It was in Europe some time ago that I learned the word “homologation” which names this process.

There is a lot to be learned from the previous regimes of testing. They have built a lot of tools and learned techniques. But robocars are different beasts, and will fail in different ways. They will definitely not fail the way human drivers do, where usually small things are always going wrong, and an accident happens when 2 or 3 things go wrong at once. The conference included a lot of people working on simulation, which I have been promoting for many years. The one good thing in the NHTSA regulations — the open public database of all incidents — may vanish in the new rules, and it would have made for a great simulator. The companies making the simulators (and the academic world) would have put every incident into a shared simulator so every new car could test itself in every known problem situation.

Still, we will see lots of simulators full of scenarios, and also ways to parameterize them. That means that instead of just testing how a car behaves if somebody cuts it off, you test what it does if it gets cut off with a gap of 1cm, or 10cm, or 1m, or 2m, and by different types of vehicles, and by two at once etc. etc. etc. The nice thing about computers is you can test just about every variation you can think of, and test it in every road situation and every type of weather, at least if your simulator is good enough,

Yoav Hollander, who I met when he came as a student to the program at Singularity U, wrote a report on the approaches to testing he saw at the conference that contains useful insights, particularly on this question of new and old thinking, and what regulations drive vs. liability and fear of the public. He puts it well — traditional and certification oriented testing has a focus on assuring you don’t have “expected bugs” but is poor at finding unexpected ones. Other testing is about finding unexpected bugs. Expected bugs are of the “we’ve seen this sort of thing before, we want to be sure you don’t suffer from it” kind. Unexpected bugs are “something goes wrong that we didn’t know to look for.”

Avoiding old thinking

I believe that we are far from done on the robocar safety question. I think there are startups who have not yet been founded who, in the future, will come up with new techniques both for promoting safety and testing it that nobody has yet thought of. As such, I strongly advise against thinking that we know very much about how to do it yet.

A classic example of things going wrong is the movement towards “explainable AI.” Here, people are concerned that we don’t really know how “black box” neural network tools make the decisions they do. Car regulations in Europe are moving towards banning software that can’t be explained in cars. In the USA, the draft NHTSA regulations also suggest the same thing, though not as strongly.

We may find ourselves in a situation where we take to systems for robocars, one explainable and the other not. We put them through the best testing we can, both in simulator and most importantly in the real world. We find the explainable system has a “safety incident” every 100,000 miles, and the unexplainable system has an incident every 150,000 miles. To me it seems obvious that it would be insane to make a law that demands the former system which, when deployed, will hurt more people. We’ll know why it hurt them. We might be better at fixing the problems, but we also might not — with the unexplainable system we’ll be able to make sure that particular error does not happen again, but we won’t be sure that others very close it it are eliminated.

Testing in sim is a challenge here. In theory, every car should get no errors in sim, because any error found in sim will be fixed or judged as not really an error or so rare as to be unworthy of fixing. Even trained machine learning systems will be retrained until they get no errors in sim. The only way to do this sort of testing in sim will be to have teams generate brand new scenarios in sim that the cars have never seen, and see how they do. We will do this, but it’s hard. Particularly because as the sims get better, there will be fewer and fewer real world situations they don’t contain. At best, the test suite will offer some new highly unusual situations, which may not be the best way to really judge the quality of the cars. In addition, teams will be willing to pay simulator companies well for new and dangerous scenarios in sim for their testing — more than the government agencies will pay for such scenarios. And of course, once a new scenario displays a problem, every customer will fix it and it will become much less valuable. Eventually, as government regulations become more prevalent, homologation companies will charge to test your compliance rate on their test suites, but again, they will need to generate a new suite every time since everybody will want the data to fix any failure. This is not like emissions testing, where they tell you that you went over the emissions limit, and it’s worth testing the same thing again.

The testing was interesting, but my other main focus was on the connected car and security sessions. More on that to come.

New more laissez-faire robocar rules may arise

While very few details have come out, Reuters reports that new proposed congressional bills on self-driving cars will reverse many of the provisions I critiqued in the NHTSA regulations last year.

One big change is a reversal of the new idea of pre-market regulation. Today, new car technologies are not regulated before they are deployed, but NHTSA proposed giving itself the power to regulate technologies even before they exist. Currently most car technologies like adaptive cruise control, autopilots, forward collision avoidance, lanekeeping and the like remain unregulated after a decade or more of deployment with few, if any, problems.

This is important because the old doctrine of “We don’t regulate until we see a problem the industry own’t fix on its own” is a much better one for innovation, and the speed of innovation is key in deciding which countries and companies lead this technology. The opposite approach of “we try to imagine what might go wrong and ban it ahead of time” may seem safer, but it’s definitely an impediment to innovation and may actually result in far more deaths through the delay of life-saving technologies.

Harder to judge is the preemption of state rules. While states are also attempting to pre-regulate, having a laboratory of 50 different competing states can also be good for innovation on the legal side. There is not one answer, and while it’s more complex to deal with 50 sets of regulations instead of one, it’s not that much more complex.

One of the few interesting and good ideas in the NHTSA regs may also vanish. NHTSA wanted all vendors to make available all sensor logs from all incidents. As I predicted, companies pushed back on this — their testing logs and the resulting test suites are very important competitive assets. The company with the best test suite is the furthest on the path to the safety needed for deployment. On the other hand, sharing this data would let everybody get further on that path, faster.

There has been lots of other news during the long road-trip I am on in Europe. This includes more entrants in the race, the retirement of Google’s 3rd generation “koala” car, lots more at Uber and more. Plus I will report from the Autonomous Car Testing and Development conference in Stuttgart starting Tuesday.

The DSRC/V2V/Connected Car Emperor has no clothes

Plans are underway to ask for a legal mandate to install radio communications devices in all new cars, starting around 2020. These radios would do “vehicle to vehicle” (v2v) and vehicle to infrastructure communication using a wifi-derived protocol called DSRC.

These plans began long ago, when all of us wondered, “wouldn’t it be cool if computers in cars could talk to other cars?” It seemed like it should be cool but in fact, after decades of trying, very few useful applications have actually shown up. However, that has not stopped fans of the idea. They had almost given up when robocars came along. As the hype built over robocars they realized that they might have an application there, and this application could make their solution finally find its problem. Since then, there have been many declarations that V2V communication is important or even essential for robocars. That what this is all really about is the “connected car.” Whole conferences and industry groups push heavily on the connected car concept.

Of course robocars will be connected, but barely. They will want updates to maps and on road conditions and events — the same things you see if you run programs like Waze. When parked, they will also want updates to their software and more detailed map data. But no sane designer plans to have them depend on real time connectivity. It might provide useful information, but it often won’t, and you need to depend on things you built and tested that work 100% of the time. Everything else is just a little gravy.

I have written many comments on the issues with v2v and related technologies. Recently, the DSRC fans got a proposal in place for the government to mandate all new cars come with DSRC radios which will, among other things, constantly broadcast their position and what they are doing. The government will mandate a decade old radio technology that is already obsolete, and which probably will never work, and certainly won’t work as well as other technologies which are arriving without government help in mobile phones and data networks.

There was a comment period. I wrote up a commentary, and have expanded it into an essay on:

Why the V2V “emperor” has no clothes.

Connected Autonomous Vehicles — Pick 2.

Waymo starts pilot in Phoenix, Apple gets more real and other news

Waymo (Google) has announced a pilot project in Phoenix offering a full ride service, with daily use, in their new minivans. Members of the public can sign up — the link is sure to be overwhelmed with applicants, but it has videos and more details — and some families are already participating. There’s also a Waymo Blog post. I was in Phoenix this morning as it turns out, but to tell real estate developers about robocars, not for this.

There are several things notable about Waymo’s pilot:

  1. They are attempting to cover a large area — they claim twice the size of San Francisco, or 90 square miles. That’s a lot. It’s enough to cover the vast majority of trips for some pilot users. In other words, this is the first pilot which can test what it’s like to offer a “car replacement.”
  2. They are pushing at families, which means even moving children, including those not of driving age. The mother in the video expects to use it to send some children to activities. While I am sure there will be safety drivers watching over things, trusting children to the vehicles is a big milestone. Google’s safety record (with safety drivers) suggests this is actually a very safe choice for the parents, but there is emotion over trusting children to robots (other than the ones that go up and down shafts in buildings.)
  3. In the videos, they are acting like there are no safety drivers, but there surely are, for legal reasons as well as safety.
  4. They are using the Pacifia minivans. The Firefly bubble cars are too slow for anything but neighbourhood operation. The minivans feature motorized doors, a feature which, though minor and commonplace, meets the image of what you want from a self-driving car.

Apple is in the game

There has been much speculation recently because of some departures from Apple’s car team that they had given up. In fact, last week they applied for self-driving car test plates for California. I never thought they had left the game.  read more »

How to do a low bandwidth, retinal resolution video call

Not everybody loves video calls, but there are times when they are great. I like them with family, and I try to insist on them when negotiating, because body language is important. So I’ve watched as we’ve increased the quality and ease of use.

The ultimate goals would be “retinal” resolution — where the resolution surpasses your eye — along with high dynamic range, stereo, light field, telepresence mobility and VR/AR with headset image removal. Eventually we’ll be able to make a video call or telepresence experience so good it’s a little hard to tell from actually being there. This will affect how much we fly for business meetings, travel inside towns, life for bedridden and low mobility people and more.

Here’s a proposal for how to provide that very high or retinal resolution without needing hundreds of megabits of high quality bandwidth.

Many people have observed that the human eye is high resolution on in the center of attention, known as the fovea centralis. If you make a display that’s sharp where a person is looking, and blurry out at the edges, the eye won’t notice — until of course it quickly moves to another section of the image and the brain will show you the tunnel vision.

Decades ago, people designing flight simulators combined “gaze tracking,” where you spot in real time where a person is looking with the foveal concept so that the simulator only rendered the scene in high resolution where the pilot’s eyes were. In those days in particular, rendering a whole immersive scene at high resolution wasn’t possible. Even today it’s a bit expensive. The trick is you have to be fast — when the eye darts to a new location, you have to render it at high-res within milliseconds, or we notice. Of course, to an outside viewer, such a system looks crazy, and with today’s technology, it’s still challenging to make it work.

With a video call, it’s even more challenging. If a person moves their eyes (or in AR/VR their head) and you need to get a high resolution stream of the new point of attention, it can take a long time — perhaps hundreds of milliseconds — to send that signal to the remote camera, have it adjust the feed, and then get that new feed back to you. There is no way the user will not see their new target as blurry for way too long. While it would still be workable, it will not be comfortable or seem real. For VR video conferencing it’s even an issue for people turning their head. For now, to get a high resolution remote VR experience would require sending probably a half-sphere of full resolution video. The delay is probably tolerable if the person wants to turn their head enough to look behind them.

One opposite approach being taken for low bandwidth video is the use of “avatars” — animated cartoons of the other speaker which are driven by motion capture on the other end. You’ve seen characters in movies like Sméagol, the blue Na’vi of the movie Avatar and perhaps the young Jeff Bridges (acted by old Jeff Bridges) in Tron: Legacy. Cartoon avatars are preferred because of what we call the Uncanny Valley — people notice flaws in attempts at total realism and just ignore them in cartoonish renderings. But we are now able to do moderately decent realistic renderings, and this is slowly improving.

My thought is to combine foveal video with animated avatars for brief moments after saccades and then gently blend them towards the true image when it arrives. Here’s how.

  1. The remote camera will send video with increasing resolution towards the foveal attention point. It will also be scanning the entire scene and making a capture of all motion of the face and body, probably with the use of 3D scanning techniques like time-of-flight or structured light. It will also be, in background bandwidth, updating the static model of the people in the scene and the room.
  2. Upon a saccade, the viewer’s display will immediately (within milliseconds) combine the blurry image of the new target with the motion capture data, along with the face model data received, and render a generated view of the new target. It will transmit the new target to the remote.
  3. The remote, when receiving the new target, will now switch the primary video stream to a foveal density video of it.
  4. When the new video stream starts arriving, the viewer’s display will attempt to blend them, creating a plausible transition between the rendered scene and the real scene, gradually correcting any differences between them until the video is 100% real
  5. In addition, both systems will be making predictions about what the likely target of next attention is. We tend to focus our eyes on certain places, notably the mouth and eyes, so there are some places that are more likely to be looked at next. Some portion of the spare bandwidth would be allocated to also sending those at higher resolution — either full resolution if possible, or with better resolution to improve the quality of the animated rendering.

The animated rendering will, today, both be slightly wrong, and also suffer from the uncanny valley problem. My hope is that if this is short lived enough, it will be less noticeable, or not be that bothersome. It will be possible to trade off how long it takes to blend the generated video over to the real video. The longer you take, the less jarring any error correction will be, but the longer the image is “uncanny.”

While there are 100 million photoreceptors in the whole eye, but only about a million nerve fibers going out. It would still be expensive to deliver this full resolution in the attention spot and most likely next spots, but it’s much less bandwidth than sending the whole scene. Even if full resolution is not delivered, much better resolution can be offered.

Stereo and simulated 3D

You can also do this in stereo to provide 3D. Another interesting approach was done at CMU called pseudo 3D. I recommend you check out the video. This system captures the background and moves the flat head against it as the viewer moves their head. The result looks surprisingly good.  read more »

Luminar unstealths their 1.5 micron LIDAR

Luminar, a bay area startup, has revealed details on their new LIDAR. Unlike all other commercial offerings, this is a LIDAR using 1.5 micron infrared light. They hope to sell it for $1,000.

1.5 micron LIDAR has some very special benefits. The lens of your eye does not focus medium depth infrared light like this. Ordinary light, including the 0.9 micron infrared light of the lasers in most commercial LIDARS is focused to a point by the lens. That limits the amount of power you can put in the laser beam, because you must not create any risk to people’s eyes.

Because of this, you can put a lot more power into the 1.5 micron laser beam. That, in turn, means you can see further, and collect more points. You can easily get out to 250 meters, while regular lidars are limited to about 100m and are petering out there.

What doesn’t everybody use 1.5 micron? The problem is silicon sensors don’t react to this type of light. Silicon is the basis of all mass market electronics. To detect 1.5 micron light, you need different materials, which are not themselves that hard to find, but they are not available cheap and off the shelf. So far, this makes units like this harder to build and more expensive. If Luminar can do this, it will be valuable.

Why do you need to see 250m? Well, you don’t for city driving, though it’s nice. For highway driving, you can get by with 100m as well, and you use radar to help you perceive, at very low resolution, what’s going on beyond that. Still, there are things that radar can’t tell you. Rare things, but still important. So you need a sensor that sees further to spot things like stalled cars under bridges. Radar sees those, but can’t tell them from the bridge.

To this point, Google has been the only company to say they have a long range LIDAR, but it has not been for sale. And as we all know, there is a famous lawsuit underway accusing Uber/Otto of copying Google’s LIDAR designs.

The Luminar point clouds are impressive. This will be a company to watch. (In the interests of disclosure, I am an advisor to Quanergy, another LIDAR startup.)

United Part 2: Misconceptions and realities

There’s a lot of bad information circulating on the famous United/Republic “passenger drag” so I wanted to consolidate a 2nd post with some of them.

Myth: This was an oversold flight

It turns out the flight was probably not oversold. A UA spokesman said it wasn’t. It was a fully sold flight, but a sudden need arose to move 4 flight attendants to SDF (Louisville) and they arrived at the gate after the flight had boarded. In United’s contract of carriage, it defines an oversold flight as a flight where there are more passengers with confirmed reservations checked in by the check-in deadline than they have seats on the plane. That does not appear to be the case on this flight, but Republic and UA got confused about it.

That, in turn, means Republic did not have the right to invoke the clauses of the contract for oversold flights. If so, they are just plain in the wrong, and this becomes a case with far less interesting nuance. United has changed their tune (of course due to public pressure) and are going full mea culpa.

Airline reservation computers oversell all the time, and carefully calculate exactly how much to oversell. It looks like the algorithms decided to not oversell this flight. And they were right — when they called for volunteers, nobody accepted, even at a very high price ($800 to $1,000) for a flight where most tickets are under $200. The algorithms performed perfectly.

Myth: This was United Airlines

Technically it was Republic Airline, a small regional airline dba “United Express.” However, United sells and and manages the tickets and they use the brand, and it’s under United’s contract, so United certainly gets a lot of the responsibility. And I am impressed that UA has not tried to throw Republic under the bus here.

Republic Airways actually operates lots of regional flights for United, AA and Delta, so this could have probably happened to any of them. I don’t know if they have a lot of airline specific training on bumping procedure for their teams. United may have just gotten some very bad luck of the draw here — and then made it worse by defending it at first. And it may be that the bumping policies UA gives to Republic might have made this more likely than the ones Delta and AA give it, but I don’t think they are tremendously different. Some hinges on whether the flight crew was a Republic crew, or a United crew.

But still, though it was not United, the buck stops with United, and at least now, they are not resisting that at all.

Myth: On an oversold flight, they can pull passengers off the plane.

If this had been an oversold flight, their contract still does not let them remove passengers from the plane involuntarily. It says they can “deny boarding.” Deny boarding does not mean remove — there is another section of the contract on removal. More bad news for United/Republic, but again, it makes the case less interesting as it’s an example of something you sort of expect — junior employees of a regional affiliate not being properly trained on what to do in an unusual situation and thus screwing up. That happens in 100 different ways all the time, but each particular incident is rare and probably does not indicate a systemic problem. That’s good — but it is only systemic problems that are of interest to the public, and which would make you boycott a company. If the junior employees make mistakes like this too often, then you have a systemic problem to worry about. (United does not have a good reputation on this count, of course.)

Update: These flight attendants were “must ride” passengers

New information reveals the flight crew declared themselves “must ride.” I don’t have a lot of details, but this is a special designation in the law (not the UA contract) which declares the crew are needed somewhere to avoid cancellation of a flight. Once a passenger is declared “must ride” the plane is required, reports say, to do everything possible to get that passenger to their destination, including delaying the plane and apparent, yes, even involuntary bumping. I am waiting for more information on this status, which would invalidate partly what I say above. They can’t pull you for an oversell, but they may be able to pull you for a must-ride. The law is there to keep the aviation system humming. Once flight crews don’t get to flights, it can mean disruption to more than just that flight.

Myth: If the doctor had just handed one of the police officers a Pepsi, it all would have been defused.

No, but that’s the best joke on this that I’ve seen.

Myth: It’s overselling that’s the problem

With the mistaken impression that overselling was the cause here, a lot of people are stating overselling is evil, and Chris Christie has even called to prohibit it. That’s a big mistake. Overselling is very good for airlines and the flying public. I explained that in yesterday’s post but I will go into more details below. You want an airline that does at least some overselling, though one can debate how much you want.

Myth: The airline prioritized employees over paying customers

In this situation, it needed to move those employees to crew a flight first thing out of SDF. If they had not gotten a crew there, that flight gets cancelled. Roughly 70 paying passengers get stranded against their will. While clearly nobody wants to be stranded against their will, the hard truth is you want to fly an airline that will strand (with good compensation) 4 people to avoid doing it to 70. (Here I am talking about the normal approach, which is to deny boarding to 4 people, not to try do drag people off the plane.) Still, I have to view it as prioritizing 70 passengers over 4, not employees over passengers.

Maybe: They could have driven the flight attendants there or chartered a jet

This is possibly true but possibly not. First of all, these airlines are all about procedure. They don’t authorize junior employees to be innovative or authorize them to spend money. So chances are if somebody thought of that, they had no system with which to do it. That is a fault of the airline but the sad norm of corporate bureaucracy.

Secondly, while I don’t know this to be true, all flight crew operate under a set of complex rules about required rest. You don’t want a sleepy pilot landing your plane, or a sleepy flight attendant helping people get onto the evacuation slides. These rules are very hard and fast. I suspect trying to sleep in a car doesn’t count, and an overnight ground ride is out of the question. Had they acted very quickly, and had a system in place, they could have probably gotten the crew there a bit after 11 — not long after the flight actually landed due to the chaos — so that might have worked, in hindsight.

They could have offered the passengers a limo ride, but again probably had no way to do something that out of the ordinary.

The same applies to an air charter. Getting an air charter on short notice is difficult, but they could have gotten one for the flight crew (or another flight crew) in the early morning if they had a system in place. This is very expensive of course, and so not likely to be in their playbook.

Maybe: They should never have gotten to the point where it was so urgent to get that flight crew moved

Airlines move flight crews a lot. There are airline pilots who live on one coast and mostly work on the other, commuting by “deadheading” on one of their airlines planes.

When you design a system that needs various parts — planes and flight crew — you have to “overprovision,” which is to say leave some wiggle room. That means you have some number of planes, crews and other resources sitting idle or on call, and you use them when something else fails. Everybody does it because you don’t want to run so close to the wire all the time. If you do, the slightest problem causes a cascade of cancellations. Airlines have to worry not just about small problems but even big ones like storms that cancel or delay many flights.

It’s not practical, however, to overprovision to the point that you never fail. You can do it, but it’s really expensive. You have to waste a lot of money, and you don’t have a competitive company. So every systems designer tries to figure out how to overprovision just the right amount. An amount that will have a few failures, but not too many. On top of that, you try to plan so you handle those failures with the least amount of pain, but you accept they will still happen.

What that means — and I don’t have any specific facts about this flight — is that sometimes you will be skating the edge, and sometimes you will fail. Sometimes you will find that crew are not going to make a flight unless you do something a little extra.

The bumping law, I think, is where the airlines find their “extra.” They don’t want to bump paying customers — it’s expensive and hurts customer relations. But they don’t want to cancel flights even more. So every so often, every airline has to find a solution. The bumping law offers them that solution. They can legally deny boarding to paying passengers against their will to make room for crew. This is much more workable, and under their control, than other options like using charter jets, or if distances are short, ground service.

True, but: Just about anything would be cheaper than the hit they’ve taken

That’s true — but only in hindsight. No playbook for these situations is going to say, “If you have to, spend $10,000 rather than bumping passengers just in case it turns into the PR nightmare of the year.” By definition, nobody knew that would happen.

In reality, airlines involuntarily bump 50,000 pax per year and while they grumble, this is the first time it’s ever gone done like this, with eviction from the plane, blood, camera phones and Facebook. So I don’t blame them for not seeing this could happen. I do blame them, however, for not understanding that any time you bring the police into a situation you bump the risk of something bad happening.

True, but: They should have known this would happen once they called the goons.

They should have known, but I can suspect why they didn’t — because they actually do this all the time and don’t have PR problems. Flight crews face unruly passengers reasonably often. They have training for it and procedures. And those procedures do call for getting the police, even knowing how that can go south. What those plans obviously did not account for was doing this when it was completely clear the passenger was the victim, that they only removed him because they wanted his seat. The rules for removing passengers mostly deal with safety issues. When they declare a passenger a safety risk, and the passenger makes trouble and even (rarely) causes a scuffle they are protected if the passenger was really a safety risk, or they can even come up with a credible lie why they thought he was a safety risk. No such story is possible here. Sure, the law says anybody who refuses a flight crew order can be removed from the plane. Technically it says this. In reality, it’s insane to think you can remove somebody for refusing the order “leave the plane” when the order is not given for a valid reason. The law says obey, but every sense of justice goes the other way. In fact, more than that, I don’t think a court would convict somebody for refusing that order, even if they are guilty, because society does not intend to grant the airlines that sort of power.

Put another way, three things are true:

  1. They can’t order you off the plane just to take your seat (but they didn’t know that.) We don’t want airlines to have that power.
  2. Once somebody refuses a flight crew order, you can then order them off the plane.

As such, it’s clear that “we removed him because he disobeyed our order to leave” is a loophole that would never stand up to scrutiny.

Myth: I should worry this can happen to me.

Well, I have to concede this is true — part of this did happen to me! The first flight I took with Kathryn, the airline came up to us after we had boarded, and insisted she give up her seat for a deadheading pilot. The pilot never sat there — instead he went up to use the jumpseat in the cockpit. We were quite angry, especially when her later flight lost an engine in the middle of the Pacific.

But a lot had to go wrong for this to happen. Here’s my guess as to the list of things that went wrong:

  • Something failed in the planned movement of flight crew, and they needed to get a crew to SDF for a Monday Morning flight. They looked over their options, and decided to try to get on UA3411
  • They decided that very late, so the flight had already boarded full when the flight crew came to the gate and said they needed to be on that plane. (I don’t know why they selected this one over the next, I presume both were full, or the next one might even have been oversold. You want to avoid the last flight in any event.)
  • They tried the normal approach — offer an incentive for volunteers. They got to $800. (UA says $1,000.) It failed. Nobody bit. This is a flight where everybody needed to get to SDF.
  • They didn’t know their contract well, and decided they could do involuntary bump to solve their problem. Why not, it’s what they usually do, right? They got mean, declaring the plane would not fly until 4 got off.
  • They really didn’t know their contract well, and figured they could involuntary bump by removing passengers from the plane. They can’t, but they told people they had to leave.
  • Usually that works. In fact, I suspect it’s worked pretty much every time for decades. Not this time. One man refuses to leave. Now they had a passenger refusing flight crew orders.
  • A non-compliant passenger is something they are trained for. They follow their procedure. He won’t leave. They follow their procedure and call in airport cops.
  • The airport cops are thugs. They manhandle him, injure him and drag him. All recorded on camera phones.
  • It explodes on the social networks. The company has no idea how to handle it, and botches that too.

Because so many things had to go wrong, the particular situation is not important. Rare things go wrong all the time. Junior staff at small airlines are not fully trained on contract nuances. Because things had never gone south like this before (and not in the way the plane was supposed to literally fly south) nobody had ever thought to write up procedures to remind gate crews that they can’t remove passengers, and that they can’t bump at all if it’s not actually oversold.

Those of us writing so much about this online only really want to care about systemic problems. What is wrong with the system, not just one gate crew or flight crew. If there is a pattern of errors, what can be done to fix it.

Myth: That poor doctor!

I am hesitant to include this one, because I don’t want to give the impression that I am defending in any way what happened to him, but it is an important fact. I am not saying anybody should be forcibly removed from a plane because the airline wants his seat. This was not just your ordinary passenger. Reports claim Dr. Dao lost his licence to practice medicine from 2003 to 2016 because he was convicted of trading prescription painkillers for sex, and his psych evaluations listed him as having anger management issues. One reason this escalated is that normally nobody dares to defy orders from the flight crew and especially from police. The orders were improper, and the bumped passengers deserve lots of compensation, but you have to attribute some portion of the blame for how far it escalated to Dr. Dao.

So, is overbooking evil or good?

The big question I have found most interesting is the subject of overbooking. Almost all airlines sell more seats on a plane than it actually has. They give you what they call a “confirmed reservation” and that name certainly makes people imagine they have a guaranteed seat on the plane. They don’t, but they almost do, and that’s as I will explain, a good thing for the flying public.

One basic statistic — the no-show rate on flights is around 8%. So a plane with 100 seats, if it is considered “sold out” after 100 reservations. On average, with no overselling or standby pax, it would take off with 8 empty seats. The number is not the same for every flight. Complex algorithms predict the actual number based on history of that flight and the passengers.

Myth: The airlines primarily do this as a fraud to make money by selling the same seat twice

Turns out, when people don’t fill their seat, only rarely does the airline get any money, or a profit from them. Airlines do make money from overbooking, but not the way you think. Most of those no-shows are because of late or cancelled connections. Those are money losers for the airline, big time. They have to rush to find another flight for that passenger, and get no money. Some of them are people who did free same-day changes or otherwise switched off the flight for low fee. A few have tickets with no change fees. A few more did a late flight change and paid a change fee. The change fee is sometimes as high as the ticket, but sometimes it’s much less. The airline pockets the change fee, but not without cost — the biggest one being they turned away passengers they would not have turned away because of the booking.  read more »

What went wrong and how could United do better on bumping a passenger?

Update: More careful reading of United’s Contract suggests both that this didn’t fit the definition of an oversold flight, and that even if it did, they only have the power to “deny boarding” to a bumped passenger, not to remove them from an aircraft. If this is true, then this case is simple and much less interesting: UA/Republic should admit fault and compensate those involved and retrain staff. End of that part of the story. Later-update: This might might have involved a special “Must ride” classification put on the flight crew which changes the rule yet again.

I have a follow-on post on misconceptions and realities about these issues.

The viral video of the day is that of police pulling a main from a United Airlines flight. He doesn’t want to go, and they pull him out, and bash his head on the armrest, then drag out his unconscious body. It’s a nightmare for everybody, and the video sends clear chills into every viewer. (Once, after I changed my flight to fly home from Hawai`i with Kathryn, they involuntarily removed her from the plane for a crew member. I spent the flight next to an empty seat as the crew member went to the cockpit jumpseat, and she flew on a later flight that lost an engine. We’ve never flown on that airline again.)

In spite of that, I have some sympathy for both sides, and while clearly things went very wrong here, as even United will eventually admit, the more interesting question for me is “what should airlines do to make this work better”? I do believe that UA clearly didn’t want this to happen, though their policies created a small risk that it would. I am sure they don’t want it to happen again. So if you were the person writing the policy for these situations, what would you do?

The situation:

  • This was UA3411, UA’s 2nd last flight from ORD to Louisville. UA (or rather Republic airlines, a small regional flying under the United Express logo) had 4 flight crew who were needed for an early flight from Louisville and, I presume, had no other option for getting them there. (The next flight was obviously more oversold.) If they don’t get there, and sleep the legally required amount, that flight is canceled and a whole lot of people don’t fly, and a bunch of other flights are affected too. Aviation rules are strict on this.
  • In an unusual situation, the four flight attendants are not expected. It is quite common for flight crew moving to their next job to be on flights and displace paying passengers, but unusual for it to be a surprise, to happen after the passengers have already boarded a full flight.
  • So they ask ( as is required by law) for people to volunteer to get off in exchange for a reward. Unfortunately, all they can offer is a flight Monday afternoon. Nobody wants that, apparently, and the offer gets up to $800 plus hotel. Tickets on this 90 minute flight are only $187, but nobody wants the offer. That’s also unusual.
  • The law then gives the airline another option, involuntary bump. They tell the passengers they will do this if nobody volunteers. They select a pool of “low priority” passengers (those who took super-discount fares, removing elites and the disabled and a few others.) They pick 4 at random.
  • 3 of those selected get off. The law requires they get a compensation of around $800 but in cash, not coupons. One, a doctor, refuses. He tells some people he has to see patients in the morning.
  • They say the plane can’t take off until this passenger leaves. He won’t. They call the airport cops. The airport cops come to his seat to remove him.
  • You can see what happens next on the video. He won’t go. They physically try to pull him out. He screams and clings to the seat. They pull harder. He hits his head on the opposite armrest and is knocked out.
  • They drag his limp form from the plane — you can see that on video.
  • Amazingly, he somehow gets back on the plane, bloodied and a bit confused. He keeps repeating, “I have to get home.” He does not appear to be wearing leggings.

New information reveals that a whole bunch of things went wrong at once, which does not excuse police manhandling a passenger, but helps us understand why it went pear-shaped.

First, understanding overselling — and why the flying public wants it

Most flights these days are oversold, because a lot of people don’t show for their flights. The system of overselling, then calling for volunteers when too many show up makes the planes fly mostly full these days on many routes. It’s a fact of flying and allowed in the law. It makes flight more efficient, perhaps 5-10% more. On competitive routes, that makes tickets cheaper for everybody. It has another benefit to the flying public — more people get to fly on the flight they want, because the airline is willing to sell you a seat on a “full” flight, knowing that 99% of the time you and everybody else will actually get to fly. The alternative is that an empty seat flies, and you wastefully take another flight. Passengers really like more availability, though they don’t directly see how it happens. The reality is many of the flights you see in your web search are technically oversold. If it is really sold out, it’s actually oversold past their limit.

Airlines could elect to not oversell, or not oversell as much, but that comes with a cost. More people denied the flight they want. More expensive tickets. More emissions per passenger. The world doesn’t want that, so the world allows and the law regulates, overselling.

Of course, there is a way to avoid ever being bumped. Pay more for your ticket, or be an elite flyer, as I am. (In fact, as an elite, they actually guarantee me a seat on “really, really sold out” flights 24 hours in advance, which really means they push their oversell percentage by plus-one for elites. If I do this — I never have — they just decide it is cheaper to pay a volunteer to get off the flight than to deny one of their elites the flight they need.)

So the most obvious solution, “Don’t oversell,” comes with a cost I don’t think the airlines or flying public actually want. Consider it this way. A flight you need with 100 seats has had 100 bookings. The airline knows that on average 7 of them won’t show up. Do you want the airline to let you “reserve” on that plane, or tell you “sorry, fly the next day?” Do you want them to only offer you a standby ticket because other people, who paid far less than you for their tickets and who barely fly on their airline, got there first? (And yes, those people who buy late pay a premium.) The airline hates taking off with an empty seat, but you hate being told you can’t get on a flight that ended up with empty seats even more.

Airlines are getting quite good at it. In 2015, only .09% of passengers were bumped, and only .01% involuntarily.

The public wants bumping for flight crew, too!

Turns out, it’s in the public interest that flight crew needed for another flight have higher priority than we do, even to the point of removing us from planes we already boarded. That may not be allowed, but one has to consider the difference between one person removed (voluntarily or not) with compensation and the very large group of people who will have their flight cancelled (sometimes with no compensation) if the flight crew doesn’t get there, properly rested and ready. You don’t want to be either, and utilitarianism is not always the right philosophy, but here the numbers are overwhelming. One guy doesn’t fly or 70 people don’t. So we want a system where that can happen, but smoothly and ideally voluntarily.

Understand involuntary bumping

Usually, the system of offering fat compensation — $800, a hotel and meals for a $180 flight is a pretty good deal — works fine. There are people who actually relish it. I met one guy who says he deliberately tries to get bumped the day before Thanksgiving — when the offers get very high. But nobody was taking it. Most would miss a day of work, which is not an easy thing to do.

The law then allows the airlines to do an involuntary bumping. They have an algorithm that picks people and they are “denied boarding.” The law specifies compensation. In this case 4 times the ticket price and other compensations. And this is cash, not flight coupons. Cash is worth a lot more.

This law is one of the culprits here. The law effectively puts a cap on the offer you will get. The airlines, in a move they thought at first was rational, don’t want to offer you a lot more than the price the law defines for an involuntary bump. Why give a passenger $2,000 when you can do it for $1,000 under the law. Well, one reason is bad PR — which is true in spades here.

The airlines don’t want to do this. About 1 in 1,000 passengers are bumped, and 1 in 10,000 are involuntarily bumped, and has been going down as they get better at working their systems. But it happens.

Without the involuntary rule, the airline might have considered the next solution…

Make better offers for voluntary bumping

This problem would have been defused if they had kept increasing the offer until somebody took it. (Those who took it early will of course be upset, but that’s how it goes.) While there is a practical limit, a volunteer should be found long before it.

They could also consider other things that are not money. Often bump offers come with things like first class upgrades which can be cheap for the airline and very nice to the passenger. They could offer a very coveted thing to some passengers — elite qualification. At the extreme, if they offered 20,000 elite qualification miles or a full-tier bump in elite status, I could see even elite passengers jumping up to volunteer. We don’t usually. We know we will never get involuntarily bumped. We usually have places to go. But we crave that elite status so much that some people fly “mileage runs” — flights to nowhere just to accumulate miles — to keep and increment it. If UA said, “get off this plane and we’ll make you 1K” they would have had a line out the door of volunteers.  read more »

No, Detroit is not winning the robocar race.

A new report from Navigant Research includes the chart shown below, ranking various teams on the race to robocar deployment. It’s causing lots of press headlines about how Ford is the top company and companies like Google and Uber are far behind.

I elected not to buy the $3800 report, but based on the summary I believe their conclusions are ill founded to say the least.

This ordering smacks of old-world car industry thinking. Saying that Ford and GM are ahead of Waymo/Google is like saying that Foxconn is ahead of Google or Apple in the smartphone market. Foxconn makes the iPhone of course, and makes lots of money at a modest profit margin of a few percent. Apple and Google don’t make their phones, they design them and the software platform.

Ford and GM might feel good reading this report, but they should not. I do actually like Ford’s plan quite a bit — especially their declaration that they will not sell their robocar to end-users. I also like Daimler’s declaration that they want to have a taxi style service called “Car2Come” after their Car2Go one-way on-demand car rental service. (Americans giggle at the name, Germans are never bothered by such things. :-)

GM does not belong high on the list, other than for its partnership with Lyft. They were wise to acquire a company like Cruise — and I know the folks at Cruise, this is not a criticism of them — but it’s not enough to catapult you to the front of the list.

A similar article came out a few months ago, declaring that Silicon Valley was sure to lose to Detroit, because Detroit knows how to make cars, and Silicon Valley doesn’t. The report went further and declared that Google was falling behind because they had said they did not plan to make a car. The author had mistakenly thought Google had plans to make a car — Google never said anything like that — and so decided that the announcement that they would not make one was a big retreat on their part.

Companies like Google, Apple and Uber have never stated they wished to make cars, or felt they were any good at it. If they want to make cars, they have the cash to go buy a car company, but there is no need to do that. There are a couple of dozen companies around the world who are already very good at making cars, and if you come to them with an order for 100,000 cars to your specification, they will jump to say “yes, sir!” Some of the companies, the big leaders like Toyota or BMW, might well refuse that order, not wanting to be the supplier for a threat to their existence. But it won’t help them. Somebody will be that supplier. If not a German, Japanese or U.S. company, then a Korean company, or failing that a Chinese company. In fact, Foxconn has said it is interested in making cars, and Apple is designing them, so the Apple-Foxconn relationship may be far more than a metaphor for this situation.

When you summon an Uber, you don’t care what nameplate is on the car. When you summon UberSelect, you don’t care if it’s Lexus, or Mercedes or BMW. Uber was your brand, and you aren’t buying the car for 15 years, you are buying it for 15 minutes. Brand plays a completely different role.

Companies like Waymo, Apple, Uber, Zoox and others would be foolish to manufacture cars, unless they want a car so radically different that nobody knows how to make it. (Then, they might decide to be the first to figure it out.) The car manufacturers would be foolish to turn down the giant purchase order, or partnerships with whoever has the best technology.

The winner of the transportation game of the future will be the company that thinks outside the car. That doesn’t mean the big car companies can’t do that. It’s just harder for them to do.

The chart’s not entirely wrong. Honda is pretty far behind — but PSA is even further behind. BMW, Daimler and Ford are among the best of the car companies, but Tesla and Volvo deserve higher ranking. Hyundai is not ahead of Toyota, and Tesla, while not ahead of Waymo, is in a pretty good place. Bosch is a surprising absentee from the list. FCA should be on it, just very low on the chart, along with the smaller Japanese vendors and many Chinese vendors.

But be clear. Making the car is essential, but it’s also old and a commodity. The value will lie in those building the self-driving software systems and sensors, and those putting services together around the technologies. The big automaker’s advantages — nameplate and reputation, reliability, manufacturing skill and capacity, retail channel experience — these are all less valuable or commoditized. They have to act fast to move to new business models that will make it in the future. Of course, one plan is to own the important components I name above, and several companies like BMW, Daimler, Ford, Nissan and Volvo are trying to do that. But they’re behind Waymo by a fair distance.

Syndicate content