Brad Templeton is Chairman Emeritus of the EFF, Singularity U founding computing faculty, software architect and internet entrepreneur, robotic car strategist, futurist lecturer, photographer and Burning Man artist.

This is an "ideas" blog rather than a "cool thing I saw today" blog. Many of the items are not topical. If you like what you read, I recommend you also browse back in the archives, starting with the best of blog section. It also has various "topic" and "tag" sections (see menu on right) and some are sub blogs like Robocars, photography and Going Green. Try my home page for more info and contact data.

Waymo starts pilot in Phoenix, Apple gets more real and other news

Waymo (Google) has announced a pilot project in Phoenix offering a full ride service, with daily use, in their new minivans. Members of the public can sign up — the link is sure to be overwhelmed with applicants, but it has videos and more details — and some families are already participating. There’s also a Waymo Blog post. I was in Phoenix this morning as it turns out, but to tell real estate developers about robocars, not for this.

There are several things notable about Waymo’s pilot:

  1. They are attempting to cover a large area — they claim twice the size of San Francisco, or 90 square miles. That’s a lot. It’s enough to cover the vast majority of trips for some pilot users. In other words, this is the first pilot which can test what it’s like to offer a “car replacement.”
  2. They are pushing at families, which means even moving children, including those not of driving age. The mother in the video expects to use it to send some children to activities. While I am sure there will be safety drivers watching over things, trusting children to the vehicles is a big milestone. Google’s safety record (with safety drivers) suggests this is actually a very safe choice for the parents, but there is emotion over trusting children to robots (other than the ones that go up and down shafts in buildings.)
  3. In the videos, they are acting like there are no safety drivers, but there surely are, for legal reasons as well as safety.
  4. They are using the Pacifia minivans. The Firefly bubble cars are too slow for anything but neighbourhood operation. The minivans feature motorized doors, a feature which, though minor and commonplace, meets the image of what you want from a self-driving car.

Apple is in the game

There has been much speculation recently because of some departures from Apple’s car team that they had given up. In fact, last week they applied for self-driving car test plates for California. I never thought they had left the game.  read more »

How to do a low bandwidth, retinal resolution video call

Not everybody loves video calls, but there are times when they are great. I like them with family, and I try to insist on them when negotiating, because body language is important. So I’ve watched as we’ve increased the quality and ease of use.

The ultimate goals would be “retinal” resolution — where the resolution surpasses your eye — along with high dynamic range, stereo, light field, telepresence mobility and VR/AR with headset image removal. Eventually we’ll be able to make a video call or telepresence experience so good it’s a little hard to tell from actually being there. This will affect how much we fly for business meetings, travel inside towns, life for bedridden and low mobility people and more.

Here’s a proposal for how to provide that very high or retinal resolution without needing hundreds of megabits of high quality bandwidth.

Many people have observed that the human eye is high resolution on in the center of attention, known as the fovea centralis. If you make a display that’s sharp where a person is looking, and blurry out at the edges, the eye won’t notice — until of course it quickly moves to another section of the image and the brain will show you the tunnel vision.

Decades ago, people designing flight simulators combined “gaze tracking,” where you spot in real time where a person is looking with the foveal concept so that the simulator only rendered the scene in high resolution where the pilot’s eyes were. In those days in particular, rendering a whole immersive scene at high resolution wasn’t possible. Even today it’s a bit expensive. The trick is you have to be fast — when the eye darts to a new location, you have to render it at high-res within milliseconds, or we notice. Of course, to an outside viewer, such a system looks crazy, and with today’s technology, it’s still challenging to make it work.

With a video call, it’s even more challenging. If a person moves their eyes (or in AR/VR their head) and you need to get a high resolution stream of the new point of attention, it can take a long time — perhaps hundreds of milliseconds — to send that signal to the remote camera, have it adjust the feed, and then get that new feed back to you. There is no way the user will not see their new target as blurry for way too long. While it would still be workable, it will not be comfortable or seem real. For VR video conferencing it’s even an issue for people turning their head. For now, to get a high resolution remote VR experience would require sending probably a half-sphere of full resolution video. The delay is probably tolerable if the person wants to turn their head enough to look behind them.

One opposite approach being taken for low bandwidth video is the use of “avatars” — animated cartoons of the other speaker which are driven by motion capture on the other end. You’ve seen characters in movies like Sméagol, the blue Na’vi of the movie Avatar and perhaps the young Jeff Bridges (acted by old Jeff Bridges) in Tron: Legacy. Cartoon avatars are preferred because of what we call the Uncanny Valley — people notice flaws in attempts at total realism and just ignore them in cartoonish renderings. But we are now able to do moderately decent realistic renderings, and this is slowly improving.

My thought is to combine foveal video with animated avatars for brief moments after saccades and then gently blend them towards the true image when it arrives. Here’s how.

  1. The remote camera will send video with increasing resolution towards the foveal attention point. It will also be scanning the entire scene and making a capture of all motion of the face and body, probably with the use of 3D scanning techniques like time-of-flight or structured light. It will also be, in background bandwidth, updating the static model of the people in the scene and the room.
  2. Upon a saccade, the viewer’s display will immediately (within milliseconds) combine the blurry image of the new target with the motion capture data, along with the face model data received, and render a generated view of the new target. It will transmit the new target to the remote.
  3. The remote, when receiving the new target, will now switch the primary video stream to a foveal density video of it.
  4. When the new video stream starts arriving, the viewer’s display will attempt to blend them, creating a plausible transition between the rendered scene and the real scene, gradually correcting any differences between them until the video is 100% real
  5. In addition, both systems will be making predictions about what the likely target of next attention is. We tend to focus our eyes on certain places, notably the mouth and eyes, so there are some places that are more likely to be looked at next. Some portion of the spare bandwidth would be allocated to also sending those at higher resolution — either full resolution if possible, or with better resolution to improve the quality of the animated rendering.

The animated rendering will, today, both be slightly wrong, and also suffer from the uncanny valley problem. My hope is that if this is short lived enough, it will be less noticeable, or not be that bothersome. It will be possible to trade off how long it takes to blend the generated video over to the real video. The longer you take, the less jarring any error correction will be, but the longer the image is “uncanny.”

While there are 100 million photoreceptors in the whole eye, but only about a million nerve fibers going out. It would still be expensive to deliver this full resolution in the attention spot and most likely next spots, but it’s much less bandwidth than sending the whole scene. Even if full resolution is not delivered, much better resolution can be offered.

Stereo and simulated 3D

You can also do this in stereo to provide 3D. Another interesting approach was done at CMU called pseudo 3D. I recommend you check out the video. This system captures the background and moves the flat head against it as the viewer moves their head. The result looks surprisingly good.  read more »

Luminar unstealths their 1.5 micron LIDAR

Luminar, a bay area startup, has revealed details on their new LIDAR. Unlike all other commercial offerings, this is a LIDAR using 1.5 micron infrared light. They hope to sell it for $1,000.

1.5 micron LIDAR has some very special benefits. The lens of your eye does not focus medium depth infrared light like this. Ordinary light, including the 0.9 micron infrared light of the lasers in most commercial LIDARS is focused to a point by the lens. That limits the amount of power you can put in the laser beam, because you must not create any risk to people’s eyes.

Because of this, you can put a lot more power into the 1.5 micron laser beam. That, in turn, means you can see further, and collect more points. You can easily get out to 250 meters, while regular lidars are limited to about 100m and are petering out there.

What doesn’t everybody use 1.5 micron? The problem is silicon sensors don’t react to this type of light. Silicon is the basis of all mass market electronics. To detect 1.5 micron light, you need different materials, which are not themselves that hard to find, but they are not available cheap and off the shelf. So far, this makes units like this harder to build and more expensive. If Luminar can do this, it will be valuable.

Why do you need to see 250m? Well, you don’t for city driving, though it’s nice. For highway driving, you can get by with 100m as well, and you use radar to help you perceive, at very low resolution, what’s going on beyond that. Still, there are things that radar can’t tell you. Rare things, but still important. So you need a sensor that sees further to spot things like stalled cars under bridges. Radar sees those, but can’t tell them from the bridge.

To this point, Google has been the only company to say they have a long range LIDAR, but it has not been for sale. And as we all know, there is a famous lawsuit underway accusing Uber/Otto of copying Google’s LIDAR designs.

The Luminar point clouds are impressive. This will be a company to watch. (In the interests of disclosure, I am an advisor to Quanergy, another LIDAR startup.)

United Part 2: Misconceptions and realities

There’s a lot of bad information circulating on the famous United/Republic “passenger drag” so I wanted to consolidate a 2nd post with some of them.

Myth: This was an oversold flight

It turns out the flight was probably not oversold. A UA spokesman said it wasn’t. It was a fully sold flight, but a sudden need arose to move 4 flight attendants to SDF (Louisville) and they arrived at the gate after the flight had boarded. In United’s contract of carriage, it defines an oversold flight as a flight where there are more passengers with confirmed reservations checked in by the check-in deadline than they have seats on the plane. That does not appear to be the case on this flight, but Republic and UA got confused about it.

That, in turn, means Republic did not have the right to invoke the clauses of the contract for oversold flights. If so, they are just plain in the wrong, and this becomes a case with far less interesting nuance. United has changed their tune (of course due to public pressure) and are going full mea culpa.

Airline reservation computers oversell all the time, and carefully calculate exactly how much to oversell. It looks like the algorithms decided to not oversell this flight. And they were right — when they called for volunteers, nobody accepted, even at a very high price ($800 to $1,000) for a flight where most tickets are under $200. The algorithms performed perfectly.

Myth: This was United Airlines

Technically it was Republic Airline, a small regional airline dba “United Express.” However, United sells and and manages the tickets and they use the brand, and it’s under United’s contract, so United certainly gets a lot of the responsibility. And I am impressed that UA has not tried to throw Republic under the bus here.

Republic Airways actually operates lots of regional flights for United, AA and Delta, so this could have probably happened to any of them. I don’t know if they have a lot of airline specific training on bumping procedure for their teams. United may have just gotten some very bad luck of the draw here — and then made it worse by defending it at first. And it may be that the bumping policies UA gives to Republic might have made this more likely than the ones Delta and AA give it, but I don’t think they are tremendously different. Some hinges on whether the flight crew was a Republic crew, or a United crew.

But still, though it was not United, the buck stops with United, and at least now, they are not resisting that at all.

Myth: On an oversold flight, they can pull passengers off the plane.

If this had been an oversold flight, their contract still does not let them remove passengers from the plane involuntarily. It says they can “deny boarding.” Deny boarding does not mean remove — there is another section of the contract on removal. More bad news for United/Republic, but again, it makes the case less interesting as it’s an example of something you sort of expect — junior employees of a regional affiliate not being properly trained on what to do in an unusual situation and thus screwing up. That happens in 100 different ways all the time, but each particular incident is rare and probably does not indicate a systemic problem. That’s good — but it is only systemic problems that are of interest to the public, and which would make you boycott a company. If the junior employees make mistakes like this too often, then you have a systemic problem to worry about. (United does not have a good reputation on this count, of course.)

Update: These flight attendants were “must ride” passengers

New information reveals the flight crew declared themselves “must ride.” I don’t have a lot of details, but this is a special designation in the law (not the UA contract) which declares the crew are needed somewhere to avoid cancellation of a flight. Once a passenger is declared “must ride” the plane is required, reports say, to do everything possible to get that passenger to their destination, including delaying the plane and apparent, yes, even involuntary bumping. I am waiting for more information on this status, which would invalidate partly what I say above. They can’t pull you for an oversell, but they may be able to pull you for a must-ride. The law is there to keep the aviation system humming. Once flight crews don’t get to flights, it can mean disruption to more than just that flight.

Myth: If the doctor had just handed one of the police officers a Pepsi, it all would have been defused.

No, but that’s the best joke on this that I’ve seen.

Myth: It’s overselling that’s the problem

With the mistaken impression that overselling was the cause here, a lot of people are stating overselling is evil, and Chris Christie has even called to prohibit it. That’s a big mistake. Overselling is very good for airlines and the flying public. I explained that in yesterday’s post but I will go into more details below. You want an airline that does at least some overselling, though one can debate how much you want.

Myth: The airline prioritized employees over paying customers

In this situation, it needed to move those employees to crew a flight first thing out of SDF. If they had not gotten a crew there, that flight gets cancelled. Roughly 70 paying passengers get stranded against their will. While clearly nobody wants to be stranded against their will, the hard truth is you want to fly an airline that will strand (with good compensation) 4 people to avoid doing it to 70. (Here I am talking about the normal approach, which is to deny boarding to 4 people, not to try do drag people off the plane.) Still, I have to view it as prioritizing 70 passengers over 4, not employees over passengers.

Maybe: They could have driven the flight attendants there or chartered a jet

This is possibly true but possibly not. First of all, these airlines are all about procedure. They don’t authorize junior employees to be innovative or authorize them to spend money. So chances are if somebody thought of that, they had no system with which to do it. That is a fault of the airline but the sad norm of corporate bureaucracy.

Secondly, while I don’t know this to be true, all flight crew operate under a set of complex rules about required rest. You don’t want a sleepy pilot landing your plane, or a sleepy flight attendant helping people get onto the evacuation slides. These rules are very hard and fast. I suspect trying to sleep in a car doesn’t count, and an overnight ground ride is out of the question. Had they acted very quickly, and had a system in place, they could have probably gotten the crew there a bit after 11 — not long after the flight actually landed due to the chaos — so that might have worked, in hindsight.

They could have offered the passengers a limo ride, but again probably had no way to do something that out of the ordinary.

The same applies to an air charter. Getting an air charter on short notice is difficult, but they could have gotten one for the flight crew (or another flight crew) in the early morning if they had a system in place. This is very expensive of course, and so not likely to be in their playbook.

Maybe: They should never have gotten to the point where it was so urgent to get that flight crew moved

Airlines move flight crews a lot. There are airline pilots who live on one coast and mostly work on the other, commuting by “deadheading” on one of their airlines planes.

When you design a system that needs various parts — planes and flight crew — you have to “overprovision,” which is to say leave some wiggle room. That means you have some number of planes, crews and other resources sitting idle or on call, and you use them when something else fails. Everybody does it because you don’t want to run so close to the wire all the time. If you do, the slightest problem causes a cascade of cancellations. Airlines have to worry not just about small problems but even big ones like storms that cancel or delay many flights.

It’s not practical, however, to overprovision to the point that you never fail. You can do it, but it’s really expensive. You have to waste a lot of money, and you don’t have a competitive company. So every systems designer tries to figure out how to overprovision just the right amount. An amount that will have a few failures, but not too many. On top of that, you try to plan so you handle those failures with the least amount of pain, but you accept they will still happen.

What that means — and I don’t have any specific facts about this flight — is that sometimes you will be skating the edge, and sometimes you will fail. Sometimes you will find that crew are not going to make a flight unless you do something a little extra.

The bumping law, I think, is where the airlines find their “extra.” They don’t want to bump paying customers — it’s expensive and hurts customer relations. But they don’t want to cancel flights even more. So every so often, every airline has to find a solution. The bumping law offers them that solution. They can legally deny boarding to paying passengers against their will to make room for crew. This is much more workable, and under their control, than other options like using charter jets, or if distances are short, ground service.

True, but: Just about anything would be cheaper than the hit they’ve taken

That’s true — but only in hindsight. No playbook for these situations is going to say, “If you have to, spend $10,000 rather than bumping passengers just in case it turns into the PR nightmare of the year.” By definition, nobody knew that would happen.

In reality, airlines involuntarily bump 50,000 pax per year and while they grumble, this is the first time it’s ever gone done like this, with eviction from the plane, blood, camera phones and Facebook. So I don’t blame them for not seeing this could happen. I do blame them, however, for not understanding that any time you bring the police into a situation you bump the risk of something bad happening.

True, but: They should have known this would happen once they called the goons.

They should have known, but I can suspect why they didn’t — because they actually do this all the time and don’t have PR problems. Flight crews face unruly passengers reasonably often. They have training for it and procedures. And those procedures do call for getting the police, even knowing how that can go south. What those plans obviously did not account for was doing this when it was completely clear the passenger was the victim, that they only removed him because they wanted his seat. The rules for removing passengers mostly deal with safety issues. When they declare a passenger a safety risk, and the passenger makes trouble and even (rarely) causes a scuffle they are protected if the passenger was really a safety risk, or they can even come up with a credible lie why they thought he was a safety risk. No such story is possible here. Sure, the law says anybody who refuses a flight crew order can be removed from the plane. Technically it says this. In reality, it’s insane to think you can remove somebody for refusing the order “leave the plane” when the order is not given for a valid reason. The law says obey, but every sense of justice goes the other way. In fact, more than that, I don’t think a court would convict somebody for refusing that order, even if they are guilty, because society does not intend to grant the airlines that sort of power.

Put another way, three things are true:

  1. They can’t order you off the plane just to take your seat (but they didn’t know that.) We don’t want airlines to have that power.
  2. Once somebody refuses a flight crew order, you can then order them off the plane.

As such, it’s clear that “we removed him because he disobeyed our order to leave” is a loophole that would never stand up to scrutiny.

Myth: I should worry this can happen to me.

Well, I have to concede this is true — part of this did happen to me! The first flight I took with Kathryn, the airline came up to us after we had boarded, and insisted she give up her seat for a deadheading pilot. The pilot never sat there — instead he went up to use the jumpseat in the cockpit. We were quite angry, especially when her later flight lost an engine in the middle of the Pacific.

But a lot had to go wrong for this to happen. Here’s my guess as to the list of things that went wrong:

  • Something failed in the planned movement of flight crew, and they needed to get a crew to SDF for a Monday Morning flight. They looked over their options, and decided to try to get on UA3411
  • They decided that very late, so the flight had already boarded full when the flight crew came to the gate and said they needed to be on that plane. (I don’t know why they selected this one over the next, I presume both were full, or the next one might even have been oversold. You want to avoid the last flight in any event.)
  • They tried the normal approach — offer an incentive for volunteers. They got to $800. (UA says $1,000.) It failed. Nobody bit. This is a flight where everybody needed to get to SDF.
  • They didn’t know their contract well, and decided they could do involuntary bump to solve their problem. Why not, it’s what they usually do, right? They got mean, declaring the plane would not fly until 4 got off.
  • They really didn’t know their contract well, and figured they could involuntary bump by removing passengers from the plane. They can’t, but they told people they had to leave.
  • Usually that works. In fact, I suspect it’s worked pretty much every time for decades. Not this time. One man refuses to leave. Now they had a passenger refusing flight crew orders.
  • A non-compliant passenger is something they are trained for. They follow their procedure. He won’t leave. They follow their procedure and call in airport cops.
  • The airport cops are thugs. They manhandle him, injure him and drag him. All recorded on camera phones.
  • It explodes on the social networks. The company has no idea how to handle it, and botches that too.

Because so many things had to go wrong, the particular situation is not important. Rare things go wrong all the time. Junior staff at small airlines are not fully trained on contract nuances. Because things had never gone south like this before (and not in the way the plane was supposed to literally fly south) nobody had ever thought to write up procedures to remind gate crews that they can’t remove passengers, and that they can’t bump at all if it’s not actually oversold.

Those of us writing so much about this online only really want to care about systemic problems. What is wrong with the system, not just one gate crew or flight crew. If there is a pattern of errors, what can be done to fix it.

Myth: That poor doctor!

I am hesitant to include this one, because I don’t want to give the impression that I am defending in any way what happened to him, but it is an important fact. I am not saying anybody should be forcibly removed from a plane because the airline wants his seat. This was not just your ordinary passenger. Reports claim Dr. Dao lost his licence to practice medicine from 2003 to 2016 because he was convicted of trading prescription painkillers for sex, and his psych evaluations listed him as having anger management issues. One reason this escalated is that normally nobody dares to defy orders from the flight crew and especially from police. The orders were improper, and the bumped passengers deserve lots of compensation, but you have to attribute some portion of the blame for how far it escalated to Dr. Dao.

So, is overbooking evil or good?

The big question I have found most interesting is the subject of overbooking. Almost all airlines sell more seats on a plane than it actually has. They give you what they call a “confirmed reservation” and that name certainly makes people imagine they have a guaranteed seat on the plane. They don’t, but they almost do, and that’s as I will explain, a good thing for the flying public.

One basic statistic — the no-show rate on flights is around 8%. So a plane with 100 seats, if it is considered “sold out” after 100 reservations. On average, with no overselling or standby pax, it would take off with 8 empty seats. The number is not the same for every flight. Complex algorithms predict the actual number based on history of that flight and the passengers.

Myth: The airlines primarily do this as a fraud to make money by selling the same seat twice

Turns out, when people don’t fill their seat, only rarely does the airline get any money, or a profit from them. Airlines do make money from overbooking, but not the way you think. Most of those no-shows are because of late or cancelled connections. Those are money losers for the airline, big time. They have to rush to find another flight for that passenger, and get no money. Some of them are people who did free same-day changes or otherwise switched off the flight for low fee. A few have tickets with no change fees. A few more did a late flight change and paid a change fee. The change fee is sometimes as high as the ticket, but sometimes it’s much less. The airline pockets the change fee, but not without cost — the biggest one being they turned away passengers they would not have turned away because of the booking.  read more »

What went wrong and how could United do better on bumping a passenger?

Update: More careful reading of United’s Contract suggests both that this didn’t fit the definition of an oversold flight, and that even if it did, they only have the power to “deny boarding” to a bumped passenger, not to remove them from an aircraft. If this is true, then this case is simple and much less interesting: UA/Republic should admit fault and compensate those involved and retrain staff. End of that part of the story. Later-update: This might might have involved a special “Must ride” classification put on the flight crew which changes the rule yet again.

I have a follow-on post on misconceptions and realities about these issues.

The viral video of the day is that of police pulling a main from a United Airlines flight. He doesn’t want to go, and they pull him out, and bash his head on the armrest, then drag out his unconscious body. It’s a nightmare for everybody, and the video sends clear chills into every viewer. (Once, after I changed my flight to fly home from Hawai`i with Kathryn, they involuntarily removed her from the plane for a crew member. I spent the flight next to an empty seat as the crew member went to the cockpit jumpseat, and she flew on a later flight that lost an engine. We’ve never flown on that airline again.)

In spite of that, I have some sympathy for both sides, and while clearly things went very wrong here, as even United will eventually admit, the more interesting question for me is “what should airlines do to make this work better”? I do believe that UA clearly didn’t want this to happen, though their policies created a small risk that it would. I am sure they don’t want it to happen again. So if you were the person writing the policy for these situations, what would you do?

The situation:

  • This was UA3411, UA’s 2nd last flight from ORD to Louisville. UA (or rather Republic airlines, a small regional flying under the United Express logo) had 4 flight crew who were needed for an early flight from Louisville and, I presume, had no other option for getting them there. (The next flight was obviously more oversold.) If they don’t get there, and sleep the legally required amount, that flight is canceled and a whole lot of people don’t fly, and a bunch of other flights are affected too. Aviation rules are strict on this.
  • In an unusual situation, the four flight attendants are not expected. It is quite common for flight crew moving to their next job to be on flights and displace paying passengers, but unusual for it to be a surprise, to happen after the passengers have already boarded a full flight.
  • So they ask ( as is required by law) for people to volunteer to get off in exchange for a reward. Unfortunately, all they can offer is a flight Monday afternoon. Nobody wants that, apparently, and the offer gets up to $800 plus hotel. Tickets on this 90 minute flight are only $187, but nobody wants the offer. That’s also unusual.
  • The law then gives the airline another option, involuntary bump. They tell the passengers they will do this if nobody volunteers. They select a pool of “low priority” passengers (those who took super-discount fares, removing elites and the disabled and a few others.) They pick 4 at random.
  • 3 of those selected get off. The law requires they get a compensation of around $800 but in cash, not coupons. One, a doctor, refuses. He tells some people he has to see patients in the morning.
  • They say the plane can’t take off until this passenger leaves. He won’t. They call the airport cops. The airport cops come to his seat to remove him.
  • You can see what happens next on the video. He won’t go. They physically try to pull him out. He screams and clings to the seat. They pull harder. He hits his head on the opposite armrest and is knocked out.
  • They drag his limp form from the plane — you can see that on video.
  • Amazingly, he somehow gets back on the plane, bloodied and a bit confused. He keeps repeating, “I have to get home.” He does not appear to be wearing leggings.

New information reveals that a whole bunch of things went wrong at once, which does not excuse police manhandling a passenger, but helps us understand why it went pear-shaped.

First, understanding overselling — and why the flying public wants it

Most flights these days are oversold, because a lot of people don’t show for their flights. The system of overselling, then calling for volunteers when too many show up makes the planes fly mostly full these days on many routes. It’s a fact of flying and allowed in the law. It makes flight more efficient, perhaps 5-10% more. On competitive routes, that makes tickets cheaper for everybody. It has another benefit to the flying public — more people get to fly on the flight they want, because the airline is willing to sell you a seat on a “full” flight, knowing that 99% of the time you and everybody else will actually get to fly. The alternative is that an empty seat flies, and you wastefully take another flight. Passengers really like more availability, though they don’t directly see how it happens. The reality is many of the flights you see in your web search are technically oversold. If it is really sold out, it’s actually oversold past their limit.

Airlines could elect to not oversell, or not oversell as much, but that comes with a cost. More people denied the flight they want. More expensive tickets. More emissions per passenger. The world doesn’t want that, so the world allows and the law regulates, overselling.

Of course, there is a way to avoid ever being bumped. Pay more for your ticket, or be an elite flyer, as I am. (In fact, as an elite, they actually guarantee me a seat on “really, really sold out” flights 24 hours in advance, which really means they push their oversell percentage by plus-one for elites. If I do this — I never have — they just decide it is cheaper to pay a volunteer to get off the flight than to deny one of their elites the flight they need.)

So the most obvious solution, “Don’t oversell,” comes with a cost I don’t think the airlines or flying public actually want. Consider it this way. A flight you need with 100 seats has had 100 bookings. The airline knows that on average 7 of them won’t show up. Do you want the airline to let you “reserve” on that plane, or tell you “sorry, fly the next day?” Do you want them to only offer you a standby ticket because other people, who paid far less than you for their tickets and who barely fly on their airline, got there first? (And yes, those people who buy late pay a premium.) The airline hates taking off with an empty seat, but you hate being told you can’t get on a flight that ended up with empty seats even more.

Airlines are getting quite good at it. In 2015, only .09% of passengers were bumped, and only .01% involuntarily.

The public wants bumping for flight crew, too!

Turns out, it’s in the public interest that flight crew needed for another flight have higher priority than we do, even to the point of removing us from planes we already boarded. That may not be allowed, but one has to consider the difference between one person removed (voluntarily or not) with compensation and the very large group of people who will have their flight cancelled (sometimes with no compensation) if the flight crew doesn’t get there, properly rested and ready. You don’t want to be either, and utilitarianism is not always the right philosophy, but here the numbers are overwhelming. One guy doesn’t fly or 70 people don’t. So we want a system where that can happen, but smoothly and ideally voluntarily.

Understand involuntary bumping

Usually, the system of offering fat compensation — $800, a hotel and meals for a $180 flight is a pretty good deal — works fine. There are people who actually relish it. I met one guy who says he deliberately tries to get bumped the day before Thanksgiving — when the offers get very high. But nobody was taking it. Most would miss a day of work, which is not an easy thing to do.

The law then allows the airlines to do an involuntary bumping. They have an algorithm that picks people and they are “denied boarding.” The law specifies compensation. In this case 4 times the ticket price and other compensations. And this is cash, not flight coupons. Cash is worth a lot more.

This law is one of the culprits here. The law effectively puts a cap on the offer you will get. The airlines, in a move they thought at first was rational, don’t want to offer you a lot more than the price the law defines for an involuntary bump. Why give a passenger $2,000 when you can do it for $1,000 under the law. Well, one reason is bad PR — which is true in spades here.

The airlines don’t want to do this. About 1 in 1,000 passengers are bumped, and 1 in 10,000 are involuntarily bumped, and has been going down as they get better at working their systems. But it happens.

Without the involuntary rule, the airline might have considered the next solution…

Make better offers for voluntary bumping

This problem would have been defused if they had kept increasing the offer until somebody took it. (Those who took it early will of course be upset, but that’s how it goes.) While there is a practical limit, a volunteer should be found long before it.

They could also consider other things that are not money. Often bump offers come with things like first class upgrades which can be cheap for the airline and very nice to the passenger. They could offer a very coveted thing to some passengers — elite qualification. At the extreme, if they offered 20,000 elite qualification miles or a full-tier bump in elite status, I could see even elite passengers jumping up to volunteer. We don’t usually. We know we will never get involuntarily bumped. We usually have places to go. But we crave that elite status so much that some people fly “mileage runs” — flights to nowhere just to accumulate miles — to keep and increment it. If UA said, “get off this plane and we’ll make you 1K” they would have had a line out the door of volunteers.  read more »

No, Detroit is not winning the robocar race.

A new report from Navigant Research includes the chart shown below, ranking various teams on the race to robocar deployment. It’s causing lots of press headlines about how Ford is the top company and companies like Google and Uber are far behind.

I elected not to buy the $3800 report, but based on the summary I believe their conclusions are ill founded to say the least.

This ordering smacks of old-world car industry thinking. Saying that Ford and GM are ahead of Waymo/Google is like saying that Foxconn is ahead of Google or Apple in the smartphone market. Foxconn makes the iPhone of course, and makes lots of money at a modest profit margin of a few percent. Apple and Google don’t make their phones, they design them and the software platform.

Ford and GM might feel good reading this report, but they should not. I do actually like Ford’s plan quite a bit — especially their declaration that they will not sell their robocar to end-users. I also like Daimler’s declaration that they want to have a taxi style service called “Car2Come” after their Car2Go one-way on-demand car rental service. (Americans giggle at the name, Germans are never bothered by such things. :-)

GM does not belong high on the list, other than for its partnership with Lyft. They were wise to acquire a company like Cruise — and I know the folks at Cruise, this is not a criticism of them — but it’s not enough to catapult you to the front of the list.

A similar article came out a few months ago, declaring that Silicon Valley was sure to lose to Detroit, because Detroit knows how to make cars, and Silicon Valley doesn’t. The report went further and declared that Google was falling behind because they had said they did not plan to make a car. The author had mistakenly thought Google had plans to make a car — Google never said anything like that — and so decided that the announcement that they would not make one was a big retreat on their part.

Companies like Google, Apple and Uber have never stated they wished to make cars, or felt they were any good at it. If they want to make cars, they have the cash to go buy a car company, but there is no need to do that. There are a couple of dozen companies around the world who are already very good at making cars, and if you come to them with an order for 100,000 cars to your specification, they will jump to say “yes, sir!” Some of the companies, the big leaders like Toyota or BMW, might well refuse that order, not wanting to be the supplier for a threat to their existence. But it won’t help them. Somebody will be that supplier. If not a German, Japanese or U.S. company, then a Korean company, or failing that a Chinese company. In fact, Foxconn has said it is interested in making cars, and Apple is designing them, so the Apple-Foxconn relationship may be far more than a metaphor for this situation.

When you summon an Uber, you don’t care what nameplate is on the car. When you summon UberSelect, you don’t care if it’s Lexus, or Mercedes or BMW. Uber was your brand, and you aren’t buying the car for 15 years, you are buying it for 15 minutes. Brand plays a completely different role.

Companies like Waymo, Apple, Uber, Zoox and others would be foolish to manufacture cars, unless they want a car so radically different that nobody knows how to make it. (Then, they might decide to be the first to figure it out.) The car manufacturers would be foolish to turn down the giant purchase order, or partnerships with whoever has the best technology.

The winner of the transportation game of the future will be the company that thinks outside the car. That doesn’t mean the big car companies can’t do that. It’s just harder for them to do.

The chart’s not entirely wrong. Honda is pretty far behind — but PSA is even further behind. BMW, Daimler and Ford are among the best of the car companies, but Tesla and Volvo deserve higher ranking. Hyundai is not ahead of Toyota, and Tesla, while not ahead of Waymo, is in a pretty good place. Bosch is a surprising absentee from the list. FCA should be on it, just very low on the chart, along with the smaller Japanese vendors and many Chinese vendors.

But be clear. Making the car is essential, but it’s also old and a commodity. The value will lie in those building the self-driving software systems and sensors, and those putting services together around the technologies. The big automaker’s advantages — nameplate and reputation, reliability, manufacturing skill and capacity, retail channel experience — these are all less valuable or commoditized. They have to act fast to move to new business models that will make it in the future. Of course, one plan is to own the important components I name above, and several companies like BMW, Daimler, Ford, Nissan and Volvo are trying to do that. But they’re behind Waymo by a fair distance.

To fix gerrymandering, a test is needed -- or an interstate compact

One of the key flaws in the US political system is gerrymandering. I have written about this before even proposing my own method of redistricting, but such proposals only have a limited utility.

In this article I present why court solutions have had trouble, and a potentially new approach using an interstate compact.

Gerrymandering is particularly bad in the USA, but it’s a general “bug” in many democratic systems. The flaw is often summed up with the phrase “The politicians pick the voters instead of vice versa.” When the incumbent legislators and parties can draw the districts, they can bias the system heavily in their favour. In the USA, the house of representatives is currently highly biased towards the Republican party. It is often cited that the Republicans won 49.9% of popular votes for congress but got 55% of the seats. You can’t actually add the individual house votes, because people vote (or rather stay away) differently in safe districts than they do in contested one, but the margin is large enough that the trend is clear.

This is in large part due to Operation Redmap which is documented in the book Ratfucked. It truly fits the description “fiendishly clever plan” and exploits the bug to the level of making it close to permanent.

How districts are drawn is left to the states both in the constitution and the law. Some states have moved to create more fair districting rules, the sort of rules you would make up if you were doing it from a nonpartisan standpoint. However, the hard fact is that those states which do this are chumps. It does not make the system more fair if one side stops cheating — and I do think of gerrymandering as cheating — and the other side keeps on cheating. It just assures victory for the cheating side going forward. At the same time, having all sides cheat indefinitely is not a good solution either.

The constitution says very little about districting. In fact, it doesn’t even demand districts! States could have, if they chose, selected their representatives in a statewide proportional vote. Later federal laws, however, have demanded each person have one congress member, which demands geographic districts. About half the states require the districts be contiguous, but the others don’t. The voting rights act and other principles have forbidden drawing the lines on racial or minority grounds, but not on the grounds of “this helps incumbents keep their seats” — that’s still largely within the rules.

In any event, as long as gerrymandering is benefiting the GOP, they are not going to commit political suicide to remove it. States controlled strongly by one party or the other will resist willfully hurting their own parties, though there are exceptions when states have ballot resolutions. The supreme court ruled, barely, that the public can supersede the legislatures on this matter with a ballot proposition, and so that has happened. While the public belong to parties, they are actually more interested in fairness than party loyalty.

A constitutional amendment could fix this, but that’s not going to happen. And strong federal law could probably fix it, but that’s not coming from houses controlled by the people which benefit from the cheating.

As such, the solution can only come from the courts, or ballot propositions in a balanced set of states.

A good summary of the rules around districting in the different states can be found at this site.

But it’s not actually fair play, say the courts

Justices of the supreme court have reportedly all denounced gerrymandering to cement political control. They agree that it violates the principles of the constitution of one person one vote and equal protection, as it effectively eliminates for partisan reasons the voting power of many. Even agreeing with this, for now they feel powerless to stop it.

We can all see gerrymandering happen, but for the courts to do something about it, they would need to define fair and unbiased test which says when it is happening. This is hard, as courts are reluctant to write sets of rules like that — that is the province of the other branches of government. Courts don’t make the rules, they just decide if people are playing fairly by the rules that the other branches created.

So while it’s easy for you or I to propose fair rules for districting — rectangular districts or my own convexity test above — these just aren’t the sort of rules courts are willing to make up. You can’t extract them from the constitution. A court can look at a crazily shaped district and know “this is unfair” but it has to come up with a way that the states can objectively know what is fair and what isn’t, without being the author of its own rules.

One proposed rule that’s been advocated is the voting efficiency gap. Here, they try to measure how many votes were “wasted” because of district design. If a district went 80% for one party and 20% for the other, 30% of party A’s votes are wasted, and20% of party Bs, and the difference between these numbers tells how biased that election was.

It’s a nice test but one can see immediate flaws. For example, in a state biased 55% to 45%, a “perfect” districting where every district has the same balance as the state would result in 100% of seats for the dominant party. Since one party is strong in cities and the other strong in the country, any geographic set of districts is going to have these “inefficiencies” with inner cities voting 80% Democratic in the same state as a rural district votes 80% Republican — without any intent to cheat in how the lines are drawn. As noted, proportional non-geographic districts are not going to happen.

The courts, if they are to help us, need a test which will clearly let them tell states, “If you don’t draw your districts to match this test, they will be ruled invalid.” It’s easy to come up with fair, non-partisan tests to use, but the problem is that it is easy and so there are several you could use — and why should one be chosen over another? The legislatures can choose one option from many, but the courts are not to be arbitrary in that way. Their test has to clearly match some principle they find in the law.

You can propose convexity, or straight lines, or random selection — but none of them answer the question of “why does the law demand that particular one, vs. another?” They will ask this because any system, even if non-partisan, will benefit one party more than a different choice and thus have the appearance of being chosen from the pool for a partisan reason. And perhaps more than the appearance.

Ballot propositions and a State Compact

Individual states deciding to play fair just cede their power. Perhaps another option is possible — through a compact of states dedicated to fair districting.  read more »

Flying cars, electogliding and noise

The recently released national noise map makes it strikingly clear just how much air travel contributes to the noise pollution in our lives. In my previous discussion of flying cars I expressed the feeling that the noise of flying cars is one of their greatest challenges. While we would all love a flying car (really a VTOL helicopter) that takes off from our back yards, we will not tolerate our neighbour having one if there is regular buzzing and distraction overhead and in the next yard.

Helicopters are also not energy efficient, so real efforts for flying cars are fixed wing, using electric multirotors to provide vertical take-off but converting in some way to fixed wing flight, usually powered by those same motors in a different orientation. If batteries continue their path of getting cheaper, and more importantly lighter, this is possible.

Fixed wing planes can be decently efficient — particularly when they travel as the crow flies — though they can have trouble competing with lightweight electric ground vehicles. Almost all aircraft today fly much faster than their optimum efficiency speed. There are a lot of reasons for this. One is the fact that maintenance is charged by the hour, not the mile. Another is that planes need powerful engines to take off, and people are in a hurry and want to use that powerful engine to fly fast once they get up there.

Typical powered planes have a glide ratio (which is a good measure of their aerodynamic efficiency) around 10:1 to 14:1. That means for every foot they drop, they go forward 10 to 14 feet. Gliders, more properly known as “sailplanes” are commonly at a 50:1 glide ratio today and go even higher. Sailplane pilots can use that efficiency to enter slowly rising columns of air found over hot spots on the ground and “soar” around in a circle to gain altitude, staying up for hours. Silent flying is great fun, though the tight turns to rise in a thermal can cause nausea. Efficient sailplanes are also light and can have fairly bumpy rides. (Note as well that the extra weight of energy storage and motors and drag of propellers means a lower glide ratio.)

It is the silent flight that is interesting. An autonomous high efficiency aircraft, equipped with redundant electric motors and power systems, need not run its engines a lot of the time. While you would never want to be constantly starting and stopping piston powered aircraft engines, electric engines can start and stop and change speed very quickly. The motors provide tremendous torque for fast response times. It would be insane to regularly land your piston powered aircraft without power, figuring you can just turn on the engine “if you need it.” It might not be that crazy to do it in an electric aircraft when you can get the engine up and operating in a fraction of a second with high reliability, and you have multiple systems, so even the rare failures can be tolerated.

Both passengers and people on the ground would greatly appreciate planes that were silent most of the time, including when landing at short airstrips. It could make the difference for acceptance.

For a more radical idea, consider my more futuristic proposal of airports that grab and stop planes with robotic platforms on cables. Such a system would even allow for mostly silent takeoff in electric aircraft.

Making efficient aircraft VTOL is a challenge. They tend to have large wingspans and are not so suitable for backyards, even if they can hover. But the option for redundant multirotor systems makes possible something else — aircraft wings that unfold in the air. There are “flying cars” with folding wings which fold the wings up so the car can get on the road, but unfolding in the air is one of those things that is insane for today’s aircraft designs. A VTOL multirotor could rise up, unfold its wings, and if they don’t unfold properly, it can descend (noisily) on the VTOL system, either to where it took off form, or a nearby large area if the wings unfolded but not perfectly. An in-flight failure of the folding system could again be saved (uncomfortably but safely) by the VTOL system.

We don’t yet know how to make powered vertical takeoff or landing quiet enough. We might make the rest of flight fairly silent, and make the noisy part fairly brief. The neighbours don’t all run their leaf blower several times per day. But a combination of robocars that take you on the first and last kilometer to places where aircraft can make noise without annoyance if they do it briefly might be a practical alternative.

Planes that fly silently would not fit well with today’s air traffic control regiments that allocate ranges of altitude to planes. A plane with a 50:1 ratio could travel 10 miles while losing 1,000 feet of altitude, then climb back up on power for another silent pass. But constant changing of altitude would freak out ATC. A computerized ATC for autonomous planes could enable entirely different regimens of keeping planes apart that would allow this, and it would also allow long slow glides all the way to the runway.

LIDAR (lasers) and cameras together -- but which is more important?

Recently we’ve seen a series of startups arise hoping to make robocars with just computer vision, along with radar. That includes recently unstealthed AutoX, the off-again, on-again efforts of comma.ai and at the non-startup end, the dedication of Tesla to not use LIDAR because it wants to sell cars today, before LIDARs can be bought at automotive quantities and prices.

Their optimism is based on the huge progress being made in the use of machine learning, most notably convolutional neural networks, at solving the problems of computer vision. Milestones are dropping quickly in AI and particularly pattern matching and computer vision. (The CNNs can also be applied to radar and LIDAR data.)

There are reasons pushing some teams this way. First of all, the big boys, including Google, already have made tons of progress with LIDAR. There right niche for a startup can be the place that the big boys are ignoring. It might not work, but if it does, the payoff is huge. I fully understand the VCs investing in companies of this sort, that’s how VCs work. There is also the cost, and for Tesla and some others, the non-availability of LIDAR. The highest capability LIDARs today come from Velodyne, but they are expensive and in short supply — they can’t make them to keep up with the demand just from research teams!

Note, for more detailed analysis on this, read my article on cameras vs. lasers.

For the three key technologies, these trends seem assured:

  1. LIDAR will improve price/performance, eventually costing just hundreds of dollars for high resolution units, and less for low-res units.
  2. Computer vision will improve until it reaches the needed levels of reliability, and the high-end processors for it will drop in cost and electrical power requirements.
  3. Radar will drop in cost to tens of dollars, and software to analyse radar returns will improve

In addition, there are some more speculative technologies whose trends are harder to predict, such as long-range LWIR LIDAR, new types of radar, and even a claimed lidar alternative that treats the photons like radio waves.

These trends are very likely. As a result, the likely winner continues to be a combination of all these technologies, and the question becomes which combination.

LIDAR’s problem is that it’s low resolution, medium in range and expensive today. Computer Vision (CV)’s problem is that it’s insufficiently reliable, depends on external lighting and needs expensive computers today. Radar’s problem is super low resolution.

Option one — high-end LIDAR with computer vision assist

High end LIDARs, like the 32 and 64 laser units favoured by the vast majority of teams, are extremely reliable at detecting potential obstacles on the road. They never fail (within their range) to differentiate something on the road from the background. But they often can’t tell you just what it is, especially at a distance. It won’t know a car from a pickup truck, or 2 pedestrians from 3. It won’t read facial expressions or body language. It can read signs but only when they are close. It can’t see colours, such as traffic signals.

The fusion of the depth map of LIDAR with the scene understanding of neural net based vision systems is powerful. The LIDAR can pull the pedestrian image away from the background, and then make it much easier for the computer vision to reliably figure out what it is. The CV is not 100% reliable, but it doesn’t have to be. Instead, it can ideally just improve the result. LIDAR alone is good enough if you take the very simple approach of “If there’s something in the way, don’t hit it.” But that’s a pretty primitive result that make brake too much for things you should not brake for.

Consider a bird on the road, or a blowing trash bag. It’s a lot harder for the LIDAR system to reliably identify those things. On the other hand, the visions systems will do a very good job at recognizing the birds. A vision system that makes errors 1 time every 10,000 is not adequate for driving. That’s too high an error rate as you encounter thousands of obstacles every hour. But missing 1 bird out of 10,000 means that you brake unnecessarily for a bird perhaps once every year or two, which is quite acceptable.

Option two — lower end LIDAR with more dependence on vision

Low end lidars, with just 4 or so scanning planes, cost a lot less. Today’s LIDAR designs basically need to have an independent laser, lens and sensor for each plane, and so the more planes, the more cost. But that’s not enough to identify a lot of objects, and will be pretty deficient on things low to the ground or high up, or very small objects.

The interesting question is, can the flaws of current computer vision systems be made up for by a lower-end, lower cost LIDAR. Those flaws, of course, include not always discerning things in their field. They also include needing illumination at night. This is a particular issue when you want a 360 degree view — one can project headlights forward and see as far as they see, but you can’t project headlights backward or to the side without distracting drivers.

It’s possible one could use infrared headlights in the other directions (or forward for that matter.) After all, the LIDAR sends out infrared laser beams. There are eye safety limits (your iris does not contract and you don’t blink to IR light) but the heat output is also not very high.

Once again, the low end lidar will eliminate most of the highly feared false negatives (when the sensor doesn’t see something that’s there) but may generate more false positives (ghosts that make the vehicle brake for nothing.) False negatives are almost entirely unacceptable. False positives can be tolerated but if there are too many, the system does not satisfy the customer.

This option is cheaper but still demands computer vision even better than we have today. But not much better, which makes it interesting.

Other options

Tesla has said they are researching what they can do with radar to supplement cameras. Radar is good for obstacles in front of you, especially moving ones. Better radar is coming that does better with stationary objects and pulls out more resolution. Advanced tricks (including with neural networks) can look at radar signals over time to identify things like walking pedestrians.

Radar sees cars very well (especially licence plates) but is not great on pedestrians. On the other hand, for close objects like pedestrians, stereo vision can help the computer vision systems a lot. You mostly need long range for higher speeds, such as the highways, where vehicles are your only concern.

Who wins?

Cost will eventually be a driver of robocar choices, but not today. Today, safety is the only driver. Get it safe, before your competitors do, at almost any cost. Later make it cheap. That’s why most teams have chosen the use of higher end LIDAR and are supplementing in with vision.

There is an easy mistake to make, though, and sometimes the press and perhaps some teams are making it. It’s “easy” on the grand scale to make a car that can do basic driving and have a nice demo. You can do it with just LIDAR or just vision. The hard part is the last 1%, which takes 99% of the time, if not more. Google had a car drive 1,000 miles of different roads and 100,000 total roads in the first 2 years of their project back in 2010, and even in 2017 with by far the largest and most skilled team, they do not feel their car is ready. It gets easier every day, as tech advances, to get the demo working, but that should not be mistaken for the real success that is required.

California New Regs, Intel buys MobilEye, Waymo sues Uber

California has published updated draft regulations for robocars whose most notable new feature is rules for testing and operating unmanned cars, including cars which have no steering wheel, such as Google, Navya, Zoox and others have designed.

This is a big step forward from earlier plans which would have banned testing and deploying those vehicles. That that they are ready to deploy, but once you ban something it’s harder to un-ban it.

One type of vehicle whose coverage is unclear are small unmanned delivery robots, like we’re working on at Starship. Small, light, low speed, inherently unmanned and running mostly on the sidewalks they are not at all a fit for these regulations and presumably would not be covered by them — that should be made more explicit.

Another large part of the regulations cover revoking permits and the bureaucracy around that. You can bet that this is because of the dust-up between the DMV and Uber/Otto a few months ago, where Uber declared that they didn’t need permits (probably technically true) but the DMV found it not at all in the spirit of the rules and revoked the licence plates on the cars. The DMV wants to be ready to fight those who challenge its authority.

Intel buys MobilEye

Intel has paid over $15B to buy Jerusalem based MobilEye. MobilEye builds ASIC-based camera/computer vision systems to do ADAS and has been steadily enhancing them to work as a self-driving sensor. They’ve done so well the stock market already got very excited and pushed them up to near this rich valuation — the stock traded at close to this for a while, but fell after ME said it would no longer sell their chips to Tesla. (Tesla’s first autopilot depended heavily on the MobilEye, and while ME’s contract with Tesla explicitly stated it did not detect things like cross-traffic, that failure to detect played a role in the famous Tesla autopilot fatal crash.

In a surprising and wise move, Intel is going to move its other self-driving efforts to Israel and let MobilEye run them, rather than gobble them up and swallow/destroy them. ME is a smart company, fairly nimble, though it has too much focus on making low-cost sensors in a world where safety at high cost is better than less safety at low cost. (Disclaimer: I own some MBLY and made a nice profit on it in this sale.)

MobilEye has been the leader in doing ADAS functions with just cameras and cameras+radar. Several other startups are attempting this, and of course so is Tesla in their independent effort. However, LIDAR continues to get cheaper (with many companies, including Quanergy, whom I advise, working hard on that.) The question may be shifting from will it be cameras or lasers? to “will it be fancy vision systems with low-end LIDAR, or will it be high-end LIDAR with more limited vision systems?” In fact, that question deserves another post.

Waymo and Uber Lawsuit

I am not going to comment a great deal on this lawsuit, because I am close with both sides, and have NDAs with both Otto and formerly with Google/Waymo. There are lots of press reports on the lawsuit, filed by Waymo accusing Anthony Levandowski (who co-founded Otto and helped found the car team at Google) of stealing a vast trove of Google’s documents and designs. This fairly detailed Bloomberg report has a lot of information, including reports that at an internal meeting, Anthony told his colleagues that any downloading he did was simply to allow work from home.

The size of the lawsuit is staggering. Since Otto sold for 1% of Uber stock (worth over $750M) the dollar values are huge, particularly if, as Google alleges, they can demonstrate Uber encouraged wrongdoing. At the same time, if Google doesn’t prove their allegations, Otto and Anthony could file for what might be the largest libel lawsuit in history, since Google published their accusations not just in court filings, but in their blog.

One reason that might not happen is that Uber is seeking to force arbitration. Like almost all contracts these days, the contracts here included clauses forcing disputes to go to arbitrators, not courts. That will mean that the resolution and other data remain secret.

It’s very serious for both sides. Some have said it’s mission critical for Uber, though I have disputed that, pointing out that even if Uber fails to develop good self-drive technology, they remain free to buy it from other people. That’s something the other players can’t do — even Lyft which has bound itself up with GM for now.

At the same time, Uber should fear something else. Uber is nothing, a $0 company, without iPhone and Android. (There is a Windows mobile app but it’s very low penetration.) Uber could push all drivers to iPhone, but if they ever found themselves unable to use Android for customers, they would lose more than they can afford.

I am not suggesting Google would go as far as to pull or block the Uber app on Android if it got into a battle. Aside from being unethical that might well violate antitrust regulations. But don’t underestimate the risk of betting half your business on a platform controlled by a company you go to war with. There are tricks I can think of (but am not yet publishing here) which Google could do which would not be seen as unfair or anti-competitive but which could potentially ruin Uber. Uber and Google will both have to be cautious in any serious battle.

In other Uber news, leaked reports say their intervention rate is still quite high. Intervention figures can be hard to interpret. Drivers are told to intervene at the smell of trouble, so the rate of grabbing the wheel can be much higher than the rate of actual problems. These leaks suggest, however, a fairly high rate of actual problems. This should remind people that while it’s pretty easy for a skilled team to get a car on the road and doing basic driving in a short time, there is a reason that Google’s very smart team has been at it 9 years and is still not ready to ship. The last 1% of the work takes 99% of the time.

Digitizing your papers, literally, for the future, with 4K video

I have so much paper that I’ve been on a slow quest to scan things. So I have high speed scanners and other tools, but it remains a great deal of work to get it done, especially reliably enough that you would throw away the scanned papers. I have done around 10 posts on digitizing and gathered them under that tag.

Recently, I was asked by a friend who could not figure out what to do with the papers of a deceased parent. Scanning them on your own or in scanning shops is time consuming and expensive, so a new thought came to me.

Set up a scanning table by mounting a camera that shoots 4K video looking down on the table. I have tripods that have an arm that extends out but there are many ways to mount it. Light the table brightly, and bring your papers. Then start the 4K video and start slapping the pages down (or pulling them off) as fast as you can.

There is no software today that can turn that video into a well scanned document. But there will be. Truth is, we could write it today, but nobody has. If you scan this way, you’re making the bet that somebody will. Even if nobody does, you can still go into the video and find any page and pull it out by hand, it will just be a lot of work, and you would only do this for single pages, not for whole documents. You are literally saving the document “for the future” because you are depending on future technology to easily extract it.  read more »

Only the Republicans can bring down Trump

Sooner than most expected, the Trump administration is in trouble. Many are talking about how to end it, or hasten that end.

  1. The Democrats don’t have the power to take down Trump prior to 2020. Not even after 2018.
  2. The revolt against Trump almost surely has to come from within his own party.
  3. While many Republicans dislike Trump, revolt within a party is extremely difficult and goes against all party instincts.
  4. Republicans will strongly resist fighting Trump as the left would like, or in a way which benefits the left.
  5. As such, the more the left approves of a method of fighting Trump, the less likely it is the Republicans would use it.
  6. This suggests a very different anti-Trump strategy than the obvious one followed by most.

Many in the GOP would prefer not to have Trump, and are ready to be disloyal to him as their leader. They are not, however, prepared to be disloyal to their party and their movement. Career party members of both sides often will put loyalty to party ahead of loyalty to country, even though they would never admit that. This means that if the GOP does this, it must be for their own reasons, not the left’s, and it must clearly not appear to serve the left except in the broadest way.

This creates a conundrum for the left fighting Trump. If they rally around something, such as a Trump error, they push the right to reluctantly defend Trump on that issue. Many GOP can’t stand Trump but support him because the alternative is victory for the left, and injury for their party. As such, the best strategy for the left may be to pull back, or stick only to issues that are clearly their own.

The Democrats might consider strategies that are victories for the GOP. Conceding important items in congress in exchange for impeachment. The Republicans know the Democrats will vote for impeachment, so only a minority of Republicans need support it, but for them, a party divided like that is no victory. This may mean offering support for portions of Pence’s or the party’s agenda. Something so that the entire GOP can see it as a victory for their party. The Democrats lost in 2016, and they must accept that, and give up the hope that Trump’s fall would be good for the Democratic Party. They must accept only that it will be good for the country and neutral, or even slightly negative for the party.

It’s a common human foible but politicians cringe from ever admitting they were wrong. Those who supported Trump, even holding their noses, won’t see themselves as having failed. They won’t go, “Oh, you Democrats were right, sorry about that.” The reason will need to be something new, something few people knew or talked about before now. People are just less likely to do the right thing if they know it’s what their opponents want them to do.

The Democrats, however, are not a cohesive force. Even if “hold back and let the GOP do it” is the right plan, they will not embrace it in large numbers. Thus they will slow down the fall of Trump. This was a frequent mistake made during the election — the unprecedented level of contempt by the left for Trump and in particular for Trump supporters brought the Trump supporters together and made them stronger, rather than weakening them. It was a strong contributor to the Trump victory.

This advice does not mean, “Only complain about Trump in a way that the right-wing will understand.” Normally that is the best approach. Here, the problem is that as soon as a complaint is seen as coming from the left, there will be resistance to acting on it.

2018

Some hold out for a change of Congress in 2018. It is quite normal for the President’s party — especially an unpopular President — to lose seats in the mid-terms. Unfortunately, the senate seats up in 2018 are far from likely to swing the senate to the Democrats. In fact, only 9 Republican seats are up for re-election with only Nevada at risk, and many of the Democrat incumbents are in pro-Trump states. It would take an immense voter revolt to not have the Senate become more Republican. In the House, Operation Redmap has assured Republican control short of a very major shift, and it also seems to mostly assure — absent some sort of court ruling against Gerrymandering — that they will get to draw the lines again in 2020 and continue it for another decade.

The Deep State

One group that can take down Trump aside from the Republicans is the intelligence agencies. Many speculate that this is already underway. This is extremely troubling to me. A coup d’état by the intelligence agencies is still a coup, even if it meets some test of “being a coup that needed to happen.” This is a bad precedent because the truth is the intelligence agencies have deep dirt on everybody, so it becomes up to them to decide which coups need to happen and which don’t. (Indeed, we saw the Russian agencies use this power already.)

There are already checks and balances for this. If the agencies find evidence of treason or malfeasance by one branch, they should present it to the other branch to act. All evidence should go to the congressional intelligence committees. But that means that again, the Republicans must decide whether to take down their own.

The Press

The press can play a role, but mainly the right-wing or right-of-center press. Again, it is their criticism of Trump that would enable the Republicans to break party loyalty, not criticisms found in media even perceived to be left or otherwise inherently anti-Trump. This is one reason Trump has worked to push more media into that classification, because it means their attacks will not be respected by his base and his party.

Trump’s base is not the mainstream GOP

The strongest counter to this approach is that Trump won the GOP nomination (and election ) due to support outside the mainstream GOP, merged with support from the party-loyal factions in the mainstream GOP. He has a tool to use against his opponents within the mainstream GOP, the same tool he used to defeat them in the nomination process. So even they must take care, for while they care most about alienating their own base, and least about alienating the progressive left, they are worried about alienating the “outsider right” contingency that Trump stumbled upon. They ideally want to be seen as having done the best thing for the party and the country in any efforts they make to block, or remove, the President.

Electrify Caltrain? Or could robocars do it for less than 1.5 billion?

Caltrain is the commuter rail line of the San Francisco peninsula. It’s not particularly good, and California is the land of the car commuter, but a plan was underway to convert it from diesel to electric. This made news this week as the California Republican house members announced they want to put a stop to both this project, and the much larger California High Speed Rail that hopes to open in 2030. For various reasons they may be right about the high speed rail but stop the electric trains? Electric trains are much better than diesel; they are cleaner and faster and quieter. But one number stands out in the plan.

To electrify the 51 miles of track, and do some other related improvements is forecast to cost over 1.5 billion dollars. Around $30M per mile.

So I started to ask, what other technology could we buy with $1.5 billion plus a private right-of-way through the most populated areas of silicon valley and the peninsula? Caltrain carries about 60,000 passengers/weekday (30,000 each way.) That’s about $50,000 per rider. In particular, what about a robotic transit line, using self-driving cars, vans and buses?

Paving over the tracks is relatively inexpensive. In fact, if we didn’t have buses, you could get by with fairly meager pavement since no heavy vehicles would travel the line. You could leave the rails intact in the pavement, though that makes the paving job harder. You want pavement because you want stations to become “offline” — vehicles depart the main route when they stop so that express vehicles can pass them by. That’s possible with rail, but in spite of the virtues of rail, there are other reasons to go to tires.

Fortunately, due to the addition of express trains many years ago, some stations already are 4 tracks wide, making it easy to convert stations to an express route with space by the side for vehicles to stop and let passengers on/off. Many other stations have parking lots or other land next to them allowing reasonably easy conversion. A few stations would present some issues.

Making robocars for a dedicated track is easy; we could have built that decades ago. In fact, with their much shorter stopping distance they could be safer than trains on rails. Perhaps we had to wait to today to convince people that one could get the same safety off of rails. Another thing that only arrived recently was the presence of smartphones in the hands of almost all the passengers, and low cost computing to make kiosks for the rest. That’s because the key to a robotic transit line would be coordination on the desires of passengers. A robotic transit line would know just who was going from station A to station J, and attempt to allocate a vehicle just for them. This vehicle would stop only at those two stations, providing a nonstop trip for most passengers. The lack of stops is also more energy efficient, but the real win is that it’s more pleasant and faster. With private ROW, it can easily beat a private car on the highways, especially at rush hour.

Another big energy win is sizing the vehicles to the load. If there are only 8 passengers going from B to K, then a van is the right choice, not a bus. This is particularly true off-peak, where vast amounts of energy are wasted moving big trains with just a few people. Caltrain’s last train to San Francisco never has more than 100 people on it. Smaller vehicles also allow for more frequent service in an efficient manner, and late night service as well — except freight uses these particular rails at night. (Most commuter trains shut down well before midnight.) Knowing you can get back is a big factor in whether you take a transit line at night.

An over-done service with a 40 passenger bus every 2 seconds would move 72,000 people (but really 30,000) in one hour in one direction to Caltrain’s 30,000 in a day. So of course we would not build that, and there would only be a few buses, mainly for rush hour. Even a fleet of just 4,000 9 passenger minvans (3 rows of 3) could move around 16,000 per hour (but really 8,000) in each direction. Even if each van was $50,000 each, we’ve spent only $200M of our $1.5B, though they might wear out too fast at that price, so we could bump the price and give them a much longer lifetime.

Energy

These vans and cars could be electric. This could be done entirely with batteries and a very impressive battery swap system, or you could have short sections of track which are electrified — with overhead rails or even third rails. The electric lines would be used to recharge batteries and supercapacitors, and would only be present on parts of the track. Unlike old 3rd rail technology, which requires full grade separation, there are new techniques to build safe 3rd rails that only provide current in a track segment after getting a positive digital signal from the vehicle. This is much cheaper than overhead wires. Inductive charging is also possible but makes pavement construction and maintenance much more expensive.

Other alternatives would be things like natural gas (which is cheap and much cleaner than liquid fuels, though still emits CO2) because it can be refilled quickly. Or hydrogen fuel cell vehicles could work here — hydrogen can be refilled quickly and can be zero emissions. Regular fossil fuel is also an option for peak times. For example the rush hour buses might make more sense running on CNG or even gasoline. The lack of starts and stops can make this pretty efficient.

Stations, anywhere

In such a system, you can also add new “stations” anywhere the ROW is wide enough for a side-lane and a small platform. You don’t need the 100m long platform able to hold a big train, just some pavement big enough to load a van. You can add a new station for extremely low cost. Of course, with more stations, it’s harder to group people for nonstop trips, and more people would need to take two-hop trips — a small van or car that takes them from a mini-station to a major station, where they join a larger group heading to their true destination.

Of course, if you were designing this from scratch, you would make the ROW with a shoulder everywhere that allowed vehicles to pull off the main track at any point to pick up a passenger and there would barely be “stations” — they would be closer to bus stops.

Getting off the track

Caltrain’s station in San Francisco is quite far from most of the destinations people want to go to. It’s one of the big reasons people don’t ride it. Vans on tires, however, have the option of keeping going once they get to the station. Employers could sponsor vehicles that arrive at the station and keep driving to their office tower. Vans could also continue to BART or more directly to underground Muni, long before the planned subway is ready. Likewise on the peninsula, vans and buses would travel from stations to corporate HQ. Google, Yahoo, Apple and many other companies already run transit fleets to bring employees in — you can bet that given the option they would gladly have those vans drive the old rail line at express speeds. On day one, they could have a driver who only drives the section back and forth between the station and the corporate office. In the not too distant future, the van or bus would of course drive itself. It’s not even out of the question that one of the passengers in a van, after having taken a special driving test, could drive that last mile, though you may need to assure somebody drives it back.

At-grade crossings

I noted above that capacity would be slightly less than half of full. That’s because Caltrain has 40 at-grade crossings on the peninsula. The robotic vehicles would coordinate their trips to travel in bunches, leaving gaps where the cross-street’s light can be turned green. If any car was detected trying to run the red, the signal could be uploaded to allow all the robotic vans to slow or even brake hard. Unlike trains, they could brake in reasonable amounts of time if somebody stalls on the old track. You would also detect people attempting to drive on the path or walk on it. Today’s cameras and cheap LIDARs can make that affordable. The biggest problem is the gaps must appear in both directions (more on that in the comments.)

Over time, there is also the option in some places to build special crossings. Because the vans and cars would all be not very high, much less expensive underpasses could be created under some of the roads for use only by the smaller vehicles. Larger vehicles would still need to bunch themselves together to leave gaps for the cross-traffic. One could also create overpasses rated only for lightweight vehicles at much lower cost, though those would still need to be high enough for trucks to go underneath. In addition, while cars can handle much, much steeper grades than trains, it could get disconcerting to handle too much up and down at 100mph. And yes, in time, they would go 100mph or even faster. And in time, some would even draft one another to both increase capacity and save energy — creating virtual trains where there used to be physical ones.

And then, obsolete

This robotic transit line would be much better than the train. But it would also be obsolete in just a couple of decades! As the rest of the world moves to more robocars, the transit line would switch to being just another path for the robocars. It would be superior, because it would allow only robocars and never have traffic congestion. You would have to pay extra to use it at rush hour, but many vehicles would, and large vehicles would get preference. The stations would largely vanish as all vehicles are able to go door to door. Most of the infrastructure would get re-used after the transit line shuts down.

It might seem crazy to build such a system if it will be obsolete in a short time, but it’s even crazier to spend billions on shoring up 19th century train.

What about the first law?

I’ve often said the first law of robocars is you don’t change the infrastructure. In particular, I am in general against ideas like this which create special roads just for robocars, because it’s essential that we not imagine robocars are only good on special roads. It’s only when huge amounts of money are already earmarked for infrastructure that this makes sense. Now we are well on the way to making general robocars good for ordinary streets. As such, special cars only for the former rail line run less risk of making people believe that robocars are only safe on dedicated paths. In fact, the funded development would almost surely lead to vehicles that work off the path as well, and allow high volume manufacturing of robotic transit vehicles for the future.

Could this actually happen?

I do fear that our urban and transit planners are unlikely to be so forward looking as to abandon a decades old plan for a centuries old technology overnight. But the advantages are huge:

  • It should be cheaper
  • Many companies could do it, and many would want to, to fund development of other technology
  • It would almost surely be technology from the Bay Area, not foreign technology, though vehicle manufacturing would come from outside
  • They could also get money for the existing rolling stock and steel in the rails to fund this
  • The service level would be vastly better. Wait times of mere minutes. Non-stop service. Higher speeds.
  • The energy use would be far lower and greener, especially if electric, CNG or hydrogen vehicles are used

The main downside is risk. This doesn’t exist yet. If you pave the road to retain the rails embedded in them, you would not need to shut down the rail line at first. In fact, you could keep it running as long as there were places that the vans could drive around trains that are slowing or stopping in the stations. Otherwise you do need to switch one day.

Free speech theory explained

There’s been a lot of talk this week on the nature of free speech. I’m a very strong defender of free speech, so I felt it would be worth laying out some of the reasons why “the first amendment is not just the law, it’s a good idea.” While I am not speaking for any particular organization, and am not a lawyer nor giving legal advice, my background includes things like:

  • Being the subject of the first big internet censorship battle, in 1987.
  • Being a plaintiff in ACLU v. Reno, which we won 9-0 in the supreme court, for which I was named a “Champion of Free Speech” by the ACLU.
  • 20 years with the Electronic Frontier Foundation, including 10 as chairman.

Two recent events has caused much debate. A viral video of somebody punching Richard B. Spencer, a man who gathers attention by promoting neo-nazi and whites-first rules has caused people to ask, “Isn’t it OK to punch a Nazi?” You see Spencer declaring “Hail Trump” and people doing Nazi salutes in one famous video.

There have also been two attempts by Breitbart writer Milo Yiannopoulos to speak at UC Davis and UC Berkeley that have been met with protests, calls that he be banned from speaking, and cancellations of his talks due to fear of violence. At UCB, a large group of apparent “black bloc” anarchists invaded a peaceful protest with violent acts and resulted in chaos and cancellation of the talk.

For a free speech supporter, the situation is fairly clear. No, it’s not OK to punch a Nazi (or in this case a wannabe neo-nazi) simply for what he says or what he is, even if it’s so-called “hate speech.” (In fact, that we don’t punch people for what they say is one of the important things that makes us better than Nazis.) And universities should not distinguish among speakers who are legitimately invited by members of the university community because of the content of their messages, even if it is hugely unpopular, offensive and hateful.

Here’s why:

Speech can be evil. But censorship is more evil.

It is a common mistake of those who say, “I am all in favour of free speech, but….” to imagine that we support free speech because speech is pure and can’t cause harm. This is the “sticks and stones” philosophy, but if you follow it, then it follows that if you can show that some speech is, unlike most speech, actually harmful, it is then OK to ban it.

While some speech is indeed harmless, important speech is powerful. It evokes change in the world, for good or ill. Speech can do great good and great harm. Consider the book “The Communist Manifesto” which advocates that to bring about an ideal communist society, one must begin with armed revolution and a “dictatorship of the proletariat” that uses draconian methods to work towards the pure goal. That idea has been used to create such dictatorships, and they have all been horrors. These dictatorships (particularly Stalin and Mao) perverted the ideas but used the ideals to justify acts which killed many tens of millions — leaving the Nazi holocaust in the dust. You can’t get much more evil or more proven harm. Yet we don’t ban that book.

Lots of speech is evil, but we have found no way to determine that reliably or in advance. As such, giving any entity the power to decide what speech is good and what is evil is a more dangerous proposition than just allowing all speech. For just as the idea in The Communist Manifesto have led to the death of millions, so much of the good in the world is also attributable to other ideas and books, including ones which were banned. We can’t grant an agency the power to decide what is good or bad without having them stamp out too much of the good. Nobody has the crystal ball that can do this, and history shows the terrible record of censorship agencies in the places that allow them.

There is also a practical angle. Censorship is only moderately effective. It’s probably slightly better at crushing good ideas than bad ones, but either way, for all the pain we get from censorship, it rarely actually stops the bad (or offensive or blasphemous) ideas from getting out. In fact, it is often of negative value, causing more publicity and support for the thing to be censored. (This worked for me when they tried to shut down my newsgroup, and later against Barbara Streisand to the extent that the principle was given her name.) In fact, I strongly suspect that the protests (even the peaceful ones) are doing precisely what Yiannopoulos wants. You think he cares that much about giving a talk to UC students? Or instead about the chance to be banned on the campus famous for the Free Speech movement of the 60s?

If we decide it’s going to be OK to punch some people for what they say, but not others, you need an arbiter who decides which speech is evil enough to warrant punching. And having that arbiter is a worse idea than letting the offensive person speak.

We have other ways to deal with bad speech

While there is bad speech, there is some merit to the “sticks and stones” argument in that people must be driven to action by the bad speech in order to get the harm. There, history shows that countering bad speech with good speech is a better, and certainly less dangerous counter-weapon than censorship. The answer to bad speech is more speech and more education.

There is a difference between speech and action

I will often hear people say that clearly some times of speech must be stopped — “what about shouting ‘fire!’ in a crowded theatre?”

That example is wrong for two important reasons. First, it’s fairly clear that shouting fire like this is not merely speech, but an action. It is the setting of a false fire alarm. It is like pulling the lever on the electronic fire alarm, which is easily seen as an action, and we can regulate actions. It is illegal to do a false fire alarm, particularly if it could cause a stampede.

Secondly, it’s a great demonstration of the evils of censorship. That argument became famous in the supreme court case Schenck v. United States. The case revolved around distributing leaflets which opposed the Draft in WWI. The court considered promoting resistance to the draft as akin to shouting fire in a crowded theatre. With our modern sensibility, we now see the debate about the merits of the draft to be an important one in a free society, one where all voices should be heard. Back then, they decided that the “incorrect” anti-draft position was so terrible it was like setting a false fire alarm. The reason why we can’t trust any agency to decide what speech is good and what is bad becomes very clear if you examine this case.

Generally, free speech law has allowed actions to be regulated but not speech. So setting a false fire alarm can be regulated. In addition, restrictions on the time and manner of speech can be regulated. They can make a rule prohibiting megaphones, but they can’t make a rule which ends up prohibiting megaphones based on what is said through them.

They can also make rules against conspiring to commit crimes. “Let’s attack John Smith” is more than speech, it is conspiracy to commit assault. “John Smith deserves assault” is not necessarily conspiracy, and the courts examine the circumstances in the borderline cases to see if the speech was also a threat, incitement or conspiracy. And yes, saying “It is OK to punch a Nazi” is speech when it’s an intellectual exercise, but more than speech when it turns into “let’s go down to the rally and punch Richard Spencer.” To count as incitement, the incited violent acts must be imminent, the path between the words and the violence must be clear and direct.

Hate speech is protected speech, at least in the USA

In many places, there have been efforts to define a special class of speech called “hate speech” and then to ban it. A number of countries, including Canada, have such laws. They are controversial and as predicted above, they have from time to time been used to attack political opponents of those in power rather than just shut down the Nazis and racists the way they are supposed to.

In the USA however, courts have consistently protected hate speech the same as any other speech.

Universities are held to an even higher standard

Many have been upset with universities allowing hate speakers to speak on campus. There are times when a student or professor wants to express an unpopular view, but more uproar comes when an outsider, like Yiannopoulos, is going to give an address.

Outsiders can’t generally come to universities, but often they get invites from people who are insiders. Yiannopoulos was invited by student Republican clubs, for example.

In the USA, the 1st amendment stops the government from censoring. The University of California is a state school, but it’s also a private institution, so there is debate on to what extent the 1st amendment governs it. (It does not govern totally private entities, such as a private club which can indeed decide what messages are allowed at club meetings.)

I’m not going to speak to that debate; rather I am going to invoke something much older than the 1st amendment, namely the traditions of academic freedom. For centuries, longer than any government or constitution has existed, universities have taken the principles of academic freedom as sacred. These principles declare an even higher bar. Universities are supposed to be the places that welcome controversial and dangerous views, views even the most enlightened governments of the world are afraid of. This has given us concepts like tenure, which assure faculty they will not be fired for expressing controversial views. History has taught us that so many of the most valuable ideas ever put forward began as controversial and banned thoughts in mainstream society.

As such, over and above any 1st amendment duties, universities, if they wish to honour their traditions, must set rules for who speaks based not at all on the message said by the speaker. They can limit locations and times. They can require external speakers to get an invitation from an accredited member of their community, but they must not treat a speaker of one message differently from another.

Indeed, there is an argument that if a speaker is so controversial, even within their own community, that there is fear of violence, that they should go the extra mile to provide extra protection rather than shy away in fear.

This does mean that a few dickheads will get to speak at universities to spout gibberish. That’s better than the alternative.

So is it OK to punch a Nazi?

Usually those asking this question point out that had the world punched/fought the real Nazis early on, the great horrors of the 20th century might have been averted. It is important to realize that this is clearly only obvious in hindsight. The people of the day did not have that vision at all. The Nazis, of course, got violent quite early on, so there were plenty of reasons to meet them with force if people had the will do do so. It was not a lack of moral clarity about “punching” them.

Indeed, at the end of the war, when the allies had almost all the Nazis captive, they tried them, and those who could be proven involved in the war crimes were executed or jailed. The others, in spite of killing many allied soldiers and civilians in battle, were set free. Including many members of the Nazi party.

Even when we had actual Nazis to deal with, the answer was not to punch them for what they were or what they said. They were punished if they were involved in the atrocities. Not for talking about them. If the actual victims of the real Nazis could do that, it seems odd for people today to claim to be wiser about it.

While the real Nazis are best known for killing people for their ethnicity and religion, they were also ready to do it for ideology, politics or sexual orientation, and many communists or simple political opponents were persecuted, rounded up and executed for it. Punching people for their beliefs is what Nazis do, not us. Instead, counter their ideology with better ideology, and be wary; for if they take up arms in their cause, it is certainly appropriate to respond with force.

California publishes robocar "intervention" reports -- Google/Waymo so far ahead it's ridiculous

California published its summary of all the reports submitted by vendors testing robocars in the state. You can read the individual reports — and they are interesting, but several other outlines have created summaries of the reports calculating things like the number of interventions per mile.

On these numbers, Google’s lead is extreme. Of over 600,000 autonomous miles driven by the various teams, Google/Waymo was 97% of them — in other words 30 times as much as everybody else put together. Beyond that, their rate of miles between disengagements (around 5,000 — a 4x improvement over 2015) is one or two orders of magnitude better than the others, and in fact for most of the others, they have so few miles that you can’t even produce a meaningful number. Only Cruise, Nissan and Delphi can claim enough miles to really tell.

Tesla is a notable entry. In 2015 they reported driving zero miles, and in 2016 they did report a very small number of miles with tons of disengagements from software failures (one very 3 miles.) That’s because Tesla’s autopilot is not a robocar system, and so miles driven by it are not counted. Tesla’s numbers must come from small scale tests of a more experimental vehicle. This is very much not in line with Tesla’s claim that it will release full autonomy features for their cars fairly soon, and that they already have all the hardware needed for that to happen.

Unfortunately you can’t easily compare these numbers:

  • Some companies are doing most of their testing on test tracks, and they do not need to report what happens there.
  • Companies have taken different interpretations of what needs to be reported. Most of Cruise’s disengagements are listed as “planned” but in theory those should not be listed in these reports. But they also don’t list the unplanned ones which should be there.
  • Delphi lists real causes and Nissan is very detailed as well. Others are less so.
  • Many teams test outside California, or even do most of their testing there. Waymo/Google actually tests a bunch outside California, making their numbers even bigger.
  • Cars drive all sorts of different roads. Urban streets with pedestrians are much harder than highway miles. The reports do list something about conditions but it takes a lot to compare apples to apples. (Apple is not one of the companies filing a report, BTW.)

One complication is that typically safety drivers are told to disengage if they have any doubts. It thus varies from driver to driver and company to company what “doubts” are and how to deal with them.

Google has said their approach is to test any disengagement in simulator, to find out what probably would have happened if the driver did not disengage. If there would have been a “contact” (accident) then Google considers that a real incident, and those are more rare than is reported here. Many of the disengagements are when software detects faults with software or sensors. There, we do indeed have a problem, but like human beings who zone out, not all such failures will cause accidents or even safety issues. You want to get rid of all of them, to be sure, but if you are are trying to compare the safety of the systems to humans, it’s not easy to do.

It’s hard to figure out a good way to get comparable numbers from all teams. The new federal guidelines, while mostly terrible, contain an interesting rule that teams must provide their sensor logs for any incident. This will allow independent parties to compare incidents in a meaningful way, and possibly even run them all in simulator at some level.

It would be worthwhile for every team to be required to report incidents that would have caused accidents. That requires a good simulator, however, and it’s hard for the law to demand this of everybody.

Uber talks of deal with Daimler which shows Uber's great advantage

I generally pay very little attention when companies issues a press release about an “alliance.” It’s usually not a lot more than a press release unless there are details on what will actually be built.

The recent announcement that Uber plans to buy some self-driving cars from Daimler/Mercedes is mostly just such an announcement — a future intent, when Mercedes actually builds a full self-driving car, that Uber will buy some. This, in spite of the fact that Uber has its own active self-driving system in development, and that it paid stock worth $760M to purchase freshly-minted startup Otto to accelerate that.

This shows a special advantage that Uber has over other players here. Their own project is very active, but unlike others, it doesn’t cripple Uber if it fails. Uber’s business is selling rides, and it will continue to be. If Uber can’t do it with its own cars, it can buy somebody else’s. Uber does not have the intention to make cars (neither does Google and that’s probably true of most other non-car companies.) There are many companies who will make cars to order for you. But if Google’s self-drive software (and hardware) project fails, they are left with very little. If Uber’s fails, they are still very much in business, but not as much in control of the underlying vehicles. As long as there are multiple suppliers for Uber to choose from, they are good.

One nightmare for the car companies is the reduction in value of their brands. If you summon “UberSelect” (the luxury Uber) you don’t care if it is a Lexus or Mercedes that shows up. As long as it’s a decent luxury car, you are good, because you are not buying the car, you are using it for 20 minutes. Uber is the brand you are trusting — and car companies fear that. I presume one thing that Daimler wants from this announcement is to remind people that they are a leader and may well be the supplier of cars to companies like Uber. But will they be in charge of the relationship? I doubt it.

Lyft should have the same advantage — but it took a $500M investment from GM which strongly pressures it to use whatever solution GM creates. Of course, if GM’s project fails, Lyft still has the freedom to use another, including Mercedes.

Other News

A lawsuit from Tesla against former Tesla autopilot team leader Sterling Anderson and former head of Google Chauffeur (now Waymo) Chris Urmson reveals little, other than the two have a company which will get a lot of attention in the space. But that’s enough. Google’s project is the most advanced one in the world. I was there and worked for Chris in its early days. Tesla’s is not necessarily the most advanced technologically — it has no LIDAR development — but it’s way ahead of others in terms of getting out there and deploying to gain experience, which has given it a headstart, especially in camera/radar based systems. The leaders of the two projects together will cause a stir in the auto business.

Flying cars are coming, what will they mean?

Earlier I posted my gallery of CES gadgets, and included a photo of the eHang 184 from China, a “personal drone” able, in theory, to carry a person up to 100kg.

Whether the eHang is real or not, some version of the personal automated flying vehicle is coming, and it’s not that far away. When I talk about robocars, I am often asked “what about flying cars?” and there will indeed be competition between them. There are a variety of factors that will affect that competition, and many other social effects not yet much discussed.

The VTOL Multirotor

There are two visions of the flying car. The most common is VTOL — vertical takeoff and landing — something that may have no wheels at all because it’s more a helicopter than a car or airplane. The recent revolution in automation and stability for multirotor helicopters — better known as drones — is making people wonder when we’ll get one able to carry a person. Multirotors almost exclusively use electric motors because you must adjust speed very quickly to get stability and control. You also want the redundancy of multiple motors and power systems, so you can lose a rotor or a battery and still fly.

This creates a problem because electric batteries are heavy. It takes a lot of power to fly this way. Carrying more batteries means more weight — and thus more power needed to carry the batteries. There are diminishing returns, and you can’t get much speed, power or range before the batteries are dead. OK in a 3 kilo drone, not OK in a 150 kilo one.

Lots of people are experimenting with combining multirotor for takeoff and landing, and traditional “fixed wing” (standard airplane) designs to travel any distance. This is a great deal more efficient, but even so, still a challenge to do with batteries for long distance flight. Other ideas including using liquid fuels some way. Those include just using a regular liquid fuel motor to run a generator (not very efficient) or combining direct drive of a master propeller with fine-control electric drive of smaller propellers for the dynamic control needed.

Another interesting option is the autogyro, which looks like a helicopter but needs a small runway for takeoff.

The traditional aircraft

Some “flying car” efforts have made airplanes whose wings fold up so they can drive on the road. These have never “taken off” — they usually end up a compromise that is not a very good car or a very good plane. They need airports but you can keep driving from the airport. They are not, for now, autonomous.

Some want to fly most of their miles, and drive just short distances. Some other designs are mostly for driving, but have an ability to “short hop” via parasailing or autogyro flying when desired.  read more »

NHTSA ODI report exonerates Tesla in fatal crash

NHTSA released the report from their Office of Defects Investigation on the fatal Tesla crash in Florida last spring. It’s a report that is surprisingly favorable to Tesla. So much so that even I am surprised. While I did not think Tesla would be found defective, this report seems to come from a different agency than the one that recently warned comma.ai that:

It is insufficient to assert, as you do, that the product does not remove any of the driver’s responsibilities” and “there is a high likelihood that some drivers will use your product in a manner that exceeds its intended purpose.”

The ODI report rules that Tesla properly considered driver distraction risks in its design of the product. It goes even further, noting that after the introduction of Tesla autopilot (including driving by those monitoring it properly, those who were distracted, and those who drove with it off) still had a decently lower accident rate for mile than drivers of Teslas before autopilot. In other words, while the autopilot without supervision is not good enough to drive on its own, the autopilot even with the occasionally lapsed supervision that is known to happen, combined with improved AEB and other ADAS functions, is still overall a safer system than not having the autopilot at all.

This will provide powerful support for companies developing autopilot style systems, and companies designing robocars who wish to use customer supervised driving as a means to build up test miles and verification data. They are not putting their customers at risk as long as they do it as well as Tesla. This is interesting (and the report notes that evaluation of autopilot distraction is not a settled question) because it seems probable that people using the autopilot and ignoring the road to do e-Mail or watch movies are not safer than regular drivers. But the overall collection of distracted and watchful drivers is still a win.

This might change as companies introduce technologies which watch drivers and keep them out of the more dangerous inattentive style of use. As the autopilots get better, it will become more and more tempting, after all.

Tesla stock did not seem to be moved by this report. But it was also not moved by the accident or other investigations — it actually went on a broadly upward course for 2 months following announcement of the fatality.

The ODI’s job is to judge if a vehicle is defective. That is different from saying it’s not perfect. Perfection is not expected, especially from ADAS and similar systems. The discussion about the finer points of whether drivers might over-trust the system are not firmly settled here. That can still be true without the car being defective and failing to perform as designed, or being designed negligently.

CES Gallery -- Smart Connected IoT home and more

I go to CES first to see the cars but it’s also good to see all the latest gadgets. My gallery, with captions you will see at the bottom as you page through them, provides photos and comments on interesting and stupid products and gadgets for this year.

Gallery of CES gadgets

CES always contains an amazing array of “What are they thinking?” products. This year, more than ever, we had more things that were made “smart” and “connected” for little reason one can discern. I was quite disappointed to read various media lists of top gadgets of CES 2017 and not find a single one that was actually exciting. There are a few that will be exciting one day — the clothes folding robot, the human carrying drone — but they are not here yet.

Rodney Brooks on Pedestrian Interaction, Andrew Ng on Infrastructure, and both on human attitudes

Recently we’ve seen two essays by people I highly respect in the field of AI and robotics. Their points are worthy of reading, but in spite of my respect, I have some differences of course.

The first essay comes from Andrew Ng, head of AI (and thus the self-driving car project) at Baidu. You will find few who can compete with Andrew when it comes to expertise on AI. (Update: This essay is not recent, but I only came upon it recently.)

In Wired he writes that Self-Driving Cars Won’t Work Until We Change Our Roads—And Attitudes. And the media have read this essay as being much more strong about changing the roads than he actually writes. I have declared it to be the “first law of robocars” that you don’t change the infrastructure. You improve your car to match the world you are given, you don’t ask the world to change to help your cars. There are several reasons I promote this rule:

  • As soon as you depend on a change in the world in order to drive safely, you have vastly limited where you can deploy. You declare that your technology will be, for a very long time, a limited area technology.
  • You have to depend on, and wait for others to change the world or their attitudes. It’s beyond your control.
  • When it comes to cities and infrastructure, the pace of change is glacial. When it comes to human behaviour, it can be even worse.
  • While it may seem that the change to infrastructure is clearer and easier to plan, the reality is almost assuredly the opposite. That’s because the clever teams of developers, armed with the constantly improving technologies driven by Moore’s law, have the ability to solve problems in a way that is much faster than our linear intuitions suggest. Consider measuring traffic by installing tons of sensors, vs. just getting everybody to download Waze. Before Waze, the sensor approach seemed clear, if expensive. But it was wrong.

As noted, Andrew Ng does not actually suggest that much change to the infrastructure. He talks about:

  • Having road construction crews log changes to the road before they do them
  • Giving police and others who direct traffic a more reliable way to communicate their commands to cars
  • Better painting of lane markers
  • More reliable ways to learn the state of traffic lights
  • Tools to help humans understand the actions and plans of robocars

Logging construction

The first proposal is one I have also made, because it’s very doable, thanks to computer technology. All it requires at first blush is a smartphone app in the hands of construction crews. Before starting a project, they would know that just as important as laying out cones and signs is opening the app and declaring the start of a project. The phone has a GPS and can offer a selection of precise road locations and log it. Of course, the projects should be logged even before they begin, but because that’s imperfect, smartphone logging is good enough. You could improve this by sticking old smartphones in all the road construction machines (old phones are cheap and there are only so many machines) so that any time a machine stops on a road for very long, it sends a message to a control center. Even emergency construction gets detected this way.

Even with all that, cars still need to detect changes to the road (that’s easy with good maps) and cones and machines. Which they can do.

Police redirection

I think the redirection problem is more difficult. Many people redirect traffic, even civilians. However, I would be interested to see Ng’s prediction on how hard it is to get neural network based recognizers to understand all the common gestures. Considering that computers are now getting better at reading sign languages, which are much more complex, I am optimistic here. But in any event, there is another solution for the cases where the system can’t understand the advice, namely calling in an operator in a remote control center, which is what Nissan plans to do, and what we do at Starship. Unmanned cars, with no human to help, will just avoid data dead zones. If somehow they get to them, there can be other solutions, which are imperfect but fine when the problem is very rare, such as a way for the traffic manager to speak to the car (after all, spoken language understanding is now close to a solved problem for limited vocabulary problems.)

Better lanes

Here I disagree with Andrew. His statement may be a result of efforts to drive on roads without maps, even though Baidu has good map expertise. Google’s car has a map of the texture of the road. It knows where the cracks and jagged lane markers are. The car actually likes degrading lane markers. It’s perfectly painted straight and smooth roads which confuse it (though only slightly, and not enough to cause a problem.) So no, I think that better line painting is not on the must-do list.

Seeing lights

He’s right, seeing lights can be challenging, though the better cars are getting good at it. The simple algorithm is “you don’t go if you don’t confirm green.” That means you don’t run a red but you could block traffic. If that’s very rare it’s OK. We can consider infrastructure to solve that, though I’m wary. Fortunately, if the city is controlling its lights with a central computer, you don’t have to alter the traffic light itself (which is hard,) you can just query the city, in those rare cases, for when the light will be changing. I think that problem will be solved, but I also think it may well be solved just by better cameras. Good robocars know exactly where all the lights are, and they know where they are, and thus they know exactly what pixels in a video image are from the light, even if the sun is behind it. (Good robocars also know where the sun is and will avoid stopping in a place where there is no light they can see without the sun right behind it.)

Working with people

How cars interact with people is one of Andrew Ng’s points and the central point of Rodney Brooks’ essay Unexpected Consequences of Self Driving Cars. Already many of the car companies have had fun experimenting with that, putting displays on the outside of cars of various sorts. While cars don’t have the body language and eye contact of human drivers, I don’t predict a problem we can’t solve with good effort.

Brooks’ credentials are also superb, as founder of iRobot (Roomba) and Rethink Robotics (Baxter) as well as many accomplishments as an MIT professor. His essay delves into one of the key questions I have wondered about for some time — how to deal with a world where things do not follow the rules, and where there are lots of implicit and changing rules and interactions. Google discovered the first instant of this when their car got stuck at a 4 way stop by being polite. They had to program the car to assert its right to go in order to handle the stop. Likewise, you need to speed to be a good citizen on many of our roads today.

His key points are as follows:

  • There is a well worked out dance between pedestrians and cars, that varies greatly among different road types, with give and take, and it’s not suitable for machines yet.
  • People want to know a driver has seen them before stepping near or certainly in front of a vehicle.
  • People jaywalk, and even expect cars to stop for them when they do on some streets.
  • In snowy places, people walk on the street when the sidewalk is not shoveled.
  • Foot traffic can be so much that timid cars can’t ever get out of sidestreets or driveways. Nice pedestrians often let them out. They will hand signal their willingness to yield or use body language.
  • Sometimes people just stand at the corner or edge of the road, and you can’t tell if they are standing there or getting ready to cross.
  • People setting cars to circle rather than park
  • People might jump out of their car to do something, leaving it in the middle of the street blocking traffic, where today they would be unwilling to double park.
  • People might abuse parking spots by having a car “hold” them for quick service when they want to leave an event.
  • Cars will grab early spots to pick up children at schools.

Brooks starts with one common mistake — he has bought into the “levels” defined by SAE, even claiming them to be well accepted. In fact, many people don’t accept them, especially the most advanced developers, and I outlined recently why there is only one level, namely unmanned operation, and so the levels are useless as a taxonomy. Instead the real taxonomy in the early days will be the difference between mobility on demand services (robotaxi) and self-drive enabled high end luxury cars. Many of his problems involve privately owned cars and selfish behaviour by their owners. Many of those behaviours don’t make sense in a world with robotaxis. I think it’s very likely that the robotaxis come first, and come in large numbers first, while some imagine it’s the other way around.

Unintended consequences

Brooks is right that there will be unintended consequences, and the technology will be put to uses nobody thought of. People will be greedy, and antisocial, that can be assured. Fortunately, however, people will work out solutions, in advance, to anything you can think of or notice just by walking down the street or thinking about issues for a few days. The experienced developers have been thinking about these problems for decades now, and cars like Google’s have driven for 300 human lifetimes of driving, and that number keeps increasing. They note every unusual situation they encounter on every road they can try to drive, and the put it into the simulator if it’s important. They’ve already seen more situations than any one human will encounter on those roads, though they certainly haven’t driven all the types of road in the world. But they will, before they certify as safe for deployment on such roads.

As I noted, only the “level 4” situation is real. Level 5 is an aspirational science-fiction goal, and the others are unsafe. Key to the improved thinking on “levels” it is no longer the amount of human supervision needed that makes the difference, it is the types of roads and situations you can handle. All these vehicles will only handle a subset of roads, and that is what everybody plans. If there is a road that is too hard, they just won’t drive it. Fortunately, there are lots of road subsets out there that are very, very useful and make economic sense. For a while, many companies planned only to do highways, which are the simplest road subset of all, except for the speed. A small subset, but everybody agrees it’s valuable.

So the short answer is, solutions will be found to these problems if the roads they occur on are commercially necessary. If they are not necessary, the solutions will be delayed until they can be found, though that’s probably not too long.

As noted above, many people do expect systems to be developed to allow dialogue between robocars and pedestrians or other humans. One useful tool is gaze detection — just as a cheap flash camera causes “red eye” in photos, machines shining infrared light can easily tell if you are looking at them. Eye contact in that direction is detectable. There have been various experiments in sending information in the reverse direction. Some cars have lasers that can paint lines on the road. Others can display text. Some have an LED ribbon surrounding them that shows all the objects and people tracked by the car, so people can understand that they are being perceived. You can also flash a light back directly at people to return their eye contact — I see you and I see that you saw me.

Over time, we’ll develop styles of communication, and they will get standarized. It’s not essential to do that on day one; you just stay on the simpler roads until you know you can handle the others. Private cars will pause and pop out a steering wheel. Services like Uber will send you a human driver in the early days if the car is going somewhere the systems can’t drive, or they might even let you drive part of it. Such incrementalism is the only way it can ever work.

People taking advantage of timidity of robocars

I believe there are solutions to some of the problems laid out. One I have considered is pedestrians and others who take advantage of the naturally conservative and timid nature of a robocar. If people feel they can safely cut off or jaywalk in front of robocars, they will. And the unmanned cars will mostly just accept that, though only about 10% of all cars should be unmanned at any given time. The cars with passengers are another story. Those passengers will be bothered if they are cut off, or forced to brake quickly. They will spill their coffee. And they will fight back.

Citizen based strong traffic code enforcement

Every time you jump in front of such a car, it will of course have saved the video and other sensor data. It’s always doing that. But the passenger might tell the car, “Please save that recent encounter. E-mail it to the police.” The police will do little with it at first, but in time, especially since there are rich people in these cars, they will throw a face recognizer and licence plate recognizer on the system that gets the videos. They will notice that one person keeps jaywalking right in front of the cars and annoying the passengers. Or the guy who keeps cutting off the cars as though they are not there because they always brake. They will have video of him doing it 40 times, or 100. And at that point, they will do something. The worst offender will get identified and get an E-mail from police. We have 50 videos of you doing this. Here are 50 tickets. Then the next, and the next until nobody wants to get to the top of the list.

This might actually create pressure the other way — a street that belongs only to the cars and excludes the non-car user. A traffic code that is enforced to the letter because every person inconvenienced has an ability to file a complaint trivially. We don’t want that either, but we can control that balance.

I actually look forward to fixing one of the dynamics of jaywalking that doesn’t work. Often, people like to jaywalk and a car is approaching. They want to have the car pass at full speed and then walk behind it — everybody is more comfortable behind a car than in front of one. But the driver gets paranoid and stops, and eventually you uncomfortably cross in front, annoyed at that and that you stopped somebody you didn’t intend to stop. I suspect robocars will be able to handle this dynamic better, predicting when people might actually be on a path to enter their lane, but not slowing down for stopped pedestrians (adults at least) and trust them to manage their crossing. Children are a different matter.

People being selfish with robocars

Brooks wonders about people doing selfish things with their robocars. Here, he mostly talks about privately owned robocars, since most of what he describes would not or could not happen with a robotaxi. There will be some private cars so we want to think about this.

A very common supposition I see here and elsewhere is the idea of a car that circles rather than parking. Today, operating a car is about $20/hour so that’s already completely irrational, and even when robocar operation drops to $8/hour or less, parking is going to be ridiculously cheap and plentiful so that’s not too likely. There could be competition for spots in very busy areas (schools, arenas etc.) which don’t have much space for pick-up and drop-off, and that’s another area where a bit of traffic code could go a long way. Allow facilities to make a rule: “No car may enter unless its passenger is waiting at the pick-up spot” with authority to ticket and evict any car that does otherwise. Over time, such locations will adjust their pick-up spots to the robocar world and become more like Singapore’s airport, which provides amazing taxi throughput with no cab lines by making it all happen in parallel. Of course, cars would wait outside the zone but robocars can easily double and triple park without blocking the cars they sit in the path of. Robocars waiting for passengers at busy locations will be able to purchase waiting spaces for less than the cost of circling, and then serve their customers or owners. If necessary, market prices can be put on the prized close waiting spaces to solve any problems of scarcity.

So when can it happen?

Robocars will come to different places at different times. They will handle different classes of streets at different times. They will handle different types of interactions with pedestrians and other road users at different times. Where you live will dictate when you can use it and how you can use it. Vendors will push at the most lucrative routes to start, then work down. There will be many problems that are difficult at first, and the result will be the early cars just don’t go on those sorts of streets or into those sorts of situations. Human driving, either by the customer or something like an Uber driver, will fill in the gaps.

Long before then, teams will have encountered or thought of just about any situation you’ve seen, and any situation you’ve likely thought of in a short amount of time. They will have programmed every variation of that situation they can imagine into their simulators to see what their car does. They will use this to grow the network of roads the cars handle every day. Even if at the start, it is not a network of use to you, it won’t be too long before it becomes that, at first for some of your rides, and eventually for most or all.

Syndicate content