Archives

Date
  • 01
  • 02
  • 03
  • 04
  • 05
  • 06
  • 07
  • 08
  • 09
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28

Digitizing your papers, literally, for the future, with 4K video

I have so much paper that I’ve been on a slow quest to scan things. So I have high speed scanners and other tools, but it remains a great deal of work to get it done, especially reliably enough that you would throw away the scanned papers. I have done around 10 posts on digitizing and gathered them under that tag.

Recently, I was asked by a friend who could not figure out what to do with the papers of a deceased parent. Scanning them on your own or in scanning shops is time consuming and expensive, so a new thought came to me.

Set up a scanning table by mounting a camera that shoots 4K video looking down on the table. I have tripods that have an arm that extends out but there are many ways to mount it. Light the table brightly, and bring your papers. Then start the 4K video and start slapping the pages down (or pulling them off) as fast as you can.

There is no software today that can turn that video into a well scanned document. But there will be. Truth is, we could write it today, but nobody has. If you scan this way, you’re making the bet that somebody will. Even if nobody does, you can still go into the video and find any page and pull it out by hand, it will just be a lot of work, and you would only do this for single pages, not for whole documents. You are literally saving the document “for the future” because you are depending on future technology to easily extract it.  read more »

Only the Republicans can bring down Trump

Sooner than most expected, the Trump administration is in trouble. Many are talking about how to end it, or hasten that end.

  1. The Democrats don’t have the power to take down Trump prior to 2020. Not even after 2018.
  2. The revolt against Trump almost surely has to come from within his own party.
  3. While many Republicans dislike Trump, revolt within a party is extremely difficult and goes against all party instincts.
  4. Republicans will strongly resist fighting Trump as the left would like, or in a way which benefits the left.
  5. As such, the more the left approves of a method of fighting Trump, the less likely it is the Republicans would use it.
  6. This suggests a very different anti-Trump strategy than the obvious one followed by most.

Many in the GOP would prefer not to have Trump, and are ready to be disloyal to him as their leader. They are not, however, prepared to be disloyal to their party and their movement. Career party members of both sides often will put loyalty to party ahead of loyalty to country, even though they would never admit that. This means that if the GOP does this, it must be for their own reasons, not the left’s, and it must clearly not appear to serve the left except in the broadest way.

This creates a conundrum for the left fighting Trump. If they rally around something, such as a Trump error, they push the right to reluctantly defend Trump on that issue. Many GOP can’t stand Trump but support him because the alternative is victory for the left, and injury for their party. As such, the best strategy for the left may be to pull back, or stick only to issues that are clearly their own.

The Democrats might consider strategies that are victories for the GOP. Conceding important items in congress in exchange for impeachment. The Republicans know the Democrats will vote for impeachment, so only a minority of Republicans need support it, but for them, a party divided like that is no victory. This may mean offering support for portions of Pence’s or the party’s agenda. Something so that the entire GOP can see it as a victory for their party. The Democrats lost in 2016, and they must accept that, and give up the hope that Trump’s fall would be good for the Democratic Party. They must accept only that it will be good for the country and neutral, or even slightly negative for the party.

It’s a common human foible but politicians cringe from ever admitting they were wrong. Those who supported Trump, even holding their noses, won’t see themselves as having failed. They won’t go, “Oh, you Democrats were right, sorry about that.” The reason will need to be something new, something few people knew or talked about before now. People are just less likely to do the right thing if they know it’s what their opponents want them to do.

The Democrats, however, are not a cohesive force. Even if “hold back and let the GOP do it” is the right plan, they will not embrace it in large numbers. Thus they will slow down the fall of Trump. This was a frequent mistake made during the election — the unprecedented level of contempt by the left for Trump and in particular for Trump supporters brought the Trump supporters together and made them stronger, rather than weakening them. It was a strong contributor to the Trump victory.

This advice does not mean, “Only complain about Trump in a way that the right-wing will understand.” Normally that is the best approach. Here, the problem is that as soon as a complaint is seen as coming from the left, there will be resistance to acting on it.

2018

Some hold out for a change of Congress in 2018. It is quite normal for the President’s party — especially an unpopular President — to lose seats in the mid-terms. Unfortunately, the senate seats up in 2018 are far from likely to swing the senate to the Democrats. In fact, only 9 Republican seats are up for re-election with only Nevada at risk, and many of the Democrat incumbents are in pro-Trump states. It would take an immense voter revolt to not have the Senate become more Republican. In the House, Operation Redmap has assured Republican control short of a very major shift, and it also seems to mostly assure — absent some sort of court ruling against Gerrymandering — that they will get to draw the lines again in 2020 and continue it for another decade.

The Deep State

One group that can take down Trump aside from the Republicans is the intelligence agencies. Many speculate that this is already underway. This is extremely troubling to me. A coup d’├ętat by the intelligence agencies is still a coup, even if it meets some test of “being a coup that needed to happen.” This is a bad precedent because the truth is the intelligence agencies have deep dirt on everybody, so it becomes up to them to decide which coups need to happen and which don’t. (Indeed, we saw the Russian agencies use this power already.)

There are already checks and balances for this. If the agencies find evidence of treason or malfeasance by one branch, they should present it to the other branch to act. All evidence should go to the congressional intelligence committees. But that means that again, the Republicans must decide whether to take down their own.

The Press

The press can play a role, but mainly the right-wing or right-of-center press. Again, it is their criticism of Trump that would enable the Republicans to break party loyalty, not criticisms found in media even perceived to be left or otherwise inherently anti-Trump. This is one reason Trump has worked to push more media into that classification, because it means their attacks will not be respected by his base and his party.

Trump’s base is not the mainstream GOP

The strongest counter to this approach is that Trump won the GOP nomination (and election ) due to support outside the mainstream GOP, merged with support from the party-loyal factions in the mainstream GOP. He has a tool to use against his opponents within the mainstream GOP, the same tool he used to defeat them in the nomination process. So even they must take care, for while they care most about alienating their own base, and least about alienating the progressive left, they are worried about alienating the “outsider right” contingency that Trump stumbled upon. They ideally want to be seen as having done the best thing for the party and the country in any efforts they make to block, or remove, the President.

Electrify Caltrain? Or could robocars do it for less than 1.5 billion?

Caltrain is the commuter rail line of the San Francisco peninsula. It’s not particularly good, and California is the land of the car commuter, but a plan was underway to convert it from diesel to electric. This made news this week as the California Republican house members announced they want to put a stop to both this project, and the much larger California High Speed Rail that hopes to open in 2030. For various reasons they may be right about the high speed rail but stop the electric trains? Electric trains are much better than diesel; they are cleaner and faster and quieter. But one number stands out in the plan.

To electrify the 51 miles of track, and do some other related improvements is forecast to cost over 1.5 billion dollars. Around $30M per mile.

So I started to ask, what other technology could we buy with $1.5 billion plus a private right-of-way through the most populated areas of silicon valley and the peninsula? Caltrain carries about 60,000 passengers/weekday (30,000 each way.) That’s about $50,000 per rider. In particular, what about a robotic transit line, using self-driving cars, vans and buses?

Paving over the tracks is relatively inexpensive. In fact, if we didn’t have buses, you could get by with fairly meager pavement since no heavy vehicles would travel the line. You could leave the rails intact in the pavement, though that makes the paving job harder. You want pavement because you want stations to become “offline” — vehicles depart the main route when they stop so that express vehicles can pass them by. That’s possible with rail, but in spite of the virtues of rail, there are other reasons to go to tires.

Fortunately, due to the addition of express trains many years ago, some stations already are 4 tracks wide, making it easy to convert stations to an express route with space by the side for vehicles to stop and let passengers on/off. Many other stations have parking lots or other land next to them allowing reasonably easy conversion. A few stations would present some issues.

Making robocars for a dedicated track is easy; we could have built that decades ago. In fact, with their much shorter stopping distance they could be safer than trains on rails. Perhaps we had to wait to today to convince people that one could get the same safety off of rails. Another thing that only arrived recently was the presence of smartphones in the hands of almost all the passengers, and low cost computing to make kiosks for the rest. That’s because the key to a robotic transit line would be coordination on the desires of passengers. A robotic transit line would know just who was going from station A to station J, and attempt to allocate a vehicle just for them. This vehicle would stop only at those two stations, providing a nonstop trip for most passengers. The lack of stops is also more energy efficient, but the real win is that it’s more pleasant and faster. With private ROW, it can easily beat a private car on the highways, especially at rush hour.

Another big energy win is sizing the vehicles to the load. If there are only 8 passengers going from B to K, then a van is the right choice, not a bus. This is particularly true off-peak, where vast amounts of energy are wasted moving big trains with just a few people. Caltrain’s last train to San Francisco never has more than 100 people on it. Smaller vehicles also allow for more frequent service in an efficient manner, and late night service as well — except freight uses these particular rails at night. (Most commuter trains shut down well before midnight.) Knowing you can get back is a big factor in whether you take a transit line at night.

An over-done service with a 40 passenger bus every 2 seconds would move 72,000 people (but really 30,000) in one hour in one direction to Caltrain’s 30,000 in a day. So of course we would not build that, and there would only be a few buses, mainly for rush hour. Even a fleet of just 4,000 9 passenger minvans (3 rows of 3) could move around 16,000 per hour (but really 8,000) in each direction. Even if each van was $50,000 each, we’ve spent only $200M of our $1.5B, though they might wear out too fast at that price, so we could bump the price and give them a much longer lifetime.

Energy

These vans and cars could be electric. This could be done entirely with batteries and a very impressive battery swap system, or you could have short sections of track which are electrified — with overhead rails or even third rails. The electric lines would be used to recharge batteries and supercapacitors, and would only be present on parts of the track. Unlike old 3rd rail technology, which requires full grade separation, there are new techniques to build safe 3rd rails that only provide current in a track segment after getting a positive digital signal from the vehicle. This is much cheaper than overhead wires. Inductive charging is also possible but makes pavement construction and maintenance much more expensive.

Other alternatives would be things like natural gas (which is cheap and much cleaner than liquid fuels, though still emits CO2) because it can be refilled quickly. Or hydrogen fuel cell vehicles could work here — hydrogen can be refilled quickly and can be zero emissions. Regular fossil fuel is also an option for peak times. For example the rush hour buses might make more sense running on CNG or even gasoline. The lack of starts and stops can make this pretty efficient.

Stations, anywhere

In such a system, you can also add new “stations” anywhere the ROW is wide enough for a side-lane and a small platform. You don’t need the 100m long platform able to hold a big train, just some pavement big enough to load a van. You can add a new station for extremely low cost. Of course, with more stations, it’s harder to group people for nonstop trips, and more people would need to take two-hop trips — a small van or car that takes them from a mini-station to a major station, where they join a larger group heading to their true destination.

Of course, if you were designing this from scratch, you would make the ROW with a shoulder everywhere that allowed vehicles to pull off the main track at any point to pick up a passenger and there would barely be “stations” — they would be closer to bus stops.

Getting off the track

Caltrain’s station in San Francisco is quite far from most of the destinations people want to go to. It’s one of the big reasons people don’t ride it. Vans on tires, however, have the option of keeping going once they get to the station. Employers could sponsor vehicles that arrive at the station and keep driving to their office tower. Vans could also continue to BART or more directly to underground Muni, long before the planned subway is ready. Likewise on the peninsula, vans and buses would travel from stations to corporate HQ. Google, Yahoo, Apple and many other companies already run transit fleets to bring employees in — you can bet that given the option they would gladly have those vans drive the old rail line at express speeds. On day one, they could have a driver who only drives the section back and forth between the station and the corporate office. In the not too distant future, the van or bus would of course drive itself. It’s not even out of the question that one of the passengers in a van, after having taken a special driving test, could drive that last mile, though you may need to assure somebody drives it back.

At-grade crossings

I noted above that capacity would be slightly less than half of full. That’s because Caltrain has 40 at-grade crossings on the peninsula. The robotic vehicles would coordinate their trips to travel in bunches, leaving gaps where the cross-street’s light can be turned green. If any car was detected trying to run the red, the signal could be uploaded to allow all the robotic vans to slow or even brake hard. Unlike trains, they could brake in reasonable amounts of time if somebody stalls on the old track. You would also detect people attempting to drive on the path or walk on it. Today’s cameras and cheap LIDARs can make that affordable. The biggest problem is the gaps must appear in both directions (more on that in the comments.)

Over time, there is also the option in some places to build special crossings. Because the vans and cars would all be not very high, much less expensive underpasses could be created under some of the roads for use only by the smaller vehicles. Larger vehicles would still need to bunch themselves together to leave gaps for the cross-traffic. One could also create overpasses rated only for lightweight vehicles at much lower cost, though those would still need to be high enough for trucks to go underneath. In addition, while cars can handle much, much steeper grades than trains, it could get disconcerting to handle too much up and down at 100mph. And yes, in time, they would go 100mph or even faster. And in time, some would even draft one another to both increase capacity and save energy — creating virtual trains where there used to be physical ones.

And then, obsolete

This robotic transit line would be much better than the train. But it would also be obsolete in just a couple of decades! As the rest of the world moves to more robocars, the transit line would switch to being just another path for the robocars. It would be superior, because it would allow only robocars and never have traffic congestion. You would have to pay extra to use it at rush hour, but many vehicles would, and large vehicles would get preference. The stations would largely vanish as all vehicles are able to go door to door. Most of the infrastructure would get re-used after the transit line shuts down.

It might seem crazy to build such a system if it will be obsolete in a short time, but it’s even crazier to spend billions on shoring up 19th century train.

What about the first law?

I’ve often said the first law of robocars is you don’t change the infrastructure. In particular, I am in general against ideas like this which create special roads just for robocars, because it’s essential that we not imagine robocars are only good on special roads. It’s only when huge amounts of money are already earmarked for infrastructure that this makes sense. Now we are well on the way to making general robocars good for ordinary streets. As such, special cars only for the former rail line run less risk of making people believe that robocars are only safe on dedicated paths. In fact, the funded development would almost surely lead to vehicles that work off the path as well, and allow high volume manufacturing of robotic transit vehicles for the future.

Could this actually happen?

I do fear that our urban and transit planners are unlikely to be so forward looking as to abandon a decades old plan for a centuries old technology overnight. But the advantages are huge:

  • It should be cheaper
  • Many companies could do it, and many would want to, to fund development of other technology
  • It would almost surely be technology from the Bay Area, not foreign technology, though vehicle manufacturing would come from outside
  • They could also get money for the existing rolling stock and steel in the rails to fund this
  • The service level would be vastly better. Wait times of mere minutes. Non-stop service. Higher speeds.
  • The energy use would be far lower and greener, especially if electric, CNG or hydrogen vehicles are used

The main downside is risk. This doesn’t exist yet. If you pave the road to retain the rails embedded in them, you would not need to shut down the rail line at first. In fact, you could keep it running as long as there were places that the vans could drive around trains that are slowing or stopping in the stations. Otherwise you do need to switch one day.

Free speech theory explained

There’s been a lot of talk this week on the nature of free speech. I’m a very strong defender of free speech, so I felt it would be worth laying out some of the reasons why “the first amendment is not just the law, it’s a good idea.” While I am not speaking for any particular organization, and am not a lawyer nor giving legal advice, my background includes things like:

  • Being the subject of the first big internet censorship battle, in 1987.
  • Being a plaintiff in ACLU v. Reno, which we won 9-0 in the supreme court, for which I was named a “Champion of Free Speech” by the ACLU.
  • 20 years with the Electronic Frontier Foundation, including 10 as chairman.

Two recent events has caused much debate. A viral video of somebody punching Richard B. Spencer, a man who gathers attention by promoting neo-nazi and whites-first rules has caused people to ask, “Isn’t it OK to punch a Nazi?” You see Spencer declaring “Hail Trump” and people doing Nazi salutes in one famous video.

There have also been two attempts by Breitbart writer Milo Yiannopoulos to speak at UC Davis and UC Berkeley that have been met with protests, calls that he be banned from speaking, and cancellations of his talks due to fear of violence. At UCB, a large group of apparent “black bloc” anarchists invaded a peaceful protest with violent acts and resulted in chaos and cancellation of the talk.

For a free speech supporter, the situation is fairly clear. No, it’s not OK to punch a Nazi (or in this case a wannabe neo-nazi) simply for what he says or what he is, even if it’s so-called “hate speech.” (In fact, that we don’t punch people for what they say is one of the important things that makes us better than Nazis.) And universities should not distinguish among speakers who are legitimately invited by members of the university community because of the content of their messages, even if it is hugely unpopular, offensive and hateful.

Here’s why:

Speech can be evil. But censorship is more evil.

It is a common mistake of those who say, “I am all in favour of free speech, but….” to imagine that we support free speech because speech is pure and can’t cause harm. This is the “sticks and stones” philosophy, but if you follow it, then it follows that if you can show that some speech is, unlike most speech, actually harmful, it is then OK to ban it.

While some speech is indeed harmless, important speech is powerful. It evokes change in the world, for good or ill. Speech can do great good and great harm. Consider the book “The Communist Manifesto” which advocates that to bring about an ideal communist society, one must begin with armed revolution and a “dictatorship of the proletariat” that uses draconian methods to work towards the pure goal. That idea has been used to create such dictatorships, and they have all been horrors. These dictatorships (particularly Stalin and Mao) perverted the ideas but used the ideals to justify acts which killed many tens of millions — leaving the Nazi holocaust in the dust. You can’t get much more evil or more proven harm. Yet we don’t ban that book.

Lots of speech is evil, but we have found no way to determine that reliably or in advance. As such, giving any entity the power to decide what speech is good and what is evil is a more dangerous proposition than just allowing all speech. For just as the idea in The Communist Manifesto have led to the death of millions, so much of the good in the world is also attributable to other ideas and books, including ones which were banned. We can’t grant an agency the power to decide what is good or bad without having them stamp out too much of the good. Nobody has the crystal ball that can do this, and history shows the terrible record of censorship agencies in the places that allow them.

There is also a practical angle. Censorship is only moderately effective. It’s probably slightly better at crushing good ideas than bad ones, but either way, for all the pain we get from censorship, it rarely actually stops the bad (or offensive or blasphemous) ideas from getting out. In fact, it is often of negative value, causing more publicity and support for the thing to be censored. (This worked for me when they tried to shut down my newsgroup, and later against Barbara Streisand to the extent that the principle was given her name.) In fact, I strongly suspect that the protests (even the peaceful ones) are doing precisely what Yiannopoulos wants. You think he cares that much about giving a talk to UC students? Or instead about the chance to be banned on the campus famous for the Free Speech movement of the 60s?

If we decide it’s going to be OK to punch some people for what they say, but not others, you need an arbiter who decides which speech is evil enough to warrant punching. And having that arbiter is a worse idea than letting the offensive person speak.

We have other ways to deal with bad speech

While there is bad speech, there is some merit to the “sticks and stones” argument in that people must be driven to action by the bad speech in order to get the harm. There, history shows that countering bad speech with good speech is a better, and certainly less dangerous counter-weapon than censorship. The answer to bad speech is more speech and more education.

There is a difference between speech and action

I will often hear people say that clearly some times of speech must be stopped — “what about shouting ‘fire!’ in a crowded theatre?”

That example is wrong for two important reasons. First, it’s fairly clear that shouting fire like this is not merely speech, but an action. It is the setting of a false fire alarm. It is like pulling the lever on the electronic fire alarm, which is easily seen as an action, and we can regulate actions. It is illegal to do a false fire alarm, particularly if it could cause a stampede.

Secondly, it’s a great demonstration of the evils of censorship. That argument became famous in the supreme court case Schenck v. United States. The case revolved around distributing leaflets which opposed the Draft in WWI. The court considered promoting resistance to the draft as akin to shouting fire in a crowded theatre. With our modern sensibility, we now see the debate about the merits of the draft to be an important one in a free society, one where all voices should be heard. Back then, they decided that the “incorrect” anti-draft position was so terrible it was like setting a false fire alarm. The reason why we can’t trust any agency to decide what speech is good and what is bad becomes very clear if you examine this case.

Generally, free speech law has allowed actions to be regulated but not speech. So setting a false fire alarm can be regulated. In addition, restrictions on the time and manner of speech can be regulated. They can make a rule prohibiting megaphones, but they can’t make a rule which ends up prohibiting megaphones based on what is said through them.

They can also make rules against conspiring to commit crimes. “Let’s attack John Smith” is more than speech, it is conspiracy to commit assault. “John Smith deserves assault” is not necessarily conspiracy, and the courts examine the circumstances in the borderline cases to see if the speech was also a threat, incitement or conspiracy. And yes, saying “It is OK to punch a Nazi” is speech when it’s an intellectual exercise, but more than speech when it turns into “let’s go down to the rally and punch Richard Spencer.” To count as incitement, the incited violent acts must be imminent, the path between the words and the violence must be clear and direct.

Hate speech is protected speech, at least in the USA

In many places, there have been efforts to define a special class of speech called “hate speech” and then to ban it. A number of countries, including Canada, have such laws. They are controversial and as predicted above, they have from time to time been used to attack political opponents of those in power rather than just shut down the Nazis and racists the way they are supposed to.

In the USA however, courts have consistently protected hate speech the same as any other speech.

Universities are held to an even higher standard

Many have been upset with universities allowing hate speakers to speak on campus. There are times when a student or professor wants to express an unpopular view, but more uproar comes when an outsider, like Yiannopoulos, is going to give an address.

Outsiders can’t generally come to universities, but often they get invites from people who are insiders. Yiannopoulos was invited by student Republican clubs, for example.

In the USA, the 1st amendment stops the government from censoring. The University of California is a state school, but it’s also a private institution, so there is debate on to what extent the 1st amendment governs it. (It does not govern totally private entities, such as a private club which can indeed decide what messages are allowed at club meetings.)

I’m not going to speak to that debate; rather I am going to invoke something much older than the 1st amendment, namely the traditions of academic freedom. For centuries, longer than any government or constitution has existed, universities have taken the principles of academic freedom as sacred. These principles declare an even higher bar. Universities are supposed to be the places that welcome controversial and dangerous views, views even the most enlightened governments of the world are afraid of. This has given us concepts like tenure, which assure faculty they will not be fired for expressing controversial views. History has taught us that so many of the most valuable ideas ever put forward began as controversial and banned thoughts in mainstream society.

As such, over and above any 1st amendment duties, universities, if they wish to honour their traditions, must set rules for who speaks based not at all on the message said by the speaker. They can limit locations and times. They can require external speakers to get an invitation from an accredited member of their community, but they must not treat a speaker of one message differently from another.

Indeed, there is an argument that if a speaker is so controversial, even within their own community, that there is fear of violence, that they should go the extra mile to provide extra protection rather than shy away in fear.

This does mean that a few dickheads will get to speak at universities to spout gibberish. That’s better than the alternative.

So is it OK to punch a Nazi?

Usually those asking this question point out that had the world punched/fought the real Nazis early on, the great horrors of the 20th century might have been averted. It is important to realize that this is clearly only obvious in hindsight. The people of the day did not have that vision at all. The Nazis, of course, got violent quite early on, so there were plenty of reasons to meet them with force if people had the will do do so. It was not a lack of moral clarity about “punching” them.

Indeed, at the end of the war, when the allies had almost all the Nazis captive, they tried them, and those who could be proven involved in the war crimes were executed or jailed. The others, in spite of killing many allied soldiers and civilians in battle, were set free. Including many members of the Nazi party.

Even when we had actual Nazis to deal with, the answer was not to punch them for what they were or what they said. They were punished if they were involved in the atrocities. Not for talking about them. If the actual victims of the real Nazis could do that, it seems odd for people today to claim to be wiser about it.

While the real Nazis are best known for killing people for their ethnicity and religion, they were also ready to do it for ideology, politics or sexual orientation, and many communists or simple political opponents were persecuted, rounded up and executed for it. Punching people for their beliefs is what Nazis do, not us. Instead, counter their ideology with better ideology, and be wary; for if they take up arms in their cause, it is certainly appropriate to respond with force.

California publishes robocar "intervention" reports -- Google/Waymo so far ahead it's ridiculous

California published its summary of all the reports submitted by vendors testing robocars in the state. You can read the individual reports — and they are interesting, but several other outlines have created summaries of the reports calculating things like the number of interventions per mile.

On these numbers, Google’s lead is extreme. Of over 600,000 autonomous miles driven by the various teams, Google/Waymo was 97% of them — in other words 30 times as much as everybody else put together. Beyond that, their rate of miles between disengagements (around 5,000 — a 4x improvement over 2015) is one or two orders of magnitude better than the others, and in fact for most of the others, they have so few miles that you can’t even produce a meaningful number. Only Cruise, Nissan and Delphi can claim enough miles to really tell.

Tesla is a notable entry. In 2015 they reported driving zero miles, and in 2016 they did report a very small number of miles with tons of disengagements from software failures (one very 3 miles.) That’s because Tesla’s autopilot is not a robocar system, and so miles driven by it are not counted. Tesla’s numbers must come from small scale tests of a more experimental vehicle. This is very much not in line with Tesla’s claim that it will release full autonomy features for their cars fairly soon, and that they already have all the hardware needed for that to happen.

Unfortunately you can’t easily compare these numbers:

  • Some companies are doing most of their testing on test tracks, and they do not need to report what happens there.
  • Companies have taken different interpretations of what needs to be reported. Most of Cruise’s disengagements are listed as “planned” but in theory those should not be listed in these reports. But they also don’t list the unplanned ones which should be there.
  • Delphi lists real causes and Nissan is very detailed as well. Others are less so.
  • Many teams test outside California, or even do most of their testing there. Waymo/Google actually tests a bunch outside California, making their numbers even bigger.
  • Cars drive all sorts of different roads. Urban streets with pedestrians are much harder than highway miles. The reports do list something about conditions but it takes a lot to compare apples to apples. (Apple is not one of the companies filing a report, BTW.)

One complication is that typically safety drivers are told to disengage if they have any doubts. It thus varies from driver to driver and company to company what “doubts” are and how to deal with them.

Google has said their approach is to test any disengagement in simulator, to find out what probably would have happened if the driver did not disengage. If there would have been a “contact” (accident) then Google considers that a real incident, and those are more rare than is reported here. Many of the disengagements are when software detects faults with software or sensors. There, we do indeed have a problem, but like human beings who zone out, not all such failures will cause accidents or even safety issues. You want to get rid of all of them, to be sure, but if you are are trying to compare the safety of the systems to humans, it’s not easy to do.

It’s hard to figure out a good way to get comparable numbers from all teams. The new federal guidelines, while mostly terrible, contain an interesting rule that teams must provide their sensor logs for any incident. This will allow independent parties to compare incidents in a meaningful way, and possibly even run them all in simulator at some level.

It would be worthwhile for every team to be required to report incidents that would have caused accidents. That requires a good simulator, however, and it’s hard for the law to demand this of everybody.