Brad Ideas

Robocars & More

(Update: I had a formatting error in the original posting, this has been fixed.)

A few weeks ago when I wrote about the non deployment of SSL I touched on an old idea I had to make web transactions vastly more efficient. I recently read about Google's proposed SPDY protocol which goes in a completely opposite direction, attempting to solve the problem of large numbers of parallel requests to a web server by multiplexing them all in a single streaming protocol that works inside a TCP session.

While calling attention to that, let me outline what I think would be the fastest way to do very simple web transactions. It may be that such simple transactions are no longer common, but it's worth considering.

Consider a protocol where you want to fetch the contents of a URL like "www.example.com/page.html" and you have not been to that server recently (or ever.) You want only the plain page, you are not yet planning to fetch lots of images and stylesheets and javascript.

Today the way this works is pretty complex:

1. You do a DNS request for www.example.com via a UDP request to your DNS server. In the pure case this also means first asking where ".com" is but your DNS server almost surely knows that. Instead, a UDP request is sent to the ".com" master server.
2. The ".com" master server returns with the address of the server for example.com.
3. You send a DNS request to the example.com server, asking where "www.example.com is."
4. The example.com DNS server sends a UDP response back with the IP address of www.example.com
5. You open a TCP session to that address. First, you send a "SYN" packet.
6. The site responds with a SYN/ACK packet.
7. You respond to the SYN/ACK with an ACK packet. You also send the packet with your HTTP "GET" reqequest for "/page.html." This is a distinct packet but there is no roundtrip so this can be viewed as one step. You may also close off your sending with a FIN packet.
8. The site sends back data with the contents of the page. If the page is short it may come in one packet. If it is long, there may be several packets.
9. There will also be acknowledgement packets as the multiple data packets arrive in each direction. You will send at least one ACK. The other server will ACK your FIN.
10. The remote server will close the session with a FIN packet.
11. You will ACK the FIN packet.

You may not be familiar with all this, but the main thing to understand is that there are a lot of roundtrips going on. If the servers are far away and the time to transmit is long, it can take a long time for all these round trips.

It gets worse when you want to set up a secure, encrypted connection using TLS/SSL. On top of all the TCP, there are additional handshakes for the encryption. For full security, you must encrypt before you send the GET because the contents of the URL name should be kept encrypted.

A simple alternative

Consider a protocol for simple transactions where the DNS server plays a role, and short transactions use UDP. I am going to call this the "Web Transaction Protocol" or WTP. (There is a WAP variant called that but WAP is fading.)

1. You send, via a UDP packet, not just a DNS request but your full GET request to the DNS server you know about, either for .com or for example.com. You also include an IP and port to which responses to the request can be sent.
2. The DNS server, which knows where the target machine is (or next level DNS server) forwards the full GET request for you to that server. It also sends back the normal DNS answer to you via UDP, including a flag to say it forwarded the request for you (or that it refused to, which is the default for servers that don't even know about this.) It is important to note that quite commonly, the DNS server for example.com and the www.example.com web server will be on the same LAN, or even be the same machine, so there is no hop time involved.
3. The web server, receiving your request, considers the size and complexity of the response. If the response is short and simple, it sends it in one UDP packet, though possibly more than one, to your specified address. If no ACK is received in reasonable time, send it again a few times until you get one.
4. When you receive the response, you send an ACK back via UDP. You're done.

The above transaction would take place incredibly fast compared to the standard approach. If you know the DNS server for example.com, it will usually mean a single packet to that server, and a single packet coming back -- one round trip -- to get your answer. If you only know the server for .com, it would mean a single packet to the .com server which is forwarded to the example.com server for you. Since the master servers tend to be in the "center" of the network and are multiplied out so there is one near you, this is not much more than a single round trip.

A proposal is being floated in Europe for computerized convoys or road trains within the next decade. This is a proposal for a system where cars can hand over control to a lead car and follow in a train or convoy, without physical connection.

Recently I wrote about the desire to provide power in every sort of cable in particular the video cable. And while we'll be using the existing video cables (VGA and DVI/HDMI) for some time to come, I think it's time to investigate new thinking in sending video to monitors. The video cable has generally been the highest bandwidth cable going out of a computer though the fairly rare 10 gigabit ethernet is around the speed of HDMI 1.3 and DisplayPort, and 100gb ethernet will be yet faster.

First: I will be speaking on robocars tomorrow, Tuesday Nov 9, at 6:30 pm for the meeting of the Jewish High Tech Community in Silicon Valley. The talk is at 6:30pm at the conference center of Fenwick and West at Castro & California in Mountain View. The public is welcome to attend, there is a $10 fee for non-members.

I suggested this as a feature for my Canon 5D SLR which shoots video, but let me expand it for all video cameras, indeed all cameras. They should all include bluetooth, notably the 480 megabit bluetooth 3.0. It's cheap and the chips are readily available.

While giving a talk on robocars to a Stanford class on automative innovation on Wednesday, I outlined the growing problem of software recalls and how they might effect cars. If a company discovers a safety problem in a car's software, it may be advised by its lawyers to shut down or cripple the cars by remote command until a fix is available. Sebastian Thrun, who had invited me to address this class, felt this could be dealt with through the ability to remotely patch the software.

I just returned from Jeff Pulver's "140 Characters" conference in L.A. which was about Twitter. I asked many people if they get Twitter -- not if they understand how it's useful, but why it is such a hot item, and whether it deserves to be, with billion dollar valuations and many talking about it as the most important platform.

Some suggested Twitter is not as big as it appears, with a larger churn than expected and some plateau appearing in new users. Others think it is still shooting for the moon.

Saturday saw the dedication of a new autonomous vehicle research center at Stanford, sponsored by Volkswagen. VW provided the hardware for Stanley and Junior, which came 1st and 2nd in the 2nd and 3rd Darpa Grand Challenges, and Junior was on display at the event, driving through the parking lot and along the Stanford streets, then parking itself to a cheering crowd.

Junior continues to be a testing platform with its nice array of sensors and computers, though the driving it did on Saturday was largely done with the Velodyne LIDAR that spins on top of it, and an internal map of the geometry of the streets at Stanford.

New and interesting was a demonstration of the "Valet Parking" mode of a new test vehicle, for now just called Junior 3. What's interesting about J3 is that it is almost entirely stock. All that is added are two lower-cost LIDAR sensors on the rear fenders. It also has a camera at the rear-view mirror (which is stock in cars with night-assist mode) and a few radar sensors used in the fixed-distance cruise control system. J3 is otherwise a Passat. Well, the trunk is filled with computers, but there is no reason what it does could not be done with a hidden embedded computer.

What it does is valet park itself. This is an earlier than expected implementation of one of the steps I outlined in the roadmap to Robocars as robo-valet parking. J3 relies on the fact the "valet" lot is empty of everything but cars and pillars. Its sensors are not good enough to deal well with random civilians, so this technology would only work in an enclosed lot where only employees enter the lot if needed. To use it, the driver brings the car to an entrance marked by 4 spots on the ground the car can see. Then the driver leaves and the car takes over. In this case, it has a map of the garage in its computer, but it could also download that on arrival in a parking lot. Using the map, and just the odometer, it is able to cruise the lanes of the parking lot, looking for an empty spot, which it sees using the radar. (Big metal cars of course show clearly on the radar.) It then drives into the spot.

I've written a lot about how to do better power connectors for all our devices, and the quest for universal DC and AC power plugs that negotiate the power delivered with a digital protocol.

While I've mostly been interested in some way of standardizing power plugs (at least within a given current range, and possibly even beyond) today I was thinking we might want to go further, and make it possible for almost every connector we use to also deliver or receive power.

Some time ago I proposed the "School of Fish Test" as a sort of turing test for robocars. In addition to being a test for the cars, it is also intended to be a way to demonstrate to the public when the vehicles have reached a certain level of safety. (In the test, a swarm of robocars moves ona track, and a skeptic in a sportscar is unable to hit one no matter what they do, like a diver trying to touch fish when swimming through a school.)

I'm impressed with a great interactive map of the U.S. power grid produced by NPR. It lets you see the location of existing and proposed grid lines, and all power plants, plus the distribution of power generation in each state.

Robocar news:

This press release describes a European research project on various intelligent vehicle technologies which will take place next year. As I outline in the roadmap a number of pre-robocar technologies are making their way into regular cars, so they can be sold as safer and more convenient. This project will actively collect data to learn about and improve the systems.

Just back from a weeklong tour including speaking at Singularity Summit, teaching classes at Cushing Academy and a big Thanksgiving dinner (well, Thanksgiving is actually today but we had it earlier) and drive through fabulous fall colour in Muskoka.

I have several sheetfed scanners. They are great in many ways -- though not nearly as automatic as they could be -- but they are expensive and have their limitations when it comes to real-world documents, which are often not in pristine shape.

A serious proportion of the computer users I know these days have gone multi-monitor. While I strongly recommend the 30" monitor (Dell 3007WFP and cousins or Apple) which I have to everybody, at $1000 it's not the most cost effective way to get a lot of screen real estate. Today 24" 1080p monitors are down to $200, and flat panels don't take so much space, so it makes a lot of sense to have two monitors or more.

Tonight I watched the debut of FlashForward, which is based on the novel of the same name by Rob Sawyer, an SF writer from my hometown whom I have known for many years. However, "based on" is the correct phrase because the TV show features Hollywood's standard inability to write a decent time travel story. Oddly, just last week I watched the fairly old movie "Deja Vu" with Denzel Washington, which is also a time travel story.

Hollywood absolutely loves time travel. It's hard to find a Hollywood F/SF TV show that hasn't fallen to the temptation to have a time travel episode. Battlestar Galactica's producer avowed he would never have time travel, and he didn't, but he did have a god who delivered prophecies of the future which is a very close cousin of that. Time travel stories seem easy, and they are fun. They are often used to explore alternate possibilities for characters, which writers and viewers love to see.

But it's very hard to do it consistently. In fact, it's almost never done consistently, except perhaps in shows devoted to time travel (where it gets more thought) and not often even then. Time travel stories must deal with the question of whether a trip to the past (by people or information) changes the future, how it changes it, who it changes it for, and how "fast" it changes it. I have an article in the works on a taxonomy of time travel fiction, but some rough categories from it are:

• Calvinist: Everything is cast, nothing changes. When you go back into the past it turns out you always did, and it results in the same present you came from.
• Alternate world: Going into the past creates a new reality, and the old reality vanishes (at varying speeds) or becomes a different, co-existing fork. Sometimes only the TT (time traveler) is aware of this, sometimes not even she is.
• Be careful not to change the past: If you change it, you might erase yourself. If you break it, you may get a chance to fix it in some limited amount of time.
• Go ahead and change the past: You won't get erased, but your world might be erased when you return to it.
• Try to change the past and you can't: Some magic force keeps pushing things back the way they are meant to be. You kill Hitler and somebody else rises to do the same thing.

Inherent in several of these is the idea of a second time dimension, in which there is a "before" the past was changed and an "after" the past was changed. In this second time dimension, it takes time (or rather time-2) for the changes to propagate. This is mainly there to give protagonists a chance to undo changes. We see Marty Mcfly slowly fade away until he gets his parents back together, and then instantly he's OK again.

In a time travel story, it is likely we will see cause follow effect, reversing normal causality. However, many writers take this as an excuse to throw all logic out the window. And almost all Hollywood SF inconsistently mixes up the various modes I describe above in one way or another.

Spoilers below for the first episode of FlashForward, and later for Deja Vu.

Update note: The fine folks at io9 asked FlashForward's producers about the flaw I raise but they are not as bothered by it.

It seems that with more and more of the online transactions I engage in -- and sometimes even when I don't buy anything -- I will get a request to participate in a customer satisfaction survey. Not just some of the time in some cases, but with every purchase. I'm also seeing it on web sites -- sometimes just for visiting a web site I will get a request to do a survey, either while reading, or upon clicking on a link away from the site.

I have put up a gallery of panoramas for Burning Man 2009. This year I went with the new Canon 5D Mark II, which has remarkable low-light shooting capabilities. As such, I generated a number of interesting new night panoramas in addition to the giant ones of the day.

In particular, you will want to check out the panorama of the crowd around the burn, as seen from the Esplanade, and the night scene around the Temple, and a twilight shot.

Two events I will be at...

Tonight, at 111 Minna Gallery in San Francisco, we at EFF will be hosting a reading by Randall Monroe, creator of the popular nerd comic "xkcd." There is a regular ticket ($30) and a VIP reception ticket ($100) and just a few are still available. Payments are contributions to the EFF.

Two years ago, I discussed solutions for Burning Man Exodus. The problem: Get 45,000 people off the playa in 2 days, 95% of them taking a single highway south which goes through a small town which has a chokepoint capacity of about 450 cars/hour. Quite often wait times to get onto the road are 4 hours or more, though this year things were smoother (perhaps due to a lower attendance) and the number of people with 4 hour waits was lower.