Brad Ideas

Robocars & More

I've just returned from the 25th reunion of my graduating class in Mathematics at the University of Waterloo. I had always imagined that a 25th reunion would be the "big one" so I went. In addition, while I found myself to have little in common with my high school classmates, even having spent 13 years growing up with many of them, like many techie people I found my true community at university, so I wanted to see them again. To top it off, it was the 40th anniversary of the faculty and the 50th anniversary of the university itself.

But what if they had a reunion and nobody came? Or rather, out of a class of several hundred, under 20 came, many of whom I only barely remembered and none of whom I was close to?

In 2005, John Scalzi burst on the scene with a remarkable first novel, Old Man's War. It got nominated for a Hugo and won him the Campbell award for best new writer. Many felt it was the sort of novel Heinlein might be writing today. That might be too high a praise, but it's close. The third book in this trilogy has just come out, so it was time to review the set.

It's hard to review the book without some spoilers, and impossible for me to review the latter two books without spoiling the first, but I'll warn you when that's going to happen.

OMW tells the story of John Perry, a 75 year old man living on an Earth only a bit more advanced than our own, but it's hundreds of years in the future. Earth people know they're part of a collection of human colonies which does battle with nasty aliens, but they are kept in the dark about the realities. People in the third world are offered o ne way trips to join colonies. People in the 1st world can, when they turn 75, sign up for the colonial military, again a one-way trip. It's not a hard choice to make since everybody presumes the military will make them young again, and the alternative is ordinary death by old age.

The protagonist and his wife sign up, but she dies before the enlistment date, so he goes on his own. The first half of the book depicts his learning the reality of the colonial union, and boot camp, and the latter half outlines his experiences fighting against various nasty aliens.

It's a highly recommended read. If you loved Starship Troopers or The Forever War this is your kind of book. Now I'll go into some minor spoilers.

Since the dawn of the web, there has been a call for a "single sign-on" facility. The web consists of millions of independently operated web sites, many of which ask users to create "accounts" and sign-on to use the site. This is frustrating to users.

Today the general single sign-on concept has morphed into what is now called "digital identity management" and is considerably more complex. The most recent project of excitement is OpenID which is a standard which allows users to log on using an identifier which can be the URL of an identity service, possibly even one they run themselves.

Many people view OpenID as positive for privacy because of what came before it. The first major single sign-on project was Microsoft Passport which came under criticism both because all your data was managed by a single company and that single company was a fairly notorious monopoly. To counter that, the Liberty Alliance project was brewed by Sun, AOL and many other companies, offering a system not run by any single company. OpenID is simpler and even more distributed.

However, I feel many of the actors in this space are not considering an inherent paradox that surrounds the entire field of identity management. On the surface, privacy-conscious identity management puts control over who gets identity information in the hands of the user. You decide who to give identity info to, and when. Ideally, you can even revoke access, and push for minimal disclosure. Kim Cameron summarized a set of laws of identity outlining many of these principles.

In spite of these laws one of the goals of most identity management systems has been ease of use. And who, on the surface, can argue with ease of use? Managing individual accounts at a thousand web sites is hard. Creating new accounts for every new web site is hard. We want something easier.

The paradox

However, here is the contradiction. If you make something easy to do, it will be done more often. It's hard to see how this can't be true. The easier it is to give somebody ID information, the more often it will be done. And the easier it is to give ID information, the more palatable it is to ask for, or demand it.

In the 1980s, my brother Ty Templeton published his first independent comic book series, Stig's Inferno. He went on to considerable fame writing and drawing comics for Marvel, D.C. and many others, including favourite characters like Superman, Batman and Spider-Man, as well as a lot of comics associated with TV shows like The Simpsons and Ren and Stimpy. But he's still at his best doing original stuff.

First, let me introduce a new blog topic, Sysadmin where I will cover computer system administration and OS design issues, notably in Linux and related systems.

My goal is to reduce the nightmare that is system administration and upgrading.

One step that goes partway in my plan would be a special software system that would build for a user a specialized operating system "package" or set of packages. This magic package would, when applied to a virgin distribution of the operating system, convert it into the customized form that the user likes.

The program would work from a modified system, and a copy of a map (with timestamps and hashes) of the original virgin OS from which the user began. First, it would note what packages the user had installed, and declare dependencies for these packages. Thus, installing this magic package would cause the installation of all the packages the user likes, and all that they depend on.

In order to do this well, it would try to determine which packages the user actually used (with access or file change times) and perhaps consider making two different dependency setups -- one for the core packages that are frequently used, and another for packages that were probably just tried and never used. A GUI to help users sort packages into those classes would be handy. It must also determine that those packages are still available, dealing with potential conflicts and name change concerns. Right now, most package managers insist that all dependencies be available or they will abort the entire install. To get around this, many of the packages might well be listed as "recommended" rather than required, or options to allow install of the package with missing 1st level (but not 2nd level) dependencies would be used.

At our new favourite Indian buffet (Cafe Bombay) they run Bollywood videos on big screens all the time. In Bollywood, as you probably know, everybody is dancing all the time, in wonderful synchronization, like Broadway but far more. I've never been to an Indian dance club to see if people try to do that in real life, but I suspect they want to.

I started musing about a future where brain implants let you give a computer control of your limbs so you could participate in such types of dance, but I realized we might be able to do something much sooner.

Self-driving cars are still some ways in the future, but there are some things they will want that human drivers can also make use of.

Most search engines now have a search box in the toolbar, which is great, and like most people mine defaults to Google. I can change the engine with a drop down menu to other places, like Amazon, Wikipedia, IMDB, eBay, Yahoo and the like. But that switch is a change in the default, rather than a temporary change -- and I don't want that, I want it to snap back to Google.

High posting volume today. I just find it remarkable that in the last 2 weeks I've seen several incredible breakthrough level stories on health and life extension.

While I was at Tim O'Reilly's Web 2.0 Expo, I did an interview with an online publication called Web Pro News. I personally prefer written text to video blogging, but for those who like to see video, you can check out:

Video Interview on Privacy and Web 2.0

The video quality is pretty good, if not the lighting.

I wasn't going to make any special commemoration, but it seems a whole ton of other blogs are linking today to my articles on the history of Spam, so I should blog them as well.

Many years ago I got interested in the origins of the term "spam" to mean net abuse. I mean I had lived through most of its origin and seen most of the early spams myself, but it wasn't clear why people took the name of the meat product and applied it to junk mail. I knew it came from USENET, so I used the USENET search engines to trace the origins.

I really wish I could find a really good calendaring tool. I've seen many of the features I want scattered in various tools, though some are nowhere to be found. I thought it would be good to itemize some of them. I'm mostly interested in *nix -- I know that on Windows, MS Outlook is the most common choice, with Exchange for sharing.

I've been writing a lot about self-driving cars which have automatic accident avoidance and how they will change our cities. I was recently talking again with Robin Chase, whose new company, goloco attempts to set people up for ad-hoc carpools and got into the issues again. She believes we should use more transit in cities and there's a lot of merit to that case.

However, in the wealthy USA, we don't, outside of New York City. We love our cars, and we can afford their much higher cost, so they still dominate, and even in New York many people of means rely strictly on taxis and car services.

Transit is, at first glance, more energy efficient. When it shares right of way with cars it reduces congestion. Private right of way transit also reduces congestion but only when you don't consider the cost of the private right-of-way, where the balance is harder to decide. (The land only has a many-person vehicle on it a small fraction of the time compared to 1-3 passenger vehicles almost all the time on ordinary roads.)

However, my new realization is that transit may not be as energy efficient as we hope. During rush hour, packed transit vehicles are very efficient, especially if they have regenerative braking. But outside those hours it can be quite wasteful to have a large bus or train with minimal ridership. However, in order to give transit users flexibility, good service outside of rush-hour is important.

I've been remiss in updating my panoramas, so I just did some work on the site and put up a new page full of Alberta panoramas, as well as some others I will point to shortly.

As part of my series on the horrors of modern system administration and upgrading, let me propose the need for a universal API, over all operating systems, for accessing data from, and some control of the package management system.

There have been many efforts in the past to standardize programming APIs within all the unix-like operating systems, some of them extending into MS Windows, such as Posix. Posix is a bit small to write very complex programs fully portably but it's a start. Any such API can make your portability easier if it can't make it trivial the way it's supposed to.

But there has been little effort to standardize the next level, machine administration and configuration. Today a large part of that is done with the package manager. Indeed, the package manager is the soul (and curse) of most major OS distributions. One of the biggest answers to "what's the difference between debian and Fedora" is "dpkg and apt, vs. rpm and yum." (Yes you can, and I do, use apt with rpm.)

Now the truth is that from a user perspective, these package managers don't actually look very different. They all install and remove packages by name, perform upgrades, handle dependencies etc. Add-ons like apt and GUI package managers help users search and auto-install all dependencies. To the user, the most common requests are to find and install a package, and to upgrade it or the system.

Many people accumulate a lot of frequent flyer miles they will never use. Some of the airlines allow you to donate miles to a very limited set of charities. I can see why they limit it -- they would much rather have you not use the miles than have the charity use them. Though it's possible that while the donor does not get any tax credit for donated miles, the airline does.

An eBay reputation is important if you're going to sell there. Research shows it adds a decent amount to the price, and it's very difficult to sell at all with just a few feedbacks. Usually sellers will buy a few items first to get a decent feedback -- sometimes even scam items sold just for feedback. Because savvy buyers insist on selling feedback, it's harder, and sometimes sellers will also sell bogus items just for feedback as a seller. eBay has considered offering a feedback score based on the dollar volume of positive and negative transactions but has not yet done this.

Yesterday I attended the online community session of Web2Open, a barcamp-like meeting going on within Tim O'Reilly's Web 2.0 Expo. (The Expo has a huge number of attendees, it's doing very well.)

I put forward a number of questions I've been considering for later posts, but one I want to make here is this: Where has the innovation been in online discussion software? Why are most message boards and blog comment systems so hard to use?

I know this is true because huge numbers of people are still using USENET, and not just for downloading binaries. USENET hasn't seen much technical innovation since the 80s. As such, it's aging, but it shouldn't be simply aging, it should have been superseded long ago. We've gone through a period of tremendous online innovation in the last few decades, unlike any in history. Other old systems, like the Well, continue to exist and even keep paying customers in spite of minimal innovation. This is like gopher beating Firefox, or a CD Walkman being superior in some ways to an iPod. It's crazy. (The users aren't crazy, it's the fact that their choice is right that's crazy.)

The use of virtual machines is getting very popular in the web hosting world. Particularly exciting to many people is Amazon.com's EC2 -- which means Elastic Compute Cloud. It's a large pool of virtual machines that you can rent by the hour. I know people planning on basing whole companies on this system, because they can build an application that scales up by adding more virtual machines on demand. It's decently priced and a lot cheaper than building it yourself in most cases.

In many ways, something like EC2 would be great for all those web sites which deal with the "slashdot" effect. I hope to see web hosters, servers and web applications just naturally allow scaling through the addition of extra machines. This typically means either some round-robin-DNS, or a master server that does redirects to a pool of servers, or a master cache that processes the data from a pool of servers, or a few other methods. Dealing with persistent state that can't be kept in cookies requires a shared database among all the servers, which may make the database the limiting factor. Rumours suggest Amazon will release an SQL interface to their internal storage system which presumably is highly scalable, solving that problem.

As noted, this would be great for small to medium web sites. They can mostly run on a single server, but if they ever see a giant burst of traffic, for example by being linked to from a highly popular site, they can in minutes bring up extra servers to share the load. I've suggested this approach for the Battlestar Galactica Wiki I've been using -- normally their load is modest, but while the show is on, each week, predictably, they get such a huge load of traffic when the show actually airs that they have to lock the wiki down. They have tried to solve this the old fashioned way -- buying bigger servers -- but that's a waste when they really just need one day a week, 22 weeks a year, of high capacity.

However, I digress. What I really want to talk about is using such systems to get access to all sorts of platforms. As I've noted before, linux is a huge mishmash of platforms. There are many revisions of Ubuntu, Fedora, SuSE, Debian, Gentoo and many others out there. Not just the current release, but all the past releases, in both stable, testing and unstable branches. On top of that there are many versions of the BSD variants.

In many cities, the transit systems have GPS data on the vehicles to allow exact prediction of when trains and buses will arrive at stops. This is quite handy if you live near a transit line, and people are working on better mobile interfaces for them, but it's still a lot harder to use them at a remote location.