Brad Ideas

Robocars & More

I have some admiration for the PETA prize for vat-grown chicken. A winner of this prize would strongly promote PETA's ethical goals, as well as many environmental goals, for the livestock industry is hugely consumptive of land, as it takes far more grain to feed animals than it takes to feed us, per calorie.

I was recently approached by a programmer named Keith Curtis, formerly at Microsoft and now a FOSS devotee. He wants to develop a driving simulator for testing robocar systems. I think this is a very worthwhile idea -- sort of a "Second Life" for robots. We have a head start -- the world of racecar video games has already done a lot of the legwork to simulate driving, and there are two open source car racing systems.

A good simulator would bring some tremendous benefits to robocar development.

I received some criticism the other day over my own criticism of the use of haplogroups in genealogy -- the finding and tracing of relatives. My language was imprecise so I want to make a correction and explore the issue in a bit more detail.

One of the most basic facts of inheritance is that while most of your DNA is a mishmash of your parents (and all their ancestors before them) two pieces of DNA are passed down almost unchanged. One is the mitochondrial DNA, which is passed down from the mother to all her children. The other is the Y chromosome, which is passed down directly from father to son. Girls don't get one. Most of the mother's X chromosome is passed down unchanged to her sons (but not her daughters) but of course they can't pass it unchanged to anybody.

This allow us to track the ancestry of two lines. The maternal line tracks your mother, her mother, her mother, her mother and so on. The paternal line tracks your father, his father and so on. The paternal line should, in theory, match the surname, but for various reasons it sometimes doesn't. Females don't have a Y, but they can often find out what Y their father had if they can sequence a sample from him, his sons, his brothers and other male relatives who share his surname.

The ability to do this got people very excited. DNA that can be tracked back arbitrarily far in time has become very useful for the study of human migrations and population genetics. The DNA is normally passed down completely but every so often there is a mutation. These mutations, if they don't kill you, are passed down. The various collections of mutations are formed into a tree, and the branches of the tree are known as haplogroups. For both kinds of DNA, there are around a couple of hundred haplogroups commonly identified. Many DNA testing companies will look at your DNA and tell you your MTDNA haplogroup, and if male, your Y haplogroup.

Last week, I wrote about interesting experiences finding Cousins who were already friends via genetic testing. 23andMe's new "Relative Finder" product identifies the other people in their database of about 35,000 to whom you are related, guessing how close. Surprisingly, 2 of the 4 relatives I made contact with were already friends of mine, but not known to be relatives.

Many people are very excited about the potential for services like Relative Finder to take the lid off the field of genealogy. Some people care deeply about genealogy (most notably the Mormons) and others wonder what the fuss is. Genetic genealogy offers the potential to finally link all the family trees built by the enthusiasts and to provably test already known or suspected relationships. As such, the big genealogy web sites are all getting involved, and the Family Tree DNA company, which previously did mostly worthless haplogroup studies (and more useful haplotype scans,) is opening up a paired-chromosome scan service for $250 -- half the price of 23andMe's top-end scan. (There is some genealogical value to the deeper clade Y studies FTDNA does, but the Mitochondrial and 12-marker Y studies show far less than people believe about living relatives. I have a followup post about haplogroups and haplotypes in genealogy.) Note that in March 2010, 23andMe is offering a scan for just $199.

The cost of this is going to keep decreasing and soon will be sub-$100. At the same time, the cost of full sequencing is falling by a factor of 10 every year (!) and many suspect it may reach the $100 price point within just a few years. (Genechip sequencing only finds the SNPs, while a full sequencing reads every letter (allele) of your genome, and perhaps in the future your epigenome.

Discover of relatives through genetics has one big surprising twist to it. You are participating in it whether you sign up or not. That's because your relatives may be participating in it, and as it gets cheaper, your relatives will almost certainly be doing so. You might be the last person on the planet to accept sequencing but it won't matter.

One of the world's favourite (and sometimes least favourite) topics is the issue of terrorism and security. On one side, there are those who feel the risk of terrorism justifies significant sacrifices of money, convenience and civil rights to provide enough security to counter it. That side includes both those who honestly come by that opinion, and those who simply want more security and feel terrorism is the excuse to use to get it.

On the other side, critics point out a number of counter arguments, most of them with merit, including:

• Much of what is done in the name of security doesn't actually enhance it, it just gives the appearance of doing so, and the appearance of security is what the public actually craves. This has been called "Security Theatre" by Bruce Schneier, who is a friend and advisor to the E.F.F.
• We often "fight the previous war," securing against the tactics of the most recent attack. The terrorists have already moved on to planning something else. They did planes, then trains, then subways, then buses, then nightclubs.
• Terrorists will attack where the target is weakest. Securing something just makes them attack something else. This has indeed been the case many times. Since everything can't be secured, most of our efforts are futile and expensive. If we do manage to secure everything they will attack the crowded lines at security.
• Terrorists are not out to kill random people they don't know. Rather, that is their tool to reach their real goal: sowing terror (for political, religious or personal goals.) When we react with fear -- particularly public fear -- to their actions, this is what they want, and indeed what they plan to achieve. Many of our reactions to them are just what they planned to happen.
• Profiling and identity checks seem smart at first, but careful analysis shows that they just give a more free pass to anybody the terrorists can recruit whose name is not yet on a list, making their job easier.
• The hard reality is, that frightening as terrorism is, in the grand scheme we are for more likely to face harm and death from other factors that we spend much less of our resources fighting. We could save far more people applying our resources in other ways. This is spelled out fairly well in this blog post.

Now Bruce's blog, which I link to above, is a good resource for material on the don't-panic viewpoint, and in fact he is sometimes consulted by the TSA and I suspect they read his blog, and even understand it. So why do we get such inane security efforts? Why are we willing to ruin ourselves, and make air travel such a burden, and strip ourselves of civil rights?

There is a mistake that both sides make, I think. The goal of counter-terrorism is not to stop the terrorists from attacking and killing people, not directly. The goal of counter-terrorism is to stop the terrorists from scaring people. Of course, killing people is frightening, so it is no wonder we conflate the two approaches.

Bizarrely, Jonathan Zittrain turns out to be my cousin -- which is odd because I have known him for some time and he is also very active in the online civil rights world. How we came to learn this will be the first of my postings on the future of DNA sequencing and the company 23andMe.

(Follow the genetics for part two and other articles.)

23andMe is one of a small crop of personal genomics companies. For a cash fee (ranging from $400 to $1000, but dropping with regularity) you get a kit to send in a DNA sample. They can't sequence your genome for that amount today, but they can read around 600,000 "single-nucleotide polymorphisms" (SNPs) which are single-letter locations in the genome that are known to vary among different people, and the subject of various research about disease. 23andMe began hoping to let their customers know about how their own DNA predicted their risk for a variety of different diseases and traits. The result is a collection of information -- some of which will just make you worry (or breathe more easily) and some of which is actually useful. However, the company's second-order goal is the real money-maker. They hope to get the sequenced people to fill out surveys and participate in studies. For example, the more people fill out their weight in surveys, the more likely they might notice, "Hey, all the fat people have this SNP, and the thin people have that SNP, maybe we've found something."

However, recently they added a new feature called "Relative Finder." With Relative Finder, they will compare your DNA with all the other customers, and see if they can find long identical stretches which are very likely to have come from a common ancestor. The more of this they find, the more closely related two people are. All of us are related, often closer than we think, but this technique, in theory, can identify closer relatives like 1st through 4th cousins. (It gets a bit noisy after this.)

Relative Finder shows you a display listing all the people you are related to in their database, and for some people, it turns out to be a lot. You don't see the name of the person but you can send them an E-mail, and if they agree and respond, you can talk, or even compare your genomes to see where you have matching DNA.

For me it showed one third cousin, and about a dozen 4th cousins. Many people don't get many relatives that close. A third cousin, if you were wondering, is somebody who shares a great-great-grandparent with you, or more typically a pair of them. It means that your grandparents and their grandparents were "1st" cousins (ordinary cousins.) Most people don't have much contact with 3rd cousins or care much to. It's not a very close relationship.

However, I was greatly shocked to see the response that this mystery cousin was Jonathan Zittrain. Jonathan and I are not close friends, more appropriately we might be called friendly colleagues in the cyberlaw field, he being a founder of the Berkman Center and I being at the EFF. But we had seen one another a few times in the prior month, and both lectured recently at the new Singularity University, so we are not distant acquaintances either. Still, it was rather shocking to see this result. I was curious to try to figure out what the odds of it are.

One of the toughest challenges the Burning Man staff face is placing all the camps in the city. This stopped being an anarchy long ago, and the city is mapped and each camp given a precise area. The city has various "premium" locations which are valued in part for being close to things but mainly because they are high traffic for camps showing off interesting art or interactivity. Far more camps want to be in the premium locations than there is room, and almost everybody serious wants to be pre-placed somewhere so they can plan in advance and not have to race in the land-rush when the event officially opens.

(Some people like the land-rush. While you will not get a spot very close to the Esplanade or be able to be on the maps and calendars by address, you will get a bigger space for your group, because it's "take what you dare.")

Camps submit applications (this year by the start of May) describing the contribution they will offer the city, where they would like to be, and how much space they need. A team of placers (mostly volunteers with a few paid leaders) try to allocate the camps. They try to be fair, but the process is largely opaque, so any biases and mistakes are not generally visible to the community.

The process takes time, and the placement last year was announced to the community in early August, just a few weeks before the event. Camps are told only their approximate street location, and their dimensions. For reasons few have been able to fathom, the actual precise map, showing who is on corners and who is next to whom, is kept secret until the event itself. Many factors go into the decision, including camp density, past reputations, the various prime locations available, camps that want or don't wan't to be next to other camps, loudness and quality and interactivity of the art in the camp.

In 2009, the placers decided something which was a fairly big surprise to the community. They decided not to place around 120 of the camps that applied at all. Those camps were left to the land-rush, which meant a few distressing things for them:

• They could not arrive in the city before the opening to set up; some had rather extensive structures to build.
• They could not know where they would be in advance, so they could not tell their address to people in advance, or put it in the city calendar which is handed out at the gate.

The placement team decided not to place these camps because they did not want to find themselves placing the majority of the city. They wanted the city to retain some randomness and made a decision that only a limited fraction of the city would be subject to mapping in advance. If too many camps applied, those who did not make the cut would not be placed. This decision caused some controversy, and there are arguments for both sides. In addition to the non-placements, there were also many camps surprised by their placement (usually negatively) and, as would be expected in any large volunteer effort, a modest number of mistakes.

I wanted to post some follow-up to my prior post about sports where the contestants go to the edge and fall down. As I see it, the sports fall into the following rough classes:

Some notes from the bi-annual Olympics crackfest...

I'm starting to say that Curling might be the best Olympic sport. Why?

• It's the most dominated by strategy. It also requires precision and grace, but above all the other Olympic sports, long pauses to think about the game are part of the game. If you haven't guessed, I like strategy.
• Yes, other sports have in-game strategy, of course, particularly the team sports. And since the gold medalist from 25 years ago in almost every sport would barely qualify, you can make a case that all the sports are mostly mental in their way. But with curling, it's right there, and I think it edges out the others in how important it is.
• While it requires precision and athletic skill, it does not require strength and endurance to the human limits. As such, skilled players of all ages can compete. (Indeed, the fact that out-of-shape curlers can compete has caused some criticism.) A few other sports, like sharpshooting and equestrian events, also demand skill over youth. All the other sports give a strong advantage to those at the prime age.
• Mixed curling is possible, and there are even tournaments. There's debate on whether completely free mixing would work, but I think there should be more mixed sports, and more encouragement of it. (Many of the team sports could be made mixed, of course mixed tennis used to be in the Olympics and is returning.)
• The games are tense and exciting, and you don't need a clock, judge or computer to tell you who is winning.

On the downside, not everybody is familiar with the game, the games can take quite a long time and the tournament even longer for just one medal, and compared to a multi-person race it's a slow game. It's not slow compared to an even that is many hours of time trials, though those events have brief bursts of high-speed excitement mixed in with waiting. And yes, I'm watching Canada-v-USA hockey now too.

We all know how annoying it is that today's much faster computers take such a long time to boot, and OS developers are working on speeding it up. Some time ago I proposed a defragmenter that notice what blocks were read in what order at boot and put the contiguous on the disk. I was told that experiments with this had not had much success, but more recently I read reports of how the latest Linux distributions will boot as much a 3 times faster on solid state disks as on rotating ones. There are some SSDs with performance that high (and higher) but typical ones range more in the 120 mb/second rate, better than 80 mb/second HDDs but getting more wins from the complete lack of latency.

However, today I want to consider something which is a large portion of the boot time, namely the power-on-self-test or "POST." This is what the BIOS does before it gets ready to load the real OS. It's important, but on many systems is quite slow.

I propose an effort to make the POST multitask with the loading of the real OS. Particularly on dual-core systems, this would be done by having one core do the POST and other BIOS (after testing all the cores of course) and other cores be used by the OS for loading. There are ways to do all this with one core I will discuss below, but this one is simple and almost all new computers have multiple cores.

Of course, the OS has to know it's not supposed to touch certain hardware until after the BIOS is done initializing it and testing it. And so, there should be BIOS APIs to allow the OS to ask about this and get events as BIOS operations conclude. The OS, until given ownership of the screen, would output its status updates to the screen via a BIOS call. In fact, it would do that with all hardware, though the screen, keyboard and primary hard disk are the main items. When I say the OS, I actually mean both the bootloader that loads the OS and the OS itself once it is handed off to.

Next, the BIOS should, as soon as it has identified that the primary boot hard disks are ready, begin transferring data from the boot area into RAM. Almost all machines have far more RAM than they need to boot, and so pre-loading all blocks needed for boot into a cache, done in optimal order, will mean that by the time the OS kernal takes over, many of the disk blocks it will want to read will already be sitting in ram. Ideally, as I noted, the blocks should have been pre-stored in contiguous zones on the disk by an algorithm that watched the prior boots to see what was accessed and when.

Indeed, if there are multiple drives, the pre-loader could be configured to read from all of them, in a striping approach. Done properly, a freshly booted computer, once the drives had spun up, would start reading the few hundred megabytes of files it needs to boot from multiple drives into ram. All of this should be doable in just a few seconds on RAID style machines, where 3 disks striped can deliver 200mb/second or more of disk read performance. But even on a single drive, it should be quick. It would begin with the OS kernel and boot files, but then pre-cache all the pages from files used in typical boots. For any special new files, only a few seeks will be required after this is done.

On a recent trip on a plane equipped with personal inflight video screens for each seat, I decided to watch a movie quickly and then have a nap. So I started watching the movie right after settling into the seat, about 20 minutes before takeoff. I figured with that I would watch the 1:30 minute movie through the meal service and be ready for the nap about an hour into the flight. What I learned instead was a greater awareness of just how many announcements there are on a typical flight these days. That's because the in-flight system paused the video with each announcement and put it through my noise cancelling headphones.

The many announcements included:

• The routine ones about the process of takeoff. Door closing. Seatbelt sign on. Various blah-blah-blah
• The huge array of safety announcements and instructions I've seen literally hundreds of times.
• A very few useful announcements: Destination check, reasons for delay, updates on flight time.
• Some possibly useful announcements (cell phones off now, OK to use electronics now.)
• Ads: Join our frequent flyer program, get our frequent flyer card, shop from the duty free cart, buy meals, buy drinks (which did not even apply to those not in coach.)

The cacophony is getting worse, almost as bad as when you're sitting in the terminal with the endless announcements. They know people hate that in the terminals and offer the paid lounge with no announcements, but I've said they should just use cell phones instead and give us peace. On Japanese Shinkansen, they also offer a "quiet car" with no announcements -- it is up to you to set your own alarm to make sure you don't miss your stop if you want to sleep or relax. The trains are so on-time you can do this.

How about doing something like this, at least on a modern airplane where you have a personal screen for each seat?

I recently went to the DLD conference in Germany, briefly to Davos during the World Economic Forum and then drove around the Alps for a few days, including a visit to an old friend in Grenoble. I have some panoramic galleries of the Alps in Winter up already.

Each trip brings some new observations and notes.

• For the first time, I got a rental car which had a USB port in it, as I've been wanting for years. The USB port was really part of the radio, and if you plugged a USB stick in, it would play the music on it, but for me its main use was a handy charging port without the need for a 12v adapter. As I've said before, let's see this all the time, and let's put them in a few places -- up on the dashboard ledge to power a GPS, and for front and rear seats, and even the trunk. And have a plug so the computer can access the devices, or even data about the car.
• The huge network of tunnels in the alpine countries continues to amaze me, considering the staggering cost. Sadly, some seem to simply bypass towns that are pretty.
• I've had good luck on winter travel, but this trip reminded me why there are no crowds. The weather can curse you, and especially curse your photography, though the snow-covered landscapes are wonderful when you do get sun. Three trips to Lake Constance/Bodenzee now, and never any good weather!
• Davos was a trip. While there was a lot of security, it was far easier than say, flying in the USA. I was surprised how many people I knew at Davos. I was able to get a hotel in a village about 20 minutes away.

On to Part Two

This weekend I spoke at BIL, a conference that was created to play off of the famous and expensive TED conference. BIL began as an un-conference, which is to say an ad-hoc conference created on short notice where the attendees are the speakers. Such conferences tend to be free or near-free. The movement begain with Tim O'Reilly's FOO Camp. FOO camp is for Tim's friends, and he has far more friend that can come. One year, he was explaining how he rotated among people and so some of those who were not invited that particular year (including myself) had a "BAR" camp which was a tremendous success, and created a trend.

The first two BILs were a lot of fun and worked pretty well. They had a variety of sub-par speakers, as these "anybody who wants to can talk" conferences often have, but there was always tons of hall conversation or sessions in other rooms to make up for that. And a modest number of TED speakers came over and gave their TED talks for free at BIL, and various regular TED attendees came as well.

This year's BIL did not live up to the earlier standard, and the hard-working and generous organizers are fully aware of that, so this is not an attempt to criticise them, but rather to look at the problem. Many things went wrong, including a last minute need to move the conference from a Saturday and Sunday(with only Saturday morning overlap with TED) to Friday and Saturday morning, which had total overlap with TED and minimal weekend time. This change was forced because no venue could be found (cheaply enough, at least) which would offer Saturday afternoon and Sunday. However, it was a ruinous change -- attendance on the workday Friday was way down, and even lower on Saturday, and no TED speakers came though a few attendees showed up, mostly near the end in the 2 hours after TED that BIL went on. The "outdoor" post-sessions were of limited success as a conference, but OK socially (I did not attend the planned Sunday events.)

Tomorrow, I will be speaking on pre-Robocar technology at BIL an unconference that parallels the famous and expensive TED conference. This is in Long Beach, CA. Unconferences are fun, cheap and often as good as expensive conferences. I will also be attending a reception at TED tonight for Singularity University, which I lecture at, so I may see you if you're at TED as well.

Last night's EFF bash was a great success. Thanks to Adam Savage and all the others who made it go so well.

In early 2000, after a tumultuous period in the EFF's history, and the staff down to just a handful, I was elected chair of the Electronic Frontier Foundation. I had been on the board for just a few years, but had been close to the organization since it was founded, including participating with it as a plaintiff in the landmark supreme court case which struck down the Communications Decency Act in 1996.

I'll have many more observations about my recent trip to DLD, Davos and the Alps soon, but one thing I've decided I do want to find (or train) is a travel agent/helper who can assist well with unscheduled travel (ie. a road or railpass trip.)

With unscheduled travel, you don't know in the morning where you will end up that night. You only figure it out later in the day. Sometimes you just drive until it starts getting late and then you pick where you will end the night. It's hard (or expensive) to do this in high season but in low season you can always find a room, and I and many others like that sort of freedom.

So when you do pick where you want to end up you have a few options:

1. You can have a guidebook or database (such as AAA in the USA) and phone around places until you get something you like
2. You can hunt around for web access (better if you have a data plan on your phone) and use sites like TripAdvisor and the various booking search engines (like Kayak/Sidestep) to find a decent hotel at a good price.
3. You can just drive into town and look for Vacancy/Zimmer Frei signs and go in and ask the price.
4. You can find somebody to do this for you.

There are problems with all these approaches. Method 3 (especially using tripadivsor) helps you avoid turkey hotels and find the better values. However, the databases cover only a fraction of the hotels, and the online reservations systems also cover only a small fraction of hotels in an area. There will be better values out there. On the other hand, many hotels offer a better price through the internet than if you call them, or will charge even more if you just walk in.

I'm at DLD in Munich, and going to Davos tomorrow. While at DLD I made a brief mention during a panel on identity and tracking of my concept of the privacy dangers of the AIs of the future, which are able to extract things from recorded data (like faces) that we can't do today.

I mentioned a new idea, however, which is a search engine which focuses on the negative, because though advanced algorithms it can tell the difference between positive and negative content.

These days it is getting very common to make videos of presentations, and even to do live streams of them. And most of these presentations have slides in Powerpoint or Keynote or whatever. But this always sucks, because the camera operator -- if there is one -- never moves between the speaker and the slide the way I want. You can't please everybody of course.

There's a phenomenon we're seeing more and more often. A company screws over a customer, but this customer now has a means to reach a large audience through the internet, and as a result it becomes a PR disaster for the company. The most famous case recently was United Breaks Guitars where Nova Scotia musician David Carroll had his luggage mistreated and didn't get good service, so he wrote a funny song and music video about it.

This weekend is the Foresight Institute conference on molecular nanotechnology and AI. I am on the board of Foresight Institute and will be speaking on the latest developments in robocars at the conference, along with a raft of great speakers. If you are interested in futurist issues around AI, nanotech and other accelerating technologies, this is the longest running conference in the field and the place to be.