Submitted by brad on Thu, 2010-03-11 11:53.
I was recently approached by a programmer named Keith Curtis, formerly at Microsoft and now a FOSS devotee. He wants to develop a driving simulator for testing robocar systems. I think this is a very worthwhile idea — sort of a “Second Life” for robots. We have a head start — the world of racecar video games has already done a lot of the legwork to simulate driving, and there are two open source car racing systems.
A good simulator would bring some tremendous benefits to robocar development.
- Anybody, even a solo hacker in their basement, could develop and test robocar software on the cheap, and with no cost and risk from crashes. Small teams, perhaps working in car-less garages, could contribute greatly to the field.
- Testing could go faster, and include regular exposure to extreme situations not often found in the real world, like crazy drivers, strange hazards, map errors, sensor failures and more.
- Simulator testing allows the creation of new sensors which are plausible, but too expensive or too far in the future to work with in the real world. It allows teams to say, “What if we had 1cm accurate GPS? What if we had 180 line LIDAR to 100m?” and see if they can build robocar controls to use it.
- Robocar contests could be held in simulation, on the cheap and with no physical risk. The winners could then get funding to build real-world vehicles to race for a bigger prize.
- The simulator APIs for car controls and sensors can become prototype APIs for real-world interfaces, allowing easy testing and switching.
Of course, robocar simulation is nothing new. Several teams in the DARPA challenges built simulators to try out ideas. These remained proprietary efforts. Road simulation is also frequently used for traffic simulators. An open simulator would be shared, and the community (and not just the robocar development community) could contribute terrain, streets, sensors and simulators for elements such as pedestrians, human driven cars, blowing trash and new sensors, to name just a few.
Our wonderful new fast GPUs will be able to generate camera views anywhere in the 3D world for those working on machine vision. Simulating LIDAR, radar, ultrasound, odometry, accelerometers etc. is not yet done in car racing games but these things should not be hard to add. Indeed, any company selling a sensor would be well advised to build a simulated version of it.
And people hacking at home love to make 3-D maps of terrain. Existing real terrain models could be imported, or new ones made by driving around with LIDAR on real streets.
To explore this more I have written a new article on how to build a robocar driving simulator where I also point to an up and coming open source simulator called “Rigs of Rods” which actually simulates the vehicles at the physics level, treating them as a network of many connected parts.
The robocar world needs somebody ready to fun the kick-starting of such a simulator, and possibly some contests within it.
Submitted by brad on Tue, 2010-03-09 17:18.
I received some criticism the other day over my own criticism of the use of haplogroups in genealogy — the finding and tracing of relatives. My language was imprecise so I want to make a correction and explore the issue in a bit more detail.
One of the most basic facts of inheritance is that while most of your DNA is a mishmash of your parents (and all their ancestors before them) two pieces of DNA are passed down almost unchanged. One is the mitochondrial DNA, which is passed down from the mother to all her children. The other is the Y chromosome, which is passed down directly from father to son. Girls don’t get one. Most of the mother’s X chromosome is passed down unchanged to her sons (but not her daughters) but of course they can’t pass it unchanged to anybody.
This allow us to track the ancestry of two lines. The maternal line tracks your mother, her mother, her mother, her mother and so on. The paternal line tracks your father, his father and so on. The paternal line should, in theory, match the surname, but for various reasons it sometimes doesn’t. Females don’t have a Y, but they can often find out what Y their father had if they can sequence a sample from him, his sons, his brothers and other male relatives who share his surname.
The ability to do this got people very excited. DNA that can be tracked back arbitrarily far in time has become very useful for the study of human migrations and population genetics. The DNA is normally passed down completely but every so often there is a mutation. These mutations, if they don’t kill you, are passed down. The various collections of mutations are formed into a tree, and the branches of the tree are known as haplogroups. For both kinds of DNA, there are around a couple of hundred haplogroups commonly identified. Many DNA testing companies will look at your DNA and tell you your MTDNA haplogroup, and if male, your Y haplogroup. read more »
Submitted by brad on Mon, 2010-03-08 14:05.
Last week, I wrote about interesting experiences finding Cousins who were already friends via genetic testing. 23andMe’s new “Relative Finder” product
the other people in their database of about 35,000 to whom you are related, guessing how
close. Surprisingly, 2 of the 4 relatives I made contact with were already friends of
mine, but not known to be relatives.
Many people are very excited about the potential for services like Relative Finder to
take the lid off the field of genealogy. Some people care deeply about genealogy (most
notably the Mormons) and others wonder what the fuss is. Genetic genealogy offers the
potential to finally link all the family trees built by the enthusiasts and to provably
test already known or suspected relationships. As such, the big genealogy web sites are
all getting involved, and the Family Tree DNA company, which previously did mostly worthless
haplogroup studies (and more useful haplotype scans,) is opening up a paired-chromosome scan service for $250 — half the
price of 23andMe’s top-end scan. (There is some genealogical value to the deeper clade
Y studies FTDNA does, but the Mitochondrial and 12-marker Y studies show far less than
people believe about living relatives. I have a followup post about haplogroups and haplotypes in genealogy.) Note that in March 2010, 23andMe is offering a scan for just $199.
The cost of this is going to keep decreasing and soon will be sub-$100. At the same time,
the cost of full sequencing is falling by a factor of 10 every year (!) and many suspect it
may reach the $100 price point within just a few years. (Genechip sequencing only finds
the SNPs, while a full sequencing reads every letter (allele) of your genome, and perhaps in
the future your epigenome.
Discover of relatives through genetics has one big surprising twist to it. You are participating
in it whether you sign up or not. That’s because your relatives may be participating in it,
and as it gets cheaper, your relatives will almost certainly be doing so. You might be the
last person on the planet to accept sequencing but it won’t matter. read more »
Submitted by brad on Thu, 2010-03-04 17:34.
One of the world’s favourite (and sometimes least favourite) topics is the issue of terrorism and security. On one side, there are those who feel the risk of terrorism justifies significant sacrifices of money, convenience and civil rights to provide enough security to counter it. That side includes both those who honestly come by that opinion, and those who simply want more security and feel terrorism is the excuse to use to get it.
On the other side, critics point out a number of counter arguments, most of them with merit, including:
- Much of what is done in the name of security doesn’t actually enhance it, it just gives the appearance of doing so, and the appearance of security is what the public actually craves. This has been called “Security Theatre” by Bruce Schneier, who is a friend and advisor to the E.F.F.
- We often “fight the previous war,” securing against the tactics of the most recent attack. The terrorists have already moved on to planning something else. They did planes, then trains, then subways, then buses, then nightclubs.
- Terrorists will attack where the target is weakest. Securing something just makes them attack something else. This has indeed been the case many times. Since everything can’t be secured, most of our efforts are futile and expensive. If we do manage to secure everything they will attack the crowded lines at security.
- Terrorists are not out to kill random people they don’t know. Rather, that is their tool to reach their real goal: sowing terror (for political, religious or personal goals.) When we react with fear — particularly public fear — to their actions, this is what they want, and indeed what they plan to achieve. Many of our reactions to them are just what they planned to happen.
- Profiling and identity checks seem smart at first, but careful analysis shows that they just give a more free pass to anybody the terrorists can recruit whose name is not yet on a list, making their job easier.
- The hard reality is, that frightening as terrorism is, in the grand scheme we are for more likely to face harm and death from other factors that we spend much less of our resources fighting. We could save far more people applying our resources in other ways. This is spelled out fairly well in this blog post.
Now Bruce’s blog, which I link to above, is a good resource for material on the don’t-panic viewpoint, and in fact he is sometimes consulted by the TSA and I suspect they read his blog, and even understand it. So why do we get such inane security efforts? Why are we willing to ruin ourselves, and make air travel such a burden, and strip ourselves of civil rights?
There is a mistake that both sides make, I think. The goal of counter-terrorism is not to stop the terrorists from attacking and killing people, not directly. The goal of counter-terrorism is to stop the terrorists from scaring people. Of course, killing people is frightening, so it is no wonder we conflate the two approaches. read more »
Submitted by brad on Wed, 2010-03-03 16:20.
Bizarrely, Jonathan Zittrain turns out to be my cousin — which is odd because I have known him for some time and he is also very active in the online civil rights world. How we came to learn this will be the first of my postings on the future of DNA sequencing and the company 23andMe.
(Follow the genetics for part two and other articles.)
23andMe is one of a small crop of personal genomics companies. For a cash fee (ranging from $400 to $1000, but dropping with regularity) you get a kit to send in a DNA sample. They can’t sequence your genome for that amount today, but they can read around 600,000 “single-nucleotide polymorphisms” (SNPs) which are single-letter locations in the genome that are known to vary among different people, and the subject of various research about disease. 23andMe began hoping to let their customers know about how their own DNA predicted their risk for a variety of different diseases and traits. The result is a collection of information — some of which will just make you worry (or breathe more easily) and some of which is actually useful. However, the company’s second-order goal is the real money-maker. They hope to get the sequenced people to fill out surveys and participate in studies. For example, the more people fill out their weight in surveys, the more likely they might notice, “Hey, all the fat people have this SNP, and the thin people have that SNP, maybe we’ve found something.”
However, recently they added a new feature called “Relative Finder.” With Relative Finder, they will compare your DNA with all the other customers, and see if they can find long identical stretches which are very likely to have come from a common ancestor. The more of this they find, the more closely related two people are. All of us are related, often closer than we think, but this technique, in theory, can identify closer relatives like 1st through 4th cousins. (It gets a bit noisy after this.)
Relative Finder shows you a display listing all the people you are related to in their database, and for some people, it turns out to be a lot. You don’t see the name of the person but you can send them an E-mail, and if they agree and respond, you can talk, or even compare your genomes to see where you have matching DNA.
For me it showed one third cousin, and about a dozen 4th cousins. Many people don’t get many relatives that close. A third cousin, if you were wondering, is somebody who shares a great-great-grandparent with you, or more typically a pair of them. It means that your grandparents and their grandparents were “1st” cousins (ordinary cousins.) Most people don’t have much contact with 3rd cousins or care much to. It’s not a very close relationship.
However, I was greatly shocked to see the response that this mystery cousin was Jonathan Zittrain. Jonathan and I are not close friends, more appropriately we might be called friendly colleagues in the cyberlaw field, he being a founder of the Berkman Center and I being at the EFF. But we had seen one another a few times in the prior month, and both lectured recently at the new Singularity University, so we are not distant acquaintances either. Still, it was rather shocking to see this result. I was curious to try to figure out what the odds of it are. read more »
Submitted by brad on Mon, 2010-03-01 13:21.
One of the toughest challenges the Burning Man staff face is placing all the camps in the city. This stopped being an anarchy long ago, and the city is mapped and each camp given a precise area. The city has various “premium” locations which are valued in part for being close to things but mainly because they are high traffic for camps showing off interesting art or interactivity. Far more camps want to be in the premium locations than there is room, and almost everybody serious wants to be pre-placed somewhere so they can plan in advance and not have to race in the land-rush when the event officially opens.
(Some people like the land-rush. While you will not get a spot very close to the Esplanade or be able to be on the maps and calendars by address, you will get a bigger space for your group, because it’s “take what you dare.”)
Camps submit applications (this year by the start of May) describing the contribution they will offer the city, where they would like to be, and how much space they need. A team of placers (mostly volunteers with a few paid leaders) try to allocate the camps. They try to be fair, but the process is largely opaque, so any biases and mistakes are not generally visible to the community.
The process takes time, and the placement last year was announced to the community in early August, just a few weeks before the event. Camps are told only their approximate street location, and their dimensions. For reasons few have been able to fathom, the actual precise map, showing who is on corners and who is next to whom, is kept secret until the event itself. Many factors go into the decision, including camp density, past reputations, the various prime locations available, camps that want or don’t wan’t to be next to other camps, loudness and quality and interactivity of the art in the camp.
In 2009, the placers decided something which was a fairly big surprise to the community. They decided not to place around 120 of the camps that applied at all. Those camps were left to the land-rush, which meant a few distressing things for them:
- They could not arrive in the city before the opening to set up; some had rather extensive structures to build.
- They could not know where they would be in advance, so they could not tell their address to people in advance, or put it in the city calendar which is handed out at the gate.
The placement team decided not to place these camps because they did not want to find themselves placing the majority of the city. They wanted the city to retain some randomness and made a decision that only a limited fraction of the city would be subject to mapping in advance. If too many camps applied, those who did not make the cut would not be placed. This decision caused some controversy, and there are arguments for both sides. In addition to the non-placements, there were also many camps surprised by their placement (usually negatively) and, as would be expected in any large volunteer effort, a modest number of mistakes. read more »
Submitted by brad on Sun, 2010-02-21 23:41.
I wanted to post some follow-up to my prior post about sports where the contestants go to the edge and fall down. As I see it, the sports fall into the following rough classes:
- You push as hard as you can. The athlete who does the best — goes higher, stronger, faster — wins. The 100 metres is a perfect example.
- You push hard, but you must make judgments as you compete, such as how hard to push, when to pass etc. If you judge incorrectly, it lowers your final performance, but only modestly.
- You must judge your abilities and the condition of the course and your equipment, and choose what is difficult enough to score well, but which you can be confident you will execute. To win, you must skate (literally or figuratively) close to the edge of what you can do. If you misjudge, or have bad luck, you are penalized significantly, and will move from 1st place to the rear of the pack.
My main concern is with sports in the third group, like figure skating, half-pipe and many others, including most of the judged sports with degrees of difficulty. The concern is that sudden shift from leader to loser because you did what you are supposed to do — go close to the edge. Medals come from either being greatly superior, knowing your edge very well, which is the intention, or from being lucky that particular day — which I think is not the intention.
Many sports seek to get around this. In high jump and pole vault, you just keep raising the bar until you can’t get over it, and somebody gets over the highest bar. This is a good system, but difficult when a sport takes a long time or is draining to do even once. You could do a figure skating contest where they all keep trying more and more difficult moves until they all fail but one, but it would take a long time, be draining, and cause injuries as there is no foam pit like in high jump.
Other sports try to solve this by letting you do 2 or more runs, and taking the best run. This is good, but we also have an instinct that the person who can do it twice is better than the one who did it once and fell down the other time. Sports that sum up several times demand consistent performance, which is good, but in essence they also put a major punishment on a single failure, perhaps an even greater one. This approach requires you be a touch conservative, just under your edge, so you know you will deliver several very good runs, and beat somebody who dares beyond their ability, but falls in one or more runs. At least it reduces the luck.
The particular problem is this. “Big failure” sports will actually often give awards either to a top athlete who got a good run, or to the athlete who was more conservative in choosing what to do, and thus had a very high probability of a clean run. Fortunately this will not happen too often, as one of the top tier who went-for-broke will have that clean run and get 1st place. But the person who does that may not be the one who is overall most capable.
Imagine if high jump were done with each competitor choosing what height they wanted the bar to be at in advance, and getting one try at it, and getting a medal if it’s the highest, but nothing if they miss.
The sports like short-track speed skating, which are highly entertaining, have this problem in spades, for athletes who wipe out can also impede other athletes. While the rules try to make it up to the athlete who was knocked out, they have a hard time doing this perfectly. For example in the semi-finals of short-track, if you get knocked out while you were in 3rd, you are not assured to get a consolation qualification even if you were just about to try for 2nd with the strength you were saving.
In some cases the chaos is not going away because they know audiences like it. Time trials are the purest and fairest competition in most cases but are dead-boring to watch.
Submitted by brad on Sun, 2010-02-21 18:08.
Some notes from the bi-annual Olympics crackfest…
I’m starting to say that Curling might be the best Olympic sport. Why?
- It’s the most dominated by strategy. It also requires precision and grace, but above all the other Olympic sports, long pauses to think about the game are part of the game. If you haven’t guessed, I like strategy.
- Yes, other sports have in-game strategy, of course, particularly the team sports. And since the gold medalist from 25 years ago in almost every sport would barely qualify, you can make a case that all the sports are mostly mental in their way. But with curling, it’s right there, and I think it edges out the others in how important it is.
- While it requires precision and athletic skill, it does not require strength and endurance to the human limits. As such, skilled players of all ages can compete. (Indeed, the fact that out-of-shape curlers can compete has caused some criticism.) A few other sports, like sharpshooting and equestrian events, also demand skill over youth. All the other sports give a strong advantage to those at the prime age.
- Mixed curling is possible, and there are even tournaments. There’s debate on whether completely free mixing would work, but I think there should be more mixed sports, and more encouragement of it. (Many of the team sports could be made mixed, of course mixed tennis used to be in the Olympics and is returning.)
- The games are tense and exciting, and you don’t need a clock, judge or computer to tell you who is winning.
On the downside, not everybody is familiar with the game, the games can take quite a long time and the tournament even longer for just one medal, and compared to a multi-person race it’s a slow game. It’s not slow compared to an even that is many hours of time trials, though those events have brief bursts of high-speed excitement mixed in with waiting. And yes, I’m watching Canada-v-USA hockey now too. read more »
Submitted by brad on Thu, 2010-02-18 19:53.
We all know how annoying it is that today’s much faster computers take such a long time to boot, and OS developers are working on speeding it up. Some time ago I proposed a defragmenter that notice what blocks were read in what order at boot and put the contiguous on the disk. I was told that experiments with this had not had much success, but more recently I read reports of how the latest Linux distributions will boot as much a 3 times faster on solid state disks as on rotating ones. There are some SSDs with performance that high (and higher) but typical ones range more in the 120 mb/second rate, better than 80 mb/second HDDs but getting more wins from the complete lack of latency.
However, today I want to consider something which is a large portion of the boot time, namely the power-on-self-test or “POST.” This is what the BIOS does before it gets ready to load the real OS. It’s important, but on many systems is quite slow.
I propose an effort to make the POST multitask with the loading of the real OS. Particularly on dual-core systems, this would be done by having one core do the POST and other BIOS (after testing all the cores of course) and other cores be used by the OS for loading. There are ways to do all this with one core I will discuss below, but this one is simple and almost all new computers have multiple cores.
Of course, the OS has to know it’s not supposed to touch certain hardware until after the BIOS is done initializing it and testing it. And so, there should be BIOS APIs to allow the OS to ask about this and get events as BIOS operations conclude. The OS, until given ownership of the screen, would output its status updates to the screen via a BIOS call. In fact, it would do that with all hardware, though the screen, keyboard and primary hard disk are the main items. When I say the OS, I actually mean both the bootloader that loads the OS and the OS itself once it is handed off to.
Next, the BIOS should, as soon as it has identified that the primary boot hard disks are ready, begin transferring data from the boot area into RAM. Almost all machines have far more RAM than they need to boot, and so pre-loading all blocks needed for boot into a cache, done in optimal order, will mean that by the time the OS kernal takes over, many of the disk blocks it will want to read will already be sitting in ram. Ideally, as I noted, the blocks should have been pre-stored in contiguous zones on the disk by an algorithm that watched the prior boots to see what was accessed and when.
Indeed, if there are multiple drives, the pre-loader could be configured to read from all of them, in a striping approach. Done properly, a freshly booted computer, once the drives had spun up, would start reading the few hundred megabytes of files it needs to boot from multiple drives into ram. All of this should be doable in just a few seconds on RAID style machines, where 3 disks striped can deliver 200mb/second or more of disk read performance. But even on a single drive, it should be quick. It would begin with the OS kernel and boot files, but then pre-cache all the pages from files used in typical boots. For any special new files, only a few seeks will be required after this is done. read more »
Submitted by brad on Wed, 2010-02-17 20:26.
On a recent trip on a plane equipped with personal inflight video screens for each seat, I decided to watch a movie quickly and then have a nap. So I started watching the movie right after settling into the seat, about 20 minutes before takeoff. I figured with that I would watch the 1:30 minute movie through the meal service and be ready for the nap about an hour into the flight. What I learned instead was a greater awareness of just how many announcements there are on a typical flight these days. That’s because the in-flight system paused the video with each announcement and put it through my noise cancelling headphones.
The many announcements included:
- The routine ones about the process of takeoff. Door closing. Seatbelt sign on. Various blah-blah-blah
- The huge array of safety announcements and instructions I’ve seen literally hundreds of times.
- A very few useful announcements: Destination check, reasons for delay, updates on flight time.
- Some possibly useful announcements (cell phones off now, OK to use electronics now.)
- Ads: Join our frequent flyer program, get our frequent flyer card, shop from the duty free cart, buy meals, buy drinks (which did not even apply to those not in coach.)
The cacophony is getting worse, almost as bad as when you’re sitting in the terminal with the endless announcements. They know people hate that in the terminals and offer the paid lounge with no announcements, but I’ve said they should just use cell phones instead and give us peace. On Japanese Shinkansen, they also offer a “quiet car” with no announcements — it is up to you to set your own alarm to make sure you don’t miss your stop if you want to sleep or relax. The trains are so on-time you can do this.
How about doing something like this, at least on a modern airplane where you have a personal screen for each seat? read more »
Submitted by brad on Tue, 2010-02-16 19:02.
I recently went to the DLD conference in Germany, briefly to Davos during the World Economic Forum and then drove around the Alps for a few days, including a visit to an old friend in Grenoble. I have some panoramic galleries of the Alps in Winter up already.
Each trip brings some new observations and notes.
- For the first time, I got a rental car which had a USB port in it, as I’ve been wanting for years. The USB port was really part of the radio, and if you plugged a USB stick in, it would play the music on it, but for me its main use was a handy charging port without the need for a 12v adapter. As I’ve said before, let’s see this all the time, and let’s put them in a few places — up on the dashboard ledge to power a GPS, and for front and rear seats, and even the trunk. And have a plug so the computer can access the devices, or even data about the car.
- The huge network of tunnels in the alpine countries continues to amaze me, considering the staggering cost. Sadly, some seem to simply bypass towns that are pretty.
- I’ve had good luck on winter travel, but this trip reminded me why there are no crowds. The weather can curse you, and especially curse your photography, though the snow-covered landscapes are wonderful when you do get sun. Three trips to Lake Constance/Bodenzee now, and never any good weather!
- Davos was a trip. While there was a lot of security, it was far easier than say, flying in the USA. I was surprised how many people I knew at Davos. I was able to get a hotel in a village about 20 minutes away.
On to Part Two read more »
Submitted by brad on Mon, 2010-02-15 19:08.
This weekend I spoke at BIL, a conference that was created to play off of the famous and expensive TED conference. BIL began as an un-conference, which is to say an ad-hoc conference created on short notice where the attendees are the speakers. Such conferences tend to be free or near-free. The movement begain with Tim O’Reilly’s FOO Camp. FOO camp is for Tim’s friends, and he has far more friend that can come. One year, he was explaining how he rotated among people and so some of those who were not invited that particular year (including myself) had a “BAR” camp which was a tremendous success, and created a trend.
The first two BILs were a lot of fun and worked pretty well. They had a variety of sub-par speakers, as these “anybody who wants to can talk” conferences often have, but there was always tons of hall conversation or sessions in other rooms to make up for that. And a modest number of TED speakers came over and gave their TED talks for free at BIL, and various regular TED attendees came as well.
This year’s BIL did not live up to the earlier standard, and the hard-working and generous organizers are fully aware of that, so this is not an attempt to criticise them, but rather to look at the problem. Many things went wrong, including a last minute need to move the conference from a Saturday and Sunday(with only Saturday morning overlap with TED) to Friday and Saturday morning, which had total overlap with TED and minimal weekend time. This change was forced because no venue could be found (cheaply enough, at least) which would offer Saturday afternoon and Sunday. However, it was a ruinous change — attendance on the workday Friday was way down, and even lower on Saturday, and no TED speakers came though a few attendees showed up, mostly near the end in the 2 hours after TED that BIL went on. The “outdoor” post-sessions were of limited success as a conference, but OK socially (I did not attend the planned Sunday events.) read more »
Submitted by brad on Thu, 2010-02-11 11:18.
Tomorrow, I will be speaking on pre-Robocar technology at BIL an unconference that parallels the famous and expensive TED conference. This is in Long Beach, CA. Unconferences are fun, cheap and often as good as expensive conferences. I will also be attending a reception at TED tonight for Singularity University, which I lecture at, so I may see you if you’re at TED as well.
Last night’s EFF bash was a great success. Thanks to Adam Savage and all the others who made it go so well.
Submitted by brad on Tue, 2010-02-09 22:27.
In early 2000, after a tumultuous period in the EFF’s history, and
the staff down to just a handful, I was elected chair of the Electronic Frontier Foundation.
I had been on the board for just a few years, but had been close to the
organization since it was founded, including participating with it as
a plaintiff in the landmark supreme court case which struck down the
Communications Decency Act in 1996.
Having now served 10 years as chairman, it is time to rotate out, and I
am happy to report the election of John Buckman, founder of Magnatune
and Bookmooch (among other ventures) as our new chair. As a part-time
resident of Europe, John will, like me, offer an international perspective
to the EFF’s efforts. Pam Samuelson, a law professor of stunning
reputation and credentials, is the vice-chair for the coming
5-year term, replacing John Perry Barlow.
I would love to claim credit for the EFF’s tremendous growth and success
during my tenure, but the truth is that our active and star-studded board
is a board of equals. We all take an active role in setting policy
and attempting to guide the organization in its mission to protect
important freedoms in the online world. While it would shock most of
my previous employees, my board management has been very laissez-faire.
I and the other board members try to let our great team do their stuff.
After I became chairman, one of the best things we on the board
did was to re-recruit Shari Steele, our former legal director,
to become the new executive director. Shari had been with the EFF for
many years but had left to work on a new venture. We brought her back
and it’s been positive ever since. We also recruited Cindy Cohn
to be our legal director. Cindy had a long history of friendship with
the organization, having worked tirelessly with our help on the fight
to stop export controls on encryption. WIth these two appointments, I
and my fellow board members started the course for an incredible decade.
In spite of a chaotic global economy, during this period, our fundraising,
budget and staff size have more than tripled. (That may seem minor
for a dot-com but it’s great news for a non-profit.) We’ve boosted
membership and membership dontations, increased funding from foundations,
and created an endowment to assure the EFF’s future.
The EFF is now 20, so I’ve been privileged to chair it for half of
its lifetime. In that period we’ve seen dramatic victories for free
speech, privacy and freedom to program. We’ve stopped e-voting abuse
and rootkits in your music CDs. We’ve protected bloggers as journalists
and preserved anonymous speech online. We’ve stopped encryption software
from being controlled like a munition and had so many other triumphs, big
and small. We’ve also seen an expanded technical and activism program,
as our technologists have led the way in unveiling things like secret
dots generated by colour laser printers that track your printouts back
to you and network interference with filesharing by cable ISPs.
We’ve also had our failures, but even those have spoken loudly about the
quality of our team. When we took Grokster/Streamcast to the supreme
court, our client lost, but the court laid down a fairly narrow standard
that allows software developers building new generations of publishing
products to know how to stay clear of liability. Our cases against the
White House’s warrantless wiretapping program have hit major hurdles,
one of which was an act of congress created specifically to nullify our
attempts to have a court examine this program — granting a retroactive
immunity to the phone companies that did it. Bad as that was, I figure
if they have to get an act of congress to stop you, you know you’ve hit
We’ve also hit many nerves with our great FOIA team that has uncovered
all sorts of attacks on your rights, and continues to do so, and our
team of activists and our new international team are working hard to
promote our doctrine of free speech and freedom to develop technology
around the world. With all our team does, many are shocked to find it is
only around 30 people. Still, we could do much more and your donations
are still what makes it all happen. I hope that if you believe in the
duty to protect fundamental freedoms online, you will work towards that end directly,
or consider outsourcing that work with a donation to us.
I am not leaving the EFF — far from it. I will continue to be
an active boardmember. In addition, I will begin to re-explore
commercial ventures, seek new opportunities, and continue on my quest
to become a leading evangelist for one of the world’s most exciting
new technologies — robotic transportation. At my robocars site you
can see my beginnings of a book on the subject, and why it may have the
largest positive effect on the world that computer technology delivers
in the medium term. Of course with my EFF hat on you will find growing
sections on the freedom and privacy issues of the technology.
During my tenure, I have served with a tremendous group of
fellow board members, as you can see from the biographies at
the EFF board page. I will continue to work with them to
protect your rights as the world becomes digital, and I hope you will all
join with me in supporting the EFF with your thoughts and your dollars.
We’ll mark the transition tonight, Feb 10 at the special EFF 20th birthday bash
at DNA lounge. This fundraiser can be attended
with a requested $30 donation, and there is also a special VIP event earlier where you can mingle more
intimately with the special guests, such as Mythbuster Adam Savage. We have quite a program planned.
Submitted by brad on Fri, 2010-02-05 10:47.
I’ll have many more observations about my recent trip to DLD, Davos and the Alps soon, but one thing I’ve decided I do want to find (or train) is a travel agent/helper who can assist well with unscheduled travel (ie. a road or railpass trip.)
With unscheduled travel, you don’t know in the morning where you will end up that night. You only figure it out later in the day. Sometimes you just drive until it starts getting late and then you pick where you will end the night. It’s hard (or expensive) to do this in high season but in low season you can always find a room, and I and many others like that sort of freedom.
So when you do pick where you want to end up you have a few options:
- You can have a guidebook or database (such as AAA in the USA) and phone around places until you get something you like
- You can hunt around for web access (better if you have a data plan on your phone) and use sites like TripAdvisor and the various booking search engines (like Kayak/Sidestep) to find a decent hotel at a good price.
- You can just drive into town and look for Vacancy/Zimmer Frei signs and go in and ask the price.
- You can find somebody to do this for you.
There are problems with all these approaches. Method 3 (especially using tripadivsor) helps you avoid turkey hotels and find the better values. However, the databases cover only a fraction of the hotels, and the online reservations systems also cover only a small fraction of hotels in an area. There will be better values out there. On the other hand, many hotels offer a better price through the internet than if you call them, or will charge even more if you just walk in. read more »
Submitted by brad on Tue, 2010-01-26 05:28.
I’m at DLD in Munich, and going to Davos tomorrow. While at DLD I made a brief mention during a panel on identity and tracking of my concept of the privacy dangers of the AIs of the future, which are able to extract things from recorded data (like faces) that we can’t do today.
I mentioned a new idea, however, which is a search engine which focuses on the negative, because though advanced algorithms it can tell the difference between positive and negative content.
We’re quite interested in dirt. Every eBay user who looks at a seller’s feedback would like to see only the negative comments, as the positive ones tell almost no information. eBay doesn’t want to show this, they want people to see eBay sellers as positive and to bid.
But a lot of the time if we are investigating a company we might do business with or even a person, we want to focus on the negative. A company with few complaints is of interest to us. AI software will exist to find such complaints, and possibly even to do things like understand photos and know which ones might be a source of embarrassment, or read on postings on message boards and tell which ones are damning. This is hard to do well today, but will change over time.
This will have deep consequences to concepts of reputation. Those with a big online presence certainly have bad stuff written by or about them out there. Normally, however, it is buried in the large volume of stuff, and doesn’t get high search engine rankings. However, our human thirst for gossip and dirt will result in some search engines will push it to the top. In addition, there will be those wanting to game this with deliberate libel of their enemies and competitors. Today they can do this but their libels will be hidden in the large volume of information.
Some have proposed that in the future it will be necessary to pay a service to libel you, and spread lots of false material that buries and discredits any libel left by enemies (as well as true negative comments.) The AIs may be able to spot the difference, but that’s an arms race which can’t easily be predicted.
It is likely that all the bad in our lives will haunt us even more than we already fear. Efforts by some countries to pass laws which let people delete alleged libels will not work, and may bring even more attention to the materials. While you might be able to remove your tag from a photo on facebook, once that photo makes it into a system that can do face recognition the tag will come back and do so in ways beyond your control.
Submitted by brad on Thu, 2010-01-21 16:28.
These days it is getting very common to make videos of presentations, and even to do live streams of them. And most of these presentations have slides in Powerpoint or Keynote or whatever. But this always sucks, because the camera operator — if there is one — never moves between the speaker and the slide the way I want. You can’t please everybody of course.
In the proprietary “web meeting” space there are several tools that will let people do a video presentation and sync it with slides, ideally by pre-transmitting the slide deck so it is rendered in full resolution at the other end, along with the video. In this industry there are also some video players where you can seek along in the video and it automatically seeks within the slides. This can be a bit complex if the slides try to do funny animations but it can be done.
Obviously it would be nice to see a flash player that understands it is playing a video and also playing slides (even video of slides, though it would be better to do it in higher quality since it isn’t usually full motion video.) Sites like youtube could support it. However, getting the synchronization requires that you have a program on the presenting computer, which you may not readily control.
One simple idea would be a button the camera operator could push to say “Copy this frame to the slide window.” Then the camera would, when there is a new slide, move or switch over there, and the button would be pushed, and the camera could go immediately back to the speaker. Usually though the camera crew has access to the projector feed and would not need to actually point a camera, in fact some systems “switch” to the slides by just changing the video feed. A program which sends the projector feed with huge compresion (in other words, an I frame for any slide change and nothing after) would also work well. No need to send all the fancy transitions.
But it would be good to send the slides not as mpeg, but as PNG style, to be sharper if you can’t get access to the slides themselves. I want a free tool, so I can’t ask for the world, yet, but even something as basic as this would make my watching of remote presentations and talks much better. And it would make people watching my talks have a better time too — a dozen or so of them are out on the web.
I’m in O’Hare waiting to fly to Munich for DLD. More details to come.
Submitted by brad on Fri, 2010-01-15 17:50.
There’s a phenomenon we’re seeing more and more often. A company screws over a customer, but this customer now has a means to reach a large audience through the internet, and as a result it becomes a PR disaster for the company. The most famous case recently was United Breaks Guitars where Nova Scotia musician David Carroll had his luggage mistreated and didn’t get good service, so he wrote a funny song and music video about it. 7 million views later, a lot of damage was done to United Airlines’ reputation.
I’ve done this myself to companies who refuse to fix things. I will write a page about the incident sometimes, and due to my high google pagerank, the page will show up high. Do a Google search for Qwest Long Distance and you’ll see the first hit is Qwest, and the 2nd is my boring but frustrating story of bad service. I’m not the only one to have done this. Over 200 people per month visit that page — which has been up for almost a decade — and you have to assume they have lost more business than it would have cost to make things right.
Now I think all good companies should make things right whenever they can to show that the errors are rare enough that they can afford to go the extra mile and fix them. If you won’t fix them, it means you must have a lot of them.
However, companies are soon going to realize that there are a whole raft of “minor celebrities” like David Carroll and even myself who can do far more damage than they can tolerate. Companies have always given top notch service to A-list celebrities, and even to B list. Not just gift bags at the Oscars. When I was kid, my father was A-list for a time in Canada, and that meant that when he got on a plane with a coach ticket, the flight attendant escorted him to first class. That was in the days before first class was always full due to upgrades, of course.
But there are tens of thousands, maybe hundreds of thousands of people who can be a risk for a company if they piss us off. All bloggers with a decent audience (and even some who have an audience that includes the A-list bloggers.) People with high search engine rank. People who can simply write well to get their story out there — in particular people who are good at making a story funny and entertaining. And of course, musicians and people who are good at video editing and producing viral videos. Perhaps them most of all.
So I predict that before long services will spring up to enumerate these D-list and E-list celebrities and potential celebs. Everybody will get graded. And a flag will show up in the customer service computer for the top few percentiles saying, “this one is an influencer.” It will say, “you are authorized, though you are just a script monkey customer rep, to do more for this customer.” Or you might just be direct right to a more powerful rep. This “long tail elite” may just start getting better service and even better deals, so long as they identify themselves first.
Companies have done this for some time based on how good a customer you are, ie. how much you spend. If you are a big spending customer, you get the magic 800 number or just get routed to the better service due to your frequent flyer number or even caller-ID. But I’m talking about doing this not just for those who spend a lot, but for those who influence a lot of spending — or could influence it in a negative way.
And of course they are working hard to make us identify ourselves in every transaction, just not yet for this. People who review products for a living will need to be sure they are anonymous when they buy and ask for service. But oddly, negative reviews from people who review stuff for a living are becoming less important than the horror story from the negative guy. Since most product reviewers at magazines are unwilling to go through the horrors of real customer service, they call the PR flacks and get top-rated service, and then explain in the review that they did this (if they are honest.)
If you’re not in the long-tail elite, this is all a bad sign. You’ll never get much satisfaction, and the number of horror stories on the net will go down below what the true level should be. Of course you will be able to join the long tail elite if you want to, since I am sure those who track it will note the names of people who regularly show up on consumer complaint message boards that have high readership or rank. But that’s a lot of work.
It doesn’t really do a lot of good for the rest of the world if perks are given to the long tail elite. Better just for companies to get good enough that they make mistakes rarely, and thus can afford to go the extra distance to fix them when it happens.
Submitted by brad on Tue, 2010-01-12 12:40.
This weekend is the Foresight Institute conference on molecular nanotechnology and AI. I am on the board of Foresight Institute and will be speaking on the latest developments in robocars at the conference, along with a raft of great speakers. If you are interested in futurist issues around AI, nanotech and other accelerating technologies, this is the longest running conference in the field and the place to be. The whole conference is just $175 and you can register for it here.
I will also be doing my general Robocar talk on Wednesday, February 24th at the “Homebrew Robotics Club” of Silicon Valley. This is a great group of people who hack robotics as a hobby, and it meets at the CMU building at NASA Ames Research Center. This event is free and open to the public.
Finally, in 2 weeks I will be attending the DLD 10 conference in Munich Germany, and may try to make my way into some parties at the World Economic Forum in Davos while doing a little Alpine road trip after DLD.
Note for those who can’t attend the Foresight conference, you can watch it stream live
Submitted by brad on Mon, 2010-01-11 19:31.
The pharma industry is littered with cases of drugs that showed good promise, but proved to be too dangerous when they got into human trials. Dangerous side effects will cancel development for most drugs. In some cases, such as Vioxx and Fen-Phen the dangerous effects were discovered later, and the drugs pulled from the market.
Some people got better on the drugs, others had bad side effects. Sometimes those bad side effects will be the result of various environmental factors, or perhaps rarely they will be bad luck. However, I suspect that some good portion of the time, they will be due to genetic factors in the test subject.
DNA sequencing is getting cheaper every day. Even today the whole genome can be done for $5,000 wholesale and many expect it to be hundreds before long. Collections of 600,000 to 1,000,000 SNPs can be read for a few hundred dollars.
It strikes me that the drug companies will want to make efforts immediately to get DNA samples, if they don’t have them, from all the people who participated in the trials of failed drugs, particularly those who had the bad side effects. And to get those samples sequenced. Because in some cases, they may well find a connection between the bad reaction and genetic patterns. They might find cases where all the side effects had one gene and all the regular reactions had another.
If they do find this, then suddenly they will have a billion dollar drug on their hands again, with much of the work already done, presuming the FDA and other regulatory agencies accept this approach. With the gene identified, making a test for it would be very cheap, and suddenly a useful drug might be available to those who have no problems with it. This might leave the people with the reacting DNA out in the lurch of course, and nobody is likely to try to find a drug for them in the immediate future.
If people get large DNA scans, those scans should remain in the possession of the patient or their doctor. There are already laws forbidding insurance companies in some jurisdictions from using DNA data to adjust insurance, but there will be powerful forces trying to reverse this.
DNA results will also explain differing efficacies of the drugs. It’s already been learned that many people need different doses of the same drug, and also that some drugs
work on men but not on women, for example.
I expect this will be standard practice for future drug trials, but my point today is that since many of these people are still alive, we can reach into the past and learn the truth about long-past drug trials as well, and perhaps get a brief flood of new useful substances as long as the patient is DNA typed in advance.