Blogs

The "Forgetful Broker" is needed for Data Deposit Box

For some time I've been advocating a concept I call the Data Deposit Box as an architecture for providing social networking and personal data based applications in a distributed way that tries to find a happy medium between the old PC (your data live on your machine) and the modern cloud (your data live on 3rd party corporate machines) approach. The basic concept is to have a piece of cloud that you legally own (a data deposit box) where your data lives, and code from applications comes and runs on your box, but displays to your browser directly. This is partly about privacy, but mostly about interoperability and control.

This concept depends on the idea of publishing and subscribing to feeds from your friends (and other sources.) Your friends are updating data about themselves, and you might want to see it -- ie. things like the Facebook wall, or Twitter feed. Feeds themselves would go through brokers just for the sake of efficiency, but would be encrypted so the brokers can't actually read them.

There is a need for brokers which do see the data in certain cases, and in fact there's a need that some types of data are never shown to your friends.

Crush

One classic example is the early social networking application the "crush" detector. In this app you get to declare a crush on a friend, but this is only revealed when both people have a mutual crush. Clearly you can't just be sending your crush status to your friends. You need a 3rd party who gets the status of both of you, and only alerts you when the crush is mutual. (In some cases applications like this can be designed to work without the broker knowing your data through the process known as blinding (cryptography).)

Topic: 

Time for the fourth screen -- the always on wall computer

In media today, it's common to talk about three screens: Desktop, mobile and TV. Many people watch TV on the first two now, and tools like Google TV and the old WebTV try to bring interactive, internet style content to the TV. People like to call the desktop the "lean forward" screen where you use a keyboard and have lots of interactivity, while the TV is the "lean back" couch-potato screen. The tablet is also distinguishing itself a bit from the small screen normally found in mobile.

More and more people also find great value in having an always-on screen where they can go to quickly ask questions or do tasks like E-mail.

I forecast we will soon see the development of a "fourth screen" which is a mostly-always-on wall panel meant to be used with almost no interaction at all. It's not a thing to stare at like the TV (though it could turn into one) nor a thing to do interactive web sessions on. The goal is to have minimal UI and be a little bit psychic about what to show.

One could start by showing stuff that's always of use. The current weather forecast, for example, and selected unusual headlines. Whether each member of the household has new mail, and if it makes sense from a privacy standpoint, possibly summaries of that mail. Likewise the most recent status from feeds on twitter or Facebook or other streams. One could easily fill a screen with these things so you need a particularly good filter to find what's relevant. Upcoming calendar events (with warnings) also make sense.

Some things would show only when important. For example, when getting ready to go out, I almost always want to see the traffic map. Or rather, I want to see it if it has traffic jams on it, no need to show it when it's green -- if it's not showing I know all is good. I may not need to see the weather if it's forecast sunny either. Or if it's raining right now. But if it's clear now and going to rain later I want to see that. Many city transit systems have a site that tracks when the next bus or train will come to my stop -- I want to see that, and perhaps at morning commute time even get an audio alert if something unusual is up or if I need to leave right now to catch the street car. A view from the security camera at the door should only show if somebody is at the door.

There are so many things I want to see that we will need some UI for the less popular ones. But it should be a simple UI, with no need to find a remote (though if I have a remote -- any remote -- it should be able to use it.) Speech commands would be good to temporarily see other screens and modes. A webcam (and eventually Kinect style sensor) for gestural UI would be nice, letting me swipe or wave to get other screens.

Ride-sharing apps instead of Bus Rapid Transit?

You may have heard of Bus Rapid Transit -- a system to give a bus line a private or semi-private right-of-way, along with bus stops that are more akin to stations than bus shelters (with ticket-taking machines and loading platforms for multiple doors.) The idea is to make bus transit competitive with light-rail (LRT) in terms of speed and convenience. Aside from getting caught in slow traffic, buses also are slow to board. BRT is hoped to be vastly less expensive than light rail -- which is not hard because LRT (which means light capacity rail, not lightweight rail) has gotten up to $80 to $100M per mile. When BRT runs down the middle of regular roads, it gets signal timing assistance to help it have fewer stops. It's the "hot new thing" in transit. Some cities even give it bits of underground or elevated ROW (the Boston Silver Line) and others just want to wall off the center of a road to make an express bus corridor. Sometimes BRT gets its own highway lane or shares a special carpool lane.

At the same time just about anybody who has looked at transit and the internet has noticed that as the buses go down the street, they travel with tons of cars carrying only one person and lots of empty seats. Many have wondered, "how could we use those empty private car seats to carry the transit load?" There are a number of ride-sharing and carpooling apps on web sites and on smartphones, but success has been modest. Drivers tend to not want to take the time to declare their route, and if money is offered, it's usually not enough to counter the inconvenience. Some apps are based on social networks so friends can give rides to friends -- great when it works but not something you can easily do on demand.

But one place I've seen a lot of success at this is the casual carpooling system found in a number of cities. Here it's very popular to cross the Oakland-SF Bay Bridge, which has a $6 toll to cross into SF. It used to be free for 3-person carpools, now it's $2.50, but the carpools also get a faster lane for access to the highly congested bridge both going in and out of SF.

Almost all the casual carpool pickup spots coming in are at BART (subway) stations, which are both easy for everybody to get to, and which allow those who can't get a carpool to just take the train. There is some irony that it means that the carpools mostly take people who would have ridden BART trains, not people who would have driven, the official purpose of carpool subsidies. In the reverse direction the carpools are far fewer with no toll to be saved, but you do get a better onramp.

People drive the casual carpools because they get something big for for it -- saving over $1,000/year, and hopefully a shorter line to the bridge. This is the key factor to success in ride share. The riders are saving a similar amount of money in BART tickets, even more if they skipped driving.

Let's consider what would happen if you put in the dedicated lane for BRT, but instead of buses created an internet mediated carpooling system. Drivers could enter the dedicated lane only if:

  • They declared their exit in advance to the app on their phone, and it's far enough away to be useful to riders.
  • They agree to pick up riders that their phone commands them to.
  • They optionally get a background check that they pay for so they can be bonded in some way to do this. (Only the score of the background check is recorded, not the details.)

Riders would declare their own need for a ride, and to what location, on their own phones, or on screens mounted at "stops" (or possibly in nearby businesses like coffee shops.) When a rider is matched to a car, the rider will be informed and get to see the approach of their ride on the map, as well as a picture of the car and plate number. The driver will be signaled and told by voice command where to go and who to pick up. I suggest calling this Carpool-Rapid-Transit or CRT.

Topic: 

Watson, game 2

Not much new to report after the second game of the Watson Jeopardy Challenge. I've added a few updates to yesterday's post on Watson and the result was as expected, though Watson struggled a lot more in this game than in the prior round, deciding not to answer many questions due to low confidence and making a few mistakes. In a few cases it was saved by not buzzing fast enough even though it had over 50% confidence, as it would have answered slightly wrong.

Watson, come here, I want you

The computer scientist world is abuzz with the game show world over the showdown between IBM's "Watson" question-answering system and the best human players to play the game Jeopardy. The first game has been shown, with a crushing victory by Watson (in spite of a tie after the first half of the game.)

Tomorrow's outcome is not in doubt. IBM would not have declared itself ready for the contest without being confident it would win, and they wouldn't be putting all the advertising out about the contest if they had lost. What's interesting is how they did it and what else they will be able to do with it.

Dealing with a general question has long been one of the hard problems in AI research. Watson isn't quite there yet but it's managed a great deal with a combination of algorithmic parsing and understanding combined with machine learning based on prior Jeopardy games. That's a must because Jeopardy "answers" (clues) are often written in obfuscated styles, with puns and many idioms, exactly the sorts of things most natural language systems have had a very hard time with.

Watson's problem is almost all understanding the question. Looking up obscure facts is not nearly so hard if you have a copy of Wikipedia and other databases on hand, particularly one parsed with other state-of-the-art natural language systems, which is what I presume they have. In fact, one would predict that Watson would do the best on the hardest $2,000 questions because these are usually hard because they refer to obscure knowledge, not because it is harder to understand the question. I expect that an evaluation of its results may show that its performance on hard questions is not much worse than on easy ones. (The main thing that would make easy questions easier would be the large number of articles in its database confirming the answer, and presumably boosting its confidence in its answer.) However, my intuition may be wrong here, in that most of Watson's problems came on the high-value questions.

It's confidence is important. If it does not feel confident it doesn't buzz in. And it has a serious advantage at buzzing in, since you can't buzz in right away on this game, and if you're an encyclopedia like the two human champions and Watson, buzzing in is a large part of the game. In fact, a fairer game, which Watson might not do as well at, would involve randomly choosing which of the players who buzz in in the first few tenths of a second gets to answer the question, eliminating any reaction time advantage. Watson gets the questions as text, which is also a bit unfair, unless it is given them one word a time at human reading speed. It could do OCR on the screen but chances are it would read faster than the humans. It's confidence numbers and results are extremely impressive. One reason it doesn't buzz in is that even with 3,000 cores it takes 2-6 seconds to answer a question.

Indeed a totally fair contest would not have buzzing in time competition at all, and just allow all players who buzz in to answer an get or lose points based on their answer. (Answers would need to be in parallel.)

Watson's coders know by now that they probably should have coded it to receive wrong answers from other contestants. In one instance it repeated a wrong answer, and in another case it said "What is Leg?" after Jennings had incorrectly answered "What is missing an arm?" in a question about an Olympic athlete. The host declared that right, but the judges reversed that saying that it would be right if a human who was following up the wrong answer said it, but was a wrong answer without that context. This was edited out. Also edited out were 4 crashes by Watson that made the game take 4 hours instead of 30 minutes.

It did not happen in what aired so far, but in the trials, another error I saw Watson make was declining to answer a request to be more specific on an answer. Watson was programmed to give minimalist answers, which often the host will accept as correct, so why take a risk. If the host doesn't think you said enough he asks for a more specific answer. Watson sometimes said "I can be no more specific." From a pure gameplay standpoint, that's like saying, "I admit I am wrong." For points, one should say the best longer phrase containing the one-word answer, because it just might be right. Though it has a larger chance of looking really stupid -- see below for thoughts on that.

The shows also contain total love-fest pieces about IBM which make me amazed that IBM is not listed as a sponsor for the shows, other than perhaps in the name "The IBM Challenge." I am sure Jeopardy is getting great ratings (just having their two champs back would do that on its own but this will be even more) but I have to wonder if any other money is flowing.

Being an idiot savant

Watson doesn't really understand the Jeopardy clues, at least not as a human does. Like so many AI breakthroughs, this result comes from figuring out another way to attack the problem different from the method humans use. As a result, Watson sometimes puts out answers that are nonsense "idiot" answers from a human perspective. They cut back a lot on this by only having it answer when it has 50% confidence or higher, and in fact for most of its answers it has very impressive confidence numbers. But sometimes it gives such an answer. To the consternation of the Watson team, it did this on the Final Jeopardy clue, where it answered "Toronto" in the category "U.S. Cities."

Definition of pixels for the world's biggest photos

I shoot lots of large panoramas, and the arrival of various cheaper robotic mounts to shoot them, such as the Gigapan Epic Pro and the Merlin/Skywatcher (which I have) has resulted in a bit of a "mine's bigger than yours" contest to take the biggest photo. Some would argue that the stitched version of the Sloane Digital Sky survey, which has been rated at a trillion pixels, is the winner, but most of the competition has been on the ground.

Topic: 

Blind man drives, sort of, with a robocar

A release from the National Federation for the Blind reports a blind person driving and avoiding obstacles on the Daytona speedway. They used a car from the TORC team at Virginia Tech, one of the competitors in the Darpa Grand Challenges. In effect, the blind driver replaced the "drive by wire" component of a robocar with a more intelligent and thinking human also able to feel acceleration and make some judgements.

Topic: 

Another pedal-powered monorail: Skyride

Last year I wrote about an interesting but simple pedal powered monorail/PRT system called Shweeb which had won a prize/investment from Google. Recent announcements show they are not alone in this concept. Scott Olson, the original developer of the Rollerblade, has founded a company called Skyride Technologies to build their own version of a pedal powered suspended monorail.

Working on Robocars at Google

As readers of this blog surely know, for several years I have been designing, writing and forecasting about the technology of self-driving "robocars" in the coming years. I'm pleased to announce that I have recently become a consultant to the robot car team working at Google.

Of course all that work will be done under NDA, and so until such time as Google makes more public announcements, I won't be writing about what they or I are doing. I am very impressed by the team and their accomplishments, and to learn more I will point you to my blog post about their announcement and the article I added to my web site shortly after that announcement. It also means I probably won't blog in any detail about certain areas of technology, in some cases not commenting on the work of other teams because of conflict of interest. However, as much as I enjoy writing and reporting on this technology, I would rather be building it.

My philosophical message about Robocars I have been saying for years, but it should be clear that I am simply consulting on the project, not setting its policies or acting as a spokesman.

My primary interest at Google is robocars, but many of you also know my long history in online civil rights and privacy, an area in which Google is often involved in both positive and negative ways. Indeed, while I was chairman of the EFF I felt there could be a conflict in working for a company which the EFF frequently has to either praise or criticise. I will be recusing myself from any EFF board decisions about Google, naturally.

My phone should know when I start a trip

Every day I get into my car and drive somewhere. My mobile phone has a lot of useful apps for travel, including maps with traffic and a lot more. And I am usually calling them up.

I believe that my phone should notice when I am driving off from somewhere, or about to, and automatically do some things for me. Of course, it could notice this if it ran the GPS all the time, but that's expensive from a power standpoint, so there are other ways to identify this:

TVs should be universal, not remote controls

Like me, you probably have a dozen "universal" remote controls gathered over the years. With each new device and remote you go through a process to try to figure out special codes to enter into the remote to train it to operate your other devices. And it's never very good, except perhaps in the expensive remotes with screens and macros.

Topic: 

Comparing electricity to a gallon of gasoline

The "burning" question for electric cars is how to compare them with gasoline. Last month I wrote about how wrong the EPA's 99mpg number for the Nissan Leaf was, and I gave the 37mpg number you get from the Dept. of Energy's methodology. More research shows the question is complex and messy.

So messy that the best solution is for electric cars to publish their efficiency in electric terms, which means a number like "watt-hours/mile." The EPA measured the Leaf as about 330 watt-hours/mile (or .33 kwh/mile if you prefer.) For those who really prefer an mpg type number, so that higher is better, you would do miles/kwh.

Then you would get local power companies to publish local "kwh to gallon of gasoline" figures for the particular mix of power plants in that area. This also is not very easy, but it removes the local variation. The DoE or EPA could also come up with a national average kwh/gallon number, and car vendors could use that if they wanted, but frankly that national number is poor enough that most would not want to use it in the above-average states like California. In addition, the number in other countries is much better than in the USA.

The local mix varies a lot. Nationally it's about 50% coal, 20% gas, 20% nuclear and 10% hydro with a smattering of other renewables. In some places, like Utah, New Mexico and many midwestern areas, it is 90% or more coal (which is bad.) In California, there is almost no coal -- it's mostly natural gas, with some nuclear, particularly in the south, and some hydro. In the Pacific Northwest, there is a dominance by hydro and electricity has far fewer emissions. (In TX, IL and NY, you can choose greener electricity providers which seems an obvious choice for the electric-car buyer.)

Understanding the local mix is a start, but there is more complexity. Let's look at some of the different methods, staring with an executive summary for the 330 wh/mile Nissan Leaf and the national average grid:

  • Theoretical perfect conversion (EPA method): 99 mpg-e(perfect)
  • Heat energy formula (DoE national average): 37 mpg-e(heat)
  • Cost of electricity vs. gasoline (untaxed): 75 mpg-e($)
  • Pollution, notably PM2.5 particulates: Hard to calculate, could be very poor. Hydrocarbons and CO: very good.
  • Greenhouse Gas emissions, g CO2 equivalent: 60 mpg-e(CO2)

Designing a better, faster, secure, vastly cheaper airport with proto-robocars

Like just about everybody, I hate the way travel through airports has become. Airports get slower and bigger and more expensive, and for short-haul flights you can easily spend more time on the ground at airports than you do in the air. Security rules are a large part of the cause, but not all of it.

In this completely rewritten essay, I outline the design on a super-cheap airport with very few buildings, based on a fleet of proto-robocars. I call them proto models because these are cars we know how to build today, which navigate on prepared courses on pavement, in controlled situations and without civilian cars to worry about.

In this robocar airport, which I describe first in a narrative and then in detail, there are no terminal buildings or gates. Each plane just parks on the tarmac and robotic stairs and ramps move up and dock to all its doors. (Catering trucks, fuel trucks and luggage robots also arrive.) The passengers arrive in a perfect boarding order in robocars that dock at the ramps/steps to let them get on the plane through every entrance. Luggage is handled by different robots, and is checked and picked up not in carousels and check-in desks, but at curbs, parking lots, rental car centers and airport hotels.

The change is so dramatic that (even with security issues) people could arrive at airports for flights under 20 minutes before take-off, and get out even faster. Checked luggage would add time, but not much. I also believe you could build a high capacity airport for a tiny fraction of the cost of today's modern multi-billion dollar edifices. I believe the overall experience would also be more pleasant and more productive for all.

This essay is a long one, but I am interested in feedback. What will work here, and what won't? Would you love to fly through this airport or hate it? This is an airport designed not to give you a glorious building in which to wait but to get you through it without waiting most of the time.

The airport gets even better when real robocars, that can drive on the streets to the airport, come on the scene.

Give me your feedback on The Robocar Airport.

Key elements of the design include:

Where will 3-D cameras like Kinect lead?

This year, I bought Microsoft Kinect cameras for the nephews and niece. At first they will mostly play energetic X-box games with them but my hope is they will start to play with the things coming from the Kinect hacking community -- the videos of the top hacks are quite interesting. At first, MS wanted to lock down the Kinect and threaten the open source developers who reverse engineered the protocol and released drivers. Now Microsoft has official open drivers.

Drivers cost 1.7 million person-years every year in the USA, 3rd of all major causes

I've written frequently about how driving fatalities are the leading cause of death for people from age 5 to 45, and one of the leading overall causes of death. I write this because we hope that safe robocars, with a much lower accident rate, can eliminate much of this death.

Topic: 

Banks: Give me two passwords

Passwords are in the news thanks to Gawker media, who had their database of userids, emails and passwords hacked and published on the web. A big part of the fault is Gawker's, who was saving user passwords (so it could email them) and thus was vulnerable. As I have written before, you should be very critical of any site that is able to email you your password if you forget it.

Some of the advice in the wake of this to users has been to not use the same password on multiple sites, and that's not at all practical in today's world. I have passwords for many hundreds of sites. Most of them are like gawker -- accounts I was forced to create just to leave a comment on a message board. I use the same password for these "junk accounts." It's just not a big issue if somebody is able to leave a comment on a blog with my name, since my name was never verified in the first place. A different password for each site just isn't something people can manage. There are password managers that try to solve this, creating different passwords for each site and remembering them, but these systems often have problems when roaming from computer to computer, or trying out new web browsers, or when sites change their login pages.

The long term solution is not passwords at all, it's digital signature (though that has all the problems listed above) and it's not to even have logins at all, but instead use authenticated actions so we are neither creating accounts to do simple actions nor using a federated identity monopoly (like Facebook Connect). This is better than OpenID too.

Tags: 

How Robocars affect the City, plus Masdar & City of Apple

I decided to gather together all my thoughts on how robocars will affect urban design. There are many things that might happen, though nobody knows enough urban planning to figure out just what will happen. However, I felt it worthwhile to outline the forces that might be at work so that urban geographers can speculate on what they will mean. It is hard to make firm predictions.

Topic: 

Pages