Submitted by brad on Sun, 2013-06-30 12:50.
Yahoo announced that in a few days they will shut down the altavista web site. This has prompted a few posts on the history of internet search, to which I will add an anecdote.
The first internet search engine predated the “web” and was called Archie search engine. Archie (an archive search) was basic by today’s standards. The main protocol for getting files on the internet in those days was FTP. Many sites ran an open FTP server, which you could connect to and download files from. If you had files or software to share with people, you put it up on an FTP server, in particular one that allowed anonymous login to get public files. The Archie team (from Montreal) built a tool to go to all the open servers, read their indexes and generate a database. You could then search, and get a pointer to all the places you could get a file. It was hugely popular for the day.
(You will probably note that this is almost exactly the way Napster worked, the only difference being that Napster was a bit more sophisticated and people used it to share files that were copyrighted. FTP servers had copyrighted material, but mostly they had open source software and documents.)
Around the same time, a lot of folks were building full-text search engines for use on large collections of documents. You could find these on private databases around the world, and the WAIS protocol was developed by Brewster Kahle to make a standardized interface to text search and his own text search tools.
Not long after the web started to grow, Fuzzy Mauldin at CMU made Lycos which was a full-text search engine applied to documents gathered from the web. The ability to search the web generated much attention, and a few other competing spiders and search engines appeared. Everybody had a favourite. (To add to my long list of missed opportunities, in April of 95 I wrote a few notes to Fuzzy looking to get his spider index so we could sort web pages based on how many incoming links they had. Nothing ever came of that but as you may know that concept later had some value. :-) And I also turned down a $4M offer from Lycos to buy ClariNet (which would have turned into $40M when their stock shot up in the bubble. Sigh.)
In 1995, for many people that favourite changed to Alta Vista, a new search engine from Digital Equipment Corp. DEC was a huge name at the time, the biggest name in minicomputers, and it was just losing the Unix crown to Sun. The team at DEC put a lot of computing power into Alta Vista, and so it had two useful attributes. First, they spidered a lot more pages, and thus were more likely to find stuff. They were also fast compared to most of the other engines. In a precursor to other rapid turnarounds in the internet business, you could switch your favourite search engine in a heartbeat and many did. It was big and fast due to DEC putting a lot of fancy computer hardware on it, and DEC eventually justified the money they were spending on it (there was no revenue for search in those days) by saying it showed off just how powerful DEC’s computers with big address spaces were. Indeed the limits of Alta Vista were the limits of the architecture, using the 64-bit Alpha to address 130gb of RAM and 500gb of disk — huge for the day.
On Alta Vista’s home page, they gave you a sample query to type in the search box, to show you how to use it. That query was:
kayak sailing “san juan islands”
Indeed, if you typed that, you got a nice array of pages which talked about kayaking up in the San Juan islands, tour operators, etc. — just what you wanted to get from a query.
My devious mind wondered, “what if I put up a page on my own web site with this as the title?” I created the Kayak Sailing “San Juan Islands” home page on the rec.humor.funny site, which was already a very popular site in those days. (Indeed it’s around 1995 that RHF fell behind Yahoo as the most widely read thing on the internet, but that refers to the USENET group, not the page.)
You will note as you look at the page that it contains the words in the title and headers, and repeated many times in invisible comments. In those days the search engines were ranking higher simply based on where words were, and if they were repeated many times. So I gave it a whirl. This was an early attempt at what is now called “black hat search engine optimization” though I was doing it for fun, rather than nefarious gain.
The results didn’t change though. Alta Vista relied on huge computer power, but it only rebuilt the index by hand. It would be a month or more before Alta Vista recalculated its index. One day I went to type in the query and bingo — there was my page on the first page of search results. Along with a dozen other people who had tried the same thing, and a few pages that were articles writing about Alta Vista and giving the example query, or which were copying its search page which of course had that string.
More to the point, not a single item on the results page was about actual Kayaking! The sample query was ruined, though the results were quite amusing. Not long after, Alta Vista changed the example to Pizza “deep dish” Chicago and of course I added it to my page as well. So not much longer after that AV switched to showing different examples from a rotating and changing collection so people could not play this game any more.
While Alta Vista ruled Search, in spite of efforts from Infoseek, Inktomi/Hotbot and others, we all know that a few years later, Google was born at Stanford, and it proved again how quickly people could switch to a new favourite search engine, and lives under that fear (but with great success) to this day. And Google’s dominance turned SEO into a giant industry.
Submitted by brad on Wed, 2013-06-26 14:25.
I always feel strange when I see blog and social network posts about the death of a pet or even a relative. I know the author but didn’t know anything about the pet other than that the author cared.
So as I report the end for our kitty, Bijou, I will make it interesting by relaying a fun surveillance related story of how she arrived at our house. She had been rescued as a stray by a distant relative. When that relative died there was nobody else to take the cats, so we took two of them, even though the two would have nothing to do with each-other. Upon arrival at our house, both cats discovered that the garage was a good place to hide, but the hiding was quite extreme, and after about 4 days we still could not figure where Bijou was hiding. Somebody was coming to eat the food, but we could not tell from where.
I had a small wireless camera with an RF transmitter on it. So I set it up near the food bowl, and we went into the TV room to watch. As expected, a few minutes later, the cat emerged — from inside the bottom of the washing machine through a rather small hole. After emerging she headed directly and deliberately to the camera and as she filled the screen, suddenly the view turned to distortion and static. It was the classic scene of any spy movie, as shot from the view of the surveillance camera. The intruder comes in and quickly disables the camera.
What really happened is that the transmitter is not very powerful and you must aim the antenna. When a cat sees something new in her environment, her first instinct is to come up to it and smell it, then rub her cheek on it to scent-mark it. And so this is what she did, bumping the antenna to lose the signal, though it certainly looked like she was the ideal cat for somebody at the EFF.
It’s also a good thing we didn’t run the washing machine. But I really wish I had been recording the video. Worthy of Kittywood studios.
She had happy years in her new home (as well as some visits to her old one before it was sold) and many a sunbeam was lazily exploited and evil bright red dot creature never captured, but it could not be forever.
RIP Bijou T. Cat, 199? - 2013
Submitted by brad on Wed, 2013-06-19 11:14.
The AUVSI summit on “driverless” cars last week contained 2 days of nothing but robocars, and I reported on issues regarding Google and policy in part 1.
As noted, NHTSA released their proposal for how they want to regulate such vehicles. In it, they defined levels 0 through 4. Level 2 is what I (and GM) have been calling “super cruise” — a car which can do limited self driving but requires constant human supervision. Level 3 is a car which can drive without constant attention, but might need to call upon a human driver (non-urgently) to handle certain streets and situations. Level 4 is the fully automatic robocar.
Level 2 issues
Level 2 is coming this year in traffic jams in the Mercedes S and the BMW 5, and soon after from Audi and Volvo. GM had announced super cruise for the 2015 Cadillac line but has pulled back and delayed that to later in the decade. Nonetheless the presentation from GM’s Jeremy Salinger brought home many of the issues with this level.
GM has done a number of user studies in their super cruise cars on the test track. And they learned that the test subjects very quickly did all sorts of dangerous things, definitely not paying attention to the road. They were not told what they couldn’t do, but subjects immediately began texting, fiddling around in the back and even reading (!) while the experimenters looked on with a bit of fear. No big surprise, as people even text today without automatic steering, but the experimental results were still striking.
Because of that GM is planning what they call “countermeasures” to make sure this doesn’t happen. They did not want to say what countermeasures they liked, but in the past, we have seen proposals such as:
- You must touch the wheel every few seconds or it disengages
- A camera looks at your eyes and head and alerts or disengages if you look away from the road for too long
- A task for your hands like touching a button every so often
The problem is these countermeasures can also get annoying, reducing the value of the system. It may be the lack of ability to design a good countermeasure is what has delayed GM’s release of the product. There is a policy argument coming up about whether level 2 might be more dangerous than the harder levels 3 and above, because there is more to go wrong with the human driver and the switches between human and machine driving. (Level 4 has no such switches, level 3 has switches with lots of warning.)
On the plus side, studies on existing accidents show that accident-avoidance systems, even just forward collision avoidance, have an easy potential for huge benefits. Already we’re seeing a 15% reduction in accidents in some studies just from FCA, but studies show that in 33% of accidents, the brakes were never applied at all, and only in just 1% of accidents were the brakes applied with full force! As such, systems which press the brakes and press them hard when they detect the imminent accident may not avoid the accident entirely, but they will highly reduce the severity of a lot of accidents. read more »
Submitted by brad on Sun, 2013-06-16 09:38.
I was sadly informed this morning by Ann Lowson that transportation pioneer Martin Lowson has fallen to a stroke this weekend.
Martin had an amazing career but it was more amazing that he was still actively engaged at age 75. We shared a panel last month in Phoenix at the people-mover conference and continued our vigourous debate on the merits of cars like his on closed guideways compared to robocars.
His career included leading a large team on the Apollo project, and building the world’s fastest helicopter, as well as faculty positions at Bristol, and you can read some about it here. For me, his big contribution was to found the ULTra PRT company, the first to commercially deploy a PRT. It runs today at Heathrow, moving people between the terminal and the business parking lot.
PRT was conceived 50 years ago, and many, including Martin and myself, were fascinated by the idea. More recently, as readers know, I decided the PRT vision of personal transportation could be realized on city streets by robocars. It’s easier to do it today on dedicated guideway, but the infrastructure costs tell me the future lies off the guideway.
That doesn’t diminish the accomplishment of being the first to make it work on the guideway. ULTra uses small cars on rubber tires, not a train on rails. They are guided by a laser rangefinder and are fully automated, with no steering wheel.
Last year I invited Martin in to give a talk to Google’s car team, and he got a ride in the car, which he quite enjoyed, even though it didn’t convince him that they were the future. But unlike other skeptics, I gave him the deepest respect for his skill and experience. People who can found companies and lead engineering and public acceptance breakthroughs while senior citizens are a very rare thing, and the world will miss him.
Submitted by brad on Sat, 2013-06-15 15:43.
This week I attended AUVSI’s “Driverless Car Summit” in Detroit. This year’s event, the third, featured a bigger crowd and a decent program, and will generate more than one post.
I would hardly call it a theme, but two speakers expressed fairly negative comments about Google’s efforts, raising some interesting subjects. (As an important disclaimer, the Google car team is a consulting client of mine, but I am not their spokesman and the views here do not represent Google’s views.)
The keynote address came from Bryan Reimer of MIT, and generated the most press coverage and debate, though the recent NHTSA guidelines also created a stir.
Reimer’s main concern: Google is testing on public streets instead of a test track. As such it is taking the risk of a fatal accident, from which the blowback could be so large it stifles the field for many years. Car companies historically have done extensive test track work before going out on real streets. I viewed Reimer’s call as one for near perfection before there is public deployment.
There is a U-shaped curve of risk here. Indeed, a vendor who takes too many risks may cause an accident that generates enough backlash to slow down the field, and thus delay not just their own efforts, but an important life-saving technology. On the other hand, a quest for perfection attempts what seems today to be impossible, and as such also delays deployment for many years, while carnage continues on the roads.
As such there is a “Goldilocks” point in the middle, with the right amount of risk to maximize the widescale deployment of robocars that drive more safely than people. And there can be legitimate argument about where that is.
Reimer also expressed concern that as automation increases, human skill decreases, and so you actually start needing more explicit training, not less. He is as such concerned with the efforts to make what NHTSA calls “level 2” systems (hands off, but eyes on the road) as well as “level 3” systems (eyes off the road but you may be called upon to drive in certain situations.) He fears that it could be dangerous to hand driving off to people who now don’t do it very often, and that stories from aviation bear this out. This is a valid point, and in a later post I will discuss the risks of the level-2 “super cruise” systems.
Maarten Sierhuis, who is running Nissan’s new research lab (where I will be giving a talk on the future of robocars this Thursday, by the way) issued immediate disagreement on the question of test tracks. His background at NASA has taught him that you “fly where you train and train where you fly” — there is no substitute for real world testing if you want to build a safe product. One must suspect Google agrees — it’s not as if they couldn’t afford a test track. The various automakers are also all doing public road testing, though not as much as Google. Jan Becker of Bosch reported their vehicle had only done “thousands” of public miles. (Google reported a 500,000 mile count earlier this year.)
Heinz Mattern, research and development manager for Valeo (which is a leading maker of self-parking systems) went even further, starting off his talk by declaring that “Google is the enemy.” When asked about this, he did not want to go much further but asked, “why aren’t they here? (at the conference)” There was one Google team employee at the conference, but not speaking, and I’m not am employee or rep. It was pointed out that Chris Urmson, chief engineer of the Google team, had spoken at the prior conferences. read more »
Submitted by brad on Sun, 2013-06-09 18:13.
I’m off for AUVSI’s “Driverless Car Summit” in Detroit. I attended and wrote about last year’s summit, which, in spite of being put on by a group that comes out of the military unmanned vehicle space, was very much about the civilian technology. (As I’ve said before, I have a dislike for the term “driverless car” and in fact at the summit last year, the audience expressed the same dislike but could not figure out what the best replacement term was.)
I’ll be reporting back on events at the summit, and making a quick visit to my family in Toronto as well. I will also attend the Transportation Research Board’s conference on automated vehicles at Stanford in July.
Then I’m back for the opening of our Singularity University Graduate Studies Program for 2013 at NASA Ames Research Park this coming weekend. My students will get some fun lectures on robocars, as well as many other technologies. Early bird tickets for the opening ceremony are still available.
On June 20, I will give a talk at a meeting of the new Silicon Valley Autonomous Vehicle Enthusiasts group. This group has had one talk. The talks are being hosted at Nissan’s new research lab in Silicon Valley, where they are researching robocars. I just gave a 10 minute version of my talk at Fujitsu Labs’ annual summit last week, this will be the much longer version!
SU consumes much of my summer. In the fall, you’ll see me giving talks on Robocars and other issues in Denmark, London, Milan as well as at our new Singularity University Summit in Budapest in November, as well as others around the USA.
Submitted by brad on Thu, 2013-06-06 14:20.
There have been a wide variety of announcements of late giving the impression that somebody has “solved the problem” of making a robocar affordable, usually with camera systems. It’s widely reported how the Velodyne LIDAR used by all the advanced robocar projects (including Google, Toyota and many academic labs) costs $75,000 (or about $30,000 in a smaller model) and since that’s more than the cost of the car, it is implied that is a dead-end approach.
Recent stories include a ride in MobilEye’s prototype car by the New York Times, a number of reports of a claim from the Oxford team (which uses LIDAR today) that they plan to do it for just $150 and many stories about a Romanian teen who won the Intel science fair with a project to build a cheaper self-driving car.
I have written an analysis of the issues comparing LIDARS (which are currently too expensive, but reliable in object detection) and vision systems (which are currently much less expensive, but nowhere near reliable enough in object detection) and why different teams are using the different technologies. Central is the question of which technology will be best at the future date when robocars are ready to be commercialized.
In particular, many take the high cost of the Velodyne, which is hand-made in small quantities, and incorrectly presume this tells us something about the cost of LIDARs a few years down the road, with the benefits of Moore’s Law and high-volume manufacturing. Saying the $75,000 LIDAR is a dead-end is like being in 1982, and noting that small disk drives cost $3,000 and declaring that planning for disk drive storage of large files is a waste of time.
Cameras or Lasers in the robocar
I will add some notes about Ionut Budisteanu, the 19-year old Romanian. His project was great, but it’s been somewhat exaggerated by the press. In particular, he mistakenly calls LIDAR “3-D radar” (an understandable mistake for a non-native English speaker) and his project was to build a lower-cost, low-resolution LIDAR, combining it with cameras. However, in his project, he only tested it in simulation. I am a big fan of simulation for development, learning, prototyping and testing, but alas, doing something in simulation, particularly with vision, is just the first small step along the way. This isn’t a condemnation of Mr. Budisteanu’s project, and I expect he has a bright future, but the press coverage of the event was way off the mark.