Best Of Blog

Is Green U.S. Transit a whopping myth?

As part of my research into robotic cars, I’ve been studying the energy efficiency of transit. What I found shocked me, because it turns out that in the USA, our transit systems aren’t green at all. Several of the modes, such as buses, as well as the light rail and subway systems of most towns, consume more energy per passenger-mile than cars do, when averaged out. The better cities and the better modes do beat the cars, but only by a little bit. And new generation efficient cars beat the transit almost every time, and electric scooters beat everything hands down.

I encourage you to read the more detailed essay I have prepared on whether green U.S. transit is a myth. I’ve been very surprised by what I’ve found. It includes links to the sources. To tease you, here’s the chart I have calculated on the energy efficiency of the various modes. Read on, and show me how these numbers are wrong if you can!

I have added a follow-up post on the comparison between lots of small personal ultralight vehicles and larger shared transit vehicles.

Note: If you want to comment on the cyclist figure, there is different thread on the fossil fuel consumption in human food which details these numbers and invites comments.

Are botnets run by spy agencies?

A recent story today about discussions for an official defense Botnet in the USA prompted me to post a question I’ve been asking for the last year. Are some of the world’s botnets secretly run by intelligence agencies, and if not, why not?

Some estimates suggest that up to 1/3 of PCs are secretly part of a botnet. The main use of botnets is sending spam, but they are also used for DDOS extortion attacks and presumably other nasty things like identity theft.

But consider this — having remote control of millions of PCs, and a large percentage of the world’s PCs seems like a very tempting target for the world’s various intelligence agencies. Most zombies are used for external purposes, but it would be easy to have them searching their own disk drives for interesting documents, and sniffing their own LANs for interesting unencrypted LAN traffic, or using their internal state to get past firewalls.

Considering the billions that spy agencies like the NSA, MI6, CSEC and others spend on getting a chance to sniff signals as they go over the wires, being able to look at the data all the time, any time as it sits on machines must be incredibly tempting.

And if the botnet lore is to be accepted, all this was done using the resources of a small group of young intrusion experts. If a group of near kids can control hundreds of millions of machines, should not security experts with billions of dollars be tempted to do it?

Of course there are legal/treaty issues. Most “free nation” spy agencies are prohibited from breaking into computers in their own countries without a warrant. (However, as we’ve seen, the NSA has recently been lifted of this restriction, and we’re suing over that.) However, they are not restricted on what they do to foreign computers, other than by the burdens of keeping up good relations with our allies.

However, in some cases the ECHELON loophole may be used, where the NSA spies on British computers and MI-6 spies on American computers in exchange.

More simply, these spy agencies would not want to get caught at this, so they would want to use young hackers building spam-networks as a front. They would be very careful to assure that the botting could not be traced back to them. To keep it legal, they might even just not take information from computers whose IP addresses or other clues suggest they are domestic. The criminal botnet operators could infect everywhere, but the spies would be more careful about where they got information and what they paid for.

Of course, spy agencies of many countries would suffer no such restrictions on domestic spying.

Of all the spy agencies in the world, can it be that none of them have thought of this? That none of them are tempted by being able to comb through a large fraction of the world’s disk drives, looking for both bad guys and doing plain old espionage?

That’s hard to fathom. The question is, how would we detect it? And if it’s true, could it mean that spies funded (as a cover story) the world’s spamming infrastructure?

Data Deposit Box instead of data portability

I’ve been ranting of late about the dangers inherent in “Data Portability” which I would like to rename as BEPSI to avoid the motherhood word “portability” for something that really has a strong dark side as well as its light side.

But it’s also important to come up with an alternative. I think the best alternative may lie in what I would call a “data deposit box” (formerly “data hosting.”) It’s a layered system, with a data layer and an application layer on top. Instead of copying the data to the applications, bring the applications to the data.

A data deposit box approach has your personal data stored on a server chosen by you. That server’s duty is not to exploit your data, but rather to protect it. That’s what you’re paying for. Legally, you “own” it, either directly, or in the same sense as you have legal rights when renting an apartment — or a safety deposit box.

Your data box’s job is to perform actions on your data. Rather than giving copies of your data out to a thousand companies (the Facebook and Data Portability approach) you host the data and perform actions on it, programmed by those companies who are developing useful social applications.

As such, you don’t join a site like Facebook or LinkedIn. Rather, companies like those build applications and application containers which can run on your data. They don’t get the data, rather they write code that works with the data and runs in a protected sandbox on your data host — and then displays the results directly to you.

To take a simple example, imagine a social application wishes to send a message to all your friends who live within 100 miles of you. Using permission tokens provided by you, it is able to connect to your data host and ask it to create that subset of your friend network, and then e-mail a message to that subset. It never sees the friend network at all.  read more »

The Glass Roots movement

Recently, while keynoting the Freedom 2 Connect conference in Washington, I spoke about some of my ideas for fiber networks being built from the ground up. For example, I hope for the day when cheap kits can be bought at local stores to fiber up your block by running fiber through the back yards, in some cases literally burying the fiber in the “grass roots.”

Doc Searls, while he was listening to the talk made up a clever term — “Glass Roots” to describe this, and other movements to deploy fiber bottom up, without waiting for telcos and city governments. Any time you can deploy a technology without permission and red tape, it quickly zooms ahead of other technology. Backyard fiber, — combined with cheaper, mass produced free-space-optics or gigabit EHF radio equipment to bridge blocks together across streets or make links to hilltops — could provide the bandwidth we want without waiting.

Because let’s face it. While wireless ISPs sound great and are indeed great for serving some types of customers, right now real bandwidth requires a wire or glass fiber in the ground, and that means monopoly telcos and cable companies as well as the hassles of city government. We want our gigabits (forget megabits) and we want them now.

There are other elements to this Glass Roots movement, though usually with city involvement. Several small towns have put in fiber based ISPs with good success. My friend Brewster Kahle, from the Internet Archive, has brought 100 megabit service to housing projects in San Francisco using some city-laid fiber and the Archive’s bandwidth. You go, Brewster.

Brough Turner has the right idea. We should get dark fiber under our streets, and lots of it, installed and leased by a company that is only in the fiber business, and not in the business of selling you video or phone service or internet. While this company might get a franchise, the important difference is that the franchised monopoly would not light the fiber. Instead, anybody could lease a fiber from their house to a major switching point, and light it any way they want. Darth Vader would tell us “you don’t understand the power of the dark fiber.”

Why is that important? While fiber and wire are basic, the technologies to “light them up” run on Moore’s law. They get obsolete very quickly. Instead of monopoly rents and long cost-plus amortization tables, you want lots of turnover in the actual electronics found at the ends. You want the option to get the latest stuff, which is usually faster and cheaper than the stuff from 2 years ago. Lots faster and lots cheaper.

If you get a lot of free market competition on what lights those endpoints, it gets even better. The result is plenty of choice in how you light it and who you get connectivity from. And that eliminates all the issues around network neutrality or walled gardens. The investment in the dark fiber can probably be amortized over a decade or two, which is long enough.

One might argue the monopoly should even just be at the level of a conduit which it’s easy to drag other things like fiber or wire through. And indeed, whoever does bury pipes under the streets should expect to pull other wires before too long. But having monopoly lockdown at any level above the glass is what slows down the advance of broadband. Get rid of that lockdown, and the real glass roots revolution can begin.

Disband Congress

No, I don’t mean dissolve congress. Rather I propose a different way to run a legislature in the modern world.

There would be no capitol. Instead, all members would work in their districts, all the time. And we would put in a bunch of extra nice HDTV videoconferencing systems. The system would be designed to handle meetings, all the way up to a full session of the house or senate, with multiple screens to show amalgamated “crowd” as well as close-up views of the important figures — the person with the floor, the person next to get the floor, the last person to have the floor, the Speaker, the party leaders etc. Members would attend sessions that way, and through a secured channel, vote. There would be screens for semi-private discussions with others during the meeting. Of course all this video would be available to the public, too.

Members, and their staffs, could also videoconference in HD with other members and their staffs, as well as any other government officials they need to talk to. And quite possibly, with a few exceptions such as classified committee meetings, all that video would be available to the public too. For those without the equipment, the old capitol would come equipped with meeting rooms that use the system. “Going up to the hill” would mean going to use one of the rooms.

Members could meet in person of course, but they would need to have a chaperone to assure they don’t make secret deals. Classified meetings would get a properly cleared chaperone.

And they could meet with lobbyists over the videoconferencing system too. And those meetings would certainly be available to the public. Registered obbyists need not and could not come to meet in person.

Of course, the members would get out of touch with beltway thinking. They would lose the serendipity of meeting the right person on the hill, the business done at exclusive beltway cocktail parties. But in exchange they and their staff would live and breathe, quite literally, their district. I can see arguments good and bad about the trade-off but it is not clear that either one is inherently superior. It would hurt the DC economy a bit, and airlines would lose some business.

Strictly speaking, all the transparency rules I describe above, where members can’t talk off the record or without chaperones, is not inherent in the idea of a videoconferenced legislature. One could do that and still allow all sorts of unrecorded conversations. They would figure out ways to have them anyway. What the video system does is enables an easy implementation of an all-transparent, all-recorded seat of government.

Food for thought, anyway.

A universal Web-USB plugin for all browsers

As our devices get more and more complex, configuring them gets harder and harder. And for members of the non-tech-savvy public, close to impossible.

Here’s an answer: Develop a simple browser plug-in for all platforms that can connect a USB peripheral to a TCP socket back to the server where the plugin page came from. (This is how flash and Java applets work, in fact this could be added to flash or Java.)

Once activated, the remote server would be able to talk to the device like its USB master, sending and receiving data from it and talking other USB protocol commands. And that means it could do any configuration or setup you might like to do, under the control of a web application that has access to the full UI toolset that web applications have. You could upload new firmware into devices that can accept that, re-flash configuration, read configuration — do anything the host computer can do.

As a result, for any new electronics device you buy — camera, TV remote control, clock, TV, DVD player, digital picture frame, phone, toy, car, appliance etc. — you could now set it up with a nice rich web interface, or somebody else could help you set it up. It would work on any computer — Mac, Linux, Windows and more, and the web UIs would improve and be reprogrammed with time. No software install needed, other than the plug-in. Technicians could remotely diagnose problems and fix them in just about anything.

So there is of course one big question — security. Of course, the plug-in would never give a remote server access to a USB device without providing a special, not-in-browser prompt for the user to confirm the grant of access, with appropriate warnings. Certain devices might be very hard to give access to, such as USB hard drives, the mouse, the keyboard etc. In fact, any device which has a driver in the OS and is mounted by it would need extra confirmation (though that would make it harder to have devices that effectively look like standard USB flash drives into which basic config is simply read and written.)

One simple security technique would be to insist the device be hot plugged during the session. Ie. the plugin would only talk to USB devices that were not plugged in when the page was loaded, and then were plugged in as the app was running. The plugin would not allow constant reloading of the page to trick it on this.

For added security, smarter devices could insist on an authentication protocol with the server. Thus the USB device would send a challenge, which the server would sign/hash with its secret key, and the USB device could then check that using a public key to confirm its talking to its manufacturer. (This however stops 3rd parties from making better configuration tools, so it has its downsides.) It could also be arranged that only devices that exhibit a standard tag in their identification would allow remote control, so standard computer peripherals would not allow this. And the plugin could even maintain and update a list of vendors and items which do or don’t want to allow this.

There are probably some other security issues to resolve. However, should we resolve this it could result in a revolution of configuring consumer electronics, as finally everything would get a big screen, full mouse and keyboard web UI. (Non portable devices like cars and TVs would require a wireless laptop to make this work, but many people have that. Alternately they could use bluetooth, and the plugin could have a similar mode for working with paired bluetooth devices. Again, doing nothing without a strong user confirmation.)

This works because basic USB chips are very cheap now. Adding a small bit of flash to your electronics device and a mini-USB socket that can read and write the flash would add only a small amount to the cost of most items — nothing to many of them, as they already have it. Whatever new toy you buy, you could set it up on the web, and if the company provides a high level of service, you could speak to a tech support agent who could help you set it up right there.

Whose call is it to say what's legal?

As many of you will know, it’s been a tumultuous week in President Bush’s battle to get congress to retroactively nullify our lawsuit against AT&T over the illegal wiretaps our witnesses have testified to. The President convinced the Senate to pass a bill with retroactive immunity for the phone companies — an immunity against not just this but all sorts of other illegal activities that have been confirmed but not explained by administration officials. But the House stood firm, and for now has refused. A battle is looming as the two bills must be reconciled. I encourage you to contact your members of congress soon to tell them you don’t want immunity.

And here, I’m going to outline in a slightly different way, why.

I’ve talked about the rule of law, and the problems with retroactive get out of jail free cards that “make it legal.” But let’s go back to when these programs started, and ask some important questions about the nature of democracy and its checks and balances.

The White House decided it wanted a new type of wiretap, and that it wouldn’t, or most probably couldn’t get a warrant from the special court convened just to deal with foreign intelligence wiretaps. They have their reasoning as to why this is legal, which we don’t agree with, but even assuming they believe it themselves, there is no denying by anybody — phone company employees, administration officials, members of congress or FISA judges — that these wiretaps were treading on new, untested ground. Wiretaps of course are an automatic red flag, because they involve the 4th amendment, and in just about every circumstance, everybody agrees they need a warrant as governed by the 4th amendment. Any wiretap without a warrant is enough to start some fine legal argument.

In the USA, the government is designed with a system of checks and balances. This is most important when the bill of rights is being affected, as it is here. The system is designed so that no one branch is allowed to interfere with rights on its own. The other branches get some oversight, they have a say.

So when the NSA came to the phone companies, asking for a new type of wiretap with no warrant, the phone companies had to decide what to do about it. The law tells them to say no, and exacts financial penalties if they don’t say no to an illegal request. The law is supposed to be simple and to not ask for too much judgment on the part of the private sector. In this situation, with a new type of wiretap being requested, the important question is who makes the call? Who should decide if the debatable orders are really legal or not?

There are two main choices. Phone company executives or federal judges. If, as the law requires, the phone company says “come back with a warrant” this puts the question of whether the program is legal in the hands of a judge. The phone company is saying, “this is not our call to make — let’s ask the right judge.”

If the administration says, “No, we say it’s legal, we will not be asking a judge, are you going to do this anyway?” then we’re putting the call in the hands of phone company executives.

That’s what happened. The phone companies made the decision. The law told them to kick it back to the judge, but the White House, it says, assured them the program was legal. And now that lawsuits like ours are trying to ask a different federal judge if the program was legal, the Senate has passed this retroactive immunity. This immunity does a lot of bad things, but among them it says that “it was right for the phone companies to be making the call.” That the pledges of the administration that the program was legal were enough. We’ve even be told we should thank the phone companies for being patriots.

But it must be understood. Even if you feel this program was necessary for the security of the nation, and was undertaking by patriots, this was not the only decision the phone company made. We’re not suing them because they felt they had a patriotic duty to help wiretap al Qaeda. We’re suing them because they took the decidedly non-patriotic step of abandoning the checks and balances that keep us free by not insisting on going to either a judge or congress or both.

Officials in the three branches take a solemn oath to defend the constitution. Phone company executives, as high minded or patriotic as they might be, don’t. So the law was written to tell them it is not their call whether a wiretap is legal, and to tell them there are heavy penalties if they try to make that decision. Those who desire immunity may think they are trying to rescue patriots, but instead they will be rewarding the destruction of proper checks and balances. And that’s not patriotic at all.

Some have argued that there was a tremendous urgency to this program, and this required the phone companies to act quickly and arrange the warrantless wiretaps. While I disagree, I can imagine how people might think that for the first week or two after the requests come in. But this wasn’t a week or two. This has gone on since 2001. There was over half a decade of time in which to consult with judges, congress or both about the legitimacy of the wiretaps. It’s not that they didn’t know — one company, Qwest, refused them at their own peril. If you argued for immunity for the actions of that first week or two, I could understand the nature of your argument. But beyond that, it’s very hard to see. For this is immunity not just for illegal wiretapping. This is immunity for not standing by the law and saying “let’s ask a judge.” For years, and years. Why we would want to grant immunity for that I just can’t understand, no matter how patriotic the goals. This system of freedom, with checks and balances, is the very core of what patriots are supposed to be defending.

A plane that goes on a train

As I noted, at DLD Lufthansa had a contest (which I won) for suggestions on how to innovate to compete with trains. They set the time horizon out 15 years, which really means a lot is possible, so while I mostly threw in ideas from this blog which are short term, I put in some longer term ones too.

One was the equivalent of “multi modal transport.” To do this, you would build new short-haul planes which consisted of an empty shell, like the cargo planes you have seen where the nose hinges up, and cargo modules are slid in on rails. This would be combine with “passenger modules” which can slide into the shell, and which can also slide into a special rail car. There might be one module on a plane, though it is also possible to have several.

Passengers would board a train normally at the train station. Then, as the train moved to the airport, they could move to the passenger module car. They would place their luggage onto a belt to put it down low into the luggage module (under the passenger module) or be assisted by a porter. They would enter the passenger module, stow their carry-ons and otherwise get ready in their seat. By the time the train got to the airport, all passengers would be in their seats, belted and ready.

The train would split up into different cars if there were several flights on it, and each would move to a terminus where the plane-shell was waiting. Yet to be invented technology would laser-align the train and the parked shell in advance, and then the passenger module would slip into the aircraft hull on special rails. Connecting passengers could board the train at the airport before it moves to the hull, and their bags could be loaded into the bottom the standard way. (Though this is for short-haul flights, so there may not be connecting passengers.) An automated system would connect power, data and air venting on the passenger modules. Water/sewage would be self-contained and processed at the train station. Catering would probably be handled there too.

The nose would come down, the pilots board via their own door and takeoff would begin shortly.  read more »

Patent reform: Apply for a patent, examine some patents

Among many patent reform proposals it is common to have a desire for better examination, and more detection of prior art and obviousness. But the patent office only has so much money for so many examiners.

So here's a simple solution. If you want to apply for a patent, you must put in some time, as an expert in your field, examining other patent applications, searching for prior art and giving opinions on the obviousness. Alternately, this duty could be given only to those who actually are granted patents, to make more sure they are "skilled in the art" of their fields.

Of course, such crowdsourced examiners would have biases. They would be expected to make a sworn statement about their biases. Making a false statement could have implications on their own patents as well as the usual penalties.

Those biased against the patent would mostly hunt for prior art -- in fact they would make the best hunters. Those unbiased could make better opinions of obviousness.

Like regular patent fees, this could be biased for small inventors. (Small inventors pay lower patent fees and get some better treatments.) Large companies might have to volunteer more time from their staff, or small inventors might get reductions in patent fees in exchange for good work. Peers would examine the work of other peers to keep them honest and to rate the quality of it. And of course, unbiased patent examiners and appeal boards would still have the final, objective say.

Other volunteers could also participate in prior art searches. But with the system described above, there should be no shortage of labour. And as the number of patents goes up, the system naturally increases the labour available to do the legwork.

eBay should support the ReBay

Best

There’s a lot of equipment you don’t need to have for long. And in some cases, the answer is to rent that equipment, but only a small subset of stuff is available for rental, especially at a good price.

So one alternative is what I would call a “ReBay” — buy something used, typically via eBay, and then after done with it, sell it there again. In an efficient market, this costs only the depreciation on the unit, along with shipping and transaction fees. Unlike a rental, there is little time cost other than depreciation.

For some items, like DVDs and Books and the like we see companies that cater specially to this sort of activity, like Peerflix and Bookmooch and the like. But it seems that eBay could profit well from encouraging these sorts of markets (while vendors of new equipment might fear it eats into their sales.)

Here are some things eBay could do to encourage the ReBay.

  • By default, arrange so that all listings include a licence to re-use the text and original photographs used in a listing for resale on eBay. While sellers could turn this off, most listings could now be reusable from a copyright basis.
  • Allow the option to easily re-list an item you’ve won on eBay, including starting from the original text and photos as above. If you add new text and photos, you must allow your buyer to use them as well.
  • ReBays would be marked however, and generally text would be added to the listing to indicate any special wear and tear since the prior listing. In general an anonymised history of the rebaying should be available to the buyer, as well as the feedback history of the seller’s purchase.
  • ReBayers would keep the packaging in which they got products. As such, unless they declare a problem with the packaging, they would be expected to charge true shipping (as eBay calculates) plus a very modest handling fee. No crazy inflated shipping or flat rate shipping.
  • Since some of these things go against the seller’s interests (but are in the buyer’s) it may be wise for eBay to offer reduced auction fees and paypal fees on a reBay. After all, they’re making the fees many times on such items, and the paypal money will often be paypal balance funded.
  • Generally you want people who are close, but for ReBaying you may also prefer to pass on to those outside your state to avoid having to collect sales tax.
  • Because ReBayers will be actually using their items, they will have a good idea of their condition. They should be required to rate it. No need for “as-is” or disclaimers of not knowing what if it works.

This could also be done inside something like Craigslist. Craigslist is more popular for local items (which is good because shipping cost is now very low or “free”) though it does not have auctions or other such functionality. Nor is it as efficient a market.

Forming a "scanner club"

I’ve accumulated tons of paper, and automated scanner technology keeps getting better and better. I’m thinking about creating a “Scanner club.” This club would purchase a high-end document scanner, ideally used on eBay. This would be combined with other needed tools such as a paper cutter able to remove the spines off bound documents (and even less-loved books) and possibly a dedicated computer. Then members of the club would each get a week with the scanner to do their documents, and at the end of that period, it would be re-sold on eBay, ie. a “ReBay.” The cost, divided up among members, should be modest. Alternately the scanner could be kept and time-shared among members from then on.

A number of people I have spoken to are interested, so recruiting enough members is no issue. The question is, what scanner to get? Document scanners can range from $500 for a “workgroup” scanner to anywhere from $1,500 to $10,000 for a “production” scanner. (There are also $100,000 scanning-house scanners that are beyond the budget. The $500 units are not worth sharing and are more modest in ability.

My question is, what scanner to get? As you go up in price, the main thing that changes is speed in pages per minute. That’s useful, but for private users not the most important attribute. (What may make it important is that if you need to monitor the scanning job to fix jams or re-feed. Then speed makes a big difference.)

To my mind the most important feature is how automatic the process is — can you put in a big stack of papers and come back later? This means a scanner which is very good at not jamming or double-feeding, and which handles papers of different sizes and thicknesses, and can tolerate papers that have been folded. My readings of reviews and spec sheets show many scanners that are good at detecting double feeds (the scanner grabs two sheets) as well as detecting staples, but the result is to stop and fix by hand. But what scanners require the least fixing-by-hand in the first place?

All the higher end units scan both sides in the same pass. Older ones may not do colour. Other things you get as you pay more will be:

  • Bigger input hoppers — up to around 500 sheets at a time. This seems very useful.
  • Higher daily duty cycles, for all-day scanning.
  • Staple detectors (stops scan) and ultrasonic double feed detectors (also stop scan.)
  • Better, fancier OCR (generating searchable PDFs) including OCR right in the hardware.
  • Automatic orientation detection
  • Ability to handle business cards. Stack up all those old business cards!
  • The VRS software system, a high end tool which figures out if the document needs colour, grayscale or threshold, discards blank pages or blank backs and so on.
  • In a few cases, a CD-burner so can be used without computer.
  • Buttons to label “who” a document is being scanned for (can double as classification buttons.)
  • Ability to scan larger documents. (Most high-end seem to do 11” wide which is enough for me.)

One thing I haven’t seen a lot of talk about is easy tools to classify documents, notably if you put several documents in a stack. At a minimum if would be nice if the units recognized a “divider page” which could be a piece of coloured paper or a piece of paper with a special symbol on it which means “start new document.” One could then handwrite text on this page to have it as a cover page for later classification at the computer, or if neatly printed, OCR is not out of the question. But even just a sure-fire way to divide up the documents makes sense here. Comments suggest such tools are common.

It may be that the most workable solution is to hire teen-agers or similar to operate the scanner, fix jams and feed and classify documents. At the speeds of these scanners (as much as 100 pages/minute for the higher end) it seems there will be something to do very often.

Anyway, anybody have experience with some of the major models and comments on which are best? The major vendors include Canon, Xerox Documate/Visioneer, Fujitsu, Kodak, Bell and Howell and Panasonic.

Photograph your shelves to catalog your library

A lot of people want to catalog their extensive libraries, to be able to know what they have, to find books and even to join social sites which match you with people with similar book tastes, or even trade books with folks.

There are sites and programs to help you catalog your library, such as LibraryThing. You can do fast searches by typing in subsets of book titles. The most reliable quick way is to get a bar code scanner, like the free CueCats we were all given a decade ago, and scan the ISBN or UPC code. Several of these sites also support you taking a digital photograph of the UPC or ISBN barcode, which they will decode for you, but it's not as quick or reliable as an actual barcode scanner.

So I propose something far faster -- take a picture with a modern hi-res digital camera of your whole shelf. Light it well first, to avoid flash glare, perhaps by carrying a lamp in your hand. Colour is not that important. Take the shelves in a predictable order so picture number is a shelf number.

What you need next is some OCR of above average sophistication, since it has to deal with text in all sorts of changing fonts and sizes, some fine print and switching orientations. But it also has a simpler problem than most OCR packages because it has a database of known book titles, authors, publisher names and other tag phrases. And it even would have, after some time, a database of actual images of fully identified book spines taken by other users. There may be millions of books to consider but that's actually a much smaller space than most OCR has to deal with when it must consider arbitrary human sentences.

Even so, it won't do the OCR perfectly on many books. But that doesn't matter so much for some applications such as search for a book. Because if you want to know "Where's my copy of *The Internet Jokebook*" it only has to find the book whose text looks the most like that from a small set. It doesn't have to get all the letters right by any stretch. If it finds more than one match it can quickly show you them as images and you can figure it out right away.

If you want a detailed catalog, you can also just get the system to list only the books it could not figure out, and you can use the other techniques to reliably identify it. The easiest being looking at the image on screen and typing the name, but it could also print out those images per shelf, and send you over to get the barcode. The right software could catalog your whole library in minutes.

This would also have useful commercial application in bookstores, especially used ones, in all sorts of libraries and on corporate bookshelves.

Of course, the photograph technique is actually worthwhile without the OCR. You can still peruse such photographs pretty easily, much more easily than going down to look at books in storage boxes. And, should your library be destroyed in a fire, it's a great thing to have for insurance and replacement purposes. And it's also easy to update. If you don't always re-shelve books in the same place (who does) it is quick to re-photograph every so often, and software to figure out that one book moved from A to B is a much simpler challenge since it already has an image of the spine from before.

New European system of Bistromathics

Bistromathics was Douglas Adams’ term for the crazy difficulty of dividing up l’addition at a restaurant properly. The very rules of math seem to go wrong, which is why they were able to make a stardrive as long as the ship had a bistro in it.

When groups go out to dinner, many people feel that “Div N” is the safest way to go. Namely divide the total bill with tip by the number of folks and everybody pays that. It has the advantage of great simplicity, avoiding the bistromathics. And it is close to a must with shared dishes and the norm for Chinese/Indian.

For many people, Div-N balances out over time, but many people resent Div-N for various reasons:

  • For non-drinkers, they are bothered at paying a bar tab that often is as big as the food tab. Sometimes two totals are given because of this.
  • For vegetarians, not only are their dishes usually cheaper, but many have an ethical problem with paying for other’s meat.
  • Dieters are as they are due to lack of self-control. Many have a compulsion that bothers them if they pay for food but don’t eat it. (Larger restaurant portions are blamed by some for the obesity epidemic.)
  • Women tend to eat less than men, causing a sex-bias.
  • Some are just plain poor, and can’t handle the high Div-N bill. Because Div-N encourages liberal ordering of expensive dishes and apetizers, it tends to raise the overall price.

Often there will be somebody (frequently of low income) who wants to break the Div-N rule and pay just for what they ordered. My rule for this now is to hand them the bill and say they are responsible for calculating and collecting the bill for everybody. I do this because there have been times when I have been the banker that people have announced they will only put in for what they ordered after much of the div-N payment has been done. While one can sympathise if they only ordered $10 of food and div-N is $25, what they are asking is that the banker now take the loss. This is why they should become the banker.

I was told last year of a new system which is gaining popularity in Europe. It works as follows. One diner is indeed the banker. The bill is passed around and each is told to put in “what they think they owe.” The banker takes the pile of money and does not count it. It is made very clear that the banker will not be counting, at least not at the table. The banker then pays the bill out of their own wallet, usually by credit card, though sometimes with cash. To avoid counting, paying with cash should typically be done by just taking out a modest number of the large bills from the stack if the banker is short.  read more »

Converting vinyl to digital, watch the tone arm

After going through the VHS to digital process, which I lamented earlier I started wondering about the state of digitizing old vinyl albums and tapes is.

There are a few turntable/cd-writer combinations out there, but like most people today, I’m interested in the convenience of compressed digital audio which means I don’t want to burn to CDs at all, and nor would I want to burn to 70 minute CDs I have to change all the time just so I can compress later. But all this means I am probably not looking for audiophile quality, or I wouldn’t be making MP3s at all. (I might be making FLACs or sampling at a high rate, I suppose.)

What I would want is convenience and low price. Because if I have to spend $500 I probably would be better off buying my favourite 500 tracks at online music stores, which is much more convenient. (And of course, there is the argument over whether I should have to re-buy music I already own, but that’s another story. Some in the RIAA don’t even think I should be able to digitize my vinyl.)

For around $100 you can also get a “USB turntable.” I don’t have one yet, but the low end ones are very simple — a basic turntable with a USB sound chip in it. They just have you record into Audacity. Nothing very fancy. But I feel this is missing something.

Just as the VHS/DVD combo is able to make use of information like knowing the tape speed and length, detecting index marks and blank tape, so should our album recorder. It should have a simple sensor on the tone arm to see as it moves over the album (for example a disk on the axis of the arm with rings of very fine lines and an optical sensor.) It should be able to tell us when the album starts, when it ends, and also detect those 2-second long periods between tracks when the tone arm is suddenly moving inward much faster than it normally is. Because that’s a far better way to break the album into tracks than silence detection. (Of course, you can also use CDDB/Freedb to get track lengths, but they are never perfect so the use of this, net data and silence detection should get you perfect track splits.) It would also detect skips and repeats this way.  read more »

Make gasoline $6/gallon, give everybody $2,000

Burning gasoline is ruining the world. It accounts for 40% of greenhouse emissions, and a large percentage of other nasty emissions including the particulate matter that kills millions each year. Getting it has driven the world to wars. When you burn it, you pollute my air, hurting me, and you owe me something for it, which is a reason that gasoline taxes make sense even in a libertarian context.

So while gas should be taxed to $5 or $6/gallon, the public won’t stand for it. So here’s an alternate idea. Tax gasoline up to $6/gallon in a revenue neutral way. That is to say, figure out how much tax revenue that raises per adult. Americans consume 140 billion gallons/year, so a $3 tax raises 420 billion (before consumption drops.) There are about 200 million adults, so this works out to just over $2,000 per adult. As such, each person (regardless of how much oil they burned) would receive a $2,000 tax credit — a refundable credit payable even if they owe no taxes.

Update: The core idea here came from an earlier comment on this blog, which I forgot about (See comments below for references.)

For people who ride transit or walk or otherwise don’t use cars, this turns into a $2,000 windfall, offset by an increase in the cost of taxis and transit. In theory, for the average gasoline user, it works out to a wash — pay about $2,000 more per year for your gasoline, but get a $2,000 tax refund. At most it’s an enforced savings program.

For heavy gasoline burners — those taking very long commutes, those electing to buy Hummers and Suburbans — it means paying lots more, and subsidizing those who don’t. Those who buy a Prius would be well rewarded, as would those who switch to transit or anything more fuel efficient.

The consequences of this would be:

  • A giant and popular win for non-drivers, and for transit systems, which would get many more passengers to offset their increased costs.
  • Everybody would file a tax return now, even those making little or no money. This would cost the IRS more, but they would probably love it for making everybody file. Not filing would become remarkably suspicious. This is both good and bad, of course.
  • There would probably be some identity theft to try to steal the refunds, this would need to be watched for.
  • It creates a major issue for illegal immigrants. Those who want to cause them trouble would like it for this, as these immigrants would now pay large fees for gas but have no means to get the refund, unless they file tax returns, which of course they are scared to do — and they have no SSNs.
  • Fuel efficient technologies would become very popular and competitive, and the market would immediately start sorting out winners.
  • Fuel consumption would drop, reducing the amount of the credit — or requiring an increase in the tax.
  • Poor people with very long commutes could face serious problems, possibly forcing them to change jobs or homes, or try to carpool.
  • People would drive into Canada and Mexico to get tanks of gas. There would also be a black market in gas smuggled from those countries.

This could be applied to all fuel use, including power plants and factories. In that case many products would increase in price, all offset by the credit.

Aside from the immigrant problem, it is also important to note how bad governments are at restraint, and there would be much temptation to not make the tax revenue neutral, and just make it a tax increase.

Would voters vote for this? Well, designed properly, if we assume that 50% of the gas is used by fewer than 50% of the people, then this is a win for more than 50% of the people, probably more than 70%. And of the top 30% of gasoline users, many of them would intellectually agree with it though it costs them more money. If people realized they would pay less, not more, under the tax, this could win voter support.

This could also be done on a state by state basis in some states. However, it would create problems on the state borders. Border gas stations would die, and need compensation. There would be a lot of smuggling from the other states. More people would risk using purple gas, as well. Enforcing is tough without some draconian system we wouldn’t like so much. It thus would be possible only in states that have few people living on their borders, mostly western rural states. California is not out of the question. It has no large cities on state borders, but does have some decent sized towns.

The positives of this idea are many, as are the negatives. But those positives are pretty valuable. In particular, this system would drive the market to work hard at producing technologies that really reduce fuel consumption, resulting in perhaps the biggest benefits of all.

My world's oldest "blog" is 20 years old tomorrow (Aug 7, 2007)

Twenty years ago Tuesday, I created the newsgroup rec.humor.funny as a moderated place for posting the funniest jokes on the net, as chosen by the editor. In light of that anniversary, I have written up a bit of history of the creation of RHF. From there you can also find links to pieces I wrote earlier about the attempt to ban RHF and how RHF led to my creation of ClariNet.

One reason people may pay a bit more attention to this anniversary is I think that RHF, with its associated web site has a claim at being the world’s longest still-running “blog.” Of course, there is much debate about the origins of blogging, and there are various contenders based on what definition you put to the word.

I provide more detailed examination of those definitional questions and the other contenders on a page about the world’s oldest blog. In short, I contend that a blog is something that is:

  • Serial (a series of publications over time)
  • Done with a personal editorial voice (rather than being news reporting)
  • On the world wide web

While most agree with that last point (since personal journals, published diaries and columns existed long before computers) many forget that when Tim Berners-Lee defined what the web was, he was very explicit about including the many media and protocols he was tying together with HTML and HTTP, including USENET, Gopher, E-mail and the rest. So the web dates back well before HTML, and so does the weblog.

I personally point to mod.ber, a short-lived moderated newsgroup from 1983 as the first blog. It was clearly the boing-boing of its day. But it doesn’t exist, so RHF may get to claim the title.

As you will know if you have followed RHF, while I continue to publish it and provide the software and systems, I only edited it for the first 5 or so years. After that Maddi Hausmann took over, and in 1995, Jim Griffith took the reigns to this day. He, however, is ready to retire shortly and we’re looking for a replacement — a note will be posted in RHF and here with more details after the anniversary.

As you’ll see in the histories, the decision to start RHF changed my life in sweeping ways. It was one of those junctures that Clarence from “It’s a wonderful life” could change if he wanted to show me a different path.

Happy 20th Birthday rec.humor.funny.

Two year contract required

I’m a big fan of making money by selling services but a disturbing trend is the requirement that customers sign a one or two (or even three) year contract in order to sign up for a service. Such contracts will have a fat termination fee if you want to end the contract early.

This is almost universal for cell phones, and of course it makes some sense when they are selling/giving you a subsidized phone. They need to be sure you will stay with them long enough to make the subsidy (From $200 to $400 if you include dealer kickbacks) back. That’s not so hard, because with many people getting cell phone plans as high as $100/month, they make it back quickly.

However, cell phone companies notoriously require a new contract for just about any change in your calling plan, including simply switching to a new plan they just started offering that you like better. Usually that’s just a one year contract. This makes much less sense. Switching your plan doesn’t cost them anything much aside from a call to customer service. They just want to put you on that contract.

DSL ISPs (and not just the phone company ones) are also notorious here. Some need it to subsidize installation or equipment, but again it’s also done simply to change price plans. In many cases you will also see major discounts offered if you commit to a contract (or of course even better if you just pay 12 months at once.)

I understand the attraction of the company for contracts. They can predict and book revenue. Quantity discounts have always had their reasons.

But they may not realize a serious negative about the contracts. They are a barrier to getting customers. In particular, a demand for a contract (when there is no major subsidy) says to me we think that without a contract, we could lose you as a customer. We fear that, if not for the contract, you would leave us. And that immediately makes me think the same thing. “What is it that makes them think they can’t keep me just by providing good service at good prices?” They already won my business, which is the hardest part. Now all they have to do is keep me happy and they will be very likely to keep it.

This recently backfired for Verizon. I’ve been off contract with them for years, though I had often debated switching to a different plan. Every time they told me I would need to sign a one year contract, and get no subsidy for doing so. (For a 2 year contract, they would have subsidized a new phone, but I wasn’t ready to do that.) So when phones broke I often picked them up on eBay rather than take their 2 year subsidy.

When it came time to really want to change plans, their demand for a new contract made them the same as all their competitors, who will also demand a new contract. And thus there was no particular reason not to switch. They encouraged me to compare all the various offers, all of which require a new contract, and all of which can offer me a phone subsidy with a 2 year contract. And all of which can keep the number, thanks to hard-won number portability. Had they been willing to let me make changes without a contract, I would have had no incentive to go shopping around at the competition. There I learned about much better deals they had, and thus left Verizon.

Perhaps they think they need a contract to keep me from the competition. But truth is, that might work temporarily but it just delays things. When a contract expires, somebody is going to be ahead, be it the competition or be it them, and they just moved the switch in time and probably locked me into the competition for their efforts.

The best company in the business shouldn’t need a contract to hold me. If the competition is offering a snazzy new subsidized phone for a contract, then my no-contract company can certainly offer that. Or, ideally, just offer me a lower monthly rate if I bring my own phone, with no need for a contract — my choice.

Over time, the public might wake up to realize that the contract is much more expensive than the phone subsidy. A typical data phone requires a plan of $60 to $80 per month, and many are on plans of $100 or more. That’s a $2400 purchase at $100/month, all to get a $200 phone subsidy. Of course most customers plan to buy from somebody over the period, so it makes sense to take the subsidy if you aren’t likely to be changing all the time, which most of us aren’t. But I am curious why all the firms feel these contracts are really in their interest.

Update: I should point out that there are reasons to get warmer to a contract when getting a new phone. Typically there is a $200 subsidy on the phone, and sometimes much more. And quite commonly, the penalty for getting out of the contract is $200, and in fact my law reduces on a pro-rata basis as you move through the life of the contract. As such, there is no reason not to sign the contract if you want that brand-new phone. In addition, there are contract trading sites (where other people will take over your contract for less than the penalty price because they don’t need a phone) to get out even cheaper.

However, you don’t want a contract without this level of quid pro quo. A contract just to change plans is ridiculous. Some carriers are getting that message.

Should we allow relative's DNA matching to prove innocence?

Earlier I wrote about the ability to find you from a DNA sample by noting it’s a near match with one of your relatives. This is a concern because it means that if relatives of yours enter the DNA databases, voluntarily or otherwise, it effectively means you’re in them too.

On a recent 60 minutes on the topic, they told the story of Darryl Hunt, who had been jailed for rape and murder. It wasn’t clear to me why, but this was done even though his blood type did not match the rapist’s DNA. Even after DNA testing improved and the non-match was better confirmed, he was still kept in jail, because he was believed to be the murderer, if not the rapist, ie. an accomplice.

Later, they did a DNA search on the rapist’s DNA and found his brother in the database, who had been entered due to a minor parole violation. So they interviewed the brothers of the near-match and found Willard Brown, who turned out to be the rapist. Once they could see he was not an associate of the rapist, Hunt was freed after 19 years of false imprisonment.

The piece also told the story of another rapist, who had raped scores of women and stolen their shoes as souvenirs, but had become a cold case. He was caught because his sister was in a DNA database due to a DUI.

Now much of our privacy law is based on having your own private data not seized and used against you without probable cause. It’s easy to answer the case of the shoe rapist. There are a wide variety of superior surveillance tools we could allow the police to use, and they would help them catch criminals, and in many cases thus prevent those criminals from committing future crimes. But we don’t give the police those tools, deliberately, because we don’t want a world where the government has such immense surveillance power. And a large part of that goal is protecting the innocent. Our rules that allow criminals to walk free when police do improver evidence gathering and surveillance to catch them are there in part to keep the police from use of those powers on the innocent.

But the innocent man who was freed presents a more interesting challenge. Can we help him, without enabling 1984? In considering this question, I asked, “What if we allowed DNA near matches to be used only when they would prove innocence?” Of course, in Hunt’s case, and many others, the innocence is proven by finding the real guilty party.

So what if, in such cases, it was ruled that while they might find the guilty party, they could not prosecute him or her? And further, that any other evidence learned as a result was considered Fruit of the poisonous tree? That’s a pretty tough rule to follow, since once the police know who the real perpetrator is, this will inspire them to find other sorts of evidence that they would not have thought to look for before, and they will find ways to argue that these were discovered independently. It might be necessary to put on a stronger standard, and just give immunity to the real perpetrator if sufficient time has passed since the crime to declare the case to be cold.

Setting out the right doctrine would be difficult. But if it frees innocents, might it be worth it?

Jobs warns knockoff iPhone "lacks many key features"

Steve Jobs of Apple Computer warned today that a rumoured cheap Chinese iPhone knockoff making its way toward America is an inferior product which lacks many of the important features of the iPhone. “It may look a bit like an iPhone, but when consumers discover all the great iPhone features that are missing from it, we think they’ll still line up at Apple Stores for the genuine article,” said Jobs in a released statement. Designed by software nerds, the knockoff, dubbed the “myPhone” by fans, has not yet been confirmed.

Apple released a list of features reported to be missing from the “myPhone.”

  • The iPhone has special software that assures you will always use the trusted AT&T cellular network. Lacking this software, the myPhone accepts any SIM card from any random network. Users may find themselves connected to a network that doesn’t have the reputation for service, trust and protecting the privacy of customers that AT&T has. Or its data speed which is almost double what we’re used to with dialup.
  • With the myPhone, users may be stuck without 2 years of guaranteed AT&T service and won’t get their price locked in for 2 years. AT&T’s EDGE network is so good “you won’t find yourself able to quit.”
  • The iPhone is configured to assure you the latest iTunes experience. The myPhone might function before you have installed the latest iTunes and registered your phone with it. Indeed, the myPhone lacks the protections that block it from being used without registering it with or reporting back to anybody, depriving the user of customer service and upsell opportunities.
  • The iPhone has special software that assures all applications run on the iPhone have been approved by Apple, which protects the user from viruses and tools that may make the user violate their licence agreements. The myPhone will run any application, from any developer, opening up the user to all sorts of risks.
  • The iPhone protects users from dangerous Flash and Java applications which may compromise their device and confuse the user experience.
  • myPhones don’t forbid VoIP software that may cause the user to accidentally make calls over wireless internet connections instead of the AT&T network. Quality on the internet is unpredictable, as is the price, which can range down to zero, causing great pricing uncertainty. With the iPhone, you always know what calls cost when in the USA.
  • The iPhone saves the user from receiving distracting instant messages over popular IM services, adding calm to your day.
  • Music and videos in the iPhone are protected by Apple FairPlay brand DRM. On the myPhone, which lacks the important DRM functionality, music can be freely copied to other devices the user owns, putting the user at risk of infringing copyrights.
  • The iPhone assures users will only play media files in approved formats, and not risky open source formats.
  • The iPhone protects the user from setting a song in their device as a ringtone, saving those around him from annoyance and protecting the user from violating music copyrights and performance rights.
  • The iPhone bluetooth functions have careful security management. Users are protected from using bluetooth to exchange files with other users (such files are risky) or accidentally printing or communicating with your computer. Bluetooth is only used for headsets and headphones as was intended. The myPhone lacks these important protections.
  • The iPhone only uses its internal flash drive. The user is protected against hard drives, which have moving parts and can put data at risk.
  • The myPhone battery has a removable door over it, which can get lost, or allow the battery to fall out or be stolen. The iPhone’s battery is solidly protected. Users are also assured they will use only Apple certified batteries and not subject to the risk of aftermarket batteries, which may explode, killing the user.
  • The iPhone is for sale only in the USA and primarily for use there. This encourages users to stay home in America which is good for the economy and their own peace of mind.
  • The iPhone, unlike the myPhone and all other cell phones, sells at a very solid markup for Apple, assuring Apple executives and stockholders will be happy, and the company will be around to support the iPhone for years to come. The myPhone, it is rumoured, will be purchasable in a wide variety of stores, confusing the buyer with too much choice, price wars and depriving them of the special experience of an Apple or AT&T store.
  • As a result, the myPhone lacks the Apple brand “coolness” which is built into the iPhone and every other Apple product. “Nobody’s going to have to spend days in line for a myPhone,” said Jobs. “You won’t have people thrusting them in your face all week to show you how cool they are.” Many iPhone users report their experience waiting in line was great fun, and that they met all sorts of new people.

MyPhones are predicted to sell for $350 without contract, $150 with a 2 year contract to the provider of your choice.  read more »

Standardize computer access in hotels, and vnc everywhere

Hotels are now commonly sporting flat widescreen TVs, usually LCD HDTVs at the 720p resolution, which is 1280 x 720 or similar. Some of these TVs have VGA ports or HDMI (DVI) ports, or they have HDTV analog component video (which is found on some laptops but not too many.) While 720p resolution is not as good as the screens on many laptops, it makes a world of difference on a PDA. As our phone/PDA devices become more like the iPhone, it would be very interesting to see hotels guarantee that their room offers the combination of:

  • A bluetooth keyboard (with USB and mini-USB as a backup)
  • A similar optical mouse
  • A means to get video into the HDTV
  • Of course, wireless internet
  • Our dreamed of universal DC power jack (or possibly inductive charging.)

Tiny devices like the iPhone won’t sport VGA or even component video out 7 pin connectors, though they might do HDMI. It’s also not out of the question to go a step further and do a remote screen protocol like VNC over the wireless ethernet or bluetooth.

This would engender a world where you carry a tiny device like the iPhone, which is all touchscreen for when you are using it in the mobile environment. However, when you sit down in your hotel room (or a few other places) you could use it like a full computer with a full screen and keyboard. (There are also quite compact real-key bluetooth keyboards and mice which travelers could also bring. Indeed, since the iPhone depends on a multitouch interface, an ordinary mouse might not be enough for it, but you could always use its screen for such pointing, effectively using the device as the touchpad.)

Such stations need not simply be in hotels. Smaller displays (which are now quite cheap) could also be present at workstations on conference tables or meeting rooms, or even for rent in public. Of course rental PCs in public are very common at internet cafes and airport kiosks, but using our own device is more tuned to our needs and more secure (though using a rented keyboard presents security risks.)

One could even imagine stations like these randomly scattered around cities behind walls. Many retailers today are putting HDTV flat panels in their windows instead of signs, and this will become a more popular trend. Imagine being able to borrow (for free or for a rental fee) such screens for a short time to do a serious round of web surfing on your portable device with high resolution, and local wifi bandwidth. Such a screen could not provide you with a keyboard or mouse easily, but the surfing experience would be much better than the typical mobile device surfing experience, even the iPhone model of seeing a blurry, full-size web page and using multitouch to zoom in on the relevant parts. Using a protocol like vnc could provide a good surfing experience for pedestrians.

Cars are also more commonly becoming equipped with screens, and they are another place we like to do mobile surfing. While the car’s computer should let you surf directly, there is merit in being able to use that screen as a temporary large screen for one’s mobile device.

Until we either get really good VR glasses or bright tiny projectors, screen size is going to be an issue in mobile devices. A world full of larger screens that can be grabbed for a few minutes use may be a good answer.

Syndicate content