I wrote before about how the fancy bags they give away at conferences very rarely get used. I have a stack in the closet, and I’m not going to use them as my bag with sponsor logos plastered all over them. The people who attend such conferences aren’t the sort who want to carry your advertising everywhere, or scream out “I’m so cheap I’m using a sponsored bag.” And you can’t give them to friends as gifts, even if they are nice bags. So I suggested that they put logos on the inside but of course that doesn’t yet happen.

So here’s a business: Decent quality sheets that one can use to cover up the logos with something else. Either a sheet with the same common “ballistic” nylon texture, or even better, a sheet that I can print out on my inkjet printer (like a iron-on T-shirt transfer) which is thick enough to cover the other logo and let me have my own image or name.

Yes, this is sort of unfair to the sponsors of the bags. But the truth is, their sponsorship really doesn’t work after the conference is over. How often do you really see bags with logos plastered all over them out in the real world beyond the flight home from a conference? So this is more a reaction to waste than a desire to cheat the sponsors.

A story from the New York Times suggests it costs over $12,000/year to store a movie in digital form.

This number is entirely bogus, and based on old thinking, namely the assumptions of offline storage on DVDs and tapes. Offline media do degrade, and you must copy them before they have a chance to degrade, which takes people, though frankly it’s still should not be as expensive as this. To do my calculations, I am going to assume a movie needs 100gb of storage with low-loss lossy compression. You can scale the numbers up if you like if you want to assume more, even at 1 TB it doesn’t change that much.

A film occupying 100gb of storage can go on about 20 dvds (or 11 dual layer,) costing about $8. It can go on 4 independent sets of 20 DVDs for $32 in media. Ideally you could rack these in a DVD jukebox, but if they are just sleeved, then once a year a person could pull out the DVDs, put them in a reader which would test them. Any that tested fine would be re-sleeved, those that did not would flag for the others to be pulled, and then copied to new media. (Probably better media, like blu-ray.) There are algorithms to distribute the data so that a large number of the disks must fail in that year to actually lose something. Of course, you use different vaults around the world. When approaching the point where failure rates go up for the media, you re-burn new copies even if the old ones still test fine.

This takes human time, though not all that much. Perhaps half an hour of actual human time swapping disks though much more real time to burn them, but you don’t do just one at a time.

However, even better is the new style of archival — online storage. Hard disks are 20 cents/gigabyte and continuing to fall. NAS boxes are more expensive now but there is no reason they won’t drop to very reasonable prices, so that a NAS case adds perhaps 5 cents/gigabyte (ie. $100 for a 4x500gb drive box which lasts for 10-15 years.) (NAS boxes are small boxes that hold a collection of drives and allow access to them over ethernet. No computer is needed.) They also cost about 2 cents/gb/year for power if on all the time, and some small amount for space, though they would tend to sit in computer centers that already exist.

Those are today’s prices, which will just get cheaper, except for the power. Much cheaper. If a drive lasts an average of 4 years before failing and a NAS lasts 10 years, this works out to 7.5 cents/gigabyte/year. Of course you will store your files redundantly, in 4 different places (which is actually overkill) and so it’s 30 cents/gigabyte/year.

Which is still just $30 for a 100gb file, or $300 for a TB.

Online storage is live. You can regularly check the integrity, all the time. You can either leave it off and spin it up every few days (to not use power) or just leave it on all the time. If one, two or three of the 4 disks fails, computers can copy the data to fresh disks in the network, and you are alive. Your disks should last 3 to 4 years but many will last much longer. You need a computer system to control all this, but you only need one for the entire cloud of NAS boxes, or at most a few. Its cost is low.

The real cost is people. But companies like Google have solved the problem of running large server farms. They tolerate single drive failures. The computers copy the data to new drives right away, and technicans go by every few days to pull old ones and slot in fresh ones for the next need — not for the same file. This takes just a few minutes of the tech’s time. And there is no rush to their work. Fore each 100gb file, you should expect to have a replacement about once every 4 years (ie. the lifetime of an average drive.)

Now all this is at today’s price of $100 for a 500gb drive. But that’s dropping fast, faster than Moore’s law. The replacements will be 1TB and 2TB drives before long, and the cost will continue to fall. And this is with 4 copies of every file. You can actually get by with less using modern data distribution algorithms which can scatter a file of 100gb into 200 1gb pieces, for which almost half must be lost before the whole file is lost. Several data centers could burn down without losing any files if things are done right. I have not accounted for bandwidth here for replacements, which usually would be done in the same data center except in unusual circumstances.

The biggest cost is the people to set all this up. However, presuming big demand, the cost per gigabyte for those people should become modest.

I’m not sure why, but beaming business cards between PDAs never caught on as much as I would have liked. Of course Palm and Wince PDAs don’t speak the same beaming language (of course) and I never saw it much in Windows anyway.

With my new fancy scanner, I can scan a stack of 60 business cards in a minute, so it’s not going to take me long to do the physical scanning. Business card scanning has been around for a while, but it still presents challenges.

People like to do funny things on their cards. They put stuff on the back (not just for foreign language contacts, where it makes sense.) They put in coloured backgrounds and pictures to make the OCR process as hard as possible. They like to do embossing, or even strange shapes. (Some people used to put rolodex tabs on their cards to make them stand out in a rolodex.) They will put lines or other OCR killers in the background too. People should start expecting their card will be scanned and OCRd, and design accordingly. That means if you put in your stylized logo, but the company name in in plain text too. (Though the need for a URL on a card helps this nowadays.)

Of course, even better to solve the OCR problem would be to put just one string in a clear, easy-to OCR format, which is the URL of a vcard. Then it doesn’t matter if I can’t OCR anything else, I can get reliable (and up to date) information from there. (One could also imagine a hosting service with a standard URL prefix to put in front of a vcard ID so you don’t have to take up that much room on your card. Another idea would be to standardize the VCARD URL so that it says something like “VCARD: S/xxxxx” where xxxxx is a semi-private string, and “S” means use the web URL found elsewhere on the card, with “std-vcard/xxxxx” appended to it. This way you don’t have to duplicate the domain name, but nor can vcards be harvested. Otherwise we could just use the E-mail to extract the vcard.)

Anyway, I came up with another idea I will try instead of beaming. “Can I take a picture of your card?” Since I plan to scan people’s cards anyway, why not save the trouble and use a small pocket camera I am carrying, and take a photo right there. You don’t even have to give me the card. Will I be rude if I don’t take the physical card?

Now admittedly, camera phone pictures may suck, and for this you really need a camera with a macro mode. On camera flash may present a giant glare spot unless you learn how to do it right, or are shooting in bright light without flash. The photo won’t be nearly as good as a scan, of course. (I suppose one could imagine putting a 2” long hand-scanner line on the side of your PDA to hand scan cards, bar codes and many other things.)

The bad news is that cell phone cameras probably can’t make the cut. They don’t have macro mode, and if they have a flash, it’s going to be very hard to get a good exposure on the card. You have to tweak what you can tweak and even then it may not be possible. (I found I had to use my cell camera’s exposure compensation to drop it by 2 stops to avoid having the LED that counts for a flash not wash out the card, and even then it wasn’t very good.)

Digital Picture Frames are finally coming down to tolerable prices and decent resolutions. We are about to give my mother one that’s 1024x768 and 15” on the diagonal. In part that’s because I never got around to building one out of a laptop though I still think a linux distro that turned an old laptop into a digital PF would be a great idea because the ability to do wireless networking to subscribe to flickr and other feeds is the waiting killer app for these frames. (Or frankly, I just want the wireless module for flat panel displays I have spoken of before.

However, turnkey appliances still have their attraction, and digital picture frames are one of the hot items for this year and probably a few to come.

However, one thing bothers me about them (and all other computer slide shows.) I take a modest number of photos in “portrait” mode, which is to say tilting the camera on its side to make a picture that is tall rather than wide. Of course I take many landscape too. And most digital picture frames are set up in landscape mode. When you see a portrait picture you lose half the resolution. You could get two frames — one arranged in portrait mode and one in landscape, but I propose making a frame where the panel and frame have a small motor on them. Every so often the motor would rotate the frame 90 degrees, and the frame would then switch to doing the pictures that are right for that orientation, and later switch back.

You would want a silent motor of course. It need not be very fast, and you could blank the screen while it turns, or even put up a clever animation that itself counterspins around the axis point so it looks still. It would not work if you only had a very small number of portrait photos, but should be fine for most folks.

Slow, quiet stepper or servo motors are not very expensive, much cheaper than a second frame, though this does add moving parts.

I’ve wanted something similar as well for projected slide shows. There the motor could turn the internal panel, or perhaps just a mirror. If these things existed, people might take more portrait pictures. Today, seeing most photos on computer screens, there seems to be no reason to shoot portrait (other than to get a wider field of view.) If you will always view on the computer, shooting portrait — for those who don’t understand its value as a compositional tool — may just seem like a waste. Now it would not be.

A card from Ty. (My brother, the comic book artist, if you didn’t know.)

I guess in the linux community it is slightly more acceptable.

This week, like many, I have gotten a bunch of invites to join people’s trust networks on the people-search/social networking site called “Spock.” Now normally I have started to mostly ignore new invites from social networking services. There are far too many, and I can’t possibly maintain accounts on them all, so a new site will have to get very, very, very compelling before I will join it.

I’m waiting for the social networking sites to figure out how how to interoperate in a meaningful way, so that I can join just one, and befriend people on others, and use apps that work over both. The new Google offering is a step in that direction but is mostly about making apps portable over networks.

However, the volume of mail from Spock was much higher than a typical new network. One blogger identified the reason, suggesting the site was designed by the evil spock from Mirror, Mirror (Star Trek). The trick is the site has already spidered other social networking sites and web sites to build profiles on people, and thus declares that almost everybody in your addressbook “already has a profile” according to Benson. This is convincing friends to authorize the semi-spam. And Wired News has discovered something even nastier about this spidering.

However, I see a deeper problem, even without these flaws in Spock’s system. We have to consider just how much we want to allow applications to “mail everybody in your address book.” This started with Plaxo and Goodcontacts, which wanted to be address book managers, and now has moved into social networking tools.

The problem is I have 1,000 or more people in my address book. If the average person engages in “mail everybody in my address book” once a year, I will get on average 3 such mails a day, and so will most others.

Facebook actually clued into that and forbids applications from mailing solicitations to everybody in your facebook profile. You are limited to a modest number per day. Even with this, it didn’t stop Zombie invitations from getting pretty annoying to people.

E-mail viruses, of course, also spread by mailing everybody in your address book, to the extent that email programs had to move to make that a more guarded operation, and antivirus programs had to detect it.

Now mailing most of your address book isn’t spam (even with commercial) because you know the people. Many of us mail a subset of it to announce parties or major events in our lives, or to send end of year letters. But we do need to generate a different ethic over mail to your whole list that is triggered by a 3rd party web site or application. With so many apps wanting to “market like a virus” this just doesn’t scale, and our boxes will become full of this spam-from-friends. (A bit like the way pyramid schemes also encourage friend spam.) It needs to be clear that this is not something apps should do, and not something our friends should let apps do without a lot of consideration.

Note: If you are on Spock, and you agree they went too far, you should delete your profile. Only be seeing people flee will they figure out they did wrong. Or, at the very least, change your profile to a stub that says you find Spock’s privacy practices unacceptable and you ask people not to network with you on it.

Update: Harry Reid has delayed the bill until 2008. Let’s hope we can keep the immunity out when it returns again next year. Let your senators know.

Usually, when you start a legal action, you consider the merits and go ahead when you have a good case. If your case is just, you should win.

You don’t usually expect your case to cause the President to personally lobby congress to grant a retroactive immunity to the parties who broke the law. You don’t usually expect to have them try to toss out your case by having an act of congress grant amnesty to those you are suing.

But this could happen tomorrow, in our battle against AT&T for letting the NSA wiretap without warrants. The house passed a bill without the amnesty the President wanted, and the Senate had two bills, but right now they’ve picked the bad one, with the amnesty, and powerful forces are pushing to make it go through quickly, and then add the amnesty to the house bill.

Senator Chris Dodd is going to show some great spine tomorrow and try to filibuster the bill and trigger debate. However, pro-amnesty forces are gathering the 60 senate votes needed to shut down the bill and grant amnesty. Your senator is probably among them. One of my senators, Dianne Feinstein, is among the worst. But it’s not too late to call your own senator and tell them not to engage in this travesty of justice.

In Star Wars: The Phantom Menace, Darth Sidious, a.k.a. Emperor Palpatine, tells his puppet trade federation to invade Naboo.

“But my lord, is that legal?” asks the trader.

“I will make it legal” says Lord Sidious.

That’s the precedent they are setting, as I’ve written before. Do what the President says, ignore checks and balances because he can make it legal, retroactively. It’s a sad say for the rule of law.

Do me a favour and call your senator and let them know what you think about this issue. Let them know their constituents will remember this action, and see if you can turn the tide.

Here’s an idea for a way to bring reputation based shopping to the brick and mortar world.

You would get a new special credit card, Visa or Mastercard. In order to use it, you would be required to rate merchants with reputation scores. You would do this when getting your online credit card bill — a random set of the merchants you purchased from would be highlighted and you would have to put in ratings. You would not have to do all of them, nor more than a set number each month and could also beg off some months to avoid it being a burden. This produces a set of ratings which are not nearly as self-selected as most rating systems, and makes it harder for the merchants to deliberately inflate their own ratings or lower competitors, because they actually have to buy stuff and don’t always rate the purchases they choose. (The system could allow manually chosen ratings but would treat them differently.) If you chargeback, your rating would also get special examination.

However, that’s just step one. The real meat comes when you use the card. You could set thresholds, and if you made a purchase at a vendor with a very poor reputation, below your threshold, the card would decline your purchase. At that point, you would have several options:

• Get the signal that the merchant is bad, and abandon the purchase
• Call the 1-800 number on the back of the card on your cell phone. It would spot your caller-ID, and immediately the computer voice would tell you the reputation of the vendor — or tell you that you hit your credit limit. You could then command it to authorize the transaction.
• Alternately, you could just have it automatically approve any second attempt at the transaction, and thus you could just say “run it again.” (Stores could know this and abuse it, however, so the call method makes more sense.)
• More simply, if you still want to purchase, you could just pull out another card, and tell them to try that one.

This would work just as well in online shopping, through frankly browser plug-ins make more sense there. However, people don’t use them so this would still work well. In this case you could go to a web URL instead of call the number. And of course it would be nice if paypal also did this, but they don’t seem inclined.

I don’t know if this would violate any bank agreements with Visa or Mastercard, or if, more to the point, they would rewrite the agreements to make it be a violation. The stores who lose business would of course hate it, but they would tend to be the scam houses that just cause lots of chargebacks anyway, so I don’t see why Visa/MC would want to come to their aid.

I was intrigued by this report of a russian chatbot fooling men into thinking it was a woman who was hot for them. The chatbot seduces men, and gets them to give personal information that can be used in identity theft. The story is scant on details, but I was wondering why this was taking place in Russia and not in richer places. As reported, this was considered a partial passing of the Turing Test.

As it turns out, programs have passed Turing’s test with unskilled chat partners for some time. As I’ve written, the test should really involve fooling a skilled AI researcher. However, as I read about this chatbot, I thought of a strategy that it might be using. (The report doesn’t say.)

A chatbot could either try to fool people in a language which is a second language to the target, and/or claim that it is using a second language for itself. With English as the lingua franca of the internet and world commerce, it’s common to see two people talk in English, even though it is not the mother tongue of either of them. It is, however, their common language.

However, when in that situation, two things will occur. First, a non-native speaker may not notice mistakes of language made by their correspondent, simply because they are not that familiar with it. Nonsensical statements may just be written off. Secondly, if the correspondent is also not expected to be fluent in the language, even a native speaker would be forgiving of errors. Especially if it’s a woman they want to seduce.

As such, you would generate a situation where a far less sophisticated program could give the appearance of humanity. It’s easier to see how a chatbot, claiming to not speak English (or some other “common” language) very well — and Russian not at all — might be able to fool a Russian whose on English is meagre. Though you have to be pretty stupid to give away important information within 30 minutes to a chat partner you know nothing about. However, such a chatbot would work far less well against native speakers of English, as forgiving as they might be of the cyberlass’ foibles.

The extended version of Razor contains this additional prophecy from the First Hybrid.

At last, they’ve come for me. I feel their lives, their destinies, spilling out before me. The denial of the one true path. To play that out on a world not their own. But will they be soon enough? Soon there will be four glorious new awakenings, struggling with the knowledge of their true selves, the pain of revelation bringing new clarity. And in the midst of confusion they will find that enemies are brought together by an awesome sense of belonging. Enemies now joined as one. The way forward, the once unthinkable, yet inevitable. And the fifth is still is in shadow, drawn toward the light, hungering for redemption, that will only come in the howl of terrible suffering. I can see them all - the seven, now six, self-described machines who believe themselves are of no sin, but in time it is sin that will consume them. They will know enmity, bitterness, the wrenching agony of the one splintering into many. And then they will join the promised land, gathered on the wings of an angel. Not an end, but a beginning.

These lines, highly prophetic, add more and more evidence that this hybrid is the cylon god, or closely connected with him. Update: the podcasts imply he is not the god himself, but is in regular communication with him.

But the line about the 5th and final Cylon seems to point at only one character. Baltar. This is not particularly satisfying, as it confuses the issue of Baltar as traitor. It makes his role entirely different. Though it does provide a good audience shocker, when compared with the old series, where he was a fairly 1 dimensional villain.

However, only one character in the show has done so much to need redemption, hungers for it, and has declared that finding out he is a Cylon would give him redemption. Sure, all the characters have done bad things and could use some redemption, but nobody like Baltar. On the whole, other characters like Adama, Roslin, Lee, Starbuck and Gaeta are heroes with a few flaws.

Ronald Moore likes redemption drama, and he seems to be preparing us for it.

Let’s consider other clues:

• When D’Anna faces the Final five, and dies, her last words are in Baltar’s arms. “So beautiful. You were right.” He asks, “About what?” but she never answers. However, the only clear thing he’s been pushing her on before this event is whether he’s a Cylon or not.
• She’s just greeted one of them with “Forgive me, I had no idea” and while she’s done ill to just about everybody, it’s Baltar she recently tortured. (Though with his nagging about it, she should have had some idea.)
• Baltar has this inner six, and she’s not just a demented dream. She knows stuff. This is the best explanation for it. She seems to have been able to physically pick him up when he was beaten down, and many think she was Shelley Godfrey, the physical six who accused Baltar of being a traitor while head-six had vanished. Godfrey turned a corner and vanished herself, and head-six was back.
• In the Hand of God Baltar randomly picks a place to bomb, and it turns out right
• He’s really smart, smarter than most colonials, smarter even than the Cylons at things like tracking clues about Earth
• The Hybrid calls him “the chosen one” and declares he is “intelligence, a mind that burns like fire.”
• In various points of the show, Baltar is shown Christ-like, in poses like Christ, with hair and beard like him. As the final Cylon, he may be their version of Christ, somehow incarnated from the Cylon God. On the base ship, he gets a very Christ-like wound.
• Inner six keeps insisting that Hera is the child of her and Baltar. Ravings? If not, it represents something like this.
• Cylons keep falling in love with him, and never kill him. In fact, he drives them crazy. The Cylons who have close contact with him are the ones who rebel, and break their compulsion not to seek the Final Five. He has sex with Tory, #6 and #3 (and of course many human women too.)
• As noted by many, he is very close to a nuclear blast at the start of the show, and while six is killed shielding him, later he shows up to get on Helo’s raptor with just a few minor scrapes.
• In season 4, he gets a religious following among the colonials, and gets thought of as a healer. He may perform miracles. He seems to be chanelling real external information about the river.
• He’s involved in everything. AI research. He’s in the middle, if not consciously, of the Cylon sabotage on Caprica. He is the one meting out clues to lead the colonials to Earth, and providing similar clues to the Cylons. Both are on their courses because of him.

Unfortunately, he’s in the “last supper” picture and it is confirmed that the final Cylon is not in that picture. Unless it’s secretly “head Baltar” in the picture, he’s out.

None of this is conclusive, but none of the other characters have nearly as many clues like this. The prophecy of the Cylon God bumps him up several notches as well.

More reflection on Razor has led to more thinking about the Cylon god, who I believe is closely tied to the prototype Hybrid we saw in Razor. Indeed that Hybrid may well be an incarnation or copy of the Cylon god. I posted a few days ago about his Starbuck prophecy but decided it was time to detail a bit more of the thinking about this very important character.

Update: the writer’s meeting podcast suggests that the Hybrid is not the Cylon god, but is in constant communication with him, and is worshipped as a god by the Guardians. Guess I didn’t get that one quite right.

For much of the show, the Cylons have spoken of their god the way Christians speak of theirs. They are monotheists, while the colonials are polytheists. The Cylons acknowledge the reality of the Lords of Kobol, but state that the colonials don’t know the real truth about them, and that they are false gods. But like the Lords of Kobol, the Cylon god may be a being with a real physical existence. Not so much a “God” like the one of the New Testament, but a “god” — a super-intelligent, super-powerful being who was involved in the creation of the Cylons, and perhaps more. However, this god might still be subject to the laws of the universe, and not supernatural as a typical religious god is. Science Fiction has often included natural gods. I particularly enjoyed the term Vernor Vinge used in A Fire Upon the Deep — “Applied theology.” In this novel, the “gods” were beings so smart they could understand a human mind the way we understand a calculator — able to build it, predict what it will do, rebuild it, invent it from scratch. Very much as we have thought of gods, but not supernatural.

The Cylon god (or something acting in that role) is certainly real, whether he’s supernatural or physical. There are various clues about that…  read more »

Computerworld has been nice enough to include me in their series on unsung innovators of the net. I should point out that I try to downplay the dot thing — to me it’s an amusing anecdote of having participated in the right mailing lists at the right time. I remain much more interested in whatever I will do next!

All over the net, a huge number of sites offer you the option of E-mailing you your password if you have forgotten it. While this seems to make sense, it is actually a dreadful security policy, and if you see it, you should complain and point them to this article or others to get them to stop. As an alternate, they should at most offer to E-mail you a new, randomly chosen temporary password, which you can use to log in and set a more memorable password.

If a site can mail you your password, it means they are keeping a copy of it. They should not be doing that. First of all, almost everybody re-uses passwords at different sites. That means if one site has a security breach — as Convio did this week for a wide variety of sites that are its clients — your password will be stolen, and it can then be used on all the other sites you use it at. (This is a good reason to always use more protected, less duplicated passwords on sites where actual damage can be done or money can be spent, like banks, eBay, paypal etc.)

Instead, they should keep a “hash” of your password. A hash is a one way function. Given the plain password, they can hash it, and store the result, but you can’t get the plain password back from the hash. So you can check to see if a password that was typed matches the password without storing what the password is. This is actually a very easy thing to do in most systems, and its main downside is the fact that they can no longer e-mail you your password. They can, however, set it to something random and mail you that. That’s a touch more work in the rare event of a lost password, but worth the trouble.

There is, oddly, one minor downside to hashed passwords. With hashed passwords, you must provide the site your real password, and they can then test it and forget it. You must trust them to forget it. The real password, however, is sent over the internet and if you don’t use an encrypted channel, like SSL/TLS/https, it could be intercepted by people tapping the line. Some password systems (included the less commonly used HTTP password system) have the browser hash the password (in a special way that is different every time) and send the hash to log in. In this case, the real password is not sent, and can’t be sniffed, but must be in storage at the remote site. However, if you use an encrypted channel (https), there is no worry about the password going over the internet, and so there’s no reason not to do it that way.

There is a better way to do all of this. With digital signature, you can prove that you’re you using a secret private key only you know. Nobody else ever gets this key, and nobody can figure it out by watching the communications you send. While this technology has been around for some time, and is in fact implemented in most browsers (though far from perfectly) it is not a common way to authenticate to web sites at all.

However, next time a site offers to E-mail your password, point them to the Convio data theft and to this page and ask them to get their act together.