Update: I now have a whole Burning Man area on the blog!

I’ve not been blogging of late because I’m at Burning Man, and while normally I don’t report breaking news in this blog, we just witnessed a strange event. Through accident or arson, the Man was set alight this evening shortly after totality began in the eclipse of the moon.

The man was not loaded with explosives or fireworks as he is before his planned burn, so it was a more sedate affair, and soon fire crews arrived to “save the man” — something we have been asking for in mock protests for years. They did put him out, and he still stands, a bit worse for wear.

I managed to get some photos of the burn….

Efforts to save the man…

The injured man, missing a hand and burnt, under the eclipsed moon…

Here are three events coming up that I will be involved with.

Burning Man of course starts next weekend and consumes much of my time. While I’m not doing any bold new art project this year, maintaining my 3 main ones is plenty of work, as is the foolishly taken on job of village organizer and power grid coordinator. I must admit I often look back fondly on my first Burning Man, where we just arrived and were effectively spectators. But you only get to do that once.

Right after Burning Man, the Singularity Institute is hosting a Singularity Summit — a futurist conference with a good rack of speakers. Last year they did it as a free event at Stanford and got a giant crowd (because it was free there were no-shows, however, making it sad that some were turned away.) This year there is a small fee, and it’s at the Palace of Fine Arts in San Francisco.

On the first weekend of November, we at the Foresight Institute will host our 2007 Vision Weekend doing half of it in “unconference” style — much more ad-hoc. It will be at Yahoo HQ in Sunnyvale, thanks to their generous sponsorship. More details on that to come.

As I noted earlier, my web site got hacked. As a result, I decided to leave my old hosting company, PowerVPS.com, and find a new host. While another VPS would probably have managed, I know a woman in San Jose who runs a hosting company, simpli.biz, who offered me a good deal on a fast dedicated server. I’ll grow into it, and in the meantime you should see much greater performance from my site.

I will make some final commentary on PowerVPS. I left for a variety of reasons, and they were certainly not 100% bad.

• They were on the other coast, so my ping times to them were 80ms or so. This was no fun for ssh and would have made running things on them impractical. I was surprised that most of the virtual hosting companies with good reputations and prices were not on the west coast.
• At first I looked for hosting in Canada. This was not simply because I was a Canadian. I thought it might be good to get hosting (in Vancouver) that was not subject to U.S. law. Not because I intend to break U.S. law, but being at the EFF we’ve been fighting some of these laws and it would be good to be on another level. And I’m Canadian. However, all the hosting offerings in Canada I tried that matched my parameters were much more expensive.
• VPSs are in general a great idea. However, it’s hard to make them swap. That means each VPS duplicates in RAM a copy of apache and mysql and the rest, which is wasteful. Dedicated servers, which swap, allow the big programs that have a lot of pages which are rarely used to swap them out to disk, while the active programs get use of all of the ram. You can’t overdo this, but it’s pretty handy. One VPS provider, Iron Mountain, does what I have been advocating — gives users access to a virtualized MySQL server on a fast machine, so you don’t have to run your own. Doing this is rare.
• They would not support Ubuntu, only Centos. I am running Ubuntu on almost all my machines. I really like the idea that I can just duplicate efforts onto my hosting server, with now learning how to do things in a different distro. And that I can compile stuff at home and just move it to the web host. CentOS is the most popular distro in the hosting world, and people have done a lot of fancy things for it (control panels, automated installs etc.) and I understand why a company will decide to only support one distro. But that just means I go to a company that picked the distro I want.
• PowerVPS screwed up when most of their customers got hacked. The hack wasn’t their fault, as far as I know, but once they realized so many of their customers were compromised, they should have E-mailed all of us immediately. Because they didn’t, I only noticed the attack when they broke some of my scripts. My site redirected unsuspecting users to a frame which might have infected them, which I regret. I should have been told about this as soon as possible.
• The kicker: When I told them I wanted to replace my server after the hack, they said I had two options. I could back up the server (many gigs of data) and they would erase it and give me a new one with a fresh Centos 4. Then I could restore the files and rebuild everything, being down during the period I did this. Or I could buy a new server, transfer, and then move the DNS or the IP as desired. They would not temporarily give me the 2nd server, and then delete the old when I was ready. They said too many people took too long, and freaked out if deleted. Being forced to buy a new server simply sent me on a shopping trip. Stupid, stupid, stupid. Why send your customers on a shopping trip?
• Another sin: When I went shopping, I looked at the list of special coupon offers various competitors offered. There I saw PowerVPS selling the same server I was paying $85 for for 30% off, lifetime discount. Be very careful when you offer new customers a much better price than existing customers get. I hate it, and I will leave you for it.

Now as I say, it was not all bad. Their support was good, and during the recent episode where I was on the digg.com homepage, they temporarily upgraded my VPS capacity — which is one of the prime things a VPS can do that a dedicated server can’t. I liked those things but the above mistakes lost a customer.

Let me know if you encounter any problems with the server move.

Updated note: After you change a server’s IP, all users should switch to a new IP after the “time to live” on the past lookup expires, which in my case was set to about 3 hours. However, turns out many people have broken (or deliberately broken) software that retains stale records for much longer. The leading culprit right now are web spiders, including googlebot, which continue to hit the old address. Actual users doing so are rare. For E-mail, a previous move found that spammers continued to use the old addresses for months after the fact. They presumably kept DNS lookup data on their CD-ROMs, or didn’t want to be subject to attempts to use DNS to block them, or had some other reason.

It was an interesting experience watching our team argue before the U.S. District Court of Appeals that the EFF’s lawsuit against AT&T for helping the NSA spy on conversations without warrants should be dismissed because it impinges on state secrets. While the judges probed both sides, I read some signs from their grilling of the U.S. Government’s lawyer that they really have some concern over the important issues. They appear to realize that we can’t leave such programs completely without judicial oversight just because an NSA official declares them to be state secrets.

As one judge said, “are we supposed to bow down” before such declarations?

Anyway, this inspired me to make up a new list of all the different classifications for secret information:

1. Unclassified (Ordinary documents)
2. Sensitive (to delay FOIA)
3. Double Super Secret (For Time Magazine Only)
4. Treated as Top Secret (Non-secret document from Vice President’s Office)
5. Leakable (Identity of covert agents married to those causing political trouble)
6. Secret
7. Top Secret
8. SCI (Sensitive Compartmented Information)
9. Embarrassing (Highest possible classification)

I get forms to fill out and sign in electronic form all the time now. Often they come as PDFs or word documents, every so often by fax, and more and more rarely on paper. My handwriting is terrible and of course I no longer have a working typewriter. But none of the various tools I have seen for the job have had a nice easy workflow.

Now some PDFs are built as forms, and in Acrobat and a few other programs you can fill out the form fairly nicely. However, it’s actually fairly rare for people to build their PDFs as fillable forms. When they do, the basic Acrobat tools generate a form which free Acrobat reader will let you fill out — but bars you from saving the form you filled it out. You can only print it! Adobe charges more, on a per form basis, to make savable forms. However, some other readers, like Foxit Reader, will let you save what you fill into forms, even if the creator didn’t pay Adobe.

You still can’t sign such forms in electronic fashion, however. And as noted, many forms of all types aren’t enabled this way. Forms that come as Microsoft Word documents can be filled out in MS Word or the free Open Office writer or abiword. And you can even insert a graphic of a signature, which gets you closer to the target.

Often however, you are relegated to taking a fax, scanned paper document or PDF converted to bitmap, and editing it in a bitmap editor. Unfortunately the major bitmap editors, like Photoshop or GIMP, tend to be aimed entirely at fancy text and they are dreadful and entering a lot of text on a form. They don’t even make it so easy as quickly clicking and typing.

I encountered a commercial package named “Form Pilot” which is for Windows only but appears to run on WINE. It’s better than the graphics editors, and it does let you click and type easily. However, it has some distance to go. Here’s what I want:

• Be smart and identify the white spaces on the form, and notably the lines or boxes. Figure a good type size if the default isn’t right.
• When I click in one of those boxes, or above a line, automatically put me at a nice position above the line for typing. This is not a hard problem, hardly even OCR, just finding borders and lines. Let me use a different click if I want to do precise manual positioning.
• When I hit TAB or some similar key, advance to the next such box or line in the form.
• If I type too much in a box, do an automating shrinking of the text so that it fits.
• Of course, let me go back and edit my text, and save the document with the text as a different layer so I can go back and change things.

Signing

Now the interesting issue of signing. For this, I would want to scan in a sheet of paper which I have placed many signatures on, and have it isolate and store them as a library of signatures.

When I wish to apply a signature, have it pick a random one. In addition, have it make some minor modifications to the signature. Modifications could include removing or adding a pixel here or there along the lines, or adjusting the aspect ratio of the signature slightly. Change the colour of the ink or thickness. There are many modifications which could generate thousands of unique signature forms. If you run out, scan another sheet.

Then make a log of the document I signed and the parameters of the signature that was added, and record that. All this is to assure the user that people who get the document can’t take the signature and copy it again to use on a different document and claim you signed it. You’ll have a log, if you want it, of just what documents were signed. Even without the log you can have assurance of uniqueness and can refute fake signatures easily.

(Refuting forged signatures is actually pretty easy on electronic documents.)

When done, let me save the document or print it, or hook up with a service so that I can easily fax it. The result should be a process of receiving a document or form, filling it out and signing it and sending it back (by fax or email of course) that’s even easier than the original method on paper.

I was surprised, by the way, at how bad all the free bitmap painters I tried were at typing. Gimp and Krita are poor. xpaint and kolourpaint seemed to have the easiest flow even though they are much older and primitive in UI. If you know of programs that do this well, let me know.

As workers search for trapped miners in Utah, having drilled a 9” hole down to what is hoped to be their area, they plan to use things like sound and detecting CO2 and O2 in the atmosphere to find the miners.

It occurs to me that it should be possible to fit one of those inflatable radio controlled blimps down such a small tube, inflating it after it gets to the bottom. There are models that support small video cameras (and LED lights would not be too hard) especially in the denser air at the bottom of a mine. You would send down a radio relay station as well, and if things were really fancy, a way for the blimp to be told to dock for recharge or exchange of battery packs. (Small butane motors might also provide better power for weight.)

It’s also possible that power could be provided by paying out a wire, if it could generate enough thrust to drag that wire. There is a high risk the wire could get caught except on smooth floors, though. One might imagine paying out wire as far as one can go, and then disconnecting, fully charged, for a modest time on internal power. These blimps are cheap, you could send down several. They could easily sail over debris a ground based robot could not handle, though they could not crawl through small holes without deflating.

Another option would be an enclosed fan hovering robot. Such a robot would be able to go through smaller holes, though it’s hard to imagine remote pilots good enough to send them through such channels with only a video camera to see by. In the future, we may well have hovering robots able to use sonar to keep themselves stable and away from obstacles. They would go on ground when they could, then use bursts of hover to get over obstacles. But the blimp is something that could certainly work in ordinary mine channels today, though only for a limited battery life.

Twenty years ago Tuesday, I created the newsgroup rec.humor.funny as a moderated place for posting the funniest jokes on the net, as chosen by the editor. In light of that anniversary, I have written up a bit of history of the creation of RHF. From there you can also find links to pieces I wrote earlier about the attempt to ban RHF and how RHF led to my creation of ClariNet.

One reason people may pay a bit more attention to this anniversary is I think that RHF, with its associated web site has a claim at being the world’s longest still-running “blog.” Of course, there is much debate about the origins of blogging, and there are various contenders based on what definition you put to the word.

I provide more detailed examination of those definitional questions and the other contenders on a page about the world’s oldest blog. In short, I contend that a blog is something that is:

• Serial (a series of publications over time)
• Done with a personal editorial voice (rather than being news reporting)
• On the world wide web

While most agree with that last point (since personal journals, published diaries and columns existed long before computers) many forget that when Tim Berners-Lee defined what the web was, he was very explicit about including the many media and protocols he was tying together with HTML and HTTP, including USENET, Gopher, E-mail and the rest. So the web dates back well before HTML, and so does the weblog.

I personally point to mod.ber, a short-lived moderated newsgroup from 1983 as the first blog. It was clearly the boing-boing of its day. But it doesn’t exist, so RHF may get to claim the title.

As you will know if you have followed RHF, while I continue to publish it and provide the software and systems, I only edited it for the first 5 or so years. After that Maddi Hausmann took over, and in 1995, Jim Griffith took the reigns to this day. He, however, is ready to retire shortly and we’re looking for a replacement — a note will be posted in RHF and here with more details after the anniversary.

As you’ll see in the histories, the decision to start RHF changed my life in sweeping ways. It was one of those junctures that Clarence from “It’s a wonderful life” could change if he wanted to show me a different path.

Happy 20th Birthday rec.humor.funny.

A few weeks ago, my site got hacked. The attacker inserted an iframe pointing to a malware site into most of my html pages. That of course is bad, but the story doesn’t end there. (I should of course have upgraded my OS from the ancient one my hosting company gave years ago, but they don’t really support that, and feel an upgrade consists of rebuilding from scratch.)

I cleaned out the entire site and searched for any remnants of the bad link. Having done this I thought all was well. However, as it turns out while the ideas.4brad.com domain and other domains were clear, the 4brad.com domain, which I don’t use for anything, still had a web server on it, pointing at a different directory far from where I keep my own web sites. (I try to never put my stuff in system directories.)

Unfortunately google, for unknown reasons, looked at 4brad.com, even though there are no links to it anywhere on the web. And found the placeholder page, with hacked link in it. From there it declared the entire site, including ideas.4brad.com, to be a malware site. I think that’s a bug, since there were never any malware links on ideas.4brad.com pages — this is a drupal site, and while the hacker’s script attempts to modify PHP scripts, it did not do so correctly, and just broke them. Running linux, I didn’t see the malware hacks on the other sites where they made the changes, but found them soon enough and removed them for now.

Alas, that means for some time people have been directed away from this blog by google. It shows up in search results, but you can’t actually click on the results, and there are warnings that going to the site may harm your computer (you get these warnings even on non-windows computers, which is reasonable, I guess, if incorrect.) I’ve asked the site stopbadware.org, which Google teams with, to confirm the hacks are gone, and now I have to rush out to rebuild the site from a fresh install. Sigh.

Update: Google reacted to the cleanup of 4brad.com very quickly and no longer lists the domain as unsafe. I did file a review request with stopbadware.org — perhaps they are much faster than they let on.

I’m shopping for hosting. I think I will upgrade to dedicated hosting, even though virtualized hosting has its merits. As I wrote before it would be great if MySQL could be virtualized independently of the OS. The ideal marriage would be a virtualized linux with access to sharable, non-virtualized services like web serving and database. The trick is memory. A typical virtual host will have 16 copies of MySQL and 16 copies of Apache and 16 copies of PHP or similar running on it. Because virtual machines don’t truly understand how much memory they have, or see the paging of the underlying OS, they can’t manage memory as well. But their ability to burst in unused capacity is a big win.

I’m a big fan of making money by selling services but a disturbing trend is the requirement that customers sign a one or two (or even three) year contract in order to sign up for a service. Such contracts will have a fat termination fee if you want to end the contract early.

This is almost universal for cell phones, and of course it makes some sense when they are selling/giving you a subsidized phone. They need to be sure you will stay with them long enough to make the subsidy (From $200 to $400 if you include dealer kickbacks) back. That’s not so hard, because with many people getting cell phone plans as high as $100/month, they make it back quickly.

However, cell phone companies notoriously require a new contract for just about any change in your calling plan, including simply switching to a new plan they just started offering that you like better. Usually that’s just a one year contract. This makes much less sense. Switching your plan doesn’t cost them anything much aside from a call to customer service. They just want to put you on that contract.

DSL ISPs (and not just the phone company ones) are also notorious here. Some need it to subsidize installation or equipment, but again it’s also done simply to change price plans. In many cases you will also see major discounts offered if you commit to a contract (or of course even better if you just pay 12 months at once.)

I understand the attraction of the company for contracts. They can predict and book revenue. Quantity discounts have always had their reasons.

But they may not realize a serious negative about the contracts. They are a barrier to getting customers. In particular, a demand for a contract (when there is no major subsidy) says to me we think that without a contract, we could lose you as a customer. We fear that, if not for the contract, you would leave us. And that immediately makes me think the same thing. “What is it that makes them think they can’t keep me just by providing good service at good prices?” They already won my business, which is the hardest part. Now all they have to do is keep me happy and they will be very likely to keep it.

This recently backfired for Verizon. I’ve been off contract with them for years, though I had often debated switching to a different plan. Every time they told me I would need to sign a one year contract, and get no subsidy for doing so. (For a 2 year contract, they would have subsidized a new phone, but I wasn’t ready to do that.) So when phones broke I often picked them up on eBay rather than take their 2 year subsidy.

When it came time to really want to change plans, their demand for a new contract made them the same as all their competitors, who will also demand a new contract. And thus there was no particular reason not to switch. They encouraged me to compare all the various offers, all of which require a new contract, and all of which can offer me a phone subsidy with a 2 year contract. And all of which can keep the number, thanks to hard-won number portability. Had they been willing to let me make changes without a contract, I would have had no incentive to go shopping around at the competition. There I learned about much better deals they had, and thus left Verizon.

Perhaps they think they need a contract to keep me from the competition. But truth is, that might work temporarily but it just delays things. When a contract expires, somebody is going to be ahead, be it the competition or be it them, and they just moved the switch in time and probably locked me into the competition for their efforts.

The best company in the business shouldn’t need a contract to hold me. If the competition is offering a snazzy new subsidized phone for a contract, then my no-contract company can certainly offer that. Or, ideally, just offer me a lower monthly rate if I bring my own phone, with no need for a contract — my choice.

Over time, the public might wake up to realize that the contract is much more expensive than the phone subsidy. A typical data phone requires a plan of $60 to $80 per month, and many are on plans of $100 or more. That’s a $2400 purchase at $100/month, all to get a $200 phone subsidy. Of course most customers plan to buy from somebody over the period, so it makes sense to take the subsidy if you aren’t likely to be changing all the time, which most of us aren’t. But I am curious why all the firms feel these contracts are really in their interest.

Update: I should point out that there are reasons to get warmer to a contract when getting a new phone. Typically there is a $200 subsidy on the phone, and sometimes much more. And quite commonly, the penalty for getting out of the contract is $200, and in fact my law reduces on a pro-rata basis as you move through the life of the contract. As such, there is no reason not to sign the contract if you want that brand-new phone. In addition, there are contract trading sites (where other people will take over your contract for less than the penalty price because they don’t need a phone) to get out even cheaper.

However, you don’t want a contract without this level of quid pro quo. A contract just to change plans is ridiculous. Some carriers are getting that message.