Announcements

Battlestar Galactica sub-blog returns to activity

Some of you may know that I started a sub-blog for my thoughts on my favourite SF TV show, Battlestar Galactica. This sub-blog was dormant while the show was off the air, but it’s started up again with new analysis as the first new episode of the final 10 (or 12) episodes airs tonight. (I will be missing watching it near-live as I will be giving a talk tonight on Robocars at the Future Salon in Palo Alto.) Reports are that one big mystery — the last Cylon — is revealed tonight.

So if you watch Battlestar Galactica, you may want to subscribe to the feed for the Battlestar Galactica Analysys Bog right here on this site. And I’ll go out on a limb and promote my two top candidates for the mystery Cylon.

Some recent posts of note:

Robocar talks at Future Salon, BIL

I’ll be giving a talk on Robocars on Friday, January 16th at the Bay Area Future Salon which is hosted at SAP, 3410 Hillview, Building D, Palo Alto CA. Follow the link for more details and RSVP information. Reception at 6, talks at 7. Eric Boyd will also talk on efficiency of transportation.

While I gave an early version of the Robocar talk at BIL (the unconference that parallels TED) last year, I think I will do an update there as well, along with a talk on the evils of cloud computing.

EFF Year in Review in Music

It’s been a remarkably dramatic year at the EFF. We worked in a huge number of areas, acting on or participating in a lot of cases. The most famous is our ongoing battle over the warrantless wiretapping scandal, where we sued AT&T for helping the White House. As you probably know, we certainly got their attention, to the point that President Bush got the congress to pass a law granting immunity to the phone companies. We lost that battle, but our case still continues, as we’re pushing to get that immunity declared unconstitutional.

We also opened a second front, based on the immunity. After all, if the phone companies can now use the excuse “we were only following orders they promised were legal” then the people who promised it was legal are culpable if it actually wasn’t. So we’ve sued the President, VP and several others over that. We’ll keep fighting.

But this was just one of many cases. The team made up a little musical animation to summarize them for you. I include it here, but encourage you to follow the link to the site and see what else we did this year. I want you to be impressed, because these are tough-times, and that also makes it tough for non-profits trying to raise money. I know most of you have wounded stock portfolios and are cutting back.

But I’m going to ask you not to cut back to zero. It’s not that bad. If you can’t give what you normally would like to give to make all this good work happen, decide some appropriate fraction and give it. Or if you are one of the few who is still flush, you may want to consider giving more to your favourite charities this year, to make up for how they’re hurting in regular donations.

The work the EFF does needs to be done. You need it to be done. You have a duty to protect your rights and the rights of others. If you can’t do the work to protect them yourself, I suggest you outsource it to the EFF. We’re really good at it, and work cheap. You’ll be glad you did.


Learn more about this video and support EFF!

Plus, we have cool T-shirts and shipping tape.

PEW study on the future of the internet

PEW Research has released their recent study on the future of the internet and technology where they interviewed a wide range of technologists and futurists, including yours truly. It’s fairly long, and the diverse opinions are perhaps too wide to be synthesized, but there is definitely some interesting stuff in there.

Comment spammers getting smarter -- user spam

Two disturbing trends are moving upwards in the area of blog comment spam.

You may want to note that I have changed the challenge question for posting comments on this blog. It is no longer my last name.

The first has been taking place for a while — it’s hand-written comment spam. Spammers are paying people, probably low-wage people in 3rd world countries, to write comments on blog posts that are very roughly on-topic. Then those comments will contain a link to the spammer’s site, with the keywords the spammer wants. Sometimes the link will just be on the userid.

The spammers do this even though I tell them that all links in comments get the “nofollow” tag which makes Google and other search engines ignore them and not assign rank to them. They are thus wasting their time, other than to get a few clickthroughs from readers here. The people they hire are smart enough to pass the Turing test and write a comment that is roughly on topic, but they either don’t understand the nofollow warning or don’t worry about it because they are paid by the comment.

Truth be known they don’t write very good comments. Any real examination will show they are not really appropriate. And more to the point, unlike the majority of comments, they have links, and of course those links are to commercial sites. Just the existence of links is enough to make the comment worthy of examination. And I now have spam filters that put posts with possible bad links into an approval queue rather that doing immediate posting, unfortunately.

Today I discovered a new type of spam on the blog. A spammer was creating userids, but not posting any comments. They just put a link to their spam pages in their user description. Userid creation does require a challenge question but at least one spammer wrote code to fill it in, since I don’t change the question every time as perhaps I should.

The userids would have names like “Brittney nude” and thus they show up in the blog user directory and are parsed by search engines. Since my pagerank is high, people are finding these userid pages for searches, and then perhaps following links to the spammers.

Mostly I want my challenge to be very simple to make it as easy as possible to participate. I don’t like image captchas, I find them a pain when I go to other sites. And most of them have been broken on the big, high-value sites. They probably would not get broken for a smaller site like mine. Other options include simple math problems (but those may get broken by code as well.)

My general rule has been that unless you are a high-value target (and perhaps I’m going up in value) you should not have to do very much. The key is not not be the same as other sites, and to not do anything like use a standard module for drupal so you are the same as all other drupal sites. As a collection, drupal sites are a high value target.

I deleted the users of course, but the interesting trick here was that since they did not post, I only noticed them by seeing referer logs coming from search engines.

Update: They are keeping at it, so I decided to put user creation on administrator approval. Truth is, not very many readers here create accounts, and there are only minor reasons to do so. If you create an account it takes away the “Not Verified” after your name and you don’t have to enter any parameters again. You can also edit and remove your comments after the fact if you post them with an account.

Upcoming: Burning Man Decompression, ENIAC Talk, Convergence, eComm

Some upcoming events I will be involved in:

Burning Man Decompression, Sunday Oct 12

As I have for the past several years, I will show off my newest giant photographs of Burning Man at the “decompression” party, which takes place from noon to midnight on Sunday, Oct 12 (this coming Sunday) on Indiana St. south of Mariposa in San Francisco.

While decompression won’t get you to understand what Burning Man truly is, it’s the closest you’ll come while staying in the city. Come by, I will be easy to find with the giant photo wall. Come in “playa wear,” which means anything out of the ordinary, to get in for only $10.

Meet Jean Bartik, one of the world’s first programmers

The world’s first software team was a group of six women recruited to program the ENIAC. Jean Bartik was one of those six, and is giving a talk at the Computer History Museum in Mountain View on October 22 at 7pm. Prior to the talk, I am helping with a special VIP reception where you can meet Bartik, and see clips of a documentary-in-progress being done about the six earliest programmers. The producer is my friend Kathy Kleiman who needs financial contributions to complete the documentary. Unfortunately 3 of the women are now gone, but video interviews were made with them for this documentary.

If you would like to attend the VIP reception, send me a note.

Convergence conference on the Future

Foresight Institute, of which I am a director, is one of the organizations sponsoring Convergence 08 a futurist gathering with both scheduled debates on issues in AI, synthetic biology and longevity. Then there’s an unconference component where the attendees make the program. I’ll deliver my robot car talk, with video. This takes place the weekend of November 15, and Foresight Institute Senior Associates are also all invited.

On a side note, while I won’t be there as I will be at Alternative Party in Finland, on Oct 25, Futurists can also attend (at a higher price) this year’s Singularity Summit in San Jose.

eComm

Further in the calendar, check out eComm a conference on emerging telephony. This conference took up the mantle from the O’Reilly conference on the same topic, and now takes the mantle of the recently deceased VON conference. To find out what’s happening in VoIP and not-plain-old-telephone-service, check it out in early March of 09. I’ll be speaking on the EFF’s battle with AT&T over wiretapping and what it means for the new generation of telephony.

Trip to Helsinki, Stockholm, St. Petersburg for Alternative Party

Coming up in a couple of weeks I will be speaking as a special guest at Alternative Party, a digital culture conference in Helsinki, Finland. I’ll be doing my main talk on October 25th plus an extra session on either the 24th or 26th depending on schedules.

After that I will head to do some touristing in Stockholm for a few days, then for my first trip to Russia to visit St. Petersburg on the 31st.

Have some recommendations in Stockholm or St. Petersburg area? Let me know. My hosts will take care of me in Finland.

Pre-Canada Day Bring your own Fireworks Party

On short notice, we’ll be having a pre-Canada Day “BYOF” (Bring your own Fireworks) party at our Pacifica place on Sunday, June 29.

Because Pacifica has so much fog to keep it moist, they allow “Safe and Sane” fireworks at private homes, and on the beach. On July 4, many hundreds will crowd the beaches, and it’s quite a sight, but we’re going to cheat and hold it on Sunday, 2 days before Canada Day (July 1.)

We’ll gather and socialize starting about 2:30pm, order in some Pizza or Chinese and other snacks around 6:30, watch the sun set over the ocean at 8:30 and set off the fireworks sometime after 9 pm.

Ok, I’ll admit it. “Safe and Sane” fireworks are sometimes a bit underwhelming. They don’t look like the picture. But it can still be fun. On the 4th, people try to set off bigger fireworks, and they run away and the police try to get them. Cops even set up a holding pen on the beach. We won’t get that, though.

About a BYOF: Key to a good BYOF is not having too many of those “combo boxes” which have a lot of little fireworks. There are only so many different ones and you end up with a lot of repeats. We don’t ask folks to spend much money, but it’s better to spend it all on one firework (or something that’s fun when repeating like sparklers) than on a combo.

There are fireworks vendors all over Pacifica, including 2 that have set up just a short walk from the house, near Manor and Highway 1.

If the crowd is small, we’ll set off at the house. If larger, we’ll do and expedition to the beach, which may involve carpools. (There is a walkable beach, but the cops want people to go to Rockway and Linda Mar, at least on July 4.)

Weather: Pleasant. The fog is just burning off now at 2:30! This webcam will show you. Also, after about 3:30pm try the partycam which is also capturing a time-lapse movie of the day.

Traffic: Do note that due to the Gay Pride parade in the morning, there will be extra traffic in downtown SF and heavier use of transit.

The party is at 231 Manor Dr. (& Perry), in Pacifica CA. Here are maps and directions. Street Parking is plentiful.

Note: This is a vacant home recently renovated by Kathryn’s mom for eventual sale. Treat it nice please! Not much furniture, which is good for a party.

Check this page before the party for any updates. Dress Canadian (ie. a cold day at the beach.) If we go to the beach, I’ll update here, and can you call my cell (408 313 BRAD) after 8:30 for details if you are not on the web.

RSVP

If you would like in on dinner food, RSVP is appreciated. It may be BYOF, but I will provide snacks and non-alcoholic drinks. You can BYOB for Alcohol. You can RSVP in one of the following ways, but please pick only one:

  • On Facebook, if you are a member at this event page
  • Via E-mail to party at mail.4brad.com
  • In the comments here, if you like

Reminder, get your credit card foreign exchange settlement tonight

Just a reminder, if you purchased things outside the USA with credit cards or used foreign ATMs the companies gouged you on exchange rates, and lost a class action case. You can go to the CCF Settlement page to fill out the form tonight, just a few hours left. Your options are:

  • Just get a plain $25
  • Report how many days you were outside the USA from 1996 to 2006 (that’s 216 days for me.) I’m guessing you might get back $2/day or so, but nobody knows.
  • Actually calculate all your foreign transactions — only easy if you have an accounting system which would make that easy to do. Get 1% to 3% of them back.

Most people are going with #2 because of how much work #3 will be. This is typical in class actions. Clearly the credit card companies know exactly how many transactions they charged you foreign exchange on, and could calculate this for you, but they arranged a settlement that worked the other way. The lawyers get their fees, though.

(I should note that the EFF has done one class action and is doing another, and learned how hard it is to get something that’s really good for the plaintiffs. However, as a civil rights foundation, we really are highly interested in the punitive nature of these cases, and any fees we get are plowed right back into more civil rights work.)

You probably got a settlement form in the mail. If you kept it, it has a number that you can key in to make this very easy. If you didn’t, you may have to disclose some info and might decide not to do so. For me, they already had most of my info from the CC databases, I just entered my refund ID #, my days outside the USA, and some questions the typical purposes of my trips. Much easier than is typical.

Charles Templeton gets own mini-room in Creation Museum

I learned today that there is an exhibit about my father in the famous creation museum near Cincinnati. This museum is a multi-million dollar project set up by creationists as a pro-bible “natural history” museum that shows dinosaurs on Noah’s Ark, and how the flood carved the Grand Canyon and much more. It’s all completely bullocks and a number of satirical articles about it have been written, including the account by SF writer John Scalzi.

While almost all this museum is about desperate attempts to make the creation story sound like natural history, it also has the “Biblical Authority Room.” This room features my father, Charles Templeton in two sections. It begins with this display on bible enemies which tells the story of how he went to Princeton seminary and lost his faith. (Warning: Too much education will kill your religion.)

However, around the corner is an amazing giant alcove. It shows a large mural of photos and news stories about my father as a preacher and later. On the next wall is an image of a man (clearly meant to be him though the museum denied it) digging a grave with the tombstone “God is Dead.” There are various other tombstones around for “Truth,” “God’s Word” and “Genesis.” There is also another image of the mural showing it a bit more fully.

Next to the painting is a small brick alcove which for the life of me looks like a shrine.

In it is a copy of his book Farewell to God along with a metal plaque with a quote from the book about how reality is inconsistent with the creation story. (You can click on the photo, courtesy Andrew Arensburger, to see a larger size and read the inscription.)

I had heard about this museum for some time, and even contemplated visiting it the next time I was in the area, though part of me doesn’t want to give them $20. However now I have to go. But I remain perplexed that he gets such a large exibit, along with the likes of Darwin, Scopes and Luther. Today, after all, only older people know of his religious career, though at his peak he was one of the most well known figures of the field. He and his best friend, Billy Graham, were taking the evangelism world by storm, and until he pulled out, many people would have bet that he, rather than Graham, would become the great star. You can read his memoir here online.

But again, this is all long ago, and a career long left behind. But there may be an explanation, based on what he told me when he was alive.

Among many fundamentalists, there is a doctrine of “Once Saved, Always Saved.” What this means is that once Jesus has entered you and become your personal saviour, he would never, ever desert you. It is impossible for somebody who was saved to fall. This makes apostacy a dreadful sin for it creates a giant contradiction. For many, the only way to reconcile this is to decide that he never was truly saved after all. That it was all fake. Only somebody who never really believed could fall.

Except that’s not the case here. He had the classic “religious experience” conversion, as detailed in his memoir. He was fully taken up with it. And more to the point, unlike most, when much later he truly came to have doubts, he debated them openly with his friends, like Graham. And finally decided that he couldn’t preach any more after decades of doing so, giving up fame and a successful career with no new prospects. He couldn’t do it because he could not feel honest preaching to people when he had become less sure himself. Not the act of somebody who was faking it all along.

However, this exhibit in the museum doesn’t try to paint it that way. Rather, it seems to be a warning that too much education by godless scientists can hurt your faith.

So there may be a second explanation. As a big-time preacher, with revival meetings filling sporting arenas, my father converted a lot of people to Christianity. He was one of the founders of Youth for Christ International, which is today still a major religious organization. I meet these converts from time to time. I can see how, if you came to your conversion through him, my father’s renunciation of it must be very hurtful — especially when combined with the once-saved-always-saved doctrine. So I have to wonder if somebody at the Creation Museum isn’t one of his converts, and thus wanted to tell the story of a man that many of the visitors to the museum will have forgotten.

Here are some other Charles Templeton links on my site:

Right now I’m in the process of scanning some of his books and will post when I have done this.

William C. Tate 1925-2008

With sadness I must report the passing of William C. Tate, my stepfather, on Thursday. Bill and my mother met and fell in love when I was a young teenager. He was a neighbour, and I had met him, and even stayed over at his house with his son before they would meet, which is a bit unusual. He was kind and generous and supported her and our family for many decades. While he died from cancer, it came upon him quite suddenly and he was fortunately strong until near the end.

Bill was a leader in Toronto’s business community. He started out in the finance dept. of the small Canadian office of Garret Corporation, a major aerospace manufacturer. Quickly he was put in charge, and built it up to a 1,400 person operation making a variety of important components for civil and military aircraft.

I have created a site with a slight longer obituary and a link to a comment/guestbook thread on this blog. Visit the Bill Tate memorial at wctate.com or the guestbook page.

Simple script to count how many read your blog

Ok, admit it, who likes blogging in to a vacuum. You want to know how many people are actually reading your blog.

I have created a simple Perl script that scans your blog’s log file and attempts to calculate how many people read the blog and the RSS feeds.

You can download the feed reader script. I release it under GPL2.

It’s a perl script, so you would go to your web server log in the shell, and type “perl feedreaders.pl logfilename” or if you like just “tail -99999 blogfilename | perl feedreaders.pl -” because you only need to scan a couple of days worth of logs to get the figures.

Here are some notes:

  • I take advantage of the fact that most blog aggregators now report how many people they are aggregating for. There is no standard but I have put in code to match the common patterns.
  • I identify common RSS feed URLs, as well as the most common “main feed” names. If you have other feeds that it doesn’t pick up on, it’s easy to add them to the list at the start of the program.
  • A reader has to fetch the feed or home page multiple times from the same IP to count
  • On the other hand, people who change IPs regularly will count multiple times. People behind caches may count just once all together.
  • I try to eliminate fetches from the most common non-RSS-aggregating spiders
  • Based on my experiences, Google Reader and Bloglines are the most popular aggregators, then NewsGator.
  • At least one aggregator identifies as Mozilla, custom code tags it.
  • It also counts people who fetch your non-RSS blog page multiple times as readers.
  • Programs that don’t say they handle multiple users get grouped among the singles.
  • Programs with only a few fetches are not counted

I invite my 1146 main blog readers to give it a whirl. (The 53 readers of the new Battlestar blog feed won’t see this notice, nor the 72 reading the comments.

Off to Munich next week for DLD

End of next week I’ll be going to Munich/München for an interesting conference called DLD. Since the flight is so long and I haven’t been to Bavaria since I was a kid, I booked a few extra days around the conference, even though it’s not exactly the high season. I welcome comments from blog readers on stuff to do there, and in surrounding Bavaria — we’ll take some day trips to the Alps and maybe to Salzberg. I know there’s a great science and tech museum we’ll go to. What scenic winter drives and train rides are recommended and still passable in January?

Some things I’ve already noted:

The 4 day Germany Rail Pass is $376 for 2 people. Is that worth it, since there is a 27 Euro Bavarian one-day roundtrip fare after 9am?

I have 2 unlocked phones so I will want to get SIMs so we can get calls and also call one another. The MVNOs seem the best deals. One service, sold by the tchibo.de cafes, is 5 cents per minute on-network and for $4/month (ie. whole trip) you get unlimited on-network. 89 cents to USA, which is lower than most, but not nearly as low as blauworld, which offers 9 cents to the USA/Canada plus 15 cents connection charge, and 19 cents to German phones including the other blauworld phone making it not as good for finding one another. Of course Eurocents are 1.4 U.S. cents now. Since use will be limited, and incoming is paid for by the caller, the services that let you get a SIM with minimal balance for 10 Euros may be better than those which are 20 Euros ($28) with $10 credit. I’ll use SIP or Skype for calls back home of any duration, and I’ll have my wifi-enabled HTC Mogul, which as a CDMA phone won’t be able to do anything else, but it’s also my PDA.

Do I have any German readers? Let me hear your thoughts.

Nice short piece in Computerworld

Computerworld has been nice enough to include me in their series on unsung innovators of the net. I should point out that I try to downplay the dot thing — to me it’s an amusing anecdote of having participated in the right mailing lists at the right time. I remain much more interested in whatever I will do next!

Eniac Programmer event postponed

I’ve been informed that the ENIAC programmer talk featuring Jean Bartik, a member of the world’s first software team, has been postponed until sometime in January. I’ll update with more information when it is worked out. Donors can transfer their seat to the later event, get a refund, or give it as a donation as they wish.

Meet one of the six women who were the world's first programmers on Nov 8

Update: Sorry to say, this event has been postponed to January

Most people know a bit about the story of the ENIAC, the first electronic computer. A few years ago, a good friend of mine named Kathy Kleimann discovered a remarkable story about the team that wrote the software for the ENIAC — the world’s first programmers. These pioneers of our field started out working during the war as “computers” — which is to say human beings and worked out military algorithms (mostly ballistics) by hand. Six of the brightest were recruited to make the first software for the quirky ENIAC, as they wanted it to do in software what they were dong by hand.

In a high-tech version of the “Rosie the Riveter” story these first software developers were all women. Math, after all, was one of those things that girls/women were allowed to be good at in that more sexist society. They worked together with the hardware engineers to get the machine calculating, and later converted it from wiring to a true stored program computer. They worked out many of the earliest concepts of software, inventing them from scratch. After the war, several of them went on to careers in software and at the early computer companies.

Their stories were not well reported, and some histories even presumed the women standing next to the ENIAC in demos were decoration rather than the coders making it go.

Unfortunately, it’s now 60 years later, and three of the team of six have died, and the other 3 are not getting younger. So Kathy set out to produce a documentary on these pioneers of our field. Time is running out. To raise some money for it, she has arranged a dinner at Google HQ for Thursday Next (Nov 8) cooked by one of Google’s exclusive internal restaurants. Folks who come will get a chance to meet Jean Bartik, one of those six pioneers of software, hear her story, see some preliminary footage and help the documentary. The mostly tax deductible donation is $100. Bartik, aside from coding for the ENIAC, also helped to design the BINAC and UNIVAC after the war.

You can read more about the project at the ENIAC Programmers and you can register for the event. If you are on Facebook, you can also mark yourself as attending through the facebook event entry though you must also register on the Google site.

Coming up: Burning Man, Singularity Summit, Foresight Vision Weekend

Here are three events coming up that I will be involved with.

Burning Man of course starts next weekend and consumes much of my time. While I’m not doing any bold new art project this year, maintaining my 3 main ones is plenty of work, as is the foolishly taken on job of village organizer and power grid coordinator. I must admit I often look back fondly on my first Burning Man, where we just arrived and were effectively spectators. But you only get to do that once.

Right after Burning Man, the Singularity Institute is hosting a Singularity Summit — a futurist conference with a good rack of speakers. Last year they did it as a free event at Stanford and got a giant crowd (because it was free there were no-shows, however, making it sad that some were turned away.) This year there is a small fee, and it’s at the Palace of Fine Arts in San Francisco.

On the first weekend of November, we at the Foresight Institute will host our 2007 Vision Weekend doing half of it in “unconference” style — much more ad-hoc. It will be at Yahoo HQ in Sunnyvale, thanks to their generous sponsorship. More details on that to come.

Blog has been moved to a new server -- notes on shopping for hosting

As I noted earlier, my web site got hacked. As a result, I decided to leave my old hosting company, PowerVPS.com, and find a new host. While another VPS would probably have managed, I know a woman in San Jose who runs a hosting company, simpli.biz, who offered me a good deal on a fast dedicated server. I’ll grow into it, and in the meantime you should see much greater performance from my site.

I will make some final commentary on PowerVPS. I left for a variety of reasons, and they were certainly not 100% bad.

  • They were on the other coast, so my ping times to them were 80ms or so. This was no fun for ssh and would have made running things on them impractical. I was surprised that most of the virtual hosting companies with good reputations and prices were not on the west coast.
  • At first I looked for hosting in Canada. This was not simply because I was a Canadian. I thought it might be good to get hosting (in Vancouver) that was not subject to U.S. law. Not because I intend to break U.S. law, but being at the EFF we’ve been fighting some of these laws and it would be good to be on another level. And I’m Canadian. However, all the hosting offerings in Canada I tried that matched my parameters were much more expensive.
  • VPSs are in general a great idea. However, it’s hard to make them swap. That means each VPS duplicates in RAM a copy of apache and mysql and the rest, which is wasteful. Dedicated servers, which swap, allow the big programs that have a lot of pages which are rarely used to swap them out to disk, while the active programs get use of all of the ram. You can’t overdo this, but it’s pretty handy. One VPS provider, Iron Mountain, does what I have been advocating — gives users access to a virtualized MySQL server on a fast machine, so you don’t have to run your own. Doing this is rare.
  • They would not support Ubuntu, only Centos. I am running Ubuntu on almost all my machines. I really like the idea that I can just duplicate efforts onto my hosting server, with now learning how to do things in a different distro. And that I can compile stuff at home and just move it to the web host. CentOS is the most popular distro in the hosting world, and people have done a lot of fancy things for it (control panels, automated installs etc.) and I understand why a company will decide to only support one distro. But that just means I go to a company that picked the distro I want.
  • PowerVPS screwed up when most of their customers got hacked. The hack wasn’t their fault, as far as I know, but once they realized so many of their customers were compromised, they should have E-mailed all of us immediately. Because they didn’t, I only noticed the attack when they broke some of my scripts. My site redirected unsuspecting users to a frame which might have infected them, which I regret. I should have been told about this as soon as possible.
  • The kicker: When I told them I wanted to replace my server after the hack, they said I had two options. I could back up the server (many gigs of data) and they would erase it and give me a new one with a fresh Centos 4. Then I could restore the files and rebuild everything, being down during the period I did this. Or I could buy a new server, transfer, and then move the DNS or the IP as desired. They would not temporarily give me the 2nd server, and then delete the old when I was ready. They said too many people took too long, and freaked out if deleted. Being forced to buy a new server simply sent me on a shopping trip. Stupid, stupid, stupid. Why send your customers on a shopping trip?
  • Another sin: When I went shopping, I looked at the list of special coupon offers various competitors offered. There I saw PowerVPS selling the same server I was paying $85 for for 30% off, lifetime discount. Be very careful when you offer new customers a much better price than existing customers get. I hate it, and I will leave you for it.

Now as I say, it was not all bad. Their support was good, and during the recent episode where I was on the digg.com homepage, they temporarily upgraded my VPS capacity — which is one of the prime things a VPS can do that a dedicated server can’t. I liked those things but the above mistakes lost a customer.

Let me know if you encounter any problems with the server move.

Updated note: After you change a server’s IP, all users should switch to a new IP after the “time to live” on the past lookup expires, which in my case was set to about 3 hours. However, turns out many people have broken (or deliberately broken) software that retains stale records for much longer. The leading culprit right now are web spiders, including googlebot, which continue to hit the old address. Actual users doing so are rare. For E-mail, a previous move found that spammers continued to use the old addresses for months after the fact. They presumably kept DNS lookup data on their CD-ROMs, or didn’t want to be subject to attempts to use DNS to block them, or had some other reason.

Yipes, badwared...

A few weeks ago, my site got hacked. The attacker inserted an iframe pointing to a malware site into most of my html pages. That of course is bad, but the story doesn’t end there. (I should of course have upgraded my OS from the ancient one my hosting company gave years ago, but they don’t really support that, and feel an upgrade consists of rebuilding from scratch.)

I cleaned out the entire site and searched for any remnants of the bad link. Having done this I thought all was well. However, as it turns out while the ideas.4brad.com domain and other domains were clear, the 4brad.com domain, which I don’t use for anything, still had a web server on it, pointing at a different directory far from where I keep my own web sites. (I try to never put my stuff in system directories.)

Unfortunately google, for unknown reasons, looked at 4brad.com, even though there are no links to it anywhere on the web. And found the placeholder page, with hacked link in it. From there it declared the entire site, including ideas.4brad.com, to be a malware site. I think that’s a bug, since there were never any malware links on ideas.4brad.com pages — this is a drupal site, and while the hacker’s script attempts to modify PHP scripts, it did not do so correctly, and just broke them. Running linux, I didn’t see the malware hacks on the other sites where they made the changes, but found them soon enough and removed them for now.

Alas, that means for some time people have been directed away from this blog by google. It shows up in search results, but you can’t actually click on the results, and there are warnings that going to the site may harm your computer (you get these warnings even on non-windows computers, which is reasonable, I guess, if incorrect.) I’ve asked the site stopbadware.org, which Google teams with, to confirm the hacks are gone, and now I have to rush out to rebuild the site from a fresh install. Sigh.

Update: Google reacted to the cleanup of 4brad.com very quickly and no longer lists the domain as unsafe. I did file a review request with stopbadware.org — perhaps they are much faster than they let on.

I’m shopping for hosting. I think I will upgrade to dedicated hosting, even though virtualized hosting has its merits. As I wrote before it would be great if MySQL could be virtualized independently of the OS. The ideal marriage would be a virtualized linux with access to sharable, non-virtualized services like web serving and database. The trick is memory. A typical virtual host will have 16 copies of MySQL and 16 copies of Apache and 16 copies of PHP or similar running on it. Because virtual machines don’t truly understand how much memory they have, or see the paging of the underlying OS, they can’t manage memory as well. But their ability to burst in unused capacity is a big win.

Photo server being dugg

Well, this site is at a crawl now because the panorama I assembled of San Francisco in 1971 is on the digg.com front page. If you haven’t seen it before it’s on the San Francisco page, the panorama of SF from the top of the Bay Bridge in 1971.

My hosting company, Defender Hosting/PowerVPS, has been kind enough to do a temporary upgrade of my server capacity to their top level, though the site’s response is still poor. This is something that virtual hosting can do that you can’t as easily do with dedicated hosting, though virtual hosting has its own costs, mostly in wasted memory.

I think it would be nice if virtual hosting companies sold this “bump” ability as a feature. When your web site gets a lot of load from a place like digg or slashdot, this could ideally be automatically detected, and more capacity made available, either free for rare use as a bonus, or for a fee. Most site owners would be glad to authorize a bit of extra payment for extra capacity in the event that they’re subject to a big swarm of traffic. (The only risk being that you might pay for capacity when under a DOS or spam attack or when being used by crackers or spammers.)

One place this might happen well is in the Amazon ec2 service, which I have yet to really try out. EC2 offers a cloud of virtual servers on demand. In this case, you would want to have a master controller which tracks load on your server, and fires up another virtual server, and then, once it’s up, starts redirecting traffic to it using DNS or proxy techniques, or both. If a web site is highly based on an SQL server, all the copies would need to use the same SQL server (or perhaps need an interesting replication strategy if not read-only) but making SQL servers scale is a well-attacked problem.

Has anybody done this yet with EC2? If not, I expect somebody will soon. The basic concept is fairly simple, though to do it perfectly you would need to do things like copy logs back after the fact and redirect any pages which want to write data to the local server to a common server if one can. For a site with static pages that don’t change due to user activity, such replication should not present too many problems.

Syndicate content