Today the word went around about mypollingplace.com, a site that helps people find their local polling station, running out of bandwidth from their provider and needing somebody to help it.

At the same time I did an interview for opinions about bitTorrent, and the idea came to me that a really useful application would be a P2P generalized web hosting tool aimed at spikes.

Volunteers who have a reachable IP address (including people who can open holes in NATs) would install software to volunteer to help host sites in need during a spike. This could be client software in a browser or permanent server software.

Then a few different things could happen. Ideally a site would install a tool which activated the sharing network when their load got high (but well before their provider cut them off.) People fetching static pages that weren't sharing bandwidth would be redirected to a port willing to share. The page would also be modified to encourage folks to join the process. Folks who were willing to share might get access or be redirected, and they would cache the page, declare where it resides and serve it there for some period of time or number of hits.

If a site didn't install the fancy redirector, they could redirect all web hits to a volunteer tracker that would do a further redirect to a real location willing to host the item.

Finally, for sites totally unaware of this, a browser plug-in could notice when access to a page is very slow or gets a "bandwidth limit exceeded" message. It could then query to see if any of the P2P folks have cached the page, and fetch from there, or offer the user a button to check or fetch from that network.

This latter mode is good because it strongly encourages having the magic plug-in with your browser. If you don't have it, you can't get at the P2P cache.

I've been informed that Coral from NYU does some of this, though it doesn't do the "on demand" aspect that leaves the web in normal state when not overloaded, and switches to cacheing only when needed. With Coral and other redirects, your Google adsense doesn't show up either!One problem is this doesn't handle dynamic pages easily, particularly those with POST forms or database fetches. In some cases, those pages can be converted to temporary static status during the peak load. As the load drops, people would be sent back to the main site.

Of course one big problem is how to prevent abuse. Which is to say, people using this as a cheap means for free web hosting of giant files at other people's expense. You need some sort of system where people can somehow "vote" a site as worth of such cacheing. The appearance of a page on slashdot or a popular blog might be enough for automatic promotion. A group of people might volunteer to serve as a committe wthat would look into calls for help. The help would be immediate once the cry went out, but shut off if the request was not legit. One would want to be sure spammers didn't find a way to host their web pages using these redirectors.

The fact that several honest users of the system all visited a page might be enough of a clue to indicate it has a need, though again people might game that. Smart sites, knowing they might get a sudden load during special events like the Olympics or an election, might ask in advance.

The redirect is ideal because it solves the question of how you know to look in the P2P cache. Otherwise you need a load distributing algorithm so that, using a hash of the domain name or some other subset of the URL, you can find just a few places to query for a cache of a particular page. Sites that generate links that will create heavy traffic can also modify those links to go directly to redirectors when they notice a target has been overloaded. Clearly there are a few problems to be solved in making this smooth.