App stores need offline interfaces

Here’s the situation: You’re in a place with no bandwidth or limited bandwidth. It’s just the place that you need to download an app, because the good apps, at least, can do more things locally and not make as much use of the network. But you can’t get to the app store. The archetype of this situation is being on a plane with wifi and video offerings over the wifi. You get on board and you connect and it says you needed to download the app before you took off and got disconnected.

There’s an obvious answer. The app stores should allow segments of themselves to be cached offline. This means that the app market app (such as iTunes or Google Play) should allow you to use a cached version of the store, as long as everything is signed and not too old. Then the plane’s server could keep copies of things like the airline app or video playing app in the cache, along with games and entertainment they want to make available to you. Mostly free stuff, though you could also allow payment with cached transactions (with a bit of trust) if need be.

Same experience for the user. They could go to the app store, search for and find the airline app, and download and install it, all without a network connection. Only if they tried to get a non-cached app would they get told they were offline.

As I wander the world, I get reminded all the time how we get a bit spoiled in our land of fast wifi and LTE phone data. You even get to understand why Google started de-ranking pages that don’t support mobile well in their mobile search results. Even as we move to having internet from drones, balloons or satellites everywhere we go, until we have gigabits everywhere, we need to design for lower connectivity environments.

Of course, the airlines could, on Android, offer you an APK file that you can manually install, but you have to check boxes and take security risks to do so, because the certification systems are centralized.

What if the city ran Waze and you had to obey it? Could this cure congestion?

I believe we have the potential to eliminate a major fraction of traffic congestion in the near future, using technology that exists today which will be cheap in the future. The method has been outlined by myself and others in the past, but here I offer an alternate way to explain it which may help crystallize it in people’s minds.

Today many people drive almost all the time guided by their smartphone, using navigation apps like Google Maps, Apple Maps or Waze (now owned by Google.) Many have come to drive as though they were a robot under the command of the app, trusting and obeying it at every turn. Tools like these apps are even causing controversy, because in the hunt for the quickest trip, they are often finding creative routes that bypass congested major roads for local streets that used to be lightly used.

Put simply, the answer to traffic congestion might be, “What if you, by law, had to obey your navigation app at rush hour?” To be more specific, what if the cities and towns that own the streets handed out reservations for routes on those streets to you via those apps, and your navigation app directed you down them? And what if the cities made sure there were never more cars put on a piece of road than it had capacity to handle? (The city would not literally run Waze, it would hand out route reservations to it, and it would still do the UI and be a private company.)

The value is huge. Estimates suggest congestion costs around 160 billion dollars per year in the USA, including 3 billion gallons of fuel and 42 hours of time for every driver. Roughly quadruple that for the world.

Road metering actually works

This approach would exploit one principle in road management that’s been most effective in reducing congestion, namely road metering. The majority of traffic congestion is caused, no surprise, by excess traffic — more cars trying to use a stretch of road than it has the capacity to handle. There are other things that cause congestion — accidents, gridlock and irrational driver behaviour, but even these only cause traffic jams when the road is near or over capacity.

Today, in many cities, highway metering is keeping the highways flowing far better than they used to. When highways stall, the metering lights stop cars from entering the freeway as fast as they want. You get frustrated waiting at the metering light but the reward is you eventually get on a freeway that’s not as badly overloaded.

Another type of metering is called congestion pricing. Pioneered in Singapore, these systems place a toll on driving in the most congested areas, typically the downtown cores at rush hour. They are also used in London, Milan, Stockholm and some smaller towns, but have never caught on in many other areas for political reasons. Congestion charging can easily be viewed as allocating the roads to the rich when they were paid for by everybody’s taxes.

A third successful metering system is the High-occupancy toll lane. HOT lanes take carpool lanes that are being underutilized, and let drivers pay a market-based price to use them solo. The price is set to bring in just enough solo drivers to avoid wasting the spare capacity of the lane without overloading it. Taking those solo drivers out of the other lanes improves their flow as well. While not every city will admit it, carpool lanes themselves have not been a success. 90% of the carpools in them are families or others who would have carpooled anyway. The 10% “induced” carpools are great, but if the carpool lane only runs at 50% capacity, it ends up causing more congestion than it saves. HOT is a metering system that fixes that problem.  read more »

If you built "Westworld" (or other robot sex) it would probably be with VR

HBO released a new version of “Westworld” based on the old movie about a robot-based western theme park. The show hasn’t excited me yet — it repeats many of the old tropes on robots/AI becoming aware — but I’m interested in the same thing the original talked about — simulated experiences for entertainment.

The new show misses what’s changed since the original. I think it’s more likely they will build a world like this with a combination of VR, AI and specialty remotely controlled actuators rather than with independent self-contained robots.

One can understand the appeal of presenting the simulation in a mostly real environment. But the advantages of the VR experience are many. In particular, with the top-quality, retinal resolution light-field VR we hope to see in the future, the big advantage is you don’t need to make the physical things look real. You will have synthetic bodies, but they only have to feel right, and only just where you touch them. They don’t have to look right. In particular, they can have cables coming out of them connecting them to external computing and power. You don’t see the cables, nor the other manipulators that are keeping the cables out of your way (even briefly unplugging them) as you and they move.

This is important to get data to the devices — they are not robots as their control logic is elsewhere, though we will call them robots — but even more important for power. Perhaps the most science fictional thing about most TV robots is that they can run for days on internal power. That’s actually very hard.

The VR has to be much better than we have today, but it’s not as much of a leap as the robots in the show. It needs to be at full retinal resolution (though only in the spot your eyes are looking) and it needs to be able to simulate the “light field” which means making the light from different distances converge correctly so you focus your eyes at those distances. It has to be lightweight enough that you forget you have it on. It has to have an amazing frame-rate and accuracy, and we are years from that. It would be nice if it were also untethered, but the option is also open for a tether which is suspended from the ceiling and constantly moved by manipulators so you never feel its weight or encounter it with your arms. (That might include short disconnections.) However, a tracking laser combined with wireless power could also do the trick to give us full bandwidth and full power without weight.

It’s probably not possible to let you touch the area around your eyes and not feel a headset, but add a little SF magic and it might be reduced to feeling like a pair of glasses.

The advantages of this are huge:

  • You don’t have to make anything look realistic, you just need to be able to render that in VR.
  • You don’t even have to build things that nobody will touch, or go to, including most backgrounds and scenery.
  • You don’t even need to keep rooms around, if you can quickly have machines put in the props when needed before a player enters the room.
  • In many cases, instead of some physical objects, a very fast manipulator might be able to quickly place in your way textures and surfaces you are about to touch. For example, imagine if, instead of a wall, a machine with a few squares of wall surface quickly holds one out anywhere you’re about to touch. Instead of a door there is just a robot arm holding a handle that moves as you push and turn it.
  • Proven tricks in VR can get people to turn around without realizing it, letting you create vast virtual spaces in small physical ones. The spaces will be designed to match what the technology can do, of course.
  • You will also control the audio and cancel sounds, so your behind-the-scenes manipulations don’t need to be fully silent.
  • You do it all with central computers, you don’t try to fit it all inside a robot.
  • You can change it all up any time.

In some cases, you need the player to “play along” and remember not to do things that would break the illusion. Don’t try to run into that wall or swing from that light fixture. Most people would play along.

For a lot more money, you might some day be able to do something more like Westworld. That has its advantages too:

  • Of course, the player is not wearing any gear, which will improve the reality of the experience. They can touch their faces and ears.
  • Superb rendering and matching are not needed, nor the light field or anything else. You just need your robots to get past the uncanny valley
  • You can use real settings (like a remote landscape for a western) though you may have a few anachronisms. (Planes flying overhead, houses in the distance.)
  • The same transmitted power and laser tricks could work for the robots, but transmitting enough power to power a horse is a great deal more than enough to power a headset. All this must be kept fully hidden.

The latter experience will be made too, but it will be more static and cost a lot more money.

Yes, there will be sex

Warning: We’re going to get a bit squicky here for some folks.

Westworld is on HBO, so of course there is sex, though mostly just a more advanced vision of the classic sex robot idea. I think that VR will change sex much sooner. In fact, there is already a small VR porn industry, and even some primitive haptic devices which tie into what’s going on in the porn. I have not tried them but do not imagine them to be very sophisticated as yet, but that will change. Indeed, it will change to the point where porn of this sort becomes a substitute for prostitution, with some strong advantages over the real thing (including, of course, the questions of legality and exploitation of humans.)  read more »

Our routers need to remove the "internet" from the "internet of things" to stop DDOS

I frequently say that there is no “internet of things.” That’s a marketing phrase for now. You can’t go buy a “thing” and plug it into the “internet of things.” IoT is still interesting because underneath the name is a real revolution from the way that computing, sensing and communications are getting cheaper, smaller and using less power. New communications protocols are also doing interesting things.

We learned a lesson on Friday though, about why using the word “internet” is its own mistake. The internet — one of the world’s greatest inventions — was created as a network of networks where anything could talk to anything, and it was useful for this to happen. Later, for various reasons, we moved to putting most devices behind NATs and firewalls to diminish this vision, but the core idea remains.

Attackers on Friday made use of growing collection of low cost IoT devices with low security to mount a DDOS attack on DYN’s domain name servers, shutting off name lookup for some big sites. While not the only source of the attack, a lot of attention has come to certain Chinese brands of IP based security cameras and baby monitors. To make them easy to use, they are designed with very poor security, and as a result they can be hijacked and put into botnets to do DDOS — recruiting a million vulnerable computers to all overload some internet site or service at once.

Most applications for small embedded systems — the old and less catchy name of the “internet of things” — aren’t at all in line with the internet concept. They have no need or desire to be able to talk to the whole world the way your phone, laptop or web server do. They only need to talk to other local devices, and sometimes to cloud servers from their vendor. We are going to see billions of these devices connected to our networks in the coming years, perhaps hundreds of billions. They are going to be designed by thousands of vendors. They are going to be cheap and not that well made. They are not going to be secure, and little we can do will change that. Even efforts to make punishments for vendors of insecure devices won’t change that.

So here’s an alternative; a long term plan for our routers and gateways to take the internet out of IoT.

Our routers should understand that two different classes of devices will connect to them. The regular devices, like phones and laptops, should connect to the internet as we expect today. There should also be a way to know that the connecting devices does not want regular internet access, and not to give it. One way to do that is for the devices to know about this, and to convey how much access they need when they first connect. One proposal for this is my friend Eliot Lear’s MUD proposal. Unfortunately, we can’t count on devices to do this. We must limit stupid devices and old devices too.  read more »

Facebook makes less than $10/user, can we find alternatives to advertising?

Facebook’s ARPU (average revenue per user, annualized) in the last quarter was just under $10, declining slightly in the USA and Canada, and a much lower 80 cents in the rest of the world. This is quite a bit less than Google’s which hovers well over $40.

That number has been mostly growing (it shrank last quarter for the first time) but it’s fairly low. I can solidly say I would happily pay $10 a year — even $50 a year — for a Facebook which was not simply advertising-free, but more importantly motivated only to please its customers and not advertisers. Why can’t I get that?

One reason is that it’s not that simple. If Facebook had to actually charge, it would not get nearly as many users as it does being free and ad-supported. It is frictionless to join and participate in FB, and that’s important with the natural monopolies that apply to social media. You dare not do anything that would scare away users.

Valley of Distraction

Being advertising supported bends how Facebook operates, as it will any company. The most obvious thing is the annoying ads. Particularly annoying are the ads which show up in my feed, often marked with “Friend X liked this company.” I am starting to warn my friends to please not like the pages of anybody who buys ads on FB, because these ads are even more distracting than regular ads. Also extra distracting are ads which are “just off the bulls-eye,” which is to say they are directed at me (based on what FB knows about me) and thus likely to distract me, but which turn out to be completely useless. That’s worse than an ad which was not well aimed and so doesn’t distract me at all with its uselessness. There is a “valley of distraction” when it comes to targeting ads:

  • Ads about things I am researching or may want to buy can be actually valuable to me, and also rewarding to the advertiser.
  • Ads about things I am interested in, but have already bought or would not buy via an ad are highly distracting but provide no value to the advertiser and negative value to me.
  • Ads about things I have no interest in tend to be only mildly distracting if they are off to the side and not blinky/flashy/pop-up style.

As sites get better at ad targeting, they generate more of the middle type.


Facebook’s need to monetize with advertising gives them strong incentives to be less protective of privacy. All social networks have an anti-privacy incentive, because the more they can get you to share with more people, the more they can make things happen on their site, and the more they can attract in other users. But advertising ads to this. Without ads, FB would focus only on attracting and retaining customers by serving them, which would be good for users.

As the old saying goes, “If you’re not paying, you’re not the customer, you’re the product.” To give credit to many web companies, in spite of the reality of this, they actually work hard to reduce the truth of this statement, but they can never do it entirely.

How we monetize the web

When I created the first internet based publication in 1989, I did it by selling subscriptions. There really wasn’t a way to do it with advertising at that time, but I lamented the eventual switch that later came which has made advertising the overwhelmingly dominant means of monetizing the web. There are a few for-pay sites but they are very few and specialized. I lament that forces pushed the web that way, and have always wished for a mechanism to make it easier, if not as easy, to monetize a web site with payment from customers. That’s why I promoted ideas like microrefunds as well as selling books in flat-rate pools like my Library of Tomorrow back in 1992. (Fortunately this concept is now starting to get some traction in some areas, like Amazon’s Kindle Unlimited.)

I’m also very interested in the way that low-friction digital currencies like Bitcoin and in particular Dogecoin have made it work workable to give donations and tips. Dogecoin started as a joke, but because people viewed it as a joke, they were willing to build easy and low security means of tipping people. The lack of value attached to Dogecoin meant people were more willing to play around with such approaches. Perhaps Bitcoin’s greatest flaw is that because its transactions are irrevocable, you must make the engine that spends them secure, and in turn, that demands it is harder to use. Easy to spend means easy to lose, or easy to steal and that’s a rule that’s hard to break. The credit card system, in order to be easy to spend, solves the problem of being easy to steal by allowing chargebacks or other human fixes when problems occur. While we can do better at making digital money easy to spend and not quite so easy to steal, it’s hard to figure out how to be perfect at that without something akin to chargebacks.

To monetize the web without advertising, we need a truly frictionless money. Advertising provides a money whose only friction is the annoyance of the advertising. To consume an ad-supported product you need do nothing but waste a little time. It’s a fairly passive thing. To consume a consumer-paid product, you must pay, and that creates three frictions:

  1. The spending itself — though if it’s low that should be tolerable
  2. The mental cost of thinking about the spending — which often exceeds the monetary cost on tiny transactions
  3. The user interface cost of your means of payment.

You can’t eliminate #1 of course, but you can realize that the monetary cost is less than the negatives introduced by advertising. Eliminating #2 and #3 in a secure way is the challenge, and indeed it is the challenge which I devised the microrefund concept to address.

Will we pay the cost?

I think lots of people would pay $10/year for Facebook, particularly if alternatives also charged money. It’s a bargain at that price. But would people pay the $50 that Google makes from them? Again, I think Google is a bargain at that price, but for a lot of the world, that could be a lot of money, and that’s Google’s average revenue, not its revenue for me. (I click on ads so rarely that I think their revenue from me is actually a lot lower.)

I already bought my ticket on Iberia!

At the same time, Google’s ads are among the least painful. The ads on search are marked and isolated, and largely text based. The only really bad ads Google is doing are the ones in the valley of distraction in Adsense. As I wrote earlier, we are all constantly seeing ads for things we already bought.

And so, even though a Google search might only cost you a couple of pennies, I doubt we could move Google to payment supported even if we could remove all the friction from it.

This is not true for many other sites, though. Video sites would be a great target for frictionless payment, since showing a 30 second video ad to watch a 2 minute video is a terrible bargain, yet we see it happen frequently. There are many sites who do much worse than Google at monetizing themselves through advertising, and who would welcome a way to get more decent revenues via payment — though of course they can’t get greedy or they friction of the payment itself will reduce their business.

In addition, there are zillions of small sites and sites about topics of no commercial value who can’t make much money from advertising at all. Some of these sites probably don’t even exist because they can’t become going concerns in the current regime of monetizing the web — what fraction of the web are we missing because we have only one practical way to monetize it?

Replacing E-mail: The calendar as communications tool

I want to begin a series of thoughts on how E-mail has failed us and what we should do about it.

Yes, E-mail has failed, and not, as we thought, because it got overwhelmed with spam. There is tons of spam but we seem to be handling it. The problem might be better described as “too much signal” rather than the signal/noise ratio. There are three linked problems:

  1. There is just too much E-mail from people we actually have relationships with. Part of this is the over-reach of businesses, who think that because you bought a tube of toothpaste that you should fill out a customer satisfaction survey and get the weekly bargains mail-out, but part of it is there really are a lot of people who want to interact with you, and e-mail makes it very easy for them to do that, particularly to “cc” you on mail you may only have a marginal interest.
  2. Because of problem 1, people are moving away from E-mail to other tools, particularly the younger generation. They (and we) are using Facebook mail and other social tools, instant messengers, texting and more.
  3. The volume means that you can’t handle it all. Important mails scroll off the main screen and are forgotten about. And some people are just not using their E-mail, so it is losing its place as the one universal and reliable way to send somebody a message.

One of the key differences the new media have is they focus on person to person communications — while there are group tools, they don’t even have the concept of a “cc” or mailing list, or even sending to two people.

I’m going to write more on these topics in the future, but today I want to talk about

The shared calendar as the communications tool

I’ve been pushing people I work with to use the calendar as the means of telling me about anything that is going to happen at a specific time. If people send me an E-mail saying, “Can we talk at 3?” I say, “don’t tell me that in an E-mail. Create an event on your calendar and invite me to it. Put the details of the conversation into the calendar entry.”

In general, I want to create a pattern of communication where if any message you send would cause the other person to put something on their calendar, you instead communicate it through the calendar by creating an event that they are an attendee of.

Our calendar and E-mail tools need to improve to make this work better. When everybody uses a shared calendar like Google Calendar, it is a lot easier, but we need tools that make it just as easy when people don’t use the same calendar tool.

When things do get into the calendar, you get a lot of nice benefits:

  • You are much less likely to forget about or miss the task or event
  • When you want to find the data on the event near the time of the event, you don’t have to hunt around for it — it is highlighted, in my case right on the home screen of my phone
  • If the event has a location, your phone typically is able to generate a map and even warn you when you need to leave based on traffic
  • If the event has a phone call/hangout/whatever, your devices can join that with a single click, no hunting for URLs or meeting codes — particularly while driving. (Google put in a tool to add one of their hangouts to any event in the calendar.)
  • Calendar events remove any confusion on time zones when people are in different zones.

Here are some features I want, some of which exist in current tools (particularly if you attach an ICS calendar entry to an E-mail) but which don’t yet work seamlessly.

  • Your email tool, when writing a message should notice if you’re talking about an event that’s not already in your calendars, and parse out dates and other data and turn it into a calendar invitation
  • Likewise your receiving tool should parse messages and figure this out, since the sender might not have done that.
  • E-mails that create calendar events should be linked together, so that from your calendar you can read all the email threads around the event, find any associated files or other resources.
  • Likewise it should be easy to contact any others tied to a calendar event by any means, not just the planned means of communication. For example, a good calendar should have a system where I can be phoned or texted on my cell phone by any other member of the event during the time around the event, without having to reveal my cell phone number. How often have you been waiting for a conference call to have somebody say, “does anybody know John’s number? Let’s find where he is.”
  • When I accept a calendar entry from outside and confirm, that should give them some access to use that calendar entry as a means of communication, even across calendar and mail platforms.

For example, when I book a flight or hotel or rent a car, the company should respond by putting that in my calendar. I might given them a token enabling that, or manually approve their invitation. Of course the confirmation numbers, links on how to change the reservation and more will be in the calendar entry. If the flight is delayed, they should be able to use this linkage to contact me — my calendar tool should know best where I am and the best ways to reach me — and push updates to me. When I get to the check-in desk, our shared calendar entry should make my phone and their computer immediately connect and make the process seamless.

When I approach the desk of a hotel, my phone should notice this, do the handshake and by the time I walk up they should say, “Good evening, Mr. Templeton, could you please sign this form? Here’s your room key, you’re in suite 1207.” (Of course, even better if I don’t have to sign the form and my phone, or any of the magstripe, chip or NFC cards I have in my wallet automatically become my room key.)

When you think this way, you start realizing that a surprisingly large amount of our E-mails are about events with times. And, as I wrote 8 years ago, most e-mails involve tasks, and E-mail and time management should be merged. Sadly my ideas of so long ago remain unrealized, and since then, E-mail has declined.

One caveat — if we do start using calendars for communication more, we must be able to prevent spam, and even over-use by people we know. We can’t do what we did with e-mail. Invitations to an event with just one or two people can be made easy — even automatic for those with authorization. Creating multi-person events needs to be a harder thing for people who aren’t whitelisted, though not impossible. The meaning of the word “invite” also needs to be more tightly understood. A solicitation for me to buy a ticket is not an invite.

Would Bitcoin fall off a cliff if it dropped to $100 or $150?

Bitcoin’s been on a long decline over the past year, and today is around $220 per coin. The value has always been based on speculation about Bitcoin’s future value, not its present value, so it’s been very hard to predict and investment in the coins has been risky.

Some thinking led me to a scary conclusion. Recent news has revealed that a number of “cloud mining” companies have shut down after the price drop. Let me explain why.

Over time, all bitcoin mining has been done using specialized ASIC hardware. The hardware is priced so that you can make a decent but not ridiculous profit with it. All the bitcoins mined go mostly into paying for mining hardware and electricity — much less goes into profit for the miners. In the past, the electricity was the big cost, but mining hardware got fast enough and expensive enough that most of the cost of mining has been paying off your mining hardware, with electricity dropping to being 20% or less of the cost.

In other words, most of the 3600 btc/day mining revenues of the bitcoin system have been going into the people making mining chips and rigs, but that’s another story.

With the drop in price, electricity is back up to being half your cost. That puts a squeeze on the cost of mining equipment. With cloud mining, as with Amazon Web Services, you rented mining equipment and power by the hour. People who bought their mining equipment will still run it as long as the revenue is more than the operating cost. For cloud mining, you need the revenue to exceed the operating and capital cost, because the capital costs are amortized into the operating cost. While cloud mining companies could cut their fees to cut their losses, some have instead just left the business. As noted, those who bought mining equipment are running it now at less profit, but as long as the mining brings in more than the electricity cost, it’s still worth running — the mining gear is all paid for, and even though you will never make back your money, it’s worse if you shut it off.

You can get a good analysis of the cost and profitability of mining rigs at this mining calculator.

What if a panic dropped a bitcoin under $100?

It’s not out of the question that a sudden panic might drop Bitcoin quickly down to $100. It probably won’t happen, but it certainly could. At this point, with current generation mining equipment, most miners then see their revenue drop below the cost of electricity. If they are rational and strictly profit-oriented, they cry into their beer and turn off the mining rig. And the cloud miners have already done that, and some other miners have done the same sooner than they expected, and the network hashrate (the measure of how much mining power there is) has had minor sustained drops for the first time in years.

(It’s worst than this. Even at $150, all but the most recent mining rigs become unprofitable to keep turned on, and so a major drop would happen with much less of a drop needed. New mining equipment expected to ship in the next few months is profitable at even lower prices, though.)

The way Bitcoin works, when they turn off the rig, it doesn’t mean more coins for the other miners. Bitcoin sets the reward rate with a “difficulty” number that makes the Bitcoin lottery problem harder the more mining capacity is out there. Your reward rate is a strict function of the difficulty and the power of your miners.

Every 2016 blocks, the difficulty adjusts based on how much capacity seems to be mining. Under normal operations, 2016 blocks is two weeks, as long as people are mining at the rate seen in the 2 weeks prior to setting the current difficulty. If large volumes of miners shut off their rigs as non-productive, the mining rate would crash. The wait for a new difficulty could be not just two weeks if this happened at the wrong time, but 4 weeks if half the miners shut down, or 8 weeks if 3/4 of them left. In terms of the Bitcoin world, it’s effectively forever, and long before that, confidence in the coin price would probably drop further, causing more miners to shut off their rigs. Only dedicated fans willing to lose money to preserve the system would keep mining.

In such a panic, the Bitcoin Foundation and others might propose an emergency modification of the Bitcoin software base which is able to do an emergency reduction of the difficulty number. Alternately they could propose bumping the mining reward back to 50 coins instead of 25. This would still take days, which I think is too long. But if they did, it’s a sticky issue. As soon as you drop the difficulty enough, all those miners come back online, and now the difficulty is too low. To do it right, an estimate would have to be made of how much mining capacity is cost effective and set the difficulty so that only some of the miners come back online, a number tied to that difficulty. For example, one might look at the various mining rigs out there, and set the difficulty such that they are (barely) profitable while others are not. Problem is, the profitability depends on the price of a bitcoin, which will be wildly fluctuating. It’s not clear how to solve this.

If the electricity cost exceeds the reward, but you still want bitcoins for future investment, the rational thing is not to mine, but to just buy bitcoins on the exchanges and keep the price up.

What would happen after such a collapse? Could it be stopped?

The collapse would probably spread to altcoins, but some might survive and become successors to Bitcoin. In addition, there are many people devoted to Bitcoin who would continue to mine, even at a loss, to get it back on its feet. After all, the early years of Bitcoin, all mining was at a loss, though it turned into a huge bonanza later and was a wise idea in hindsight. With the large number of well funded companies in the space, we could see companies willing to maintain unprofitable mining for some time if the alternative is the destruction of the thing they’ve based their business on. They might even buy up the rigs of failed miners, or pay them to mine. Perhaps, if they are ready, they could heed the warning in this message and make contracts with enough miners to say, “we’ll pay you to keep mining if a collapse happens.”

Alternately, Bitcoin users and boosters could just start deliberately leaving large transaction fees in their transactions to make the cost of mining worthwhile again. While hard to sustain long term, it is in their interest to spend their bitcoins to keep the mining system going, since those coins probably drop immensely if it falls down. It also keeps faith in the mining system since if the coin owners ran the miners, they might corrupt the network with that much power. It should be noted that it’s always been part of the plan for Bitcoin that higher transaction fees would arise as the coinbase rewards dropped, but not this early, and because the reward dropped in btc, not dollars.

The subsidy would have to be enough to overcome losses and provide a modest or even very small profit. The network cost pays 3600 bitcoins/day in mining fees (or $360K at $100/bitcoin.) The subsidy might be more in the range of $50K or $100K per day — affordable to keep the network alive for up to 14 days to survival.

Another idea would be to develop a way to make the difficulty more dynamic, or provide some mechanism for an emergency reduction. (An emergency increase would mean something was really wrong and would probably also mean somebody had more than half the mining capacity, another must-not-happen.)

What sort of events could cause such a huge drop, to 45% of the current value? That’s not been seen in a short time, but a big political event, such as a suggestion the USA or EU might forbid or impede Bitcoin could do it. But there are many other things that can cause panic. A shutdown of exchanges (a common technique in stock market panics) would probably do little, as there are exchanges all over the world and all will not shut down. A call to miners to sacrifice might work, at least for a while, to allow time to fix the problem.

Latent mining capacity

Mining rigs are shut down all the time as non-profitable, but in the past that’s always been because newer, better rigs were out there dominating the mining space and pushing up the difficulty. It would be a new idea to have rigs shut down because the dollar price dropped. When such rigs shut down, they would not be permanently useless, and unless torn down, they would be able to restart at any time. For example, if the difficulty dropped (because they all shut down) they would all start running again, and blocks would come out faster than intended. Then, 2016 blocks later, the difficulty would be recalculated up again — and they would stop again. Miners would also start and stop based on the day’s price as well, and the price might even swing around the expected rises and drops in difficulty. This seems like it would be chaos.

Once the electricity cost dominates, the important metric in mining equipment is not gigahashes/second, but gigahashes per joule. At 10 cents/kwh, you need around 2 gigahashes/joule to beat the electricity cost with $100 bitcoins and today’s difficulty number. At today’s $220 bitcoins, 0.9 gigahash/joule will do. Most miners are under 2, but there are some that do close to 3, and there is the promise of 5. If the trends in the rest of computing are an indicator, operations per joule will eventually level off, even as transistor counts continue to increase. If that happens we will stop seeing big increases in mining power and the upward spiral would end.

CES Day 2 Gallery and notes

After a short Day 1 at CES a more full day was full of the usual equipment — cameras, TVs, audio and the like and visits to several car booths.

I’ve expanded my gallery of notable things with captions with cars and other technology.

Lots of people were making demonstrations of traffic jam assist — simple self-driving at low speeds among other cars. All the demos were of a supervised traffic jam assist. This style of product (as well as supervised highway cruising) is the first thing that car companies are delivering (though they are also delivering various parking assist and valet parking systems.)

This makes sense as it’s an easy problem to solve. So easy, in fact, that many of them now admit they are working on making a real traffic jam assist, which will drive the jam for you while you do e-mail or read a book. This is a readily solvable problem today — you really just have to follow the other cars, and you are going slow enough that short of a catastrophic error like going full throttle, you aren’t going to hurt people no matter what you do, at least on a highway where there are no pedestrians or cyclists. As such, a full auto traffic jam assist should be the first product we see form car companies.

None of them will say when they might do this. The barrier is not so much technological as corporate — concern about liability and image. It’s a shame, because frankly the supervised cruise and traffic jam assist products are just in the “pleasant extra feature” category. They may help you relax a bit (if you trust them) as cruise control does, but they give you little else. A “read a book” level system would give people back time, and signal the true dawn of robocars. It would probably sell for lots more money, too.

The most impressive car is Delphi’s, a collaboration with folks out of CMU. The Delphi car, a modified Audi SUV, has no fewer than 6 4-plane LIDARs and an even larger number of radars. It helps if you make the radars, as otherwise this is an expensive bill of materials. With all the radars, the vehicle can look left and right, and back left and back right, as well as forward, which is what you need for dealing with intersections where cross traffic doesn’t stop, and for changing lanes at high speed.

As a refresher: Radar gives you great information, including speed on moving objects, and sucks on stationary ones. It goes very far and sees through all weather. It has terrible resolution. LIDAR has more resolution but does not see as far, and does not directly give you speed. Together they do great stuff.

For notes and photos, browse the gallery

Even ASIC miners of Bitcoins face security threats

Last month I wrote about paradoxes involving bitcoin and other cryptocurrency mining. In particular, I pointed out that while many people are designing alternative coins so that they are hard to mine with ASICs — and thus can be more democratically mined by people’s ordinary computers or GPUs — this generates a problem. If mining is done on ordinary computers, it becomes worthwhile to break into ordinary computers and steal their resources for mining. This has been happening, even with low powered NAS box computers which nobody would ever bother to mine on if they had to pay for the computer and its electricity. The attacker pays nothing, so any mining capacity is good.

Almost any. In Bitcoin, ASIC mining is so productive that it’s largely a waste of time to mine with ordinary CPUs even if you get them for free, since there is always some minor risk in stealing computer time. While ordinary computers are very hard to secure, dedicated ASIC mining rigs are very simple special purpose computers, and you can probably secure them.

But in a recently revealed attack thieves stole bitcoins from miners by attacking not the ASIC mining rigs, but their internet connections. The rigs may be simple, but the computers they flow their data through, and the big network routers, are less so. Using BGP redirection, it is suspected, the thieves just connected the mining rigs to a different mining pool than the one they thought they joined. And so they worked away, mining hard, and sometimes winning the bitcoin lottery, not for their chosen pool, but the thieves’ pool.

It’s not hard to imagine fixes for this particular attack. Pools and rigs can authenticate more strongly, and pools can also work to keep themselves more secure.

But we are shown one of the flaws of almost all digital money systems. If your computer can make serious money just by computing, or it can spend money on your behalf without need for a 2nd factor authentication, then it becomes very worthwhile for people to compromise your system and steal your computer time or your digital money. Bitcoin makes this even worse by making transactions irrevocable and anonymous. For many uses, those are features, but they are also bugs.

For the spending half, there is much effort in the community to build more secure wallets that can’t just spend your money if somebody takes over your computer. They rely on using multiple keys, and keeping at least one key in a more secure, even offline computer. Doing this is very hard, or rather doing it with a pleasant and happy user interface is super hard. If you’re going to compete with PayPal it’s a challenge. If somebody breaks into my PayPal account and transfers away the money there, I can go to PayPal and they can reverse those transactions, possibly even help track down the thieves. It’s bad news if a merchant was scammed but very good news for me.

One could design alternate currencies with chargebacks or refundability, but Bitcoin is quite deliberate in its choice not to have those. It was designed to be like cash. The issue is that while you could probably get away keeping your cash in your mattress and keeping a secure house, this is a world where somebody can build robots that can go into all the houses it can find and pull the cash out of the mattresses without anybody seeing.

The tide of surveys gets worse -- "would you please rate our survey?"

Five years ago, I posted a rant about the excess of customer service surveys we’re all being exposed to. You can’t do any transaction these days, it seems, without being asked to do a survey on how you liked it. We get so many surveys that we now just reject these requests unless we have some particular problem we want to complain about — in other words, we’re back to what we had with self-selected complaints. The value of surveys is now largely destroyed, and perversely, as the response rates drop and the utility diminishes, that just pushes some companies to push even harder on getting feedback, creating a death spiral.

A great example of this death spiral came a few weeks ago when I rode in an Uber and the driver had a number of problems. So this time I filled out the form to rate the driver and leave comments. Uber’s service department is diligent, and actually read it, and wrote me back to ask for more details and suggestions, which I gave.

That was followed up with:

Hi Brad Templeton,

We’d love to hear what you think of our customer service. It will only take a second, we promise. This feedback will allow us to make sure you always receive the best possible customer service experience in future.

If you were satisfied in how we handled your query, simply click this link.

If you weren’t satisfied in how we handled your ticket, simply click this link.

A survey on my satisfaction with the survey process! Ok, to give Uber some kudos, I will note:

  • They really did try to make this one simple, just click a link. Though one wonders, had I clicked I was unsatisfied, would there have been more inquiry? Of course I was unsatisfied — because they sent yet another survey. The service was actually fine.
  • At least they addressed me as “Hi Brad Templeton.” That’s way better than “Dear Brad” like the computer sending the message pretending it’s on a first-name basis with me. Though the correct salutation should be “Dear Customer” to let me know that it is not a personally written message for me. The ability to fill in people’s names in form letters stopped being impressive or looking personal in the 1970s.

This survey-on-a-survey is nice and short, but many of the surveys I get are astoundingly long. They must be designed, one imagines, to make sure nobody who values their time ever fully responds.

Why does this happen? Because we’ve become so thrilled at the ability to get high-volume feedback from customers that people feel it is a primary job function to get that feedback. If that’s your job, then you focus on measuring everything you can, without thinking about how the measurement (and over-measurement) affects the market, the customers and the very things you are try to measure. Heisenberg could teach these folks a lesson.

To work, surveys must be done on a small sample of the population, chosen in a manner to eliminate bias. Once chosen, major efforts should be made to assure people who are chosen do complete the surveys, which means you have to be able to truthfully tell them they are part of a small sample. Problem is, nobody is going to believe that when your colleagues are sending a dozen other surveys a day. It’s like over-use of antibiotics. All the other doctors are over-prescribing and so they stop working for you, even if you’re good.

The only way to stop this is to bring the hammer down from above. People higher up, with a focus on the whole customer experience, must limit the feedback efforts, and marketing professionals need to be taught hard in school and continuing education just why there are only so many they can do.

Reflections on the 25th anniversary of ClariNet and the dot-com

25 years ago, on June 8, 1989, I announced to the world my new company ClariNet, which offered for sale an electronic newspaper delivered over the internet. This has the distinction, as far as I know, of being the first business created to use the internet as a platform, what we usually call a “dot-com” company.

I know it was the first because up until that time, the internet’s backbone was run by the National Science Foundation and it had a policy disallowing commercial use of the network. In building ClariNet, I found a way to hack around those rules and sell the service. Later, the rules would be relaxed and the flood of dot-coms came on a path of history that changed the world.

A quarter of a century seems like an infinite amount of time in internet-years. Five years ago, for the 20th anniversary, I decided to write up this history of the company, how I came to found it, and the times in which it was founded.

Read The history of and the dawn of internet based business

There’s not a great deal to add in the 5 years since that prior anniversary.

  • Since then, USENET’s death has become more complete. I no longer use it, and porn, spam and binaries dominate it now. Even RSS, which was USENET’s successor — oddly with some inferiorities — has begun to fall from favour.
  • The last remnants of ClariNet, if they exist at Yellowbrix, are hard to find, though that company exists and continues to sell similar services.
  • Social media themselves are showing signs of shrinking. Publishing and discussing among large groups just doesn’t scale past a certain point and people are shrinking their circles rather than widening them.
  • We also just saw the 25th anniversary of the Web itself a few months ago, or at least its draft design document. ClariNet’s announcement in June was just that — work had been underway for many months before that, and product would not ship until later in the summer.

Many readers of this blog will not have seen this history before, and 25 years is enough of an anniversary to make it worth re-issuing. There is more than just the history of ClariNet in there. You will also find the history of other early internet business, my own personal industry history that put me in the right place at the right time with these early intentions, and some anecdotes from ClariNet’s life and times.

The endgame for Bitcoin

Bitcoin is hot-hot-hot, but today I want to talk about how it ends. Earlier, I predicted a variety of possible fates for Bitcoin ranging from taking over the entire M1 money supply to complete collapse, but the most probable one, in my view, is that Bitcoin is eventually supplanted by one or more successor digital currencies which win in the marketplace. I think that successor will also itself be supplanted, and that this might continue for some time. I want to talk about not just why that might happen, but also how it may take place.

Nobody thinks Bitcoin is perfect, and no digital currency (DigiC) is likely to satisfy everybody. Some of the flaws are seen as flaws by most people, but many of its facets are seen as features by some, and flaws by others. The anonymity of addresses, the public nature of the transactions, the irrevocable transactions, the fixed supply, the mining system, the resistance to control by governments — there are parties that love these and hate these.

Bitcoin’s most remarkable achievement, so far, is the demonstration that a digital currency with no intrinsic value or backer/market maker can work and get a serious valuation. Bitcoin argues — and for now demonstrates — that you can have a money that people will accept only because they know they can get others to accept it with no reliance on a government’s credit or the useful physical properties of a metal. The price of a bitcoin today is pretty clearly the result of speculative bubble investment, but that it sustains a price at all is a revelation.

Bitcoins have their value because they are scarce. That scarcity is written into the code — in the regulated speed of mining, and in the fixed limit on coins. There will only be so many bitcoins, and this gives you confidence in their value, unlike say, Zimbabwe 100 trillion dollar notes. This fixed limit is often criticised because it will be strongly deflationary over time, and some more traditional economic theory feels there are serious problems with a deflationary currency. People resist spending it because holding it is better than spending it, among other things.


While bitcoins have this scarcity, digital currencies as a group do not. You can always create another digital currency. And many people have. While Bitcoin is the largest, there are many “altcoins,” a few of which (such as Ripple, Litecoin and even the satirical currency Dogecoin) have serious total market capitalizations of tens or hundreds of millions of dollars(1). Some of these altcoins are simply Bitcoin or minor modifications of the Bitcoin protocol with a different blockchain or group of participants, others have more serious differences, such as alternate forms of mining. Ripple is considerably different. New Altcoins will emerge from time to time, presumably forever.

What makes one digital coin better than another? Obviously a crucial element is who will accept the coin in exchange for goods, services or other types of currency. The leading coin (Bitcoin) is accepted at more stores which gives it a competitive advantage.

If one is using digital currency simply as a medium — changing dollars to bitcoins to immediately buy something with bitcoins at a store, then it doesn’t matter a great deal which DigiC you use, or what its price is, as long as it is not extremely volatile. (You may be interested in other attributes, like speed of transaction and revocation, along with security, ease of use and other factors.) If you wish to hold the DigC you care about appreciation, inflation and deflation, as well as the risk of collapse. These factors are affected as well by the “cost” of the DigiC.

The cost of a digital currency

I will advance that every currency has a cost which affects its value. For fiat currency like dollars, all new dollars go to the government, and every newly printed dollar devalues all the other dollars, and overprinting creates clear inflation.  read more »

Birth of the World Wide Web

Yesterday, I was interviewed for the public radio program Marketplace and as is normal, 30 minutes come down to 30 seconds. So I wanted to add some commentary to that story.

As you are no doubt hearing today, it was 25 years ago that Tim Berners-Lee first developed his draft proposal for an internet based hypertext system to tie together all the internet’s protocols: E-mail, USENET, FTP, Gopher, Telnet and a potential new protocol (HTTP) to serve up those hypertext pages. He didn’t call it the web then, and the first web tools were not written for a while, and wouldn’t make it to the outside world until 1991, but this was the germ of a system that changed the internet and the world. The first wave of public attention came when the UIUC’s supercomputing center released a graphical browser called Mosaic in 1993 and CERN declared the web protocols non-proprietary. Mosaic’s main author went on to start Mozilla/Netscape, which turned into the Firefox browser you may be reading this with.

As the radio piece explains, many people are confused as to what the difference is between the internet and the web. (They also are unsure what a browser is, or how the web is distinct even from Google sometimes.) To most, the internet was an overnight success — an overnight success that had been developing for over 20 years.

I don’t want to diminish the importance of the web, or TimBL’s contribution to it. He writes a guest editorial on the Google blog today where he lays out a similar message. The web integrated many concepts from deeper internet history.

Prior to the web, several systems emerged to let you use the internet’s resources. Mailing lists were the first seat of community on the internet, starting with Dave Farber’s MSGGROUP in the 70s. In the early 80s, that seat of community moved to USENET. USENET was serial, rather than browsed, but it taught lessons about having a giant network with nobody owning it or being in control.

The large collection of FTP servers were indexed by the Archie search engine, the first internet search engine from McGill University. Greater excitement came from the Gopher protocol from the U. of Minnesota, which allowed you to browse a tree of menus, moving from site to site, being taken to pages, files, local search resources and more all over the internet.

The web was not based on menus, though. It took the concept of hypertext; the ability to put links into documents that point at other documents. Hypertext concepts go back all the way to Vannevar Bush’s famous “Memex” but the man most known for popularizing it was Ted Nelson, who wrote the popular book Comptuer Lib. Ted tried hard for decades to commercialize hypertext and saw his Project Xanadu system as the vision for the future computerized world. In Xanadu, links were to specific points in other documents, were bi-directional and also allowed for copyright ownership and billing — I could link in text from your document and you got paid when people paid to read my document. Hypertext was the base of Apple’s “Hypercard” and a few other non-networked systems.

So did TimBL just combine hypertext with internet protocols to make a revolution? One important difference with the web was that the links were one-way and the system was non-proprietary. Anybody could join the system, anybody could link to anybody, and no permission or money were needed. Embracing the internet’s philosophy of open protocols, while others had built more closed systems, this was a tool that everybody could jump aboard.

Another key difference, which allowed WWW to quickly supplant gopher, was counter-intuitive. Gopher used menus and thus was structured. Structure enables several useful things, but it’s hard to maintain and limits other things you can do. Hypertext is unstructured and produces a giant morass, what we math nerds would call a big directed graph. This “writer friendly” approach was easy to add to, in spite of the lack of plan and the many broken links.

The Web was a superset of Gopher, but by being less structured it was more powerful. This lesson would be taught several times in the future, as Yahoo’s structure menus, which made billions for its founders, were supplanted by unstructured text search from Lycos, Alta Vista and eventually Google. Wikipedia’s anybody-can-contribute approach devoured the old world of encyclopedias.

For the real explosion into the public consciousness, though, the role of Mosaic is quite important. TimBL did envision the inclusion of graphics — I remember him excitedly showing me an early version of Mosaic in 1992 he was playing with — but at the time most of us used USENET, gopher and the very early Web using text browsers, and more to the point, we liked it that way. The inclusion of graphics into web pages was mostly superfluous and slowed things down, making it harder, not easier to get to the meat of what we wanted. The broader public doesn’t see it that way, and found Mosaic to be their gateway into the internet. In addition, many companies and content producers would not be satisfied with publishing online until they could make it look the way they wanted it to look. Graphical browsers allowed for that, but at the time, people were much more interested in the new PDF format which let you publish a document to look just like paper than in the HTML format where you didn’t control the margins, fonts or stylistic elements.

(The HTML specification’s history is one of a war between those who believe you should specify the meaning of the structural elements in your documents and let the browser figure out the best way to present those, and those who want tight control to produce a specific vision. CSS has settled some of that war, but it continues to this day.)

Nobody owned the web, and while Tim is not poor, it was others like Marc Andreesen, Jerry Yang & Dave Filo who would become the early billionaires from it. The web was the internet’s inflection point, when so many powerful trends came together and reached a form that allowed the world to embrace it. (In addition, it was necessary that the Moore’s law curves governing the price of computing and networking were also reaching the level needed to give these technologies to the public.)

25 years ago, I was busy working on the code for ClariNet, which would become the first business founded on the internet when I announced it in June — I will post an update on that 25th anniversary later this year.

More about stolen bitcoins

Yesterday, I wrote about stolen bitcoins and the issues around a database of stolen coins. The issue is very complex, so today I will add some follow-up issues.

When stolen property changes hands (innocently) the law says that nobody in the chain had authority to transfer title to that property. Let’s assume that the law accepts bitcoins as property, and bitcoin transactions as denoting transfer of title, (as well as possession/control) to it. So with a stolen bitcoin, the final recipient is required on the law to return possession of the coin to its rightful owner, the victim of the theft. However, that recipient is also now entitled to demand back whatever they paid for the bitcoin, and so on down the line, all the way to the thief. With anonymous transactions, that’s a tall order, though most real world transactions are not that anonymous.

This is complicated by the fact that almost all Bitcoin transactions mix coins together. A Bitcoin “wallet” doesn’t hold bitcoins, rather it holds addresses which were the outputs of earlier transactions, and those outputs were amounts of bitcoin. When you want to do a new transaction, you do two things:

  1. You gather together enough addresses in your wallet which hold outputs of prior transactions, which together add up to as much as you plan to spend, and almost always a bit more.
  2. You write a transaction that lists all those old outputs as “inputs” and then has a series of outputs, which are the addresses of the recipients of the transaction.

There are typically 3 (or more) outputs on a transaction:

  1. The person you’re paying. The output is set to be the amount you’re paying
  2. Yourself. The output is the “change” from the transaction since the inputs probably didn’t add up exactly to the amount you’re paying.
  3. Any amount left over — normally small and sometimes zero — which does not have a specific output, but is given as a transaction fee to the miner who put your transaction into the Bitcoin ledger (blockchain.)

They can be more complex, but the vast majority work like this. While normally you pay the “change” back to yourself, the address for the change can be any new random address, and nothing in the ledger connects it to you.

So as you can see, a transaction might combine a ton of inputs, some of which are clean, untainted coins, some of which are tainted, and some of which are mixed. After coins have been through a lot of transactions, the mix can be very complex. Not so complex as the computers can’t deal with it and calculate a precise fraction of the total coin that was tainted, but much too complex for humans to wish to worry about.

A thief will want to mix up their coins as quickly as possible, and there are a variety of ways to do that.

Right now, the people who bought coins at Mt.Gox (or those who sent them there to buy other currency) are the main victims of this heist. They thought they had a balance there, and its gone. Many of them bought these coins at lower prices, and so their loss is not nearly as high as the total suggests, but they are deservedly upset.

Unfortunately, if the law does right by them and recovers their stolen property, it is likely that might come from the whole Bitcoin owning and using community, because of the fact that everybody in the chain is liable. Of particular concern are the merchants who are taking bitcoin on their web sites. Let’s speculate on the typical path of a stolen coin that’s been around for a while:

  • It left Mt.Gox for cash, sold by the thief, and a speculator simply held onto the coins. That’s the “easy” one, the person who now has stolen coins has to find the thief and get their money back. Not too likely, but legally clear.
  • It left Mt.Gox and was used in a series of transactions, ending up with one where somebody bought an item from a web store using bitcoin.
  • With almost all stores, the merchant system takes all bitcoin received and sells it for dollars that day. Somebody else — usually a bitcoin speculator — paid dollars for that bitcoin that day, and the chain continues.

There is the potential here for a lot of hassle. The store learns they sold partially tainted bitcoins. The speculator wants and is entitled to getting a portion of her money back, and the store is an easy target to go after. The store now has to go after their customer for the missing money. The store also probably knows who their customer is. The customer may have less knowledge of where her bitcoins came from.

This is a huge hassle for the store, and might very well lead to stores reversing their decisions to accept bitcoin. If 6% of all bitcoins are stolen, as the Mt.Gox heist alleges, most transactions are tainted. 6% is an amount worth recovering for many, and it’s probably all the profit at a typical web store. Worse, the number of stolen coins may be closer to 15% of all the circulating bitcoins, certainly something worth recovering on many transactions.

The “sinking taint” approach

Previously, I suggested a rule. The rule was that if a transaction merges various inputs which are variously reported as stolen (tainted) and not, then the total percentage be calculated, and the first outputs receive all the tainting, and the latter outputs (including the transaction fee, last of all) be marked clear. One of the outputs would remain partial unless the transaction was designed to avoid this. There is no inherent rule that the “change” comes last, it is just a custom, and it would probably be reversed, so that as much of the tainted fraction remains in the change as possible, and the paid amount is as clean as possible. Recipients would want to insist on that.

This allows the creation of a special transaction that people could do with themselves on discovering they have coin that is reported stolen. The transaction would split the coin precisely into one or more purely tainted outputs, and one or more fully clean outputs. Recipients would likely refuse bitcoin with any taint on it at all, and so holders of bitcoin would be forced to do these dividing transactions. (They might have to do them again if new theft reports come on coin that they own.) People would end up doing various combinations of these transactions to protect their privacy and not publicly correlate all their coin.

Tainted transaction fees?

The above system makes the transaction fee clean if any of the coin in the transaction is clean. If this is not done, miners might not accept such transactions. On the other hand, there is an argument that it would be good if miners refused even partially tainted transactions, other than the ones above used to divide the stolen coins from the clean. There would need to be a rule that allows a transaction to be declared a splitting transaction which pays its fees from the clean part. In this case, as soon as coins had any taint at all, they would become unspendable in the legit markets and it would be necessary to split them. They would still be spendable with people who did not accept this system, or in some underground markets, but they would probably convert to other currencies at a discount.

This works better if there is agreement on the database of tainted coins, but that’s unlikely. As such, miners would decide what databases to use. Anything in the database used by a significant portion of the miners would make those coins difficult to spend and thus prime for splitting. However, if they are clean in the view of a significant fraction of the miners, they will enter the blockchain eventually.

This is a lot of complexity, much more than anybody in the Bitcoin community wants. The issue is that if the law gets involved, there is a world of pain in store for the system, and merchants, if a large fraction of all circulating coins are reported as stolen in a police report, even a Japanese police report.

What if somebody steals a bitcoin?

Bitcoin has seen a lot of chaos in the last few months, including being banned in several countries, the fall of the Silk Road, and biggest of all, the collapse of Mt. Gox, which was for much of Bitcoin’s early history, the largest (and only major) exchange between regular currencies and bitcoins. Most early “investors” in bitcoin bought there, and if they didn’t move their coins out, they now greatly regret it.

I’ve been quite impressed by the ability of the bitcoin system to withstand these problems. Each has caused major “sell” days but it has bounced back each time. This is impressive because nothing underlies bitcoins other than the expectation that you will be able to use them into the future and that others will take them.

It is claimed (though doubted by some) that most of Mt.Gox’s bitcoins — 750,000 of them or over $400M — were stolen in some way, either through thieves exploiting a bug or some other means. If true, this is one of the largest heists in history. There are several other stories of theft out there as well. Because bitcoin transactions can’t be reversed, and there is no central organization to complain to, theft is a real issue for bitcoin. If you leave your bitcoin keys on your networked devices, and people get in, they can transfer all your coins away, and there is no recourse.

Or is there?

If you sell something and are paid in stolen money, there is bad news for you, the recipient of the money. If this is discovered, the original owner gets the money back. You are out of luck for having received stolen property. You might even be suspected of being involved, but even if you are entirely innocent, you still lose.

All bitcoin transactions are public, but the identities of the parties are obscured. If your bitcoins are stolen, you can stand up and declare they were stolen. More than that, unless the thief wiped all your backups, you can 99.9% prove that you were, at least in the past, the owner of the allegedly stolen coins. Should society accept bitcoins as money or property, you would be able to file a police report on the theft, and identify the exact coin fragments stolen, and prove they were yours, once. We would even know “where” they are today, or see every time they are spent and know who they went to, or rather, know the random number address that owns them now in the bitcoin system. You still own them, under the law, but in the system they are at some other address.

That random address is not inherently linked to this un-owner, but as the coins are spent and re-spent, they will probably find their way to a non-anonymous party, like a retailer, from whom you could claim them back. Retailers, exchanges and other legitimate parties would not want this, they don’t want to take stolen coins and lose their money. (Clever recipients generate a new address for every transaction, but others use publicly known addresses.)

Tainted coin database?

It’s possible, not even that difficult, to create a database of “tainted” coins. If such a database existed, people accepting coins could check if the source transaction coins are in that database. If there, they might reject the coins or even report the sender. I say “reject” because you normally don’t know what coins you are getting until the transaction is published, and if the other party publishes it, the coins are now yours. You can refuse to do your end of the transaction (ie. not hand over the purchased goods) or even publish a transaction “refunding” the coins back to the sender. It’s also possible to imagine that the miners could refuse to enter a transaction involving tainted coins into the blockchain. (For one thing, if the coins are stolen, they won’t get their transaction fees.) However, as long as some miner comes along willing to enter it, it will be recorded, though other miners could refuse to accept that block as legit.  read more »

End the redirect wrapper on links

A lot of sites, most notably search engines like Google, like to rewrite all the links on their pages. So search for this page and instead of, the link Google gives you is or similar. (I have redacted the actual codes.)

What’s happening is that when you click on the link, you really go to Google. Google records what you clicked on and other parameters related to the search so they can study just how people use their search engine, what they click on and when. It’s a reasonable thing for them to want to study, though a potential privacy invasion.

Because each click goes through Google, your clicks are slowed down. Because Google has such huge resources, and is almost never down, you usually don’t notice it, though even with Google you will see the delay on slow links, like mobile GPRS and Edge connections. It also means you can’t easily cut and paste links from search results.

Other sites are not as good. They sometimes noticeably slow own your click. Worse, they sometimes break it. For example, on my phone, when I click on links in LinkedIn messages, as well as Facebook ones, which are also redirected, it doesn’t work if I’m not currently logged in to those sites. Due to some bad code, it also wants to send the link to the mobile apps of these sites, which is not what I want. (The one for LinkedIn is particularly broken, as it doesn’t seem to know where the app is, and sends me to the Play store to install it even though it is already installed.)

In other words, these links break the web from time to time. They can also interfere with spiders. On the plus side, they can be set to protect your privacy by hiding data in the REFERER field from the target web site. For sites that have been identified ad malicious, they can provide a warning.

To fix this, sites can change all their links to be javascript. The link can be a real target, and associated onClick javascript can also send a web hit back to the server with the logging info.

A better solution would be to push use of the “ping” attribute in the HTML spec, and allow links to have both an href to the target, and another URL which gets invoked when the link is clicked. In the background, this would not slow down your click, or break it. Browsers could also elect to block it, which the sites might not like but is good for users. Links to malicious sites could be treated differently if that’s part of the service. There would also be no need to fake the status window when moving the mouse over the link, as must be done with redirects.

Let’s say no to all these redirects.

Satoshi, is now the time to consider donating lots of bitcoin to charity?

I don’t know who the person or people are who, under the name Satoshi Nakamoto, created the Bitcoin system. The creator(s) want to keep their privacy, and given the ideology behind Bitcoin, that’s not too surprising.

There can only be 21 million bitcoins. It is commonly speculated that Satoshi did much of the early mining, and owns between 1 million and 1.5 million unspent bitcoins. Today, thanks in part to a speculative bubble, bitcoins are selling for $800, and have been north of $1,000. In other words, Satoshi has near a billion dollars worth of bitcoin. Many feel that this is not an unreasonable thing, that a great reward should go to Satoshi for creating such a useful system.

For Satoshi, the problem is that it’s very difficult to spend more than a small portion of this block, possibly ever. Bitcoin addresses are generally anonymous, but all transactions are public. Things are a bit different for the first million bitcoins, which went only to the earliest adopters. People know those addresses, and the ones that remain unspent are commonly believed to be Satoshi’s. If Satoshi starts spending them in any serious volume, it will be noticed and will be news.

The fate of Bitcoin

Whether Bitcoin becomes a stable currency in the future or not, today few would deny it is not stable, and undergoing speculative bubbles. Some think that because nothing backs the value of bitcoins, it will never become stable, but others are optimistic. Regardless of that, today the value of a bitcoin is fragile. The news that “Satoshi is selling his bitcoins!” would trigger panic selling, and that’s bad news in any bubble.

If Satoshi could sell, it is hard to work out exactly when the time to sell would be. Bitcoin has several possible long term fates:

  1. It could become the world’s dominant form of money. If it replaced all of the “M1” money supply in the world (cash and very liquid deposits) a bitcoin could be worth $1 million each!
  2. It could compete with other currencies (digital and fiat) for that role. If it captured 1% of world money supply, it might be $10,000 a coin. While there is a limit on the number of bitcoins, the limit on the number of cryptocurrencies is unknown, and as bitcoin prices and fees increase, competition is to be expected.
  3. It could be replaced by one or more successors of superior design, with some ability to exchange during a modest window, and then drifting down to minimal value
  4. It could collapse entirely and quickly in the face of government opposition, competition and other factors during its bubble phase.

My personal prediction is #3 — that several successor currencies will arise which fix issues with Bitcoin, with exchange possible for a while. However, just as bitcoins had their sudden rushes and bubbles, so will this exchange rate, and as momentum moves into this currency it could move very fast. Unlike exchanges that trade bitcoins for dollars, inter-cryptocurrency exchanges will be fast (though the settlement times of the currencies will slow things down.) It could be even worse if the word got out that “Satoshi is trading his coins for [Foo]Coin” as that could cause complete collapse of Bitcoin.

Perhaps he could move some coins through randomizing services that scramble the identity association, but moving the early coins to such a system would be seen as selling them.  read more »

No, we don't want much more Fedex and UPS on Dec 24

A big story this Christmas was a huge surge in the use of rush shipping in the last 2 days before Christmas. Huge numbers of people signed up for Amazon Prime, and other merchants started discounting 2 day and overnight shipping to get those last minute sales. In turn, a lot of stuff didn’t get delivered on time, making angry customers and offers of apology discounts from merchants. This was characterized as a “first world problem” by many outside the game, of course.

When I shop, I am usually travelling outside the US and so I have to get stuff even before the 24th, and I’ve had stuff I left to the last day not delivered several times, so I know to avoid doing it. Some packages are not going to make it, and this should be expected — even desired.

While it makes sense to increase the infrastructure a bit as online shopping grows in popularity, you don’t want to go nuts at Christmas. If you need to build your infrastructure to handle every Christmas gift, you have to build it too big, and you pay for that through higher prices the rest of the year. Shippers need to figure out their real capacity, and everybody needs to plan based on it.

The failure this season was not a failure of the delivery system. Rather it was a failure of either the shippers to tell the merchants what their capacity was, and/or a failure of the merchants to communicate to customers that too much was being shipped and not everybody could be promised Dec 24 delivery.

The obvious way to fix this is first to have the shippers get a solid handle on their capacity for the various types of shipping to the various destinations. They can also identify the bottlenecks and widen them a modest amount.

The next thing is for the merchants to know just how much shipping they can buy. There can either be a live spot market — so the merchant web sites just stop offering the delivery promise when the capacity is reached, or merchants could even attempt to pre-contract for capacity, paying for it whether they need it or not (or reselling it if they know they won’t need it.) Merchants should be building their own forecasts about available capacity and querying shippers for updates on just how much more is left. Capacity isn’t a fixed thing — it depends on the size of packages and where they are going and many other things — but this is a problem computers can handle.

Finally, the shippers and the merchants can start increasing the price of the rush shipping so that demand and supply match. This can be based on accurate forecasts, or just live data. As Dec 23rd wears on, the price of next-day shipping will keep going up and up so only the serious buy it. Of course, this might reveal just how keen some people are to get items, and justify having more capacity in years to come. Indeed, as the price goes up, it may make sense for Amazon to say, “Listen, we’re just going to buy this for you at your local Wal-Mart, it will be waiting for you there.” Wal-Mart surely won’t mind that.

There are also some tricks to increase capacity. For example, most people would probably tolerate having to pick up items at a retail location — FedEx and UPS and the USPS of course have tons of those — especially if it is the only option or offers a serious discount over surge priced home delivery. (This is not as good for sending gifts to remote locations.) Temporarily contracted depots could also be used. You want to streamline these depots, as lots of people will be coming in, so you want some nice system where people bring in a bar code and everything is optimized to get them out the door with the right package quickly.

All of this will push people to shop and ship a little earlier, smoothing out the rush, and avoiding having to design the system for one peak day. I have always found it remarkable that most stores and malls have giant parking lots (back in the brick and mortar world) which are only filled in December. It’s such a waste — but something robocars will fix in the future.

Delivery to the wrong address

I had a missed delivery myself this year. In this case it was on December 14th because I went home early, and I had the gifts arriving 2 days before I left. But oddly, I got the note that the package had been delivered at 6pm — but it wasn’t. Both UPS and Amazon had very little set up to handle this. Amazon’s system insists you wait at least a day to complain about this, which was no help to me. I could have used that day to replace the items if I were sure it wasn’t coming.

After I left, the package showed up on my porch on Sunday. UPS does not operate Sunday so it seems pretty likely they had left the package with a neighbour who was perhaps away for a few days. I presume the neighbour eventually came and dropped off the package but they left no note. (Of course I wish they had done it right away — replacing the gifts in Canada cost me a bunch extra.)

Amazon had already given a refund — fairly good service there — and so I just had UPS return the package as undelivered which costs me nothing, so that all worked out, except the scramble and the extra cost of replacing the items.

I don’t know how often this happens — it’s in the Amazon FAQ so it must be often enough — but there are some obvious fixes. The UPS driver’s wand, which scans the package on delivery, should record more data, including any location from a GPS in the wand or the truck, but perhaps more easily the MACs and signal strengths of any WIFI nodes visible when the package was scanned.

That information would have both allowed UPS to say, “OK, that’s odd, this doesn’t match where the package should be going” right when it was scanned, or it would have allowed me to figure out where it went and get it right away.

You’re probably wondering, didn’t I just imagine it was stolen? I did consider that possible, though in my safe neighbourhood it doesn’t appear to be a real danger. Somebody following UPS trucks at Christmas time to steal gifts would be very Grinchey, not to say it doesn’t happen. In safe neighbourhoods, UPS and Fedex routinely just leave packages at the door. Not actually signed for, I presume they just eat the loss the rare times they are stolen, or perhaps the merchant does. It’s small enough shrinkage that the system handles it.

Having secure open wifi (Death to wifi login part 2)

In part 1 I outlined the many problems caused by wifi login pages that hijack your browser (“captive portals”) and how to improve things.

Today I want to discuss the sad state of having security in WIFI in most of the setups used today.

Almost all open WIFI networks are simply “in the clear.” That means, however you got on, your traffic is readable by anybody, and can be interfered with as well, since random users near you can inject fake packets or pretend to be the access point. Any security you have on such a network depends on securing your outdoing connections. The most secure way to do this is to have a VPN (virtual private network) and many corporations run these and insist their employees use them. VPNs do several things:

  • Encrypt your traffic
  • Send all the traffic through the same proxy, so sniffers can’t even see who else you are talking to
  • Put you on the “inside” of corporate networks, behind firewalls. (This has its own risks.)

VPNs have downsides. They are hard to set up. If you are not using a corporate VPN, and want a decent one, you typically have to pay a 3rd party provider at least $50/year. If your VPN router is not in the same geographic region as you are, all your traffic is sent to somewhere remote first, adding latency and in some cases reducing bandwidth. Doing voice or video calls over a VPN can be quite impractical — some VPNs are all TCP without the UDP needed for that, and extra latency is always a killer. Also, there is the risk your VPN provider could be snooping on you — it actually can make it much easier to snoop on you (by tapping the outbound pipe of your VPN provider) than to follow you everywhere to tap where you are.

If you don’t have a VPN, you want to try to use encrypted protocols for all you do. At a minimum, if you use POP/IMAP E-mail, it should be configured to only get and receive mail over TLS encrypted channels. In fact, my own IMAP server doesn’t even accept connections in the clear to make sure nobody is tempted to use one. For your web traffic, use sites in https mode as much as possible, and use EFF’s plugin https everywhere to make your browser switch to https wherever it can.  read more »

Canada to stop urban mail home delivery, but fails to abolish snail-mail

Here in Canada, a hot political issue (other than disgust with Rob Ford) is the recent plan by Canada Post to stop home delivery in cities. My initial reaction was, “Wow, I wish we could get that in the USA!” but it turns out all they are doing is making people go to neighbourhood mailboxes to get their mail. For many years, people in new developments have had to do this — they install a big giant mailbox out on the street, and you get a key to get your mail. You normally don’t walk further than the end of your block. However, this will save a lot of work — and eliminate a lot of jobs, which also has people upset.

But let me go back to my original reaction — I want to see home letter delivery abolished.

Why? All I, and most other people get by mail are:

  • Junk mail (the vast bulk of the mail.)
  • One or two magazines
  • Bills and communications from companies that refuse to switch to all-electronic communication
  • Official notices (from governments who refuse to switch to all-electronic communication)
  • Cheques from companies who refuse to do direct deposit (see note below.)
  • Parcels (lots of these, though many more from UPS/Fedex/etc.)
  • A tiny and dwindling number of personal cards and letters. Perhaps 2-3 personal xmas cards.

The abolition of general mail delivery would force all those parties who refuse to do electronic communication to switch to it. The concept of an official e-mail address would arise. We would also need to see a better e-cheque service, something priced like a cheque (ie. not paypal which takes 2% or more) and as easy to use (ACH is not there yet.) This would force it into existing if you could not mail a cheque.

A replacement for registered mail would need to arise — that is what is needed for legal service. Putting that into e-mail is doable though challenging, as it requires adding money to e-mail, because you want people to have to pay to use it so that you don’t get it all the time.

And of course, parcel service would continue. And people who really want to send a letter could send it via parcel service, but not for sub-dollar first class mail prices.

Magazines would have to go all-electronic. Some may not see the world ready for that, but I think the time is very near. Today, one can make cheap large tablets in the 14 to 17 inch size that would be great for magazines. They would be too heavy to handhold (though possibly if they had no batteries and used a small cord they could be light enough for that) but they could easily be held on laps and tables and replace the magazine.

Few would mourn the death of junk mail, though it might lead to more spam in e-mail boxes until that’s under control. Senders of junk mail (notably politicians) might mourn it.

So the only sad thing would be the loss of the dwindling supply of personal letters. People getting married could use the parcel companies or go electronic. Thank-you notes would go electronic, making Miss Manners spin in her grave, but spin she eventually will. Truth is, the parcel companies would probably start up a basic letter service priced higher than 1st class mail but less than their most basic parcel. The more addresses you can share the cost of a truck on, the better — until the deliverbots arrive, at least. This is not easy, though. The postal service got to use the economies of delivering several letters a day to your house, and this could pay for a person to walk the street with a bag full, while the parcel companies use trucks.

We all know this day is coming. The question is, can we do better if we force it, and shut down letter delivery sooner rather than later?

Syndicate content