For some time I've been advocating a concept I call the Data Deposit Box as an architecture for providing social networking and personal data based applications in a distributed way that tries to find a happy medium between the old PC (your data live on your machine) and the modern cloud (your data live on 3rd party corporate machines) approach. The basic concept is to have a piece of cloud that you legally own (a data deposit box) where your data lives, and code from applications comes and runs on your box, but displays to your browser directly. This is partly about privacy, but mostly about interoperability and control.

This concept depends on the idea of publishing and subscribing to feeds from your friends (and other sources.) Your friends are updating data about themselves, and you might want to see it -- ie. things like the Facebook wall, or Twitter feed. Feeds themselves would go through brokers just for the sake of efficiency, but would be encrypted so the brokers can't actually read them.

There is a need for brokers which do see the data in certain cases, and in fact there's a need that some types of data are never shown to your friends.

Crush

One classic example is the early social networking application the "crush" detector. In this app you get to declare a crush on a friend, but this is only revealed when both people have a mutual crush. Clearly you can't just be sending your crush status to your friends. You need a 3rd party who gets the status of both of you, and only alerts you when the crush is mutual. (In some cases applications like this can be designed to work without the broker knowing your data through the process known as blinding (cryptography).)

Proximity

Another example is proximity detection. You might want your tools to notice when you and a friend are physically near one another and notify you. But you don't want to send your friend a complete log of all your movements. Here's where you need a "forgetful" broker. Both of you send your location to the broker, that checks for proximity (again possibly with blinding) and informs you of the real location when you are close. Otherwise it is contractually bound to forget your location. To protect your privacy, the forgetful broker has to be smart and not let you update your location at high speed so that people can't "search" for you. (Though if they find you, you know they found you so this ability is limited.)

Invisible undo or take-back

Harder to implement is the ability to "undo" changes and messages. While a feed mechanism can include the ability to retract or change an update, nothing stops the friend from remembering the old status and in fact highlighting it because something unusual happened to it. While many people have sought an "un-send" for e-mail this just isn't possible in a P2P e-mail world. It is possible in a centralized world, if you trust the central server, and this may attract people to centralized approaches unless there is a solution for decentralized tools. This remains difficult, even with a broker, and in fact even centralized tools are not perfect.

With any tool, it's possible to write a bot which constantly checks for new data and downloads it, assuring the bot still has a copy even if it gets deleted. We depend on people to not have such bots in ordinary use, and in fact social sites try to limit the activity of bots. But short of having Turing Tests like captchas, it can't be stopped. Nonetheless, it's even easier in a system where fetching of data from feeds or brokers is meant to be automated.

Worse, there is a technical reason you want to fetch new things like E-mails immediately, so that they can be cached locally, and indexed. If you don't actually fetch the body of the email until the user opens it, you run the risk of the remote server being down, or slow. You also can't scan the mail for things in advance. There are also privacy issues -- many people do not want others to know when they have read their messages or whether they have read them. Here you might have a forgetful broker do as much of this as possible -- including providing quality service of the mail body when the time is ready -- and try to watch for bot-like activity by the user. It's hard to be perfect but one can be similar to the centralized services. This requires fairly sophisticated brokers, which adds cost as they must be paid for.

Security and money

Security also presents a problem. It may be hard to do things like distributed games, as you can't expect people to store secured values (like how many gold pieces you have) on their own server. While there are offline digital money systems, they are complex and are overkill for applications like that. Many social games have simple exchanges of virtual value going on all the time, and these will need to go through a broker run by the game, or use a more complex digital signature scheme.

In a distributed world it is a blessing and a curse that there are no restrictions on what code can do with data you feed to your friends. While facebook might have the power to ban apps that do things it doesn't like, an open world does not have that power. Facebook doesn't want you to know if somebody un-friends you but you can't stop a distributed system from making a big deal out of it. Sometimes people view that power as useful, sometimes it is a big inhibitor. On the whole, having a gatekeeper is generally a negative, but the times when it is positive can be very compelling to users.

Fortunately, most information doesn't need a fancy broker. But as developers come up with cool new apps, it is necessary that the infrastructure exist to create them.

Examples of feed items

Here's an example of some of the things that might flow in a feed to a friend. Of course, not all these would flow to every friend, and some subscribers to your feed would also be businesses. I would be interested in comments on things apps like to see about others.

• General status updates (tweets)
• Changes to generally-static personal data (ie. relationship status -> married)
• Location updates
• Messages (e-mails)
• Invitations
• Version number updates
• Changes in subscription & requests for greater access
• Broker negotiations
• Specialized application messages and updates (Crush, trip plans, event items and many, many more.)

If the list can be broken down to a small number of general categories, the need for brokers may be reduced.