Brad Ideas - Data hosting instead of data portability - Comments

Data Spaces in the Clouds

Kingsley Idehen — Thu, 08 May 2008 13:55:39 -0700

Brad,

What you describe is what I've referred to as Data Spaces in the Clouds (Fourth Platform) for a while :-)

Yes, there is some confusion about the literal interpretation of the phrase: Data Portability (free movement of data across platforms). We don't necessarily want free movement of data across realms (unless we explicity enable it in our space). Instead, I believe we seek Open Access to our Data Spaces with access control granularity.

To conclued, we do need Data Access by Reference facilitated by portable Data Containers (Data Spaces) in the Clouds. Of course, these containers can move themselves, or data from the clouds to other locations, wholesale or via replication and synchronixation. In all cases using standard protocols and existing infrastructure such as the Internet and Web.

Links:

1. OpenLink Data Space Wikipedia Page
2. OpenLink Data Spaces (Open Source Edition) Home Page
3. My Data Space Profile Page
4. EC2 installation Guide
5. How to get a Gateway into your Data Space (i.e a URI for Your Data Space) in 5 minutes or less
6. Recent WWW2008 Presentation about Data Portability and Data Accessibility (PPT)

FoF apps

brad — Wed, 07 May 2008 11:26:57 -0700

As I’ve noted, FoF apps turn out to be much less interesting than people thought at first. Do you really look at the photos of your FoFs? The main FoF app that seems to be useful is LinkedIn’s “search your network” which can answer questions like, “Who can I contact at Company X” and dating introductions. FoFoF turns out to be surprisingly non-useful.

However, I won’t proclaim that nobody can think of useful apps (or simply entertaining) apps here. So there need to be solutions, even if it turns out to be that those apps get access to large networks, but are the only ones that do. (Remember, our alternative today is zillions of apps getting access to this data.)

I don’t expect home PCs to be required here. Everybody wants an always on host for many functions. You don’t need it in the sense that I don’t think “Ask 100 hosts to search for a query” is a good implementation, but I would use the local hosts just as a way to do things efficiently for the user when the user is signed on. The cloud host would do things for others. Client data hosts and cloud data hosts would sync.

Your particular app, and other FoF apps, could be implemented, somewhat less efficiently, with data updates. That is to say, if you are using an FoF app, you would send changes to your friends, and they would forward those changes on to their friends as part of the update stream. In this case everybody is storing all the basic data (not big things like photos, just smaller stuff, including the URLs/access tokens of the photos) on their own host, and apps can operate on it. This is why it does not scale to FoFoF, but I think it could handle FoF if the updates are not large.

Of course, you, your Fs and your FoFs must be running the same application, but that is the same as saying they are all members of Flickr. To implement Flickr you need more though, and it may not even be possible to implement all apps in this manner. Still better than implementing them all in a central repository manner, though.

Interesting approach

brad — Wed, 07 May 2008 11:16:17 -0700

But I haven’t studied the underlying systems proposed enough to judge if they can do it. One concern I see immediately regards whether developers can be talked into it. Part of the sex appeal of web 2.0 (meaning apps in the cloud) is that developers get free reign to write and maintain their apps using whatever platforms and tools they like. They are no longer limited to even the constraints and problems of writing code for a user’s PC. Users at the same time love not having to install software, having somebody else maintain it all, and having to roam.

My own proposals face this problem too. These abilities are very attractive to users and developers, and as long as they can get the functionality (which javascript has now given) they will rush to them.

It is for this reason that I have decided that some compromises will be needed, that we won’t get to the level where we can run a malicious app on our data. That’s because the programming hoops required to use a system that bars malicious apps may be too involved. Happy to be proven wrong, though. I would be happy just to reach the level where apps don’t end up taking more data than they need, and don’t end up storing copies of it.

W5 Project at MIT

Evan Jones — Wed, 07 May 2008 04:51:52 -0700

The W5 project at MIT is looking at ways to solve these issues:

http://pdos.csail.mit.edu/~max/docs/w5.pdf

Searches

Radovan Semancik — Wed, 07 May 2008 01:14:33 -0700

I really wonder how you can efficiently implement this use case:

"Show me the flickr photos that my friends and the friends of my friends faved in last 2 weeks, sorted by the total number of favs", while the photos are still stored on flickr.

I can see you can implemente that with "agent" that will crawl to the hosting sites of your friends and their friends, collect the data and come back. But that would be slow, especially if some of your friends host their data in PCs that are currently offline (remember The Eight Fallacies of Distributed Computing?). Do you have any solution for that in mind?

BTW, creating a Virtual Machine you still provide API to the applications. But it is quite broad, which means it is difficult to control security.

Finding the host

brad — Tue, 06 May 2008 14:48:00 -0700

Well, in the past we have used DNS as a way to name hosts and move them around. This could be used here, but not a high level domain. I might get bradsdata.datadns.com as a subdomain that I can point at whatever data host I like. However, there could be other indirection or discovery protocols.

Typically I see a main site providing your interface to social data apps. That site would embed other web pages which are served from your data host, running code provided by app providers. The DNS would direct your own browser at your own data host of choice. (In fact, this architecture allows the data host to be your own PC, if you don’t need to roam. Your own pc at localhost:port would see a request for a social app window. The data host program on your PC would connect to the specified remote application’s server for any code updates or special data, download them if it doesn’t have them cached, execute the code and return the results to the iframe in the browser page.)

I think I’ll update about that.

Now as for the central repository: This is complex. People are saying, it seems, that they don’t want their data scattered around everywhere, both because of lack of control, but more commonly because the UI to give apps access to it is too complex. If we can develop a good UI so that it is easy to give apps just the data they need, and no more, then scattering can be good. The data hosting model does not dictate about scattering or centralization, but I agree that users will tend to centralize, just for ease of control. A central server contracted to me may be better than 30 servers with only loose bonds to me, such as 30 different social app companies each knowing different subsets of my data.

Service portability good. Centralized servers not necessary.

Joe Andrieu — Tue, 06 May 2008 12:26:43 -0700

Good post.

It's clear that the data portability model is limited, but I would even go beyond hosting portability to service portability, and I would separate the hosting company from the value-added service provider.

First, the hosting model still leaves you at the mercy of the host if/when you decide to move... you still have to update your address at all of the service providers who might access the services of your hosting company. A more robust model would provide service portability through service discovery. Service providers don't need to know where it the data is hosted, but rather where they can find the current host. That gives you a layer of indirection that lets you move your hosting company without needing to remember which service providers are currently relying on that host.

Second, you suggest that the hosting company's job is to perform actions on your data.

Why is that?

My DNS host doesn't perform functions on my DNS. Nor do I expect my webhost to perform actions on my website, although I do like to have a range of services I can easily install and run (such as installing WordPress through Fantastico).

I would argue that there is an inherent conflict of interest in the hosting company providing value-added services, and that in fact, what we should do is design an architecture where hosting is functionally distinct from value-added services. Any authorized value-add service provider should be able to access your data services, which leads to a cleaner architecture where companies that happen to provide both hosting and value-add services can do so with clear contracts and authorization.

Finally, there's no real need to have all of my services at the same hosting company, just like my DNS, my website, and my email can all painlessly be hosted anywhere I like. As long as the services can be discovered, there's no reason to have any individual's services centralized, nor a need to centralize many individuals' data in one place.

The collaborative/co-op type negotiating strategy you suggest can easily be implemented by a value-add service provider, independent of the source of the data. Users simply join that co-op or buying group and point the co-op to their discovery service. Flash-mobs rejoice.

All of which is to say, you are definitely going down the right path. More portability more better.

-j

It matters where it is hosted

brad — Tue, 06 May 2008 11:42:59 -0700

Because we want to lay down the duties of the people who host it for us, if we don’t host it ourselves. (And most people won’t host it themselves.)

If a host has just one duty, to keep the data safe and allow only authorized actions on it, then that is what the host will focus on. If the host is facebook, whose duty is to find ways to monetize its database to maximize shareholder value, you will get a different result.

That’s why the layers, and why for most people, not having too many hosts. (Some might decide to have multiple hosts to maintain multiple persona, such as a personal and work persona, but that’s their decision.)

Up to a point...

Danny Ayers — Tue, 06 May 2008 04:35:09 -0700

I agree with most of your arguments, but am not sure about the conclusions - in particular I believe we essentially have most of the necessary infrastructure. Rather than thinking in terms of data being maintained in SQL DBs on or off the web, consider the web itself as the database. It really doesn't matter where the data is hosted as long as appropriate connections are in place. See http://en.wikipedia.org/wiki/Linked_Data

Data hosting instead of data portability

brad — Mon, 05 May 2008 20:08:19 -0700

I’ve been ranting of late about the dangers inherent in “Data Portability” which I would like to rename as BEPSI to avoid the motherhood word “portability” for something that really has a strong dark side as well as its light side.

But it’s also important to come up with an alternative. I think the best alternative may lie in what I would call “data hosting.” It’s a layered system, with a data layer and an application layer on top.

A data hosting approach has your personal data stored on a server chosen by you. (You might have that server right in your own house, or pay for hosting services.) If you pay, that server’s duty is not to exploit your data, but rather to protect it. That’s what you’re paying for. You can have more than one (with different personas, if you like) but for now let’s imagine having just one.

Your data host’s job is to perform actions on your data. Rather than giving copies of your data out to a thousand companies (the Facebook and Data Portability approach) you host the data and perform actions on it, programmed by those companies who are developing useful social applications.

As such, you don’t join a site like Facebook or LinkedIn. Rather, companies like those build applications which can run on your data. They don’t get the data, rather they write code to do things with it that runs in a protected sandbox on your data host.