Brad Ideas - Best Of Blog - Comments

Pharm

brad — Thu, 15 May 2008 17:57:00 -0700

Oh, I don’t need to go beyond commercial motives to find a reason for Pharm. But more to the point, Pharm is domestic.

The hallmark of a signals intelligence espionage program would be intrusion into foreign computers, ideally non-allied computers or targeted computers, which is within the balliwick of most of these organizations.

Indeed, there might be a desire to simply scan lots of hard drives in rural Pakistan and Afghanistan. And North Korea (the few that are on the internet) and other places, staying away from spying on computers belonging to allies and domestic parties. They could write code to examine machines and determine if they are domestic, or owned by domestic companies. Or even code to say, “Does this computer look like it might be owned by a jihadi?” — and then start spying on just those computers.

The recent trend in intelligence has been to look for ways to do blanket basic surveillance and then isolate the few actual targets they want to put human beings on. Of course, in the domestic case, such as AT&T, the law says they can’t do this. But they want to do it, and in fact we allege in our lawsuit that they did do this — put in a splitter to divert all data into NSA systems. If they are doing that in the domestic arena, seems likely they are doing it overseas where there is less control.

Targetted Advertisers?

Anonymous — Thu, 15 May 2008 17:08:01 -0700

Who is behind the so-called targetted advertising systems companies like Phorm are trying to get installed on ISP internal networks? Being able to snoop 70% of the UK's clickstream is an intelligence tool worth billions. Putting aside the fact that Phorm's system breaks a dozen laws, who quality assures or positive vets companies like this? I would've thought allowing a known spyware company that games the legal system and employs foreign nationals to code its software would be a red flag for someone.

It gets worse, what if it

fnord — Thu, 15 May 2008 02:26:16 -0700

It gets worse, what if it was organized crime, instead of or in addition to spy agencies, engaging in such activities, which of course they are. Spy agencies are normally engaged in criminal activity too, so that's not what makes it worse.

What makes it worse is that ICANN has allowed itself on various levels including registrars and the DNS itself to be increasingly co-opted by organized crime. ICANN's insatiable hunger for money is largely to blame, and criminals have long known how to exploit such a weakness. Read this current article on ICANNWatch.org and follow the link in Fergie's subsequent comment, or my more accurate link to RBNBlog which follows.

The US Government has always been ICANN's overseer, have they just been asleep at the switch? Isn't it ironic that the organized crime Russian Business Network, with likely ties to the Russian government (and they have recently moved some of their activities to mainland China), is co-opting various critical levels of the internet? This is going to turn out badly. -g

2-year contracts vs. Prepaid lines

David Monget — Wed, 14 May 2008 17:14:34 -0700

Brad, your blog is just terrific!
It's great the insight you made with the contracts. I have never signed a contract before, but I felt terrified already with the experiences some friends had with termination fees and other silly monthly charges. I found this webpage trying to get some advice from other users to get a plan without signing a 1 or 2-year contract with any company. I just don't like to surrender the freedom of choice and change. I have been used prepaid from T-Mobile To-Go so far, it works well but it's pretty pricey (around an additional 50-70% and there are no free wkds or nights).
Thanks again Brad for sharing this.

data can go both ways

Anon Y. Mouse — Tue, 13 May 2008 23:48:44 -0700

First, I think it's much more likely the NSA has already
compromised one or more existing botnets, as opposed to
screwing up Windows. That's Microsoft's job!

Try this on for size. Everybody's heard about the
secret room(s) at AT&T et. al. run by the NSA and supposedly
designed to slurp up all the internet packets, telco phone
meta-data, etc. Perhaps they are covertly working with
the backbone operators to provide peering points where
they can *inject* massive amounts of traffic generated
by their own custom dedicated servers. It's more reliable
if you have your own botnet.

This is possible

brad — Tue, 13 May 2008 21:02:42 -0700

But in fact it might be simpler to just assure there are security flaws, and then build the botnets, or have them built by your front organizations, which are of course not in the USA. Or just exploiting those that are already there.

Directly compromising Windows is a dangerous thing for the NSA to do. Aside from the fact it is still not supposed to operate inside the USA at all, this could hurt the security of Americans against foreign spies. In fact, the NSA is supposed to be helping to make U.S. computers more secure, it is part of their mission. To go directly against that mission is not beyond them but scandalous if discovered.

Of course this does not apply to foreign spy agencies, they could compromise Windows without breaking their rules. But since Windows is run in so many countries, again this has the risk of scandal.

On the other hand, paying botnet rings to run secret code on non-domestic computers to spy on the owners of those computers would not be the same sort of major scandal. (Creation of the botnets directly would be a scandal, but one they can hide much more easily.) They would mostly get an “attaboy” for spying on foreign computers. They might create a problem with allies if they spied on the computers of allied governments if they were caught, but frankly everybody knows that each spy agency spies on its allies. It’s part of the game, though still not something to be caught at.

To do this, they would want to build a system that can identify honeypots and make sure never to put spy code into them. That’s hard to do for a criminal hacker ring but easily within the abilities of a big spy agency. Ideally they would use other methods to determine the IP blocks or other attributes of “computers of interest” they wish to spy on, confirm that they really have these computers, and then briefly load spy code in them to rootkit the systems and look for interesting files.

Detection of this would require a very clever honeypot that knows how to look like a “computer of interest” — once we define what a computer of interest is. I would bet that computers in rural Pakistan, for example, and Iraq, are commonly computers of interest.

Spy agencies & software

Anonymous — Tue, 13 May 2008 19:58:11 -0700

There is one other thing that would be too easy for a spy agency to buy off that it is hard to fathom that it hasn't happened: adding some kind of hook to popular closed-source operating systems to leak confidential information into covert channels. The effort required to modify kernel source code so that e.g., passwords or something are encoded into the output of some pseudorandom number generator used for picking e.g., TCP sequence numbers is low enough that a competent coder involved in the process between when the source code is checked out to do the production build and executing the makefile could slip it in with very low probability of detection and nobody would be any the wiser. If spy agencies can spend billions on covert spy satellite programs, surely they can spend a couple million to buy off a programmer involved in production builds to slip in carefully chosen patches. The presence of spy agencies in the world today with these kind of budgets almost guarantees that closed source products (and pre-built open source products) have backdoors. They would be stupid not to.

It follows that if you don't compile your own stuff, you're stuff is probably backdoored. On the positive side, keeping this stuff secret is almost certainly enough of a priority that the spy agencies will probably be keeping your secrets (unless you are doing something they are directly interested in) and third party crackers will probably not be any the wiser.

Data Spaces in the Clouds

Kingsley Idehen — Thu, 08 May 2008 13:55:39 -0700

Brad,

What you describe is what I've referred to as Data Spaces in the Clouds (Fourth Platform) for a while :-)

Yes, there is some confusion about the literal interpretation of the phrase: Data Portability (free movement of data across platforms). We don't necessarily want free movement of data across realms (unless we explicity enable it in our space). Instead, I believe we seek Open Access to our Data Spaces with access control granularity.

To conclued, we do need Data Access by Reference facilitated by portable Data Containers (Data Spaces) in the Clouds. Of course, these containers can move themselves, or data from the clouds to other locations, wholesale or via replication and synchronixation. In all cases using standard protocols and existing infrastructure such as the Internet and Web.

Links:

1. OpenLink Data Space Wikipedia Page
2. OpenLink Data Spaces (Open Source Edition) Home Page
3. My Data Space Profile Page
4. EC2 installation Guide
5. How to get a Gateway into your Data Space (i.e a URI for Your Data Space) in 5 minutes or less
6. Recent WWW2008 Presentation about Data Portability and Data Accessibility (PPT)

FoF apps

brad — Wed, 07 May 2008 11:26:57 -0700

As I’ve noted, FoF apps turn out to be much less interesting than people thought at first. Do you really look at the photos of your FoFs? The main FoF app that seems to be useful is LinkedIn’s “search your network” which can answer questions like, “Who can I contact at Company X” and dating introductions. FoFoF turns out to be surprisingly non-useful.

However, I won’t proclaim that nobody can think of useful apps (or simply entertaining) apps here. So there need to be solutions, even if it turns out to be that those apps get access to large networks, but are the only ones that do. (Remember, our alternative today is zillions of apps getting access to this data.)

I don’t expect home PCs to be required here. Everybody wants an always on host for many functions. You don’t need it in the sense that I don’t think “Ask 100 hosts to search for a query” is a good implementation, but I would use the local hosts just as a way to do things efficiently for the user when the user is signed on. The cloud host would do things for others. Client data hosts and cloud data hosts would sync.

Your particular app, and other FoF apps, could be implemented, somewhat less efficiently, with data updates. That is to say, if you are using an FoF app, you would send changes to your friends, and they would forward those changes on to their friends as part of the update stream. In this case everybody is storing all the basic data (not big things like photos, just smaller stuff, including the URLs/access tokens of the photos) on their own host, and apps can operate on it. This is why it does not scale to FoFoF, but I think it could handle FoF if the updates are not large.

Of course, you, your Fs and your FoFs must be running the same application, but that is the same as saying they are all members of Flickr. To implement Flickr you need more though, and it may not even be possible to implement all apps in this manner. Still better than implementing them all in a central repository manner, though.

Interesting approach

brad — Wed, 07 May 2008 11:16:17 -0700

But I haven’t studied the underlying systems proposed enough to judge if they can do it. One concern I see immediately regards whether developers can be talked into it. Part of the sex appeal of web 2.0 (meaning apps in the cloud) is that developers get free reign to write and maintain their apps using whatever platforms and tools they like. They are no longer limited to even the constraints and problems of writing code for a user’s PC. Users at the same time love not having to install software, having somebody else maintain it all, and having to roam.

My own proposals face this problem too. These abilities are very attractive to users and developers, and as long as they can get the functionality (which javascript has now given) they will rush to them.

It is for this reason that I have decided that some compromises will be needed, that we won’t get to the level where we can run a malicious app on our data. That’s because the programming hoops required to use a system that bars malicious apps may be too involved. Happy to be proven wrong, though. I would be happy just to reach the level where apps don’t end up taking more data than they need, and don’t end up storing copies of it.

W5 Project at MIT

Evan Jones — Wed, 07 May 2008 04:51:52 -0700

The W5 project at MIT is looking at ways to solve these issues:

http://pdos.csail.mit.edu/~max/docs/w5.pdf

Searches

Radovan Semancik — Wed, 07 May 2008 01:14:33 -0700

I really wonder how you can efficiently implement this use case:

"Show me the flickr photos that my friends and the friends of my friends faved in last 2 weeks, sorted by the total number of favs", while the photos are still stored on flickr.

I can see you can implemente that with "agent" that will crawl to the hosting sites of your friends and their friends, collect the data and come back. But that would be slow, especially if some of your friends host their data in PCs that are currently offline (remember The Eight Fallacies of Distributed Computing?). Do you have any solution for that in mind?

BTW, creating a Virtual Machine you still provide API to the applications. But it is quite broad, which means it is difficult to control security.

Finding the host

brad — Tue, 06 May 2008 14:48:00 -0700

Well, in the past we have used DNS as a way to name hosts and move them around. This could be used here, but not a high level domain. I might get bradsdata.datadns.com as a subdomain that I can point at whatever data host I like. However, there could be other indirection or discovery protocols.

Typically I see a main site providing your interface to social data apps. That site would embed other web pages which are served from your data host, running code provided by app providers. The DNS would direct your own browser at your own data host of choice. (In fact, this architecture allows the data host to be your own PC, if you don’t need to roam. Your own pc at localhost:port would see a request for a social app window. The data host program on your PC would connect to the specified remote application’s server for any code updates or special data, download them if it doesn’t have them cached, execute the code and return the results to the iframe in the browser page.)

I think I’ll update about that.

Now as for the central repository: This is complex. People are saying, it seems, that they don’t want their data scattered around everywhere, both because of lack of control, but more commonly because the UI to give apps access to it is too complex. If we can develop a good UI so that it is easy to give apps just the data they need, and no more, then scattering can be good. The data hosting model does not dictate about scattering or centralization, but I agree that users will tend to centralize, just for ease of control. A central server contracted to me may be better than 30 servers with only loose bonds to me, such as 30 different social app companies each knowing different subsets of my data.

Service portability good. Centralized servers not necessary.

Joe Andrieu — Tue, 06 May 2008 12:26:43 -0700

Good post.

It's clear that the data portability model is limited, but I would even go beyond hosting portability to service portability, and I would separate the hosting company from the value-added service provider.

First, the hosting model still leaves you at the mercy of the host if/when you decide to move... you still have to update your address at all of the service providers who might access the services of your hosting company. A more robust model would provide service portability through service discovery. Service providers don't need to know where it the data is hosted, but rather where they can find the current host. That gives you a layer of indirection that lets you move your hosting company without needing to remember which service providers are currently relying on that host.

Second, you suggest that the hosting company's job is to perform actions on your data.

Why is that?

My DNS host doesn't perform functions on my DNS. Nor do I expect my webhost to perform actions on my website, although I do like to have a range of services I can easily install and run (such as installing WordPress through Fantastico).

I would argue that there is an inherent conflict of interest in the hosting company providing value-added services, and that in fact, what we should do is design an architecture where hosting is functionally distinct from value-added services. Any authorized value-add service provider should be able to access your data services, which leads to a cleaner architecture where companies that happen to provide both hosting and value-add services can do so with clear contracts and authorization.

Finally, there's no real need to have all of my services at the same hosting company, just like my DNS, my website, and my email can all painlessly be hosted anywhere I like. As long as the services can be discovered, there's no reason to have any individual's services centralized, nor a need to centralize many individuals' data in one place.

The collaborative/co-op type negotiating strategy you suggest can easily be implemented by a value-add service provider, independent of the source of the data. Users simply join that co-op or buying group and point the co-op to their discovery service. Flash-mobs rejoice.

All of which is to say, you are definitely going down the right path. More portability more better.

-j

It matters where it is hosted

brad — Tue, 06 May 2008 11:42:59 -0700

Because we want to lay down the duties of the people who host it for us, if we don’t host it ourselves. (And most people won’t host it themselves.)

If a host has just one duty, to keep the data safe and allow only authorized actions on it, then that is what the host will focus on. If the host is facebook, whose duty is to find ways to monetize its database to maximize shareholder value, you will get a different result.

That’s why the layers, and why for most people, not having too many hosts. (Some might decide to have multiple hosts to maintain multiple persona, such as a personal and work persona, but that’s their decision.)