As part of my series on the horrors of modern system administration and upgrading, let me propose the need for a universal API, over all operating systems, for accessing data from, and some control of the package management system.

There have been many efforts in the past to standardize programming APIs within all the unix-like operating systems, some of them extending into MS Windows, such as Posix. Posix is a bit small to write very complex programs fully portably but it's a start. Any such API can make your portability easier if it can't make it trivial the way it's supposed to.

But there has been little effort to standardize the next level, machine administration and configuration. Today a large part of that is done with the package manager. Indeed, the package manager is the soul (and curse) of most major OS distributions. One of the biggest answers to "what's the difference between debian and Fedora" is "dpkg and apt, vs. rpm and yum." (Yes you can, and I do, use apt with rpm.)

Now the truth is that from a user perspective, these package managers don't actually look very different. They all install and remove packages by name, perform upgrades, handle dependencies etc. Add-ons like apt and GUI package managers help users search and auto-install all dependencies. To the user, the most common requests are to find and install a package, and to upgrade it or the system. In spite of doing the same thing for the user, they look different enough under the hood and for advanced users that they have different UIs and APIs. Portage, for example is very different under the hoood -- it always compiles packages from source, rather than pulling in binaries. That mostly work the same, though the blurry scroll of compiler messages and worse, diagnostics, can be scary.

I don't expect the world to just have one package manager for now (though it might be a good idea) but I wonder if we could not get an API to handle some of these issues. In particular, an API to ask questions like "What's installed on this machine" and "Where can I find the binary for this" or "How do I get at the mailer, or the sql database, or other resources." One universal interface to this would make it possible for more programs to easily port over all the similar OSs (including BSDs vs. linux) without a lot of changes.

This may remind you of the Windows Registry database, which performs this function there. Not everything in Windows is stupid, after all. I would not do it exactly the same way but I would borrow some ideas. There probably does have to be a database, but it can't be totally based on a full database server like mySQL because it must operate when the machine is booting, before services are started. The library for accessing this database might use direct access during boot, and switch to SQL access once the SQL server or network is up, with appropriate tools to mirror as needed.

This database might also answer many of the questions asked by the typical "configure" programs found with many software packages. Simple reliable answers to those questions can aid in portability.

A universal API to package management might also make it easier to create GUIs for the package managers, as these GUIs tend not to delve into all the detailed options of a package manager. They mostly want to do the basics.

One issue is that different distributions use different names for the same package, and they can't so easily fix this (though a rename function would be good in all package managers.) It would be nice to start some consolidation on this. For older packages, it probably would be necessary to generate a unique package ID central service which assigns unique IDs to anybody writing software which might some day be packaged. These IDs might not be visible to users but programs wanting to find out where dependencies are could use them with 100% portability and reliability. Packages that have managed to keep the same name over all distros could perhaps have that name as the unique ID.

The existence of a "where is this?" database eventually gives us something even better. Package arrangements can start to be portable over all OSs. In many cases, the differences between Ubuntu and Fedora may be simply a matter of one system putting a package or a key configuration file in one place vs. another. With an API to ask where things are, it no longer matters a great deal where each individual OS puts a file, and in fact you can put them in entirely custom places not likely to interfere with any system. For some time Debian has had the alien tool to convert rpms to debs, and this could be made more reliable and universal.

Of course, the dependency nightmare remains, but it could become possible to make a package that varies very little from distro to distro. Many software packages today are available as binary tar files with installers, instead of as packages, which is bad because you lose the common interface for install/deinstall and dependencies. They tend to be written to work with older libs so they can install over a wide range of systems. We might see these fit inside the package system, and indeed a "universal" package format, chosen to meet the intersection of the needs of major systems could exist and be understood by each package manager. And that would be a big plus for maintaining systems and make more software available in that format.