A first solution to linux dependencies part 2 -- yes, service packs

Topic: 
Tags: 

Last week I wrote about linux's problems with dependencies and upgrades and promised some suggestions this week.

There are a couple of ideas here to be stolen from (sacrilige) windows which could be a start here, though they aren't my long term solution.

Microsoft takes a different approach to updates, which consists of little patches and big service packs. The service packs integrate a lot of changes, including major changes, into one upgrade. They are not very frequent, and in some ways akin to the major distribution releases of systems like Ubuntu (but not its parent Debian ), Fedora Core and SuSE.

Installing a service pack is certainly not without risks, but the very particular combination of new libraries and changed apps in a service pack is extensively tested together, as is also the case for a major revision of a linux distribution. Generally installing one of these packs has been a safe procedure. Most windows programs also do not use hand-edited configuration files for local changes, and so don't suffer from the upgrade problems associated with this particular technique nearly as much.

There have only been 2 major service packs for 2001's Windows XP. There are software packages that require a user have upgraded to SP1 or SP2, but as noted, a fairly large amount of software typically installs fine on 1999's Windows 2000 or even Windows 95.

I wonder if something akin to service packs might help linux distributions. In most of these cases, a service pack is something very simple -- it's just a set of core packages, tied together, and tested together, and tested with old releases of the distribution.

The typical package in a service pack would be the major libraries (libc, X windows, etc.) and major applications which have more than one or two add-ons that depend on them. Anything that is widely depended on (or which has just one very popular package that depends on it) would make sense in the service pack.

The pack would be assembled and compiled for the most recent release, of course, but also for releases going back several years. Volunteers could also build the pack for releases going back as far as they like.

In addition, a strong push would be made for packages in the pack to have samless upgrade. Where possible, they would be migrated, either by their authors or the distribution maintainers, to divide their configuration up so that configuration by the author, packager and mostly importantly local machine admins and user are never kept in the same file.

If they must be in the same file, scripts should be prepared to help merge changes. These scripts would have access to the virgin configuration files from the package previously installed and the user's latest versin of it, so that they can calculate the differences. They would also have access to the new author/packager version of the file. With all 3 in hand the tool could do a decent job of creating a new merged file applying both the changes by the maintainers and the local changes. This is a similar problem to that faced in version control systems, and in fact these tools might be used for this problem. (Package authors and maintainers might even retain repositories showing all changes on their end, and encapsulte a history of all these changes with a package.)

Of course, ideally one change will be to change the master file to just say "Include localconfig/*" and put all local changes into localconfig, and do it that way from then on. This moves us closer to my eventual goal of dividing configuration according to who does the configuration instead of just what package it is.

At this point, the service pack upgrade should be fairly safe. Others should have tried that exact upgrade before, even on older systems. And new software packages can be installed on old systems with greater ease and less risk. The packs need not be monolithic. All linux package managers can identify packages in a pack that don't need to be upgraded or installed at all. With minor modification they might even record that if anything does come to depend on a package, it should be upgraded to match the pack. The packs, which are simply lists of packages tested together, can also be divided up, so there is one general pack, one for media applications, one for programming and so on.

The temptation must be avoided, however, to turn the pack into a kitchen sink. Otherwise it could turn into something akin to a major release.

Software developers and packagers, of course, should still make an effort to tune their dependencies. They should test and build packages with older releases of libraries and other dependencies unless they know for sure that they need newer ones. Thanks to virtualization tools, such as user mode linux and the low price of disk space, it is also possible to keep around older releases for testing and package building. With a bit of work, if a package says it depends on some late revision of a library, you can suspect it actually means it.

Microsoft's other feature, which they took a long time getting around to, is their tool for XP, the User Migration Tool/Settings Transfer Wizard. That is a tool designed to extract the user's custom settings from one copy of Windows (either older or the same release) and import them to a fresh install. This was not Microsoft's idea of course -- commercial products to do this existed for some time beforehand. Because fresh installs are even more needed in Windows than in *nix, this tool is even more needed there, but it takes us along what I think is the right course -- finding a way to encapsulate the changes an owner makes to a system away from changes made by all sorts of other people (software authors, OS distributions, corporate sysadmins) so they can be easily backed up and transferred. I'll write more about that in the future.

Comments

Neither apt nor yum (depending on whether your maintenance architecture uses .deb or .rpm files) suffers from dependency problems.

A given repository may not show a complete set of updates at a given time, which will cause either tool to reject the updates as incomplete (missing dependencies). This is not a problem - it's the correct decision! Once the repositories show a stable complete set of updates, they will be applied.

I've been running regular maintenance on many systems for many years, and have not experienced a problem. Understanding why maintenance updates are being rejected is the key.

There are some tools that can make incremental updates easier. for example, for yum there are a set of plugins:

yum install yum-fastestmirror yum-skip-broken yum-utils

You can then add the option '--ignore-broken' to your yum commands, and yum will try to apply as many dependency complete updates as possible (instead of giving up when encountering a missing dependency).

You can also periodically run:

package-cleanup --oldkernels --count=3

This will keep the latest three kernels, cleaning up the old ones (and dependent packages).

In short, the key to using apt or yum is regularly scheduled maintenance, and proper use of the included toolset. This causes small, incremental and easily managed changes to be applied to the systems. Trying to apply huge changesets, like a service pak, is where you will run into grief (and major system impact).

In fact, I even use it on Fedora as well as Debian and Ubuntu.

The problem, as I noted, is that new packages are tested and built only with very recent versions of their dependencies. So, in order to try out a cool new program or to get an updated version of a program I use which has bug fixes or new features, I must also get new versions of all the dependencies.

And worse than that, these new versions simply aren't made available in binary forms for releases that are just a year old in many cases. Debian, for example, as stable, testing and unstable. But in reality, unless you want no ability to install new software, you have to run unstable. Now unstable is not nearly so unstable as the name suggests -- but this is still stupid. The truth is these new software packages you want to run don't really depend on all these new libraries and tools, that's just what the guy who built the DEB or RPM file had on his system at the time.

Running a stable system with older, more tested base packages is not an invalid goal. But it shouldn't prohibit you from running new software that doesn't actually and truly need something more modern.

Again, compare it to the Windows user running 7 year old Windows 2000 and having done one update to SP2. They can download and install almost every piece of Windows software out there. Try to run a 3 year old linux and you can't even get close to that.

Anonymous: The problem isn't just the packages that are distributed. Yes, regular updates to all the packages keeps the distribution up to date, but then you have touch broken configs regularly.

I believe there are two major points that Brad is trying to solve:
1) Configuration. Some packages can only be configured by modifying the config file that comes with the package. When you install a new version of the package, your config can be lost. This is broken, and I think the very simple solution of not making the user put their changes into the distribution config file is probably the best possible first to finally fixing this.

2) Dependancy. Not every package in the world is distributed through a package manager or has proper dependancies. Especially with software that one compiles oneself, the package manager is usually out of the loop, and thus, I can easily find myself with important software that 'suddenly' is broken. As Brad has noted, this is a MAJOR pain in the posterior, and doesn't do anything to help the image of Linux, or to get us away from dependancy on M$.

One thing I'm curious about: What do the Linspire folks do? Do they simply assume that you won't run anything that didn't come from them? Since all the packages would come through them, that would at least make it theoretically possible for all the packages to be correct. Do they have a real solution for config files?

Brad, thanks for getting a conversation going on this topic. I'm not currently using Linux daily, in part because maintaining the damn thing became more effort that it was worth to me.

Thanks for your article. It's hard to find articles that go as deep in the dependency problem. For the moment I'm having this problem. I want to run kdenlive on kubuntu breezy. And I found there is no proper way to do it for a non technical linux user as I am. No breezy package in the backports and the klikable package (http://klik.atekon.de/)does not run on my system because it can't find an rpm on the packmanrepository. Sometimes I think I'm the only one who run a linuxdistribution longer than 1 year :)...and run their system with a minimum of updates because I don't have a broadband internetconnection.

Add new comment