The Utopia of Program Management

Ever since I started using computers, I’ve been baffled by the relative clumsiness of installing applications. Whether we are talking the really old days (launching the Rambo game off a tape), the ’90s (running Keen or using installers in Windows 95), or the modern days (still those installers, but now also package management and self-contained applications); it’s all relatively cumbersome, and they all have their downsides. I decided to put my money where my mouth is, and come up with my idealistic, utopian method of installing, running, updating, and uninstalling applications.

Introduction

Mac users are always quick to point out the benefits of their self-contained applications; one package to download, no installation procedures, easy to remove. While this seems ideal, there are many problems with the actual Mac OS X implementation of this idea. Applications in Mac OS X are generally not easy to remove at all, because they leave a trail of files around outside of /Applications that normal users rarely encounter. Over the course of time, this can amount to quite the mess. In addition, Mac OS X provides no way of updating applications in a central way, resulting in each application in OS X having its own updater application; hardly the user-friendly and consistent image Apple tries to adhere to.

The Windows world is not much better off – in fact, it is probably worse. Not only does it have to deal with the same problems as OS X, it also has too deal with dreadful installers. Some of them are such usability disasters they make me want to curl up in foetal position and cry. And then, when I’m done crying, I can start all over again because the uninstallation procedure is just as dreadful.

This leaves us with the Linux world. They have the centralised, easy updating application – the update application in for instance Ubuntu is an excellent example of proper balance between providing enough technical information for experts, but still hiding all that fluff from normal users. However, Linux suffers from other problems. Dependency hell, while not nearly as huge a problem as it used to be, still exists to this day. Downloading a package outside of the repositories is a risky business, but it really shouldn’t be. You are completely dependent on your distributor updating his repositories and keeping them clean – nothing is as annoying as knowing there is a new version of Super Awesome Garden Designer Ultimate Edition, only to realise all the distributions except yours already packaged it.

Basically, the proposal you’re about to read will incorporate bits and pieces from all over the operating system world, but will also include some new ideas of my own. The goal of my proposal is to make the installation, updating, and uninstallation of applications as easy as possible, while at the same time providing features advanced users and developers might appreciate.

I started working on this proposal ages ago, and as I went along, I realised I needed to alter a lot of things about the operating system itself (most notably, the filesystem layout) before I could continue working on the proposal itself. Consequently, I have created a new filesystem layout, and most likely a new filesystem – as I am not sure to what extent existing filesystems can accommodate this plan. Consequently, this article starts out with detailing the privilege model required, followed by an explanation and breakdown of the filesystem layout I devised. It will then move on to the importance and benefits of using BFS-like attributes and queries for updating and maintaining applications, ending with a conclusion (I bet you didn’t see that coming).

Privileges

It might seem a little odd to start an article on a utopian method of managing applications with a section on privileges – and you’re right, it is. The reason I’m starting with it anyway is because in my opinion, the privileges a user has is vital in the process of managing applications – or programs, as I will call them from now on (don’t worry, this arbitrary decision will become clear). Don’t pay an awful lot of attention to the names of the directories yet, I will explain those later on.

By default, a user has either User rights, or System rights. When in normal, day-to-day operation, the user and his programs/processes utilise User rights. This means the user and his programs can only write to their home directory (/Users/User 1) and their settings directory (/Settings/User 1). Write access to all other parts of the system is restricted – of course programs and processes may read contents in the restricted directories, but they may not write there. Consequently, installing and uninstalling applications requires the System password. User 1 may not read or write to files in other users’ home and settings directories.

This is similar to other operating systems, and has the usual and understood benefits: processes and programs launched by a user can only damage his or her own files, and cannot ruin the entire system or other users’ files.

Which brings us to System rights. Users can elevate their rights either on a need, or a perceived-need basis, to System rights, which gives them the ability to change system settings or access content in otherwise restricted directories. With ‘on a need basis’, I mean that programs, processes, or actions initiated in User rights mode, may require System rights at some point (changing a systemwide setting, writing a file to a restricted directory, and so on). With ‘on a perceived-need basis’, I mean that the user can switch to System rights without there being a specific need; this may come in handy when setting up the machine in question, when doing heavy system modifications, or simply when an experienced user wants to know more about the system. This is one of the complaints behind UAC: you can’t tell it to maintain System rights.

Of course, switching to System rights requires the System password at all times.

The reason I settled on the naming convention of User and System rights (instead of things like root or administrator) is because I find them clearer to the user. You either have ‘User-wide’ access (Home and Settings directories) or ‘System-wide’ access. I find ‘root’ to be an inherently meaningless term in computing, and administrator sounds too much like someone who works for the government. And nobody likes government people. In addition, these names happen to fit quite well into the new filesystem layout.

Even though these two permission levels are the default, advanced users are of course free to fine grain permissions, add different permission levels, forbid specific users from elevating permissions, create a System user account that always has access to everything (root, basically), and so on. In addition, advanced users may also sandbox applications at will, by specifically forbidding certain programs or processes from gaining System privileges. This level of control is needed in for instance larger organisations or server environments, but can also prove to be valuable for programmers who want to test drive their latest code in a secure, non-destructive manner.

Of course, this is nothing new, but in my utopian system, this should actually be easy to accomplish. I’ve been wrestling with the Group Policy editor thing in Server 2003 for a while now, and dear lord, I have absolutely no idea what I’m doing. I’m reviewing a tool that allows me to fine grain the rights of processes and programs, but it’s so utterly complicated I’m completely lost. I see the potential – it just needs to be easier to accomplish.

Let’s move on to the real deal.

Filesystem layout

The filesystem layout is actually remarkably straightforward. I hope I can keep this paragraph short, because that would mean it reached its goal – to be self explanatory. The raw data:

/System<br />
/System/Utilities

/Users<br />
/Users/User 1<br />
/Users/User 2<br />
/Users/User 3<br />
/Users/Shared

/Programs

/Settings<br />
/Settings/User 1<br />
/Settings/User 2<br />
/Settings/User 3

There are two main themes behind this layout. The first is to separate the base operating system files from everything else, and tuck them safely away in /System. This directory contains the kernel, drivers, resource files (f. ex. icons), preference panels, and so on. I also took a cue from Mac OS X by creating the specific /System/Utilities directory, where the operating system can store utilities such as the Activity Monitor, graphical Bluetooth tools, Network Utilty (graphical ping, whois, etc.), those sorts of things. What does and doesn’t go into that directory is fairly arbitrary, and is open for debate. Other binaries that usually reside in /bin on UNIX systems can also go into /System (say, something like /System/Binaries, since this is the 21st century – why use unclear acronyms).

The second theme is the separation between between /Programs, which carries the actual self-contained program directories (program bundles), and /Settings, where the program bundles can store their setting files in user-specific directories (f. ex. /Settings/User 1/Garden Designer – the name must be the same as the program bundle’s name). The idea is that these setting files are more or less human readable (preferably in a standard format, shared by all programs), so that advanced users can modify them by hand. An added benefit is that people can share settings files with one another, which might help in troubleshooting scenarios.

To prevent the mess Mac OS X has with separating settings files from the program bundles, the system should somehow ‘know’ the two belong together – this shouldn’t be too hard to realise. One option would be to use BFS-like attributes; the Garden Designer settings file could have an attribute attached to it that links it to the Garden Designer application bundle, for instance. This way, when you want to remove Garden Designer from your system, the system can ask you if you want to delete its settings files too.

The /Programs directory contains all the, well, programs. Unlike most other systems, the programs themselves will be distributed as program bundles, analogous to applications on RISC OS and Mac OS X/NEXTSTEP. This way, possible uncommon dependencies can be shipped within the program bundle itself, to prevent the idiotic situations where users of Linux are confronted with having to download and install a whole batch of gibberish that will only confuse them.

This still leaves one question unanswered: how are we going to manage all these applications? Wasn’t one of my points of criticism on Mac OS X, in the introduction, that it lacked a centralised method of updating applications? Indeed.

Cue BFS-like attributes and live queries.

Harold Kelley would be proud

This is really where the beauty of my utopian system starts to shine. Attributes in the BFS filesystem, combined with the concept of live queries, enabled you to do all sorts of crazy stuff with the files on your BeOS machine. Back in 2005, when I reviewed Zeta R1, I already explained a few interesting uses of BFS live queries:

First of all, let’s say I’m having a discussion via email with Eugenia. Because each email is stored as a single file, I can easily set-up and save a live query with all emails from Eugenia, sorted with newest on top. If the discussion continues to develop, I can refine that query even further, by adding a query on the subject of the emails. All that without ever starting an email client. Remember, these searches are instant; a lot faster than Spotlight. Also, they are ‘live’ queries, which means that they are updated automatically and instantly. You can save those queries as if they were directories. Again, this requires no extra applications or whatsoever.

Another interesting scenario to use BeFS is when you are putting songs on your MP3 player. Want all music from Bruce Springsteen? Or all songs from the Devils & Dust album? All songs from the ‘rock’ genre? You can do that without ever touching a music player or other specialized applications.

You are only limited by your imagination.

And you really are only limited by your imagination. Combine the concept of live queries and attributes with program bundles, separate setting files on a per-user basis, and the possibilities are endless. Seriously.

Each program bundle can store the basic information in attributes, the stuff you’d expect: name, publisher, date of release, country of origin, names of the main contributors, website, license, and so on and so forth. The boring stuff, but handy when you forget where you got the application from. There is also the attribute that takes care of the ‘linking’ with the settings files, but how exactly this would work, I don’t know – I’m leaving this up to the programmers and coders.

I also want to introduce a concept alongside the version number. When we think of version numbers, we tend to think of the numbers dangling at the end of releases, which are usually fairly arbitrary and don’t really say anything at all. Those would become ‘external’ version numbers. What is new is the ‘internal’ version number. How programmers order these numbers is completely up to them, as long as the second release’s internal version number is higher than that of the first release. This is essential.

Take Garden Designer. You have Garden Designer 8.0 installed on your system, which happens to carry the internal version number of ‘348’. Now, say Garden Designer 8.1 is released – the idea with program bundles is that various releases are named the same. This new version should get an internal release number that is at least 348+1. This way, the system can compare the attribute for “internal version number” and see that the program bundle Garden Designer 8.0 carries an internal version number of 348, while Garden Designer 8.1 carries 349*.

From this, the system concludes that 349>348, and will update the program bundle accordingly. However, the fun doesn’t stop here.

Who says you would want to replace version 348? What if you’re unsure about version 349? You may have heard horror stories online, you may have had a bad update experience in the past, or you may be running a production system. Well, simply tell the system to allow the program bundles for version 348 and 349 to live side-by-side, so you can continue with 348, which you know works fine, and test 349 before ditching 348. Combined with the ability to sandbox program bundles (see the privilege section) this allows you to do extensive non-destructive testing on your system. It also enables you to easily switch back to an older version – a prompt would warn you the version you are about to install is older, but if the newer version doesn’t work as expected, you can ignore the dialog and replace the newer, broken version with the older, working one. Fairly hassle-free.

A few of you are probably worrying about the settings files in /Settings right about now. How would the settings files deal with multiple program bundles of the same program? The answer is straightforward: think attributes again – the settings files can carry the same attribute for “internal version number” as the program bundles themselves, so you can easily have multiple settings files living side-by-side peacefully. Using live queries, you can search for unused settings files (using the attribute that links them to the program bundles), and clean them up accordingly in case you want to remove old settings files that you thought you might need in the future when you deleted their applications.

Now we can finally come to the centralised updating issue. Some of you will have probably already done the math, but here it goes anyway: I envision a centralised repository of program bundles, all tagged – of course – with the proper internal version number attribute. A very simple application could check all your programs’ internal version numbers using a live query, and compare them to the internal version numbers of the program bundles in the repository. You could tell it to prompt you for each update with the replace/side-by-side option, or simply have it update everything.

The centralised updating tool being built on top of a set of live queries, it could be made as complicated or as simple as the programmer would want it to be. You can do simple queries like “search for outdated programs and update them”, but also more advanced and detailed queries such as “search for outdated programs from Microsoft and update them”, or “search for program bundles with multiple versions installed and list them”, or whatever other possibility that might be useful. And the beauty of it all? Live query support is built right into the file manager in BeOS, so you can do all this manually if you don’t like the available centralised update programs. Ultimate control for experts, hassle-free updating for inexperienced users, and anything in between.

If you like a centralised method of managing applications – use the centralised updater/installer tool. Don’t like it, and prefer a more manual approach? Don’t use the centralised tool. The choice is yours.

* Why not use dates instead of these numbers? Two reasons: date formats differ across the globe, and you have different timezones that might interfere with such a system – you’d need a method of working around such a problem, which would just introduce another complication to the system.

The beauty

The beauty of this system is that it combines the benefits of centralised management with the ease of the self-contained program. You can manage your programs by using the file manager and live queries, or you can use a centralised ‘bundle manager’ like Synaptic if you are into that sort of thing – or ditch live queries altogether, and just do as you do in OS X. You can install programs from an online repository like .deb and .rpm do, or simply download a single application online – or all at once.

Since all the tools are based on live queries, gone would be the days of one updating application ‘locking’ the package database, preventing you from installing another program (like in Linux). Since live queries are, well, live, they update automatically, so there’s no database to lock in the first place. Say you are using an update manager to update the programs on your machine. While the update is taking place, you browse around and encounter an interesting application you’d like to try. Instead of having to wait until the update manager is done and unlock the package database, you can simply install the application yourself, and the live queries used by the update manager automatically adds the program to its list.

On Windows, Linux, and Mac OS X, it’s quite difficult to run different versions of the same program side-by-side, simply because those systems have an all-or-nothing approach to updating programs; either you update and completely replace the old version, or you keep the old version and do not update. Very, very inflexible. My utopia allows for a lot more flexibility – but you’re not forced into using it. ‘Grandmother’ will never have to deal with any of the advanced functions and possibilities, while advanced users can go all-out and do whatever the hell they want, not limited by the user-friendliness that appeals so much to grandmother.

In my utopian system, you’re not forced into a specific way of thinking, like Linux, Windows, and Mac OS X do. You are free to maintain your programs in a way that you, the user, like. And let’s face it, who should be the one deciding what’s good for you?

69 Comments

  1. 2008-05-05 9:10 pm
    • 2008-05-05 9:13 pm
      • 2008-05-05 9:31 pm
    • 2008-05-06 10:39 am
  2. 2008-05-05 9:31 pm
    • 2008-05-05 10:38 pm
      • 2008-05-05 10:48 pm
        • 2008-05-06 12:38 pm
          • 2008-05-06 1:04 pm
          • 2008-05-06 3:11 pm
    • 2008-05-05 11:13 pm
      • 2008-05-06 1:09 pm
        • 2008-05-06 1:54 pm
          • 2008-05-06 5:11 pm
          • 2008-05-06 8:32 pm
    • 2008-05-06 4:10 am
    • 2008-05-06 7:24 am
  3. 2008-05-05 9:32 pm
    • 2008-05-05 10:34 pm
  4. 2008-05-05 10:37 pm
    • 2008-05-06 1:08 pm
  5. 2008-05-05 11:14 pm
  6. 2008-05-05 11:16 pm
    • 2008-05-05 11:35 pm
      • 2008-05-06 7:44 am
      • 2008-05-06 8:15 am
  7. 2008-05-05 11:44 pm
    • 2008-05-06 10:05 am
      • 2008-05-06 4:54 pm
        • 2008-05-06 5:00 pm
          • 2008-05-06 9:36 pm
          • 2008-05-07 12:48 pm
  8. 2008-05-05 11:53 pm
    • 2008-05-06 9:11 am
    • 2008-05-06 10:07 am
      • 2008-05-07 9:31 pm
    • 2008-05-06 1:11 pm
  9. 2008-05-05 11:57 pm
  10. 2008-05-06 12:08 am
    • 2008-05-06 3:18 pm
      • 2008-05-06 4:14 pm
  11. 2008-05-06 1:10 am
    • 2008-05-06 10:12 am
    • 2008-05-06 3:21 pm
  12. 2008-05-06 4:09 am
  13. 2008-05-06 4:15 am
    • 2008-05-06 8:57 am
      • 2008-05-07 4:45 am
        • 2008-05-07 4:01 pm
      • 2008-05-07 4:39 pm
  14. 2008-05-06 4:22 am
    • 2008-05-06 10:24 am
      • 2008-05-06 2:24 pm
  15. 2008-05-06 5:06 am
  16. 2008-05-06 7:04 am
    • 2008-05-06 8:02 am
      • 2008-05-06 10:29 am
  17. 2008-05-06 8:04 am
    • 2008-05-06 9:50 am
      • 2008-05-06 10:37 am
  18. 2008-05-06 10:00 am
    • 2008-05-06 1:06 pm
  19. 2008-05-06 10:33 am
  20. 2008-05-06 11:12 am
  21. 2008-05-06 12:21 pm
  22. 2008-05-06 12:51 pm
  23. 2008-05-06 1:04 pm
  24. 2008-05-06 4:11 pm
  25. 2008-05-06 6:14 pm