posted by Thomas Leonard on Tue 16th Jan 2007 00:32 UTC

"Compiling, Converting Between Formats"


Freedom to modify programs requires easy access to the source code, and the ability to use your patched version of a library with existing programs. Our installation system should be able to download the source code for any program, along with any compilers, build tools or header files we need, as in this screenshot of Zero Install compiling a Pager applet:

Downloading source code

It is often the case that a program compiled against an old version of a library will still run when used with a newer version, but the same program compiled against a newer version will fail to work with the old library. Therefore, if you plan to distribute a binary it is usually desirable to compile against the oldest version of the library you intend to support. Notice how Pager in the screenshot above has asked to be compiled against the GTK 2.4 header files, even though my system is running GTK 2.8. My blog post Easy GTK binary compatibility describes this in more detail, with more screenshots in 0compile GUI.

The Pager binary package includes an XML file giving the exact versions of the libraries used to compile it. Our ability to install any version of any library we require without disturbing other programs allows us to recreate the previous build environment very closely, reducing the risk that recompiling it will have some other unintended effect.

Converting between formats

Autopackage and Klik packages are supplied as executable files. Running the script installs the software. Zero Install uses no installation instructions, only an XML file describing the software and its requirements. Most systems fall in between these two extremes, often giving dependencies declaratively, but with scripts to perform any extra setup required.

The scripting approach gives the most power to the packager. For example, supporting a new archive format only requires having the Klik server send out suitably modified shell scripts for clients to run, whereas new archive formats can only be used with Zero Install after upgrading the Zero Install software itself.

On the other hand, scripts give very little power to the user; the only thing you can do reliably with a script is execute it. It is possible to trace a particular run of a script and see what it did; CheckInstall monitors a program's "make install" command and then generates a package from the actions it observes. However, this cannot pick up dependency information or detect alternative actions the script might have taken in other circumstances.

My klik2zero script can convert Klik packages to Zero Install feeds. This works because the result of executing a Klik script is a self-contained archive with all the program's files. However, you have to host the resulting file yourself because the information about where the script got the files from is lost.

The autopackage2zero program works differently. Many Autopackages are actually shell scripts concatenated with a tar.bz2 archive, and the converter parses the start of the Autopackage script to find out where the archive starts and then creates an XML description that ignores the script entirely. This means that you can create a Zero Install feed for an existing autopackage, using Zero Install to check for updates and providing signature checking, but actually downloading the original .package file. Again, this loses any information about dependencies or other installation actions, but it does work surprisingly often.

Going the other way (converting from a declarative XML description to a script) is much easier. Zero2Bundle creates large self-contained application directories from Zero Install feeds by unpacking each dependency into a subdirectory and creating a script to set up the environment variables appropriately. Note that this is different to the normal (recommended) way to use Zero Install applications from ROX, which is using AddApp to create a tiny ROX application directory with a launcher script.

Even if people continue to get most of their programs from centralised distributions, the process of getting new versions of packages into the distributions in the first place could benefit from some kind of decentralised publishing system. A packager should be able to tell their computer to watch for new releases of programs they have packaged, download each new release, create a package for it, and notify them that it's ready for testing. Likewise, adding a new package to a distribution should require no more than confirmation from a packager. If it is not this simple, then we should be finding out why and fixing it.

Better integration between decentralised systems and native package managers would also be very useful. Users should be able to mix-and-match packages as they please.

Table of contents
  1. "Introduction, Use Cases, Traditional Distributions"
  2. "Naming, Conflicts"
  3. "Avoiding Conflicts, Dependencies"
  4. "Publishing Software, Sharing Installed Software"
  5. "Security"
  6. "Compiling, Converting Between Formats"
  7. "Summary"
e p (8)    76 Comment(s)

Technology White Papers

See More