NewsForge hosts an interview with Autopackage’s project leader Mike Hearn. The idea around Autopackage is to come up with a single software packaging/installation system that will work across all major Linux distributions, and resolve dependencies, too, so that neither developers nor users need to worry about distribution compatibility issues. OSNews hosted recently a feature article about the project.
Interview with Autopackage’s Project Leader
About The Author
Eugenia Loli
Ex-programmer, ex-editor in chief at OSNews.com, now a visual artist/filmmaker.
Follow me on Twitter @EugeniaLoli
I wonder if it is feasible to retrofit something like the NSIS (Nullsoft installer system) into a Linux environment… that would be cool. Some people don’t really like having binary installers for software separate from their distro packing, but I think it makes life a lot easier sometimes.
I will continue praying that Autopackage become a standard. It’s just too rediculous having all these packaging systems that are completely incompatible. I love Linux (More specifically Gentoo and Slackware) and the only thing I wish I could do more easily is install software and have it work right, on any distribution. Less headache for packagers as well as users. “What RPM/TGZ/DEB was I supposed to download again?!”
This system rox, if it works the way the author says and it is this friendly to developers and packagers alike it’s unbeatable.
It is a little frightening though, what will become fo lindows once this becoems the new standard. Anyway I hope it hasa very nice easy to use addd/remove programs like Windows too, than it will be perfect.
I am currently learning C++ jsut for this project =) Chp 1 Deitel+Deitel 3rd Ed almost on chapter 2 =)
Neat. Do you have any C experience? The Deitel book on C is just as good, and is a helpful foundation if you’re going into C++. I would recommend reading it if you’re seriously interested in programming. That and the book the book you are currently reading are actually the textbooks. If you’d like to quiz yourself, try doing the problems referenced here:
http://www.pierce.ctc.edu/rhorst/CIS271Winter2003/homework.htm
I guess you can’t check them, but you it is a good test.
Ack. (hiding and wishing there was a preview button)
Mario, unless you’re intending to work on the KDE front end, you do realise that this project is written in bash and C right?
I don’t know C++
Autopackage is just a big hack IMHO.
Why not:
LSB should establish standard runtime environments (REs) for linux, as well as development environments. There should be a minimum set of runtime environments that should be available on all Linux distros. Maybe separate ones for Servers and Desktops
Packages should contain both the Program, and appropriate versions of non-standard runtime environments it depends on. So the package is actually an intelligent archive of RPMS or DEBS, or it could be an archive of the RPMS for the program and some information about where to download necessary REs.
During installation, the installer should check for the presence of the appropriate dependencies. If available, it installs only the program, if absent, it installs all the dependencies and then the program.
The backend does not have to be different from RPM or APT, so experienced users can still install individual RPMS.
I have seen posts on slashdot refering to your autopackage project for the last six months- I have checked your webpage several times and have been loosely following your progress. I reread the original OSnews article you wrote and the 100+ comments and I read the interview refered to in this article. At no point did I see you really address what for me remains the most fundamental aspect of package management, even more fundamental than dependency resolution- and that is system integration and configuration. From you FAQ it would seem that you intend autopackage to only be used for end-user apps, ie. stearing clear of the fundamental system apps which make up the linux system(ie. gcc/glibc/x windows/kde/gnome etc.). The way I see it you have chose a route which is quite difficult, in terms of the adoptability of your autopackage system- and its relative appeal.
If you stear clear of fundamental system apps which need a great deal in intricate configuration in order to effectively integrate them into the rest of the underlying distro you autoackage app will be relegated to secondary use as a tool to download non-essential stuff-and this means that users will have to have command of the distro-specific system for taking care of fundamental apps and your autopackage system too. I see this problem of system integration and configuration being as important if not more important than dependancy resolution per se.
Most of the dependency resolution hell of the past year has been due to fundamental changes in gcc/glibc as regards the ABI and the slow transition to xft/fontconfig/freetype2/xfree-4.3. This transition has caused problems for every major distro- rendering version 8.0+ incompatible with everything pre-8.0(this holds for redhat/suse/mandrake(to some extent also with 9.0) and this instability has charachterized all of the minor releases post.-8.0 as well-yet this should be stabilizing within the next 2-4 months and is unlikely to recur at such a fundamental level inside of the next 2 years.
Gnome-system-tools, following on the heels of xst-tools, promises to help with system integration and configuration- they are doing this by parsing configuration files via XML using perl-based backends written for each of the major distro’s and a common GUi-frontend.
quote form their site:
“GST Structure
Each tool consists of single universal front-end which produces configuration information in XML and passes it to a back-end specific to the underlying operating system. The back-end interprets the XML configuration instructions and applies them to the specific operating system. This has three major advantages:
* The tools work on multiple platforms, but still use the native configuration systems of the host operating system. There is no new configuration system to learn, and the GNOME System Tools does not interfere with the operation of other configuration methods.
* XML configuration profiles can be archived, permitting the “undo” feature.
* XML configuration profiles can be generated on one system and exported to another, allowing an administrator to change settings for an entire workgroup at once. ”
Although GST is still in it’s infancy it seems very promising. If your project could take advantage of the GST backends your system could include the ability to manipulate configuration files and successfully integrate software into the specific context of the underlying distro. For your information- someone is working on a similiar project write now for gentoo-gnoportage- which may one day enable the portage emerge system to be controlled through nautilus with a very simple GUI-ie. the progammer, slacker, wants to implement an XML-based parsing system for portages ebuilds and couple this with various backends(perl at first, then perhaps python(the language in which portage itself is written), and possibly c/c++- and combine all this with a simple GUI frontend abstracted and independent of the various backends.
The advantage in providing such GST style support in your autopackage system is that in the end your autopackage system would be able to deal with tight integration and system configuration issues, which as it stands, do not seem to be doable within your current project plans-correct me if I am wrong. Finally autopackage should be flexible enough to allow for binary and source compilation installs- this is simply a matter of abstraction and the functional distinction between backends/frontends. If your autpackage system could translate existing rpm/dkpg/portage databases, perhaps via XML, into a autopackage database installed locally- one that would “know” not only what packages are installed via configure-like scripts which hack into the existing system by searching for specific library files, but also “know” what the present package managers “know” about the base of installed systems and if it were possible to keep these two in sync with one another-ie. develop translation tools which go from rpm/dpkg/portage to autopackage and from autopackage to rpm/dpkg/portage, which admittedly is a lot of work, you could guarantee that users who use your autopackage system don’t end up with a horribly confused distro-specific package management system which is unaware of what is locally installed.
You could also have something like an autopackage wrapper for rpm/dkpg/portage- you could simply define what is a fundamental system app which needs tight integration and system configuration to work and and when the user tries to use autopackage to update one of these fundamental system apps let autopackage then grabs the distro-specific packages, in native format, and install them using the native package management system(hidden of course from the user)- this might be a temporary solution to actually integrating system-specific integration/configuration functionality within autopackage until such has become stabilized and wide-spread….
If you can “teach” autopackage to utilize the underlying distro-specific and to take advantage of what is already locally available your autopackage system could enhance the existing functionailty without compromising the functionality of the already-in-place package management systems and your system would be trully universal. And please rethink your stance as regards source compilation-this should not be mandatory for users of autopackage, but it should be availble as an option, and if autopackage can parse/translate the portage system you could extend this functionality to systems without portage-which would be great for users. If autopackage was capable of such you would be ale to provide users with a superset of availble installable packages giving rpm users access to dpkg/portage stuff and every other combination- users would love you for this and you would change the entire linux experience….
just some food for thought….
any initiative that would result making the installationprocess on linux as easy as the one for windows.
however, i don’t know if the issue would be solved just by inreoducing a graphical installer because, maybe due to the enourmous amount of versions of all kinds in the linuxworld, just solving dependencies seems not enough-example: i recently tried via “apt-get rpm” to install the newest version of gnome on my red hat 8-result:
a total crash of gnome when trying to login despite all the dependencies seemed to have been resolved successfully.
so, this is really very frustrating for a user, especially a newbie, because you usually don’t expect such with the products from redmond.
I’d love to see a NSIS-like for linux, too. As a windows programmer I use it a lot for distribuying my projects, and am sure it would be superb for the linux newbies.
Hi, thanks for your comments.
Well, we have to split “integration and configuration” up to really understand what it is. If you disagree with my definition please say so… basically, we currently have the designs and some code for stuff like:
* Menu integration
* MIME typing
Those will both be dealt with mostly at freedesktop.org in the future. For now we abstract them behind some APIs.
* Package manager integration. This is the interesting one. We don’t have code for it yet, but the plan is to poke librpm directly (and the equivalent, assuming there is one, on debian) to essentially install a “fake” RPM. RPM is then aware of what was installed via autopackage and you can uninstall it using RPM and so on. We’re not sure how far we can go with this integration yet, we’ll have to see.
“From you FAQ it would seem that you intend autopackage to only be used for end-user apps, ie. stearing clear of the fundamental system apps which make up the linux system(ie. gcc/glibc/x windows/kde/gnome etc.). The way I see it you have chose a route which is quite difficult, in terms of the adoptability of your autopackage system- and its relative appeal.”
Actually, I think the opposite is true. If we’d simply created a new kind of RPM/DEB, who would use it? Would Red Hat? Would Debian? Would Gentoo? Certainly not, these distros are very much tied to their choice of package manager. By producing something designed to layer on top, and limiting the scope of the project so we focus on only a few types of packages and doing them well, I think we can be more easily adopted by the developer community because it doesn’t rely on having distro makers replace their existing infrastructure with our own.
“and this means that users will have to have command of the distro-specific system for taking care of fundamental apps and your autopackage system too. I see this problem of system integration and configuration being as important if not more important than dependancy resolution per se. “
In fact we intend for the “package” command to be able to drive RPM, DEB, emerge etc as well. In reality, we may only bother with RPM, as if you’re using Debian or Gentoo then a large part of the reason is probably the native packaging system anyway, we’re mostly focussed on making things easier for the more “newbie” friendly distros, which tend to to package everything and whos users aren’t happy with compiling from source.
So, you could simply never use the “rpm” program, and just use the one we provide, and it’d figure it all out for you. RPM has a truly confusing syntax imo anyway, I think it’s possible to do better.
But, I see your point. It’s not just the command line tool, it’s also package GUIs etc. Well, one of the reasons we’re working with the LSB and Free Standards Group is to deal with issues like this, trying to make things as integrated as possible. I don’t know how well it’ll work.
“Most of the dependency resolution hell of the past year has been due to fundamental changes in gcc/glibc as regards the ABI and the slow transition to xft/fontconfig/freetype2/xfree-4.3.”
I don’t agree. Dependancy hell existing long before this, and is due to fundamental problems with the way we manage the whole thing. Note, glibc never breaks the ABI, it uses symbol versions so that if your app works on one version of glibc, you know it will work on later versions.
There are exceptions to that rule with glibc, some broken (postgresql) or just unlucky (wine) apps ended up depending on facets of the implementation, and they did indeed break. We can’t do much about cases like that, they are the exception rather than the rule.
“This transition has caused problems for every major distro- rendering version 8.0+ incompatible with everything pre-8.0”
It would have been anyway, apps compiled on later distro versions pull in the latest glibc symvers, and so may or may not run on earlier versions (it’s unpredictable without analysis). That’s one thing the LSB is useful for – it standardises a more stable set of symvers and core interfaces.
“yet this should be stabilizing within the next 2-4 months and is unlikely to recur at such a fundamental level inside of the next 2 years.”
You’d still have dependancy hell unfortunately, although yes, the current transitions (mostly c++ abi related) are causing large amounts of breakage.
“Although GST is still in it’s infancy it seems very promising. If your project could take advantage of the GST backends your system could include the ability to manipulate configuration files and successfully integrate software into the specific context of the underlying distro. “
Indeed, possibly using GST is a good idea. We may well look into that, feel free to join the dev list if you’d like to help. Most of the GST tools are related to system configuration however, not something that applications should really be screwing about with anyway. Why would an installer want to alter your system clock, or change your XFree configuration? The only one off the top of my head that might be useful is runlevel configuration, which we can more easily abstract at the API level (there aren’t that many different systems), but I don’t really want to see apps inserting themselves into the runlevels system anyway, for automatic starting at login time the session manager is more appropriate.
“which as it stands, do not seem to be doable within your current project plans-correct me if I am wrong”
There’s nothing inherant in the design which says we can’t, it’s just a case of providing an API to packages to perform that integration. But for the type of (mostly desktop) applications, they simply don’t need very tight integration below the desktop layer (which is being standardised at freedesktop.org).
Remember we are going to hopefully be able to delegate to the native packages for dependancies anyway, so if a package depends on GStreamer then autopackage would install the packages native to your distro (if there are any), so that’s another way that the line can be crossed.
“Finally autopackage should be flexible enough to allow for binary and source compilation installs- this is simply a matter of abstraction and the functional distinction between backends/frontends.”
Right, it’s on the to do list. Patches are welcome.
“If your autpackage system could translate existing rpm/dkpg/portage databases, perhaps via XML, into a autopackage database installed locally”
I think you want http://www.gnupdate.org/ which does precisely this. We are taking a different approach, partly because that kind of translation is extremely hard, it’s not just a case of sprinkling some magic XML, at some layer you have to map between the dependancy names, and file vs package deps. Tricky.
“If you can “teach” autopackage to utilize the underlying distro-specific and to take advantage of what is already locally available your autopackage system could enhance the existing functionailty without compromising the functionality of the already-in-place package management systems and your system would be trully universal.”
This has been the plan for a long time now, I may not have made it clear because the article/interview were both mostly non-technical, and because we haven’t written the code yet. If you’re interested in working on this however, the design allows for it nicely and all we need is a coder to do it
As a fairly experienced C++ developer, here is my book advice.
1. I’ve had little luck finding a good reference. Every book I’ve gotten is good at some things, but terrible at others. No matter how big the book is, make sure it has a BIG index. You’ll use that the most. I’ve got Lippman and Schildt, and I’ve used Straustrup and Deitel. None are particularly good.
2. Once you have learned the language basics, buy “Effective C++” and “More Effective C++” by Meyers (and everything else this guy ever wrote about C++). These are the best C++ books I have ever seen. Read them and reread them. This guy has a deeper understanding of intricacies of C++ than I will ever have.
Good luck.
i wish autopackage all the good luck and hope it makes great progess and good sucess!!!
After all the user would have to keep track of system updates and package management through rpm and the RH package manager and then use the package program from autopackage for outside non-distro appps.
I am all for things like better dependency resolution and easier install and maintenance of packages in linux.
However, is this any easier, than rpm and apt if you throw in the fact that the user has to be familiar with two different ways on the same system to manage their applications?
Why is this any easier than making a distributed version of apt and creating better rpm packages with less outside dependencies?
After all, why in the world do we need like four different libgnomeprint and five different types of bonobo packages is completely beyond me!
Why is rpm so limiting? It seems 99% of the time my major problems are not the rpms or the rpm system but come from poor packaging of the rpms or groups of rpm sets.
Not trying to flame but are we just adding to the complexity of the overall system instead of fixing the package systems we already have?
Maybe my understanding of autopackage is completely wrong but here goes:
Is there a way to prevent autopackage from installing dependencies that come from untrusted sources?
I.e. if I want to install a package that belongs to the namespace osnews.com and if this package has dependencies that belong to a different namespace, say slashdot.org, will autopackage notify me about this before attempting to install the slashdot.org packages?
I’m thinking along the lines of java applets that only initiate connections to the same server they originate from and not to any other servers (unless you grant them permission).
This doesn’t really affect me (I’m a gentoo user), but I still think it is a fantastic project. I frequently see the postings about it on slashdot and it makes me excited. Sometimes I catch myself forgetting how bad dependencies are on other distros (not counting you Debian folks). I always think, if everyone just used Gentoo or Debian there would be no problem. Then I remember how scary installing one of these “difficult” distros is for a first time user and I realize that will never happen.
However, autopackage will take that ease of use to other distros, which is important.
Yes, you’re right, it does.
Less than ideal, isn’t it. However, remember that right now, today, people have to compile stuff from source all the time anyway, unless they never wish to upgrade their applications or install new ones from the net (having packages available for your distro is a hit and miss affair).
So really, things aren’t getting any worse with respect to package dilution, they are simply changing.
Also, I’d note that both Windows and MacOS have different installation mechanisms for OS components than 3rd party apps. The fact that most distros comes with so much stuff blurs the line somewhat.
You’re right in that it’d be possible to improve RPMs to the point where they are distro neutral. However, to do that we’d need to get everybody to agree on a standard dependancy tree. If you see the lsb-discuss archives, you’ll see we’re discussing that right now, but success is not guaranteed.
Basically, autopackage is about me hedging my bets. In the ideal world, all the distros would come together, quickly and efficiently settle their differences and a ultra-cool package management system would be designed and deployed.
In the real world, that might not happen, in fact, given the past history of such attempts, it probably won’t. If it doesn’t, we have autopackage, which requires no standardisation or agreement as it layers on top.
Finally, nothing is fixed, if we can agree on a standard middleware layer in the LSB, then maybe autopackage will be a purely behind the scenes thing, and users will only have one interface to learn. Note what I said above about autopackage “proxying” to RPM also.
@OutOfTheBlue: In short, no. Dependancies will almost always come from different domains. Security would most likely be done via package signing, as is the case today.
Thanks for your reply Mike.
Package signing is a way to confirm that the package is really created by a particular entity/person right? My question is what happens if I don’t like this entity’s work?
Say I installed some stuff from osnews.com and that created some issues with my KDE installation I got from kde.org. Now, that really pissed me off and I don’t want any more osnews.com packages on my machine ever again. Will autopackage provide a way to blacklist osnews.com and notify me before I try to install something that requires some modules from osnews.com?
I guess so, it wouldn’t be hard to add. It’d be a better idea to simply make sure you never have one package screw up another though