What follows in the article is an analysis of the "dependency hell" problem in Linux and the issues surrounding software installation. I make a proposal of my own and critique the suggested "solutions" to these problems and point my finger at "the" (my perception) source of these problems.
Post a Comment
When I read most of the first page and the author hadn't yet made his point, I couldn't be bothered to read the rest.
Anyway, IMHO ... the problem isn't dependency hell, as it seems that both apt, portage, and probably other package managers are able to approriately deal with those.
The problem as I see it is that most distros tend to want to do their own thing with package managers, and so when you pick a distro, you're automagically limited to whatever packages the distro and/or the distro's community provide you, unless you want to do configure, make, make install and hope that works.
I have said for a long time that what is needed it to define a standard for how packages should be laid out (ok, two standards - one for source and one for binary distros), and let the package mangers each do it their own way. Just like HTML doesn't care about the web browser, neither should the packages care about the package manager either.
> as it seems that both apt, portage, and probably other
>package managers are able to approriately deal with those.
I don't believe they do. There are always cases where not-so-intelligent packagers screw up the packages and deps don't resolve correctly, or they end up overwrite versions of libraries you really need for other apps. Fink (apt-based) was the latest problem I encountered the other day (I wrote about it on my blog). It sucked. I wish things were statically linked like apps are on Win/BeOS/OSX. Things just work (TM).
I still don't see a perfect solution for package management. we have apt, yum, opencarpet, etc
but i would like to see the distros duke it out a bit more and let the market decide
the last thing i want is the LSB to pick a winner. the market is better at picking winners than a committee.
I am fine with this. As long as installation of the app is as easy as it is with statically linked apps, I am willing to have a few extra KBs of filesize (which actually makes the app load faster because it doesn't try to go find all these libs one by one -- a reason for gnome apps loading not as fast as they should) and to download the full app when a new version is out (I am on a fast 512 KB/sec cable line here).
>And should be recompiled for every update. ...
From the moment that the "statically link" route is taken, that would be the job of the developer, not mine, so the "pain" wouldn't be mine, as a user.
Remember, I am not talking about statically link everything, but surely everything that is not standard on distros, e.g. libsigc++ or gtkmm. But certainly not things like libpng or libfreetype.
Gotta agree with Gabriel here. I am often forced to compile some of my code statically, esp. in re Intel's MKL or other numeric libraries. And the problem gets really annoying. Keeping around a few versions of 50+ MB executables starts getting to me.
And, has anyone ever tried to figure out what static version of popular software would bloat to? Some of the GNOME and KDE programs are already pretty big using dynamic linking.
> the last thing i want is the LSB to pick a winner.
Almost nobody cares about LSB anymore (except maybe commercial distros).
Neither debian nor gentoo are LSB-compliant. They don't even use rpm currently, so IMO there's no way they'd ever adopt a LSB-glorified package manager (unless it's apt, of course).
>Keeping around a few versions of 50+ MB executables starts getting to me.
If gcc was more clever, it should not be 50MBs of executables. I am talking about a few more KBs more for each binary package! That's negligible!
SGI's compilers KNOW how to correctly statically link only the needed symbols from libs, it's time GCC learns how to do that too instead of statically compile all the cr*p inside and bloat things.
@Eugenia
Uh, at last check GCC does only link in needed Code. I've done quite a bit of static / dynamic mixed linking with executables I've built and all of the documentation clearly states that only needed code is linked in. Perhaps you should further research your claim.
> From the moment that the "statically link" route is taken, that would be the job of the developer, not mine, so the "pain" wouldn't be mine, as a user.
s/developer/maintainer/
I'm sure no developer with limited resources would take that route, many are already only releasing source and that number would only increase with your proposal.
>I'm sure no developer with limited resources would take that route,
That's their problem. If they want their app to be used further, they should provide binaries that really work.
>Uh, at last check GCC does only link in needed Code.
Last I checked, it didn't.
" why is it that the GTK libraries end up in /usr/lib but QT/KDE does not ? why is it that dbus is dependent upon glib and how should KDE reconcile itself to such a dependency(Not Invented Here syndrome)." NOTE TO AUTHOR - read this quote from their DBUS website:- "D-BUS has no required dependencies, but includes GLib, Qt, Python and .NET APIs to the D-BUS system, if you want to build those."
So are you just trolling regarding the KDE reconcile comment?
GTK libraries in /usr/lib - good question why is it there? Does it comply with the LSB in being there?
The author seems to forget that Debian solved dependency hell about 7 years ago with the introduction of apt, and have been laughing at the increasingly fractured and broken RPM distros ever since. I'm surprised the editors of OSNews would allow such a glaring oversight get published.
> I am talking about a few more KBs more for each binary package! That's negligible!
A few KBs? Are you kidding (from the GTK tutorial; GTK version 1.2):
gebner@schnecke2 ~/tmp/scribble-simple [0] $ gcc -static scribble-simple.c `gtk-config --libs --cflags` -lpthread -o scribble-simple.static
gebner@schnecke2 ~/tmp/scribble-simple [0] $ gcc scribble-simple.c `gtk-config --libs --cflags` -lpthread -o scribble-simple.dynamic
gebner@schnecke2 ~/tmp/scribble-simple [0] $ ls -lh scribble-simple.*
-rw-r--r-- 1 gebner gebner 4.6K Jun 17 1999 scribble-simple.c
-rwxr-xr-x 1 gebner gebner 17K Aug 4 21:17 scribble-simple.dynamic
-rwxr-xr-x 1 gebner gebner 2.7M Aug 4 21:17 scribble-simple.static
gebner@schnecke2 ~/tmp/scribble-simple [0] $
Not trying to flame or any thing...
but why not copy the ports/package system from FreeBSD or the like?
It is OSS, so there is nothing stopping it from being ported and it handles dependency problems wonder fully. Except where stuff is mutually exclusive, but that is a given...
>A few KBs? Are you kidding (from the GTK tutorial; GTK version 1.2):
Gabriel, READ again the above comment about GCC not knowing how to ONLY statically link the symbols it really needs! What you linked there was the FULL gtk, NOT the symbols that are really needed!
>The author seems to forget that Debian solved dependency hell about 7 years ago
Read my comment above about Fink (apt-based OSX system).
@Eugenia
Last I checked, it didn't.
Perhaps you should check again then. GCC only links in all symbols of a particular object when linking if it was directly referenced on the command line as "foo.o", and then only on certain platforms. When linking to library code GCC only links in symbols (pointers to code / data basically) that have been referenced by the program and global symbols in the library. It does NOT just link in the entire library. Read the GCC documentation. Executables can be further shrunk after compilation by using the "strip" program supplied with their compiler toolkit.
In my experiments even the Intel Compiler kit for Linux doesn't produce smaller executables, in fact it generally produces larger ones.
Besides, we are not talking about statically link libs that are standard. GTK+ is standard. Most other third party libs that are not standard are usually much smaller.
BTW, you didn't strip your binaries.
strip --strip-all scribble-simple.static
ls -l scribble-simple.static
I wish things were statically linked like apps are on Win/BeOS/OSX. Things just work (TM).
I wonder why nobody (no distro that i know) never tried to provide statically linked versions of Gnome/KDE/etc? Is it hard to do?
Victor.
> Gabriel, READ again the above comment about GCC not knowing how to ONLY statically link the symbols it really needs! What you linked there was the FULL gtk, NOT the symbols that are really needed!
-rw-r--r-- 1 root root 3.9M Jul 30 15:03 /usr/lib/libc.a
My static libc is bigger than the static executable I presented, so gcc/ld has to strip at least something.
If you had bothered to read the article you would see that the problem I was writing about effects all Linux distributions with no exceptions. Debians apt-get is only really a solution for Debian packages-whereas what I was trying to point out is what would be necessary for third-party developers to distribute applications and libraires for any and or all distros. Although I do not use Debian I do like the package management system used by Debian(I have used it before).
This is why I specifically did not address the differing package management systems which are currently in use by different Distros- this article is not about RPM vs. APT vs. vs Portage etc. No package management system from any distro can address the needs of all distros-and when it comes to distributing and installing software for the "Linux platform" what is needed is alternatives which can dynamically adapt to the particular distro on which the application is to be installed. LSB represents one approach to this problem, for those distros which conform to the LSB- many wish to radically reduce the number of dynamic dependencies- either by statically linking everything- which is simply not viable for most software- or by trying to outright reduce the number of libraries used in software development- which means less code reusage, which is the native advantage of opens source software.
My text is an attempt to explore an alternative-autobuild environments which can compile only those specific libraries which are needed for a particular app, dynamically. No packaged binary solution can have absoulute all dependencies covered when one takes into account all of the different Linux distros. And no package management system covers absolutely all needed libraries.
So I did not overlook Debian....
LSB does not require you to statically link the libc.
> BTW, you didn't strip your binaries.
The difference is tiny:
-rwxr-xr-x 1 gebner gebner 11K Aug 4 21:31 scribble-simple.dynamic
-rwxr-xr-x 1 gebner gebner 2.4M Aug 4 21:31 scribble-simple.static
compared to:
-rwxr-xr-x 1 gebner gebner 17K Aug 4 21:17 scribble-simple.dynamic
-rwxr-xr-x 1 gebner gebner 2.7M Aug 4 21:17 scribble-simple.static
As I told you, your sample is not representative. GTK is a big library and it is standard. Try to compile a real app that has 2-3 deps (or more) to non-standard third party libs (which are usually small libs). Even if the binary ends up be big, it well worth the pain of installing these additional libs by hand or by a package management system that is prone to screw up things regularly.
> or by a package management system that is prone to screw up things regularly.
If you use such a package management system, that's your fault. I've never had any problems with apt/debian with regard to this.
"Perhaps you should check again then. GCC only links in all symbols of a particular object when linking if it was directly referenced on the command line as "foo.o", and then only on certain platforms. When linking to library code GCC only links in symbols (pointers to code / data basically) that have been referenced by the program and global symbols in the library. It does NOT just link in the entire library. Read the GCC documentation. Executables can be further shrunk after compilation by using the "strip" program supplied with their compiler toolkit. "
1. GCC is not a linker.
2. Do you have any idea how linkers work?
3. Care to explain how a linker could do dependency analysis, figure out what pieces of data a piece of code needs, as well as what other procedures it might call, and then only extracts the relevant code from the library?
Gabriel, I have used pacman, apt-get, yum, Fink, urpmi, BSD ports and Gentoo among others. I am not a newbie around here. I have found problems with ALL of these package manangements that they simply try to resolve the impossible and they try to solve the WRONG problem.
I am against dependency-resolution package managers. It is a solution for the wrong problem.
Now, you can either respect my opinion as I do yours and give an end to this discussion, or try to be sarcastic and caustic as you were earlier and that won't help you around here.
Unfortunately sometimes static linking isn't an option, either due to licensing issues (LGPL requires a dynamic executable) or platform problems.
For example, I've been maintaining the Linux port of the ags (Adventure Game Studio) runtime engine for a while now. I originally distributed binaries that were statically linked to Allegro, and Allegro in turn would dynamically load a module for arts support on the system. However, when it does this on newer Linux distributions it seg faults. When I filed a big with one Linux distribution vendor I was told in no uncertain terms that statically linking to a library that dynamically loaded libraries was unsupported and strongly discouraged.
> 3. Care to explain how a linker could do dependency analysis, figure out what pieces of data a piece of code needs, as well as what other procedures it might call, and then only extracts the relevant code from the library?
1. Figure out the entry points of the code.
2. Traverse all references from the entry points.
3. Only link those functions for which references were found.
Am I too naive?
@tore
"1. GCC is not a linker."
I know that, but it does manage the linking process.
"2. Do you have any idea how linkers work?"
Enough to use it and work with its idiosynchrasies (at least as far as the GNU Toolkit is concerned).
"3. Care to explain how a linker could do dependency analysis, figure out what pieces of data a piece of code needs, as well as what other procedures it might call, and then only extracts the relevant code from the library? ""
Care to explain why the GNU documentation explicitly states that when liking to library code only referenced symbols from that library are linked in?
I don't pretend to understand every single step it performs, but I know enough from reading the documentation and having done several different types of linking and maintaining a Linux port of a game engine to know that it doesn't link an entire library in.
@Gabriel
Thank you, thank you, thank you 
And yes, BTW that's how it works from what I've read...
The FHS, which is a Linux standard for directory layout,
indicates that /usr/bin is where an executable goes unless
one of:
a. it is only run by root (use /usr/sbin then)
b. it is required at boot (use /bin then)
c. both "a" and "b" (use /sbin then)
d. it comes with XFree86 (may use path with "X11R6" in it)
e. it doesn't come with the system -- see below
You use $HOME/bin for personal stuff, /usr/local/bin for
stuff the admin compiled up for all users, and a place
under /opt for add-on packages (sold separately).
By this, we conclude that KDE is either a distinct add-on
package that doesn't ship with the OS (it is usually not)
or that KDE is getting installed in the wrong place.
When I tried to install a program (rpm based) and I found out I had to download other little pieces of software on the internet, I gave it up.
Then I tried my first debian based distribution.
I honestly think the debian "way" of installing programs is the solution.
This basically what gentoo does with portage/emerge. But this won't work because of the same reasons that the different BSD's port systems are not 100% compatible. Not being up to speed on this I believe the Open and Net ones are compatible. As the BSD's are similar in distro's that they patch and edit al included software to a degree to cope with for example kernel and device system differences. The problem would also occur if you downloaded a random (Ports is source based? isn't it?) that is binary for say open BSD and insert it in a net BSD. A few behavioural differences between the two where they store files would give the same troubles.
But the reason this dependency hell exists is partly also due that packages depend on other packages and not the files that those packages contain. This is the central flaw of the package system and is also why the auto package system works so well. This means that an RPM package named AAA says I need package AAB and does not say I need file libAAB.so. This is the major shortcoming of the package system and also the reason why this problem occurs as different distro's put files in different package. So in SuSE the package is named AAS.rpm and in Red Hat AAR.rpm a distro's alien package wants the package AAA.rpm as this is the one it expects will provide libAAA.so and does not know that both the SuSE and Red Hat Package also provide the library libAAA.so.
You must not forget that a package is merely some binary program (often) with a description of what that program requires to work and where it should put it self.
And that is why even as Ports has been ported to gentoo it won't solve dependancy hell as it is as the major commercial distro's are linked very much to their rpm/deb based way of doing things and they don't yet have the time or commercial luxury to change things. As doe the ISV who dislike any changes at all. I hope you find my points interesting. Feel free to reply or feel free to assign this to the garbage.
Jerven
@velox:
but why not copy the ports/package system from FreeBSD or the like?
I thought Gentoo was special in the Linux world, because it uses a BSD like ports system.
And I am such used to building everything on my FreeBSD system from the latest source tree (kernel, usertools, apps) that I seldom trust one of my generated binaries to run on another system, because I fear it is not in sync with the libs there. :-)
Regards,
Marc
...is to decide on one packaging format (.dpkg or .rpm or a new one perhaps) but there's no guaranteeing that all the distros will support it.
I think software makers should just make their own packages, like an installer. People like Real and NVidia do this already. Should it really be the distro's responsibility to make sure that the installation of 3rd party software packages run smoothly?
I installed freeBSD on my laptop recently (dual boot with Mandrake Linux), and suffered through the interminable cvsup routine, updating the OS, downloading and building the ports. Due to dependency problems in the ports collection, I had to repeat that part of the process a couple times (something to do with updating perl to 5.8.5, etc). The bottom line is my poor laptop sat spinning furiously for the better part of 2 weeks. All in all, I bet it compiled code for 40-60 hours or more. The poor thing was hotter than a fire-cracker, and I bet I took a good 6 months off it's life. Yeah, I want to go through that again! I'm thinking the laptop HP now sells pre-loaded with Linux sound better every day.
Well I was just checking out Autopackage and they just added(dont know when) support for sealed installers!!!
So now Autopackage can work like the old loki installers with no need for network dependency resolution. Very nice.
I prefer to build from source. Though Slackware has a pretty nice package tool, I normally will download the source tarball and go from there.
./configure
make
make install
That easy. And if you are missing a dependency, the configure script tells you.
Uhm, dude, that's the whole point... That dependency then needs another dependencies, which, in turn, have dependencies of their own and so on and so forth.
./configure-make-make install is not the solution to this problem.
Although I have to admit that 'back in the day' when I used Linux, I also tended to use compile-from-source.
Eugenia is right, GCC is very lazy when it comes to staticly linking. GCC doesn't look further then the object file (there are multiple object files in a library) to check what has to be included. The result is very very _very_ large staticly linked binaries.
But this lazyness of GCC can be worked around like the guy of dietlibc did by making every single function its own object file (plus other size saving hacks). He has some links to applications (grep, ldap server, etc) that use his library staticly linked that are of sizes you can't get a staticly linked glibc hello world binary.
... but I wouldn't wish to see that. I for one would absolutely not be happy if some commercial rpm based distromaker would make the rest stick with them.
I for one enjoy (how peculiar that may seem to many of you) the diversity of package management systems. It just is part of what the open development process means. Or should I say "this is who we are".
I have to tell you, I always thought that package dependency hell is only a problem of the lazy (some of you will take this offensive, hey, I can live with that). There is no problem in linux what a caring mind cannot solve (yeah, you might also argue on this as long as the Nile goes).
I have to tell I am a debian man in the first place (though not exclusively). I have never ever encountered such a hell that couldn't be changed into joy over some nights and by some beers (okay okay, I know how that sounds ). I'm not telling I always like the solving process, but the joy when being done has always beeing worth it.
So for me, I don't care if someone will introduce some standard on a rainy afternoon someday, I just wish I will still have the good old opportunity to use what I want.
<grin> How about using a tool that does not need to be told three times where the dynamic libraries are supposed to be, after which it still doesn't feel like doing the right thing</grin>. Seriously, even though this would be a major architectural change, why not use, reimplement, or copy the dynamic linking features of Darwin?
Autobuild sounds like a great idea. I think I'd like [binary + source] packages that can be _rebuilt_ on the fly, or whenever the admin so whishes. If the binaries work, if you're not paranoid, or you're low on CPU - no need to compile. If they don't work, or you can afford it - have the system rebuild the package(s) seamlessly.
Source trees are great for developers, but I'd rather not have hundreds of thousands of files consuming filesystem time or space. Keep the user happy and the admin sane by keeping it simple.
Another idea might be to separate metadata from the common binary packages, and have an Internet-service that pushes updates to package metadata - dependencies, conflicts, etc, as well as storing all ever published packages so you'll never ever be missing one because it's not available anymore.
If this can be tied to some kind of failsafe install/remove mechanism where packages aren't really uncompressed in common locations, but kept apart from each other, in the form they came, more or less, I'd be thrilled. Perhaps something like what QNX has. (Until recently I think. Did they remove it?).
DragonFly BSD has plans for a new package system:
http://www.livebsd.com/cgi-bin/trac.cgi/wiki/dflypkg
I am against dependency-resolution package managers. It is a solution for the wrong problem.
I'd be very curious if you would explain what you think is the problem.
Resolving installation dependancies and application library dependancies are completely separate problems. Static binaries are a possible solution for small programs, but not for programs that take tens of MB of space. Its also not a particularly nice solution if a major flaw is found in a critical library (e.g., the libz flaw a couple years ago, and I'm pretty sure glibc has had such flaws, openSSL has recently had such problems).
The big problem is that the major libraries: libz, glibc, etc. are different on major distros. Sometimes these differences are incompatible, even between distro minor versions. When you have the source to a program, this isn't really too much of an issue: you simply (may not so simply depending on the distro) recompile.
But, when you have a binary only program (i.e., the Intel Fortran Compiler), there is no way to do such. Now, "yous takes your chances" is the relevant theme, but many people desire such programs to work in the Linux world. Ideology aside, its a "good thing" to help them.
The question of how is very, very hard on an operating system where most applications are assumed to be source compatible, not binary compatible. Face it, there is no quick-and-dirty solution. The only real solution for these vendors is if their primary platform targets cooperate and agree to make the job easier. I don't how they can do this and expect things to work in Gentoo/Slack/Jo-Blow Linux as well. Its probably and intractable problem.
Face it, GNU operating systems were designed to be source compatible, not binary compatible. Them's the breaks. It opens up a whole world of possiblities, but there will probably never be a way for any random binary application to work perfectly across all Linux distros. There's too much variety, but at least a subset will work. The LSB will help that subset identify a target for vendors, and it may help out-of-set distros, but it can't be responsible for everyone.
Hmm, now that I think of it, one possiblity is that the LSB should not insist on particular library versions. Instead, the LSB should insist on certain implimentation functionality. Maybe specifying such functionality as "version X" of a library.
Now, when a distro includes incompatible versions of a library, they must include an LSB library (libLSB) that provides the specified functionality. For instance, if glibc changes to provide a newer version of "foo", then libLSB should contain the older "foo", which will be linked against.
I'll admit to not thinking about this for more than 5 minutes, so any objective criticism is welcome. This also means that all LSB-compliant distros (something done by certification) will end up supplying their own versions of libLSB. That seems like a reasonable compromise to me, as it allows for strict compliance with several LSB versions as well as letting source-compatible applications follow the moving targets.
At first, I was inclined to agree with your opinion of Eugenia, but then I did some testing.
1. The gnu manuals do indeed say that only the necessary symbols are loaded for static linking.
2. The gnu manuals lie.
---a.h---
#include "b.h"
int foo(int x);
int bar(int x);
int baz(int x);
---a.c---
int foo(int x) {return x + 42;}
int bar(int x) {return x * foo(x);}
int baz(int x) {return bar(x) - wombat(x);}
---b.h---
int wombat(int x);
---b.c---
int wombat(int x) {return x*x;}
---main.c---
#include "a.h"
int main() {return bar(57);}
gcc -c a.c
gcc -c b.c
ar rc libFoobar.a a.o b.o
ranlib libFoobar.a
gcc -c main.c
gcc -o main-static main.o -L`pwd` -lFoobar
My gcc-3.2.3 puts all 4 functions (foo, bar, baz, wombat) into the executable, even though only 2 are actually used.
You are correct I stand corrected. I found this information out myself shortly after having written the article. This was a thorny debate issue a while back-and furthermore it serves as an example of such issues. It was not my intent to troll about KDE or GNOME- I am glad that DBUS is being written in such a way as to have no dependencies and allowing glib and KDE to provide wrappers. But this was not the first such issue and probably won't be the last.
My example was slightly stupid, because main.c #included all the 4 functions. Should have done 1 more minute of thinking.
Anyway, it seems that gcc includes all the symbols that it finds the program accessing, even if it's just through an #include and the corresponding function never gets used. So, for example, if you changed my files so that a.h does not include b.h, you will find that wombat doesn't show up in the executable, but baz (which never gets called) does show up.
Exactly.
This is why I said that GCC doesn't do it right, no matter what the manuals say. SGI's compilers, I know, they do it more correctly.
It maybe that GNU systems wered designed to be source compatible-but if you look at the build environments of the most popular distros all you find is complete and utter chaos(Redhat, Fedora, Mandrake, SuSE).
There are many issues responsible for this: a) maintaing build environments which build a humongous library of software on multliple different architectures and most importantly b) the "branding" needs of distributors in the form of specific "features" brought on through patching and specific compile time options.
The build environments present in the major distributions make the possibility of an "autobuild" solution quite difficult- the build environment needs to be relatively clean and coherent. But if all of the mature distributors could provide clean and coherent build environments they could standardize on a kind of "autobuild" system which would allow third-party application developers to let such a system dynamically resolve dependencies not managed by the package management system itself.
Their is probably a direct relationship between how clean and coherent the build environment of a distribution is and how much it would profit from such a "autobuild" system. Debian and Gentoo would only profit slightly from such because their package management systems have solved most of the problems with dependencies- whereas Redhat, SuSE, Fedora, Mandrake, Lycoris,Linspire,Xandros etc. could profit immensely from such. But the point of such a system would to be to facilitate work for third-party developers.
At any rate a key to solving these problems is rigorous adherance to library naming conventions: we need:
liblib-Major.Minor.Bugfix.PackageSystemFix
The libraries as released from the authors can only include:
liblib-Major.Minor.Bugfix
But given that most binary distributors also change their versions of the third-party libraries over time to enchance integration and ease of management-a system needs to be built which documents which patches and compile options were used by the distributor so that third-party developers can write their code independent of the particular hacking of particular distributors....
A "major" release number indicates changes in the API which may break backwards compatibility. "Minor" indicates incremental changes in a given API which expose new functionality not included in the older API. "Bugfixes" explains itself- ie. has this or that exploit been fixed?. And finally "PackageSystemFix" should describe the changes made in the .deb/.ebuild/.rpm spec file which allow for integration of the library into a package manangement system.
liblib-Major.Minor.Bugfix.PackageSystemFix
I personally would prefer the Gentoo naming convention; it is often convenient to explicitly state whether a given package/library is beta or alpha or release candidate or whatever.
http://www.gentoo.org/proj/en/devrel/handbook/handbook.xml?part=2&a...
The dependency problem is _solved_ lots of times ago. apt exist. statically linked packages exist (see the opera deb package in opera webpage).
Dude, dependecy problems are _already_ solved.
The author suggest yet another source-based package system. As far as I know, source-based packages systems are _dead_. From the controversy generated in the latest year, one can very well conclude that _nobody_ is happy with either binary packages or source-base compilations. Everyone wants both. And since binary packages must have _some_ way of being compiled from source, they're the ones who really satisfice all the users.
The "dynamic dependency" crap (aka USE flag) has no sense. Good software allows you to configure at *runtime* its settings not at compěle time. If I've to choose at compile time if X program is going to use OSS or alsa, well, that software is _broken_, fix it. Decent software uses OSS or alsa or esd or arts or...., depending on what is available, not having to choose what one you'll use at runtime in a fucking environment variable.
i think the solution partly lies in packages not only asking for packages but for specific files so that if the package is not around the file can be looked for by the installer.
then there is the "problem" of multiple library versions. is there a way to make it smarter so that you can have multiple versions of the same library installed and the app can tell what lib version it wants to the system? sure it will lead to install bloat in that you have multiple versions of the same library installed but that is unavoidable, either you build all yourself or you risk a bit of install bloat. sure this is similar to static linking but it will lead to smaller installs overall (compared to pure static links) as when you staticly link you get a new version of the lib for every app you install. with the system above you only get multiple versions of you have a app that must have the new version but allso have a app that need the older version.
and what if the installer is a bit smarter so that when you uninstall/upgrade something it allso parses the dependency tree for what your uninstalling/upgradeing and cleans up the tree for packages that are no longer in use (go down one level, do a scan for installed packages that depend on this package, if none then remove/upgrade it and check the dependencys of that package in the same way until you hit rock bottom). sure if you mix it with source installed stuff this may break but thats allso why i try to avoid source installs as mutch as i can.
as for distros useing modified libs and compiles well that is i fear unavoidable as the text points out that while its new school in the way the source is handled it still old school business style when it comes to user lock-in, this will allways be the case i fear. but then if apps could request diffrent lib versions and the system could install it without breaking stuff allready in the system the power of lock-in would be gone. just install a set of "standard" libs on top and be happy
if a lib or app was changed compared to the default one then they should carry a tag saying so in the package name. like say rh have a modified version of lib foo. then it would be named libfoo.x.x.x-rh.xxx and you would know that this is a home grown version. if a package then requested libfoo.x.x.x.xxx it would be installed and the user told the reason is that the -rh one is a rh specific variation and the app he is installing is requesting the clean one.
"what one you'll use at runtime in a fucking environment variable." -> compiletime, obviously
BTW, I very much agree with other people here.
"Sorry, I've debian stable, I try to use gnome 2.7.53-alpha3.2 packages from GodKnowsWhere and I've dependendy problems". Of course you've dependendy problems. The thing, is, a normal user wouldnt want to do stupid things and he wouldn't install those packages.
Its the fact that "backward compatible" isn't in the average OSS developers vocabulary. Why does every version of a library break applications written for a previous one? Newer should always be compatible, but improved.
It is the same problem with the kernel. Modules won't migrate from one kernel to the next. Not even through a simple recompile in almost all cases.
The other thing (that autopackage works around) is the way libraries are linked. Unless you use a special linking procedure, just having a compatible library on the system still may not work.
Packaging is the LEAST of our worries.
(Sorry if this is a rehash, I wasn't interested in reading 50 of the same post)
You are kidding? right?
Typical example for you: lots of simple qt applications can be trivially made to look and behave like kde applications by replacing QApplication, QFileDialog, etc. with KApplication, KFileDialog, etc., since the function parameters match. Typically you do it like this:
#ifdef USE_PRETTY_KDE
KFileDialog dialog(dirname);
#else /* use ugly qt */
QFileDialog dialog(dirname);
#endif
Now, are you suggesting that instead of this elegant mechanism (which is directly exploitable by Gentoo USE flags), an application should always compile against kde (and which version of kde? there are odd incompatibilities) and using some complex fragile system-dependent magic degrade to qt if kde is not found?
A pity you can't read and/or comprehend . If you would have bothered to read and try to understand what I wrote you would see that I am not talking about a new source based package management system. For your info source based systems are actually quite new in the Linux World, with the exception of LFS. And they are far, far from dead. But *my point* is that the binary distributions could benefit from using some of the technology found in source based distributions.
Firstly source based distribtuions provide clean and coherent build environments-they must or there is no software to run. Secondly source based distributions like Gentoo already know how to deal with multiple versions of the same library. Now if binary distributions could provide a kind of "autobuild" environment-standard on all distributions- third party developers could write their application and let "autobuild" only compile those libraries not already provided for in their own package management systems.
"Dependency Hell" may be too strong a word-but most downloadable binary applications only run on a handful of Linux distributions without problem. 90% of the time they require that certain libraries which may be present on the system were compiled with specific options which may or may not present. This is "dependency hell" if someone needs this application. Further I am not talking about USE flags- although I think it is important that the system have a way of documenting which compilation options were used during compilation-and which patches were used. Lastly I am not talking about forcing people to do ./configure, make and make install.
What I am talking about is distributors commiting themselves to providing clean and coherent build environments and to implement some kind of "autobuild" system, which could be some kind of new standard, which would allow for dynamic compilation of libraries which export the exact symbols needed by a binary pacakage and which would not conflict with other already installed libraries.
Certain major applications have provided their own build environments for years-X, Mozilla and OpenOffice to name a few. These build environments are massive undertakings for third-party developers and few rarely compile such large systems on their own machines. But until very recently if one wanted a version of Mozilla which actually used the gtk libraries and the local fonts one had to actually compile the program on their own machine unless and/or until their distributor offered a binary package for that specific release of that particular version of their Linux distribution.
What I am arguing for is that the distributors themselves take on a larger part of the burden for insuring that open source software is trully portable across distribution lines.
Please, for the love of god would people stop saying "everyone drop what your doing and switch to this brand new package manager that I've just invented but haven't implemented and which won't work unless everyone uses it."
Red Hat won't drop RPM, Debian won't drop DEBs and Slack won't drop TGZs. To fix "dependency hell" in an inter-distro fashion, create a machine readable way of describing dependencies that package authors can use to complement their packages and distro authors can use to make their hacks work. Then write some software to use that info - not too hard.
Mostly these problems are solvable on a distro-by-distro basis and it is up to software authors to ensure their target market can use their software. This is something commercial software vendors do well.
There is no reason why people who are scared of make should be compiling from source - it gives very little beneift in terms of speed (I use LFS). I wish software authors would provide FHS compliant, distro-independent binaries but rpms can usually be made to work.
More libraries should be modules and loaded as needed if present not this --with-png --with-gtk. If the application is compiled with 'hooks' to allow for loading module libraries than we can cut down on dependency hell significantly. Though, we might run into module version ABI/API incompatabilities...
but it gets us a little further ahead.
Howdy
Has anyone actually filed a bug report telling the GCC folks that it is not working as expected? .. just a thought.
Static linking if done correctly would aleviate some of the more funky one-off libs that you need and maybe just leave larger ones to be dynamic (KDE and Gnome etc etc).
I have just installed Gentoo 2004.1 and was compiling KDE-3.2.3 the other day and the little dependencies it wanted were incredible!
I have compiled the same KDE using Konstruct on Redhat 8.0 and most deps didn`t exist! (and no they weren`t previously installed) so i`m not sure what was happening there but maybe the Gentoo ebuild specified different params to the make file???
SuSE/Redhat use RPM, and without apt-rpm, dependency hell is almost guaranteed, and apt-rpm still needs to be installed.
Debian uses DEB, and with apt-get and use of only the stable
repositories, it is possible to avoid dependency hell, but the stable branch is way too old. I have hosed more than one machine in short order trying to use the unstable branch, simply by doing an 'apt-get upgrade'.
Gentoo, on the other hand, has in 'portage' taken the BSD ports metaphor and taken it to another level of ease of use and configurability. Gentoo supports both source and binary packages and has been nearly bulletproof regarding package updates in my experience. Most packages are nearly bleeding edge, yet ebuilds currently marked stable are 99.99% of the time exactly that - stable and "just work". It is trivial to update/upgrade/uninstall packages. The only cost is CPU time to compile the source, and it's almost impossible to irreparably break your install just by upgrading, which is something I can't say for RPM or even DEB based distros.
We don't have to re-invent the wheel - we simply need to take a look at what Gentoo has done correctly and learn.
Kudos to Daniel Robbins and the whole Gentoo team!
Check out:
http://www.dragonflybsd.org/goals/packages.cgi
With all the ranting that is going on about gcc makes me think that we should have at least two quality opensource c/c++ compilers. Competition between the two should produce better compilers much like competition between kde and gnome pushed both projects farther.
With all the ranting that is going on about gcc makes me think that we should have at least two quality opensource c/c++ compilers. Competition between the two should produce better compilers much like competition between kde and gnome pushed both projects farther.
Actually, there is a second compiler under development...TenDRA...it's BSD Licensed too for those who swing that way. (Like myself :p)
http://www.tendra.org/
I haven't personally tried it, apparently ANSI C works pretty well, and C++ support is partially there.
It would be interesting to see an article comparing GCC, TenDRA, and Intel's C/C++ Compiler. *hint*
To be on topic...perhaps I'm the exception rather than the rule, but since using APT on Fedora Core, I have had almost no dependency problems. Perhaps you guys just go off the beaten path and work around the packaging system more often than I do? Anyway...with GCC in its current state, static linking every application would be insane.
Just my thoughts.
look..this is a moot point among 'nix users.....If you are a gentoo user you use portage..or compile from source.. if you use slack.. you use .tgz..or compile from source.. If you use debian youuse ports and .debs..if you use redhat.. you use redhat rpms or suffer.. same with fedora and mandrake.. the problem doesnt seem to be with 'nix users.. it seems to be with the press who make this into more of a problem than it really is...
Ive been running the same slack box for over 2 yrs.. and i have software on it that frankly ive forgottne about.. and have never really experienced problems with mainstream applications..such as oOo.org the gimp..mozilla.. or the like. Sure..there will always be some wild little proggie that need sspazlib or some other esoteric library to run.. but its sort of the same on winboxes.. they may need some proprietary or little used .dll to run....Really.. its time to get off this.. when an enterprise decides to go with a particular distro.. they will know which packages to use.. and there will be no prob.. nuff said.
I've heard of and toyed with tendra. Anything that has partial c++ support is hardly production quality. TenDRA has a lot of potential but it isn't there yet. I seem to recall it was hanging in limbo for a while. Has work on the project picked up?
I wish things were statically linked like apps are on Win/BeOS/OSX.
Um, no they're not (well, they might have been on BeOS - I doubt it though - but they certainly aren't on Windows and OS X). Windows and OS X apps certainly *can* be statically linked, but as a rule, they are not.
You are, perhaps, thinking of the way Windows and OS X apps are often *distributed* with any non-standard libraries and store them "locally" in the application directory, but that is *very* different to static linking.
It does not matter how, as long it works. The problem with unix is that if these apps place their libs on /usr/lib or /usr/local/lib they will overwrite existing same libs (of different version), and that's not what you want. So, there are two things to do:
1. Either statically link
2. Don't use /usr/l*
In any case, the core of the matter is that the user should not see dependency problems whatsoever.
When I read most of the first page and the author hadn't yet made his point, I couldn't be bothered to read the rest.
Well, you should have. This is one of the most intelligent proposals I've seen so far about dependency resolution and vendor-neutral software distribution.
Of course, what he proposes his quite a task, but it's definitely doable. The question is, would the various distros get with it, or would they fight it...
Really, you should have RTF[riendly]A. Even though the style may be a bit dry at times, there is truth in those words.
1. the (gnu)linker's working unit is the .o file not C functions. b.o was linked in because it's refered to by a.o 's baz().
2. it's unlikely any other linker would do better with the example (a.c b.c and main.c) above unless that linker can do code-elimination at the function level (which is arguably NOT the linker's job. Plus, in the case of asm code, how do you determine where functions start and end).
3. In the above example, just put one function in one file to get the desired effect. Or make baz() static and gcc-3.4.x will remove it (then the linker won't link in b.o).
4. In the old days, I was taught to put each C function in its own file. But now a day: disks are cheap, ADSL is cheap and OSes do demand-paging. All become a NON-issue.
5. Most ppls on OSNEWS are more qualified for debating GNOME-vs-KDE than saying coherent things about linker/compiler.
I am against dependency-resolution package managers. It is a solution for the wrong problem.
While I personally haven't had any real problems with urpmi, I can definitely understand the arguments for a standardization of software management on Linux.
So, it's clear you don't like dependency-resolution package managers, but what I'd be curious to know is what you think of Karl's proposal. It strikes me as a very interesting approach, as it would - in my view - promote standardization while allowing for a transition for distros that use lots of patched and quirky libraries. It's not as much a revolution as an evolution - which means it also fits the open nature of Linux.
Anyway, I was just curious to heard what you thought of it.
Dude, dependecy problems are _already_ solved.
The author suggest yet another source-based package system.
Dude, RTFA. It's clear from your comments that you didn't.
Dependency problems in Linux are relatively minor, especially for an OS that has such a complex developer ecosystem. They are certainly not crippling problems, and if you stay within the confines of the packages available for your distro, then dependency issues are pretty much non-existant (due to the dependency-resolving package managers Eugenia dislikes).
But start mixing and matching distro rpms with that of other distros, or deb packages, and source packages...dependency problems do occur. What the author suggest is an interesting way to reduce these problems even further.
Newer should always be compatible, but improved.
At some point, compatibility means bloat, stagnation or instability. Look at what MS has had to do to make sure that backward compatibility was maintained, such as coding in dozens exceptions for specific third-party software products. The Windows codebase has become monstrous, I can understand why they consider dropping some old cruft in the next version.
The point is, in order to remain compatible, you have to allow for exceptions for older apps, but at the same time you don't want to penalize people who don't use these apps. As you can see, the situation isn't quite as clear-cut as you indicate.
So, there are two things to do:
There are other options:
A 3rd option that some of us are going towards: Including precompiled versions of any "obscure" or "critical" libraries we use and installing them into the application's directory. Most Linux distributions honour LD_LIBRARY_PATH so an app can simply install libs into it's own directory and use those instead of the ones located in the system.
Bioware does this with neverwinter nights, they have a local library directory that's used for the Miles Sound Design libraries and a precompiled version of SDL that they link against. This eliminates the dependency problem for the user, but bloats the executable, and sometimes the library won't work on the system.
One problem with Eugenia's response about how the developer is responsible for depedencies is that graphics library (especially those like SDL, Allegro, etc.) and audio libraries cannot be compiled on one distribution and be guaranteed to work on other distributions.
For example, the version of SDL included with Bioware's NWN works fine with RedHat, and Fedora but flakes out and has other problems on SuSE 9.1. Additionally, there are licensing considerations and other technical limitations that often prevent the developer from being able to statically link against code or include a precompiled version. So, not even the 3rd option is a complete solution either.
A 4th option that developers have is to limit their official support to one or two Linux distributions (much like Oracle does).
A 5th option is to go through the hellish experience of distributing a different binary for each Linux distribution they want to support.
With a commercial package an autobuild system is not a viable solution.
"Dependency hell occurs when an application requires a multitude of tightly interdependent libraries and those versions expected by the application conflict with previously installed software on the system or when other applications require the same libraries but with different version numbers."
I think the "two" scenario's described are actually the same
A needs xyzlib v1
B needs xyzlib v2
xyzlib v1 and v2 may pose a conflict - meaning one will override/erase the other on installation.
But would this still be a problem if v2 is backward compatible with v1 e.g. if v2 includes v1 unmodified - or if the libraries are named differently and only known as such throughout the system e.g. no symlinking using a common name?
Personally, it has not been a big issue. I've seen conflicts but not many.
I think another thing is that the packagers sometimes require higher versions than what's actually necessary. I think this happens because the dependency requirements are derived from whatever libs happen to be on the system on which the packaging is done. And more often than not these libs are the very latest.
Dude, you are aware there is absolutely no OS called BSD? And that Net/Free/OpenBSD, Darwin, and ect are not the same OS?
Yeah, the ports system is source based, but the package system is not, but works with it.
So what is wrong with something being mutually exclusive? It happens on every OS, including windows. Don't see any reason to whine pointless over that.
BTW the FreeBSD port/package system was NOT ported to gentoo. It is similar, but a entirely different beast. Of which does not have a problem with comercial software installs requiring stuff. It just requires a port for that software to install it and ect. There are several in the FreeBSD ports tree like that and require a disk to install it.
Then you really did not do something right... try reading the docs and ect next time...
I'm glad to see discussion about problems surrounding linux installation and package management because I've been complaining and suggesting change for a long time.
I'm to the point that I've given up pushing Linux and I'm just waiting for something better to come along which deserves my attention and support effort.
I really wish all Linux supporters luck as I believe in the ideals, I just don't like the flavor.
Windows solved the problem by simply installing different software (including libs) to different independent folders. System is looking at program's folder first when looking for dynamic lib (so called "dll"). And that's it. Program can be uninstalled simply by removing program's folder. Now, I know some people there came with all sorts of stupid ideas like OCX (like dlls, but depends on registry), dlls in system folders, using registry etc. But we can ignore it. I think Gobo Linux learned that lesson right.
Statically linking libs is something that should have died 20 years ago. In fact, the only proponents and users of static linking are commercial vendors. It doesn't make sense to use in free and open source software where sharing and reuse are standard and second nature.
While stactically linked packages would make installation easier, I doubt it is a solution to the "supposed dependency hell" people are experiencing. I haven't experienced one of those myself in years.
From