From the debian-devel mailing list:
debootstrap in unstable can now install with merged-/usr, that is with
/bin, /sbin, /lib* being symlinks to their counterpart in /usr.
LWN.net published an article in January 2016 going into this then-proposed change.
Debian is the latest Linux distribution to consider moving away from the use of separate /bin, /sbin, and /lib directories for certain binaries. The original impetus for requiring these directories was due to space limitations in the first Unix implementations, developers favoring the change point out. But today, many of the services on a modern Linux system impose requirements of their own on the partition scheme – requirements that make life far simpler if /bin, /sbin, and /lib can be symbolic links to subdirectories within a unified /usr directory. Although some resistance was raised to the change, the project now seems to be on track to make “merged /usr” installations a supported
option. And perhaps more importantly, the arguments favoring the merge suggest that many Debian developers would like to see that configuration eventually become the default.
Any steps to clean up Linux’ FHS implementation – no matter how small – is cause for widespread celebration all across the land. Call it forth!
Never understood the drive behind “fixing” the FHS. Systems outside Unix land never had any rules for where to put files. My Atari and Amiga filesystems are a mess. macOS seems to have multiple layers of where crap is intalled, when they could have just left it alone.
There still are use cases for smaller / split up directories, mainly in the embedded world. Realistically, if your path is set up with sane defaults, you shouldn’t have to care where the binaries are, and generally you don’t care where the libraries are. Most will only care if their crap works, and if they can find the important things, like Documents, Pictures, etc. And most desktop environments set those up on ~\ by default.
I agree – I really don’t think the FHS is that complex or difficult to understand. The problem comes when it is varied depending on distribution, but even there the differences aren’t huge in my mind.
has a nice an clean structure.
And with the help of the famous “assign” command and the use of logical volumes it gives you also a great deal of flexibility.
AmigaOS has (IMO) a very clean structure. LIBS: DEVS: C: S: L: HELP: FONTS: and so on, they could be on different partitions/volumes but the default was directorys under SYS: (the volume you booted from)
And most programs don’t really mess with the system-directories except for perhaps copying a library to LIBS:
Ah yes, the Amiga really is clean, until you start putting your own things on there. Granted I guess this is handled by Linux distributions via the package manager, whereas even Windows has the nasty habit of not sorting through things in any sane standardized way.
One thing I could never figure out for the Amiga is what the difference between a Tool and a Utility. Seems to me they are rather interchangeable, and so when I’d install something to either one, I end up having to check both when I look for the program again.
Of course the Atari ST line pretty much had an AUTO folder, and later versions of TOS supported .ACC files and CPX folder for their control panel modules. But it was the wild west otherwise.
Agree with the ‘assign’ command on AmigaOS, it’s very useful, I also really like that you don’t really have drive letters there, much like Unix.
I have mixed feelings about the merge of /bin and other top level directories into /usr. On most Linux distributions it probably makes sense because on most Linux distributions, the executables in the top level directories are just more dynamically linked binaries. They could be just about anywhere that is commonly mounted early on and it would be fine. Putting them in /usr tidies up the file system and I like that.
My issue with this situation is the files in /bin and /sbin should not be dynamically linked. I’ve been on Linux systems where something broke and any dynamically linked files (like /bin/cp or /bin/mv) stopped working. On other UNIX-like systems programs in top level directories are generally statically linked and won’t break due to a changing dependency or corrupted C library. This allows the admin to fix things. But on Linux, the admin needs to take the system off-line and fire up a rescue disc.
I’ve been through this sort of thing a few times and it bothers me most Linux distros don’t bother using static executables. By comparison, deciding whether to put them in /bin or /usr/bin seems like a minor issue.
“Static only binaries in /bin & /sbin” hasn’t been true for some time. It’s also true that, largely, we no longer care so much if a machine eats itself badly enough that dynamically linked executables no longer work; it’s normally (I hope, for your sake) quicker & easier to blow away the instance/re-install the machine than to spend your time trying to nurse it back to health.
Spoken like a true Windows admin.
Spoken by someone who’s been a Linux sysadmin for the past 20 years. I have thousands of machines to look after, and everything is n+1; I literally couldn’t give a single crap about any individual one of them. And nor should I; re-installing takes a few minutes (you have a provisioning system, right?), dicking about with a horked filesystem and/or shared libraries takes hours and may never actually work.
So yeah, if you want to waste your time & your employers money, go right ahead.
Static binaries *on Linux* has not been the case for a long time. At least gong back to around 2000, which is when the bug (and I consider dynamically linked programs in /bin a bug) first affected me. However, some other systems have maintained the tradition of statically linked programs in the top level directories and it’s a welcome feature. I wish more systems still shipped this way. I’d much rather re-install one broken library than re-install an entire operating system.
Agreed. If a *NIX-based system other than GNU/Linux gets hosed up badly enough to where a reinstall is needed, then you’ve done something seriously wrong. If you know admin basics you can, for instance, keep a Solaris or *BSD system running and ready for years and never even have to think the word “reinstall.” Not all Linux-based systems are so easily hosed, and you can do the same with RHEL/CentOS or even Debian stable (stress on the stable part). However there’s no denying that stuff in Linuxland has gotten far more complicated than other *NIX variants even so.
Sadly after many years of using Debian, this is the first time I’m going to have to restructure/reinstall one of my systems… but it’s not Debian’s fault really, one of the drives in the JBOD/LVM exhibited some random read errors, even though SMART says it’s completely fine, and it caused a nasty loop of fsck checks that then completely hosed he file structure.
So some things like /var/lib/dpkg is missing…
Fortunately most of my actual files are still there, I just need to put them in the right place after copying them off. But I wasn’t going to deal with this crap a second time, so ended up ordering a proper raid controller!
Right, but this is an article & discussion about Linux, so thought the context was pretty clear.
Okay, maybe there really is somebody out there using Debian/Hurd who cares about the distinction, but they can cope.
Some people won’t be happy until everything gets stashed in a giant /System folder ala osx i guess …
Personally, i think FHS itself is quite well structured, just almost always poorly implemented/adhered to.
I don’t want to see it go.
I’d just like the current mess of people doing whatever the f*ck they want to go away.
I second that. People will stop using the symlinks as a performance bump and then we’ll start seeing scripts hard coded for /usr/bin/sh and other crazy non portable things.
This is going to make portability of scripts even worse than it is now across Linux & BSD systems. Even Linux to Linux will be hard now. (not that it’s great as it is)
I use which or env
Aside from that, hope that $PATH is set correctly
Edited 2016-11-23 15:26 UTC
/sbin, /bin, /lib are binary files, right? Seems to me that putting them in a /system folder would make a lot more sense than /usr. To me, ‘usr’ signifies ‘user’, which is where I’d guess all a user’s personal files go. IRIC, I think the user’s files go in /home, but that’s not the point …
/lib contains shared objects. Shouldn’t contain executable binaries.
/usr does in effect contain things belonging to the user. It contains binaries and objects that are non-critical tot the system, and support the applications/platform that the user(s) run on it.
/home does indeed contain the personal files, profile, etc of the system’s users.
Not anymore. Did you miss the point of this article? They want /usr to be everything except profiles and config data. Next we’ll hear about /usr/var, /usr/etc. I’d say we’d have /usr/home, but I do believe it was BSD/OS back in the days that used that.
Ah well, at least it’s not Solaris. Remember all those /usr subdirectories like /usr/sfw/bin, /usr/xpg4/bin, /usr/contrib/bin…
Personally, I think OpenBSD has the most logic. /bin, /lib, etc contain system-critical files. /usr contains non-critical system binaries (though I still think usr is a misname). /usr/local contains everything that the user has installed, either via packages or compilation themselves (makes it really, really simple to separate the system from the added software).
And don’t even get me started on /opt.
They want this, but it’s not the case now, so for now, it’s still as described.
Solaris is a special kind of special lol
Inhad the same misconception about usr.
It doesnt represent “user” but “Unix System Resources”. (Maybe it needs a name change eventually.)
the Users folder equivalents in linux are “home/*”
Having all major distributions with a merged usr has benefits in documentation where documentation from one will more likely work on another distribution.
it also allows for simpler “resets” of a system where user configuration can be removed deleting everything but usr.
LOL, well what more should I expect from an OS that puts all its important configuration files in a directory named ‘etc’, which doesn’t actually mean ‘et cetera’. I mean, do they TRY to make it a pain in the ass to grok? Do they sit around and have meetings, where they say, ‘What else could we possibly do to make the operating system as unapproachable as possible to newcomers?’
And why should newcomers know about the things stored in /etc ? Do newcomers of MS Windows play around the registry ? These things are important only to people that want or need to deal with OS roots, most should not, the ones that really must will learn about those messy things and, BTW, I prefer a lot to play around /etc but, frankly, it is every day more uncommon to do so.
Edited 2016-11-24 12:38 UTC
Do you know the history of UNIX? Shorthand commands and directories were created because teletypes were slow. I prefer the short names. It makes seeing full paths and typing commands much easier. Spelling everything out is great for readability but horrible for usability. Have you used Powershell? It’s very frustrating.
That still doesn’t excuse putting critical system files in a directory named ‘etc’. Yes, I know what it stands for, but the point is that it’s NOT intuitive. As in, if I had just installed Linux and was poking around, that is the LAST directory I would look in for such files.
The problem with that is USR stands for Universal System Resources.
Edited 2016-11-26 22:39 UTC
There’s pointless stupidity in Linux FHS, which could be simplified out. But, it’s fairly easy to make whatever adjustments you want. I think this is less of an issue than some people want it to be and that’s coming from someone who prefers things to be tidy rather than spread all over the place in a big mess, and isn’t impressed with FHS.
For historical reasons it would be correct to remove usr altogether and make sure there would only be /bin, /sbin, /lib.
OsNews have covered this issue before back in 2012:
http://www.osnews.com/story/25556/Understanding_the_bin_sbin_usr_bi…
And in here it is pointed out that /usr came to be because second disc had to be mounted somewhere and it was full of user stuff (hence the usr). Then at some point the OS grew so big, that it couldn’t accomodate all the binaries in /bin and stuff was moved over into /usr. The third disc was then mounted to /home and user stuff moved away from /usr.
So now that everything can fit into /bin then so it should be. The /sbin would be perfect place to put static binaries.
However what about /usr/local? It should live in /opt and so should all the extra binaries and stuff that does not fit into /bin
I agree.
For a Linux system, it would make more sense to remove /usr completely and merge things back to the / filesystem.
/etc for configuration data.
/bin for dynamically-linked binaries
/sbin for statically-linked binaries (like it was originally intended)
/lib for libraries
/var for logs, caches, etc.
/opt for third-party software
/home for all users’ data
/local can be created to replace /usr/local for locally compiled and installed software.
For non-Linux systems, I like the FreeBSD layout:
/ everything needed to boot the system to single-user mode
/usr everything needed to use the system after boot
/usr/local software installed via the official packaging system
Nice, neat, simple, and well-separated by function.
Edited 2016-11-23 20:23 UTC
Linux distros messed up the FHS by not paying attention to what must go where.
That is one reason I moved years ago to FreeBSD. Everything has its place in the file system hierarchy under FreeBSD. Base OS goes to one place, user installed apps go into another – things *never* get mixed. Even the ‘etc’ and ‘rc.d’ directories are separated between base system and user installed. I always know where to find anything. Long live FreeBSD!
If you’re ditching the separation of “/” and “/usr” why not just ditcch “/usr” too? I mean, in that scheme what purpose does “/usr” actually serve?
Soulbender,
I agree. From a user point of view it shouldn’t matter if software is from a distro’s repo, another repo, github or my own. Why should software organization change based on how software was installed or even whether it’s a static binary?
It just seems unnecessary and arbitrary to me. Just because the binaries were compiled differently or by someone else doesn’t mean we need to segregate them in our hierarchy. Oh these binaries have debug symbols, lets put them in /usr/dbg/bin. These ones are optimized for core2, they go /usr/core2/bin. It’s just seems pointless to me.
IMHO the only reason to separate binaries is when there is a technical need for them to be mounted on separate volumes for bootstrapping reasons. However modern distros don’t do this.
I should note that in my own linux distro, I tried to create as simple a hierarchy as it made sense for me to do. It’s funny because this was before I heard of gobolinux, yet our efforts turned out to be fairly similar. Even though I feel there are better alternatives to FHS, there are definite perks to siding with the majority, and sadly I reverted my hierarchy because changing all the packages was creating too much of a maintenance burden.
Well, I understand that most cases are not like mine but I do like separation for some reasons:
– To not interfere with what comes from main repositories (for some stuff, it is not uncommon that I install newer/development versions, generally on /opt or /usr/local);
– To “avoid” names clash;
– To restrict access with easy, instead of just relying on ACLs (actually, in most cases I use both);
Anyway, from my point of view, if they want to ditch things, /usr should be the logical (?) thing to go. Why opt to the easy kill instead of what seems the more appropriate ? In the long run, what seems hard usually is what gives the best rewards.
acobar,
To be honest, I don’t really care that much about the specific names of directories. My interest is for it to be cleaned up, consistent, and not have a design bias for a centralized repo.
Edited 2016-11-26 18:06 UTC
For the, usually big, applications that need to have more flexibility there are two solutions currently: Ubuntu snap and Red Hat flatpak. I think they solve most of the problems you have and throw in sandboxing as a bonus.
For the other things there is nothing best than a central repo on my humble opinion: you can upgrade to a new version of your distribution in one go with a few commands.
You would if you had to support different users with different versions of applications with different glitches for which you collect fix recipes.
acrobar,
I’ve done this, and I used to be a fan of it. But after years of being a sysadmin I’ve grown to hating it because the stable repo can often be outdated and testing can sometimes have regressions, meaning neither stable nor testing are appropriate for the whole system. This wouldn’t be a problem if the repos weren’t mutually exclusive, but they aren’t made to work together and each repo will fight over the dependencies for the whole system.
I’ve been stuck in a scenario where I absolutely needed a new version, yet the only way to do it (using the official managed repos) was to upgrade the entire system to the testing repo. At least that’s easy enough to do. Once I thought everything worked, but other clients using the shared server started calling about problems with unrelated services, and sure enough there were regressions as a result of the testing repo.
Now just to be clear, I take responsibility for switching repos and things not working. However the distro’s centralized repos unequivocally deserve 100% of the blame for not being able to cope with selecting a single package from the new repo. It has bitten me several times over the years and I’ve grown very wary of the exclusive choice between updating the whole system at once to a non-stable repo, versus having to manually compile & manage software for the stable repo.
Sorry about the rant, I know some people can get by entirely within one repo and for them it’s easy, but it can cause lots of stress when we have to stray from the central repo. Just because many linux distros have a central repo doesn’t mean we should ignore the problems associated with installing software from other sources. Windows, for all it’s other problems, doesn’t have this problem at all.
Edited 2016-11-26 22:12 UTC
I have seen this happen only when needing a new version of some program with a hard dependency on new features of newer versions of libc and/or glib, most of other things are not so dramatic.
For many years I kept updating a huge bash script that I assembled to easy the pain of having to support a combination of Slackware, Debian, FreeBSD, CentOS and openSUSE. It was used to assembly the native packages (using rpmbuild, dpkg, pkg*, etc). I gave it the name of PASM – Package ASsembly Mule. It is ugly as hell and I don´t care about it anymore (almost, it makes my life easier when needed, even if it is seldom used). I also lowered my attention deficit disorder duo to too much interest span by sticking only to Red Hat/CentOS, openSUSE and Debian.
The last thing to strike me happened with Zeal, a document browser counterpart of Kapeli´s Dash. When trying to browse Qt5 Documentation (and one another I don´t remember) Zeal would segfault. Analyzing the code gave me no insights of what was wrong. Firing GDB gave me hints, even though it is a Qt5 application (confirmed by ldd), for some strange reason on openSUSE 13.1 the loader/dynamic linker would link it against libQtGui instead of libQt5Gui dynamically, no matter how I tried to enforce only Qt5 libs by ld options, so I may understand your pain. My last resort was a script that renamed /usr/lib64/libQtGui.so.4 to something else, fire Zeal (by nohup) and then rename libQtGui back. Ugh! (but works !)
Corner cases on open software are really a hell paradise.
Anyway, for all its faults, I still prefer a centralized repository for most things. The other cases are appropriately handled now with Flatpak and Snap in my opinion.
acobar,
I’m not quite suggesting we get rid of all centralized repos, only that the package management should support installation of packages from multiple repos without conflict. Different repos should have the option of being in different namespaces such that updates in one don’t force undesirable updates and removals in another repo, which can be infuriatingly wrong.
These namespaces would solve my repo problems without causing new problems for existing repos.
Edited 2016-11-27 05:51 UTC
Even though I agree with you on general lines, I don’t think that it is easily achievable, perhaps, it is even harder than the current proposed methods. Do you have something written down, like a draft specification? Would be a nice reading.
Edited 2016-11-27 18:00 UTC
acobar,
I would link it for you if I had one!
If I had the opportunity to redesign the repos myself, I’d start with the most trivial solution that makes no attempt to eliminate the duplicate dependencies between repos.
1. Begin with a normal install from a stable repo into the default namespace.
2. Add a new repo into a new namespace.
3. Installing a new package (like nginx) from the new repo would also install it’s dependencies from the new repo, even if they are duplicated in the first repo.
4. When running software from the new repo, the linux loader would automatically satisfy dependencies with dependencies from the new repo as well (it might be as simple as changing the order of the library/binary paths in the LD_* environment variables).
I’m not claiming this is ideal, but the important thing is that it works without too much effort up front. And it opens up a path forward for the harder task of optimizing to eliminate cross-repo library duplication. For example, a perl dependency in the new repo would result in a second perl interpreter being installed for packages from the new repo.
Here’s a quasi-fictitious simplified example for illustration…
(I’m sticking with the symlink approach from my easier example because, as I was trying to argue earlier, it allows arbitrary repos to co-exist without overstepping each other in FHS designated directories.)
Assume repo_1 = stable base system
Assume repo_2 = testing/devel
So, assume due to functional requirements, we need “lm-sensors” from the testing repo. However lm-sensors has a perl dependency. So, the repo is clueless about whether repo_2’s lm-sensors is compatible with repo_1’s perl, so it could just install a second perl interpreter from repo_2.
/bin/perl -> /pkg/repo_1/bin/perl
/bin/sensors-detect -> /pkg/repo_2/bin/sensors-detect
/bin/sensors -> /pkg/repo_2/bin/sensors
/pkg/repo_1/bin/perl
/pkg/repo_2/bin/perl
/pkg/repo_2/bin/sensors-detect
/pkg/repo_2/bin/sensors
When I login and run perl, I see the original perl from the stable repo (because I didn’t ask the package manager to update perl for my system). I run sensors-detect, it runs the version I explicitly chose from the testing repo, and because it has no knowledge about stable compatibility, it also runs the perl interpreter from the testing repo. It might be unnecessary to have another perl interpreter, but at least the repo explicitly knows that version works with sensors-detect.
I use this example because it is not optimal and we can see that the perl interpreter is probably extra baggage, but at least it works and avoids the update cascade situation that makes multiple repos incompatible today.
Now it is very hard to solve this optimization this without sophisticated heuristics, more metadata, or manual intervention. For a start, I’d say manual intervention is probably fine: “I see you’re trying to pull in a new perl from repo_2, but I’ll veto that, just default to the perl that’s already installed”. If something like this was implemented by package managers, it would solve all my multi-repo problems.
This approach extends to arbitrarily many repos (however many are reasonably warranted
). If a legacy package was dropped from stable, and I’m not ready replace it, then I could create a new namespace for repo_3 and use the same technique to get it working along side my current software.
Back to the topic of FHS, I hope I’ve explained enough of my thought process that you can understand why I don’t like the idea of one special central repo in charge of system directories. If people happen to only use one repo, then that’s fine, but as far as package management is concerned, it’s just an arbitrary repo that happens to be installed by itself. It shouldn’t be considered a special case from any other repos that might need to be installed.
Since all repos would be treated the same way, this might even be a reasonable upgrade strategy to slowly migrate to new repos with less risk than a giant system-wide update.
Edited 2016-11-27 20:02 UTC
Some people might enjoy an article written on osnews about the same topic from 2012:
Understanding the /bin, /sbin, /usr/bin, /usr/sbin Split
http://www.osnews.com/story/25556
I don’t get how the FHS is a problem. When has the current directory structure caused problems? How is throwing everything into one directory solving anything? It’s usually just people who recently switched to Linux who complain about this because they are not used to it. It amazes me that so many people complain about changes like systemd, which actually fix known issues, but insist on fixing something that isn’t broken.
Long story shorted: many programs on /sbin and /bin have a dependency on things on /usr/*bin or /usr/lib* and if your /usr resides on a different partition and it cannot be mounted on boot you would not be able to start recovery.
See:
“https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMe…
“https://wiki.gentoo.org/wiki//usr_move“
“https://fedoraproject.org/wiki/Features/UsrMove“
And, finally, if things are already merged, why pollute the root with not really needed /bin and /sbin ?