Finally something really interesting to talk about. If you’ve used UNIX or any of its derivatives, you’ve probably wondered why there’s /bin, /sbin, /usr/bin, /usr/sbin in the file system. You may even have a rationalisation for the existence of each and every one of these directories. The thing is, though – all these rationalisations were thought up after these directories were created. As it turns out, the real reasoning is pretty damn straightforward.
I’ve never made a secret of the fact that I absolutely detest the UNIX directory structure. The names are non-descriptive and often entirely arbitrary, they require a book to properly understand, and everyone seems to have their own ideas about what goes where. And heck, does it show – even among Linux distributions there’s no consistency about what goes where.
It’s a total and utter nightmare that even defiled my beloved BeOS.
Solutions so far tend to be just layers upon layers upon layers to hide the mess of the directory structure. Mac OS X is especially egregious in this regard – open a terminal and check the directory listing at root – it’s an even bigger mess than regular UNIX. Load up a finder window, and you’re looking at an entirely different directory structure.
This is like buying a beautiful car, only to realise the engine is made of cake and rotting brocolli.
Whenever I complained about the UNIX directory structure in the past, I (and those who agreed with me) were always pummelled with explanations on why it makes sense, why it’s best to spread stuff across /bin, /sbin, /usr/bin, /usr/sbin, and so on. The funny thing always was – these explanations were never particularly consistent between each other.
Late last week, I came across a link at HackerNews giving some intriguing insight into how /bin, /sbin, /usr/bin, and /usr/sbin came to be. Many of you will be surprised to learn that no, there is no divine plan behind all this separation.
Back on November 30 2010, David Collier wondered on the BusyBox mailing list why “kill is in /bin and killall in /usr/bin”. He “[doesn’t] have a grip on what might be the logic for that”. Rob Landley replied, and offered an interesting insight into all this.
The issue was that when Ken Thompson and Dennis Ritchie upgraded from a PDP-7 (on which they created UNIX in 1969) to a PDP-11 in 1971, they were confronted with not one 1.5MB hard drive, but two. Amazingly, they now had an insane amount of megabytes (3 of ‘m). The first disk contained the operating system, while the second one contained all the user stuff. This second disk, with all the user stuff, was mounted at /usr (/home was invented later).
Data Center image via Shutterstock
At some point, the operating system grew too big for the first disk, and had to spill over to the second disk. As a result, Thompson and Ritchie replicated the system directory structure (/bin, /sbin, /lib, /tmp, and so on) on this second disk under /usr. When they got a third disk, they moved all the user stuff from /usr to the third disk, mounted under /home.
This forced them to come up with a number of rules, such as that a command like mount couldn’t be installed in /usr/bin, since mount was needed to mount the second disk (/usr) in the first place.
“The /bin vs /usr/bin split (and all the others) is an artifact of this, a 1970’s implementation detail that got carried forward for decades by bureaucrats who never question _why_ they’re doing things,” Landley explains, “It stopped making any sense before Linux was ever invented, for multiple reasons.”
He then continues to details these reasons. First, Linux already has a temporary system that takes care of the ‘this file is needed before that one’-problem. Second, shared libraries solved the issues caused by static linking (which was the norm back when UNIX was created). Third, hard drive space hit the 100MB mark back in 1990, so small disks are no longer an issue.
“Standards bureaucracies like the Linux Foundation (which consumed the Free Standards Group in its ever-growing accretion disk years ago) happily document and add to this sort of complexity without ever trying to understand why it was there in the first place,” he adds, “‘Ken and Dennis leaked their OS into the equivalent of home because an RK05 disk pack on the PDP-11 was too small’ goes whoosh over their heads.”
In a way, this feels like vindication. All those silly rationalisations people are bandying about – they’re all made up after the fact, for reasons that haven’t made sense in at least 30 years – and heck, that never made any sense in the Linux world at all.
Arguing that the UNIX directory structure is a horrible, horrible mess that defiles an otherwise elegant system is like trying to convince a floor tile to flip over. People are so used to their knee-jerk responses about how it all supposedly makes sense, they often refuse to even think about redesigning it for the modern age. Since the geek is a proud and stubborn creature, there’s little to no chance of this ever changing in my lifetime.
“I’m pretty sure the busybox install just puts binaries wherever other versions of those binaries have historically gone. There’s no actual REASON for any of it anymore. Personally, I symlink /bin /sbin and /lib to their /usr equivalents on systems I put together,” Landley, currently working on embedded Linux, concludes, “Embedded guys try to understand and simplify…”
I’m with the embedded guys – I really like to understand things like this!
This also vindicates my first thought when I started using *nix – that “/usr” looks and sounds like “user”. That name never made sense to me.