Unix Archive
Maybe its pervasiveness has long obscured its origins. But Unix, the operating system that in one derivative or another powers nearly all smartphones sold worldwide, was born 50 years ago from the failure of an ambitious project that involved titans like Bell Labs, GE, and MIT. Largely the brainchild of a few programmers at Bell Labs, the unlikely story of Unix begins with a meeting on the top floor of an otherwise unremarkable annex at the sprawling Bell Labs complex in Murray Hill, New Jersey. I acknowledge the importance of UNIX – who doesn’t – but I hate how it has become a huge roadblock to any meaningful rethinking and improvement in lower-level operating system design. The best we can do seems to be to hide the ’60s guts underneath ever more layers, instead of addressing the actual shortcomings of such an old design. But hey, I’ve learned over the years that criticizing UNIX is akin to drowning kittens, so maybe I should just fall in line and parrot the party line – UNIX is great, UNIX is perfect, and UNIX needs zero modernisation because it was instantly perfect.
Commodore built this prototype UNIX workstation/server computer in the same time frame as the Amiga and their PC-Clone and then decided that they only had production capacity for two out of three, and the CBM900 lost. All the approx 300-500 prototypes were recalled for destruction, but due to some kind of “mistake” this particular machine, which was on loan to a favored customer in Denmark, never made it back. The machine resurfaced when this company cleaned up their basement, and sent 3 euro-pallets of Commodore artifacts our way. I never knew Commodore tried to build a UNIX workstation. I shouldn’t be surprised though; virtually everyone dabbled in UNIX workstations in the ’80s. This page has more information about the CBM900.
Killing processes in a Unix-like system can be trickier than expected. Last week I was debugging an odd issue related to job stopping on Semaphore. More specifically, an issue related to the killing of a running process in a job. Here are the highlights of what I learned. Interesting technical dive.
This post is about vintage gaming in vintage unusual operating systems, focused on Xenix/x86. Tried Hampa’s turnkey xenix86 images while they had been tested in fake86, 8086tiny and other emulators. The installation was surprisingly easy, because most software packages in floppy/tape images are basically in .tar format, so let’s check GAMES 360k floppy image’s content on host. I can’t get enough of articles like these.
RetroBSD is a port of 2.11BSD Unix intended for embedded systems with fixed memory mapping. The current target is Microchip PIC32 microcontroller with 128 kbytes of RAM and 512 kbytes of Flash. PIC32 processor has MIPS M4K architecture, executable data memory and flexible RAM partitioning between user and kernel modes.
In what's never going to be a regular occurance, I'm linking to a Twitter thread. Chris Espinosa tweets:
Just as I was wrapping up an email and getting ready to leave work, a co-worker rolled his chair over to show me an "interesting" thing.
Go ahead, read it.
UNIX, man. Not even once.
AT&T has a
YouTube channel, where a few times a week they post old videos from the glory days. A few years ago, they posted a cool video from 1982 called
The UNIX System: Making Computers More Productive. It's worth a watch. There's lots of other gems on the channel. For example, how about an
interview with Arthur C Clarke from 1976?
But today's breakthroughs would be nowhere and would not have been possible without what came before them - a fact we sometimes forget. Mainframes led to personal computers, which gave way to laptops, then tablets and smartphones, and now the Internet of Things. Today much of the interoperability we enjoy between our devices and systems - whether at home, the office or across the globe - owes itself to efforts in the 1980s and 1990s to make an interoperable operating system (OS) that could be used across diverse computing environments - the UNIX operating system.
As part of the standardization efforts undertaken by IEEE, it developed a small set of application programming interfaces (APIs). This effort was known as POSIX, or Portable Operation System Interface. Published in 1988, the POSIX.1 standard was the first attempt outside the work at AT&T and BSD (the UNIX derivative developed at the University of California at Berkeley) to create common APIs for UNIX systems. In parallel, X/Open (an industry consortium consisting at that time of over twenty UNIX suppliers) began developing a set of standards aligned with POSIX that consisted of a superset of the POSIX APIs. The X/Open standard was known as the X/Open Portability Guide and had an emphasis on usability. ISO also got involved in the efforts, by taking the POSIX standard and internationalizing it.
A short look at the history of UNIX standardisation and POSIX.
It has often been told how the Bell Labs law department became the first non-research department to use Unix, displacing a newly acquired stand-alone word-processing system that fell short of the department's hopes because it couldn't number the lines on patent applications, as USPTO required. When Joe Ossanna heard of this, he told them about roff and promised to give it line-numbering capability the next day. They tried it and were hooked. Patent secretaries became remote members of the fellowship of the Unix lab. In due time the law department got its own machine.
Fascinating bit of Unix history.
Programs written to run on conventional operating systems typically depend on OS abstractions like processes, pipes, signals, sockets, and a shared file system. Compiling programs into JavaScript, asm.js, or WebAssembly with tools like Emscripten or GopherJS isn't enough to successfully run many programs client-side, as browsers present a non-traditional runtime environment that lacks OS functionality. Porting these applications to the web currently requires extensive rewriting or paying to host significant portions of code in the cloud.
Browsix is our answer to these challenges.
Neat.
The history and evolution of the Unix operating system is made available as a revision management repository, covering the period from its inception in 1970 as a 2.5 thousand line kernel and 26 commands, to 2016 as a widely-used 27 million line system. The 1.1GB repository contains about half a million commits and more than two thousand merges. The repository employs Git system for its storage and is hosted on GitHub. It has been created by synthesizing with custom software 24 snapshots of systems developed at Bell Labs, the University of California at Berkeley, and the 386BSD team, two legacy repositories, and the modern repository of the open source FreeBSD system. In total, about one thousand individual contributors are identified, the early ones through primary research. The data set can be used for empirical research in software engineering, information systems, and software archaeology.
The project aims to put in the repository as much metadata as possible, allowing the automated analysis of Unix history.
In a sane world, Unix vendors would have either replaced their version of more with the clearly superior less or at least updated their version of more to the 4.3 BSD version. Maybe less wouldn't have replaced more immediately, but certainly over say the next five years, when it kept on being better and most people kept preferring it when they had a choice. This would have been Unix evolving to pick a better alternative. In this world, basically neither happened. Unix fossilized around more; no one was willing to outright replace more and even updating it to the 4.3 BSD version was a slow thing (which of course drove more and more people to less). Eventually the Single Unix Specification came along and standardized more with more features than it originally had but still with a subset of less's features (which had kept growing).
This entire history has led to a series of vaguely absurd outcomes on various modern Unixes.
The latest problem I was working out was how to run Unix on the Atari ST. The Tramiels had somehow wrangled a license for AT&T's SVR-something-or-other version of Unix (might have been SVR3, but this was in the bad old days when AT&T was actively fucking up Unix, and it could have been just about any version, including SVR666). The license was for a mind boggling, nay, jaw-dropping ten bucks a seat. The problem was that the ST didn’t have any kind of memory management hardware, just a raw CPU flinging real addresses at naked DRAM, and the machine's cheap-ass vanilla 68000 was incapable of recovering from a fault unless you cheated.
On a related note, there's MiNT.
The subject of process management, supervision and init(8) for Unix-like systems is one plagued by a large degree of ahistoricity and "pop culture" explanations. This leads to a lot of confusion and misunderstanding surrounding feature sets and how one formulates the problems surrounding reliable process management on Unix in general, making it a ripe topic for demagogues of all persuasions.
The purpose of this article is to set the record straight on the history of attempts to create "modern" init systems, where we define "modern" somewhat broadly as anything that tries to improve the classical BSD and System V styles of initialization and service management.
We missed this earlier this year, but Coherent has been released as open source. Coherent is a UNIX clone originally developed for the PDP-11, but later ported to a number of other platforms, including the IBM PC. It was developed by the Mark Williams Company, and despite an official investigation by AT&T, no signs of copied code were ever found.
Mark Williams Company closed in 1995. In 2001, Bob Swartz asked me to archive the hard disks containing the Mark Williams source repository; the command and system sources here are from that repository. I have long intended to catalog and organize these sources, but in the meantime they are posted here as is. MWC's documentation guru Fred Butzen provided the MWC documentation sources.
An old (2010) story from former Sun employee Jeremy Allison.
David Miller wrote (at the end of a long email explaining how Sparc Linux used cache optimizations to beat Solaris on performance):
"One final note. When you have to deal with SunSOFT to report a bug, how "important" do you have (ie. Fortune 500?) to be and how big of a customer do you have to be (multi million dollar purchases?) to get direct access to Sun's Engineers at Sun Quentin? With Linux, all you have to do is send me or one of the other SparcLinux hackers an email and we will attend to your bug in due time. We have too much pride in our system to ignore you and not fix the bug."
To which Bryan Cantrill replied with this amazing retort:
"Have you ever kissed a girl?"
Talk about missing the point and underestimating the competition.
The article offers an interesting look at why Sun eventually failed. I stumbled upon this story because Rob Landley mentioned the girl comment in his email about BSD/SysV.
Rob Landley:
So gcc's library bindings strongly preferring System V system calls to BSD was due to the flood of Solaris end-users dominating the late 80's gcc development community to avoid paying Ed Zander extra for Sun's Solaris compiler. This combined with Linus reading Sun workstation manuals to get a system call list gave linux a very System V flavor.
Interesting - if opinionated - view on the whole situation. Not sure if all the dates check out, but it's a fun read nonetheless.
grep is a Unix command line utility (well most Unix utilities are command line) that searches the input files for pattern and prints lines that contain the pattern. If you are reading this you, you are probably no stranger to grep.
grep was written by Ken Thompson, the same guy who wrote Unix. grep first appeared in Unix v4 with limited features as compared to today's grep.
I've used grep so much over the years. One of the countless little utilities that's the staple of all UNIX-like systems that you never really think about, but use all the time.
Modern microcontrollers are becoming quite beefy. The Microchip PIC32 line is actually an implementation of the MIPS32 4K architecture - and with 512K of flash and 128K of RAM you can even run Unix! RetroBSD is a port of BSD 2.11 for the PIC32. You might not be able to run X11, but it is still very useful and a great reminder of how small Unix used to be - and how far it has come.
"Few tools are more indispensable to my work than Unix. Manipulating data into different formats, performing transformations, and conducting exploratory data analysis (EDA) is the lingua franca of data science.1 The coffers of Unix hold many simple tools, which by themselves are powerful, but when chained together facilitate complex data manipulations. Unix's use of functional composition eliminates much of the tedious boilerplate of I/0 and text parsing found in scripting languages. This design creates a simple and succinct interface for manipulating data and a foundation upon which custom tools can be built. Although languages like R and Python are invaluable for data analysis, I find Unix to be superior in many scenarios for quick and simple data cleaning, idea prototyping, and understanding data. This post is about
how I use Unix for EDA."
Guest post by 
Guest post by 
Submitted by 
Guest post by