General Unix Archive

Vintage gaming on Xenix

This post is about vintage gaming in vintage unusual operating systems, focused on Xenix/x86. Tried Hampa’s turnkey xenix86 images while they had been tested in fake86, 8086tiny and other emulators. The installation was surprisingly easy, because most software packages in floppy/tape images are basically in .tar format, so let’s check GAMES 360k floppy image’s content on host. I can’t get enough of articles like these.

The enviable pedigree of UNIX and POSIX

But today's breakthroughs would be nowhere and would not have been possible without what came before them - a fact we sometimes forget. Mainframes led to personal computers, which gave way to laptops, then tablets and smartphones, and now the Internet of Things. Today much of the interoperability we enjoy between our devices and systems - whether at home, the office or across the globe - owes itself to efforts in the 1980s and 1990s to make an interoperable operating system (OS) that could be used across diverse computing environments - the UNIX operating system.

As part of the standardization efforts undertaken by IEEE, it developed a small set of application programming interfaces (APIs). This effort was known as POSIX, or Portable Operation System Interface. Published in 1988, the POSIX.1 standard was the first attempt outside the work at AT&T and BSD (the UNIX derivative developed at the University of California at Berkeley) to create common APIs for UNIX systems. In parallel, X/Open (an industry consortium consisting at that time of over twenty UNIX suppliers) began developing a set of standards aligned with POSIX that consisted of a superset of the POSIX APIs. The X/Open standard was known as the X/Open Portability Guide and had an emphasis on usability. ISO also got involved in the efforts, by taking the POSIX standard and internationalizing it.

A short look at the history of UNIX standardisation and POSIX.

How Unix made it to the top

It has often been told how the Bell Labs law department became the first non-research department to use Unix, displacing a newly acquired stand-alone word-processing system that fell short of the department's hopes because it couldn't number the lines on patent applications, as USPTO required. When Joe Ossanna heard of this, he told them about roff and promised to give it line-numbering capability the next day. They tried it and were hooked. Patent secretaries became remote members of the fellowship of the Unix lab. In due time the law department got its own machine.

Fascinating bit of Unix history.

Unix in your browser tab

Programs written to run on conventional operating systems typically depend on OS abstractions like processes, pipes, signals, sockets, and a shared file system. Compiling programs into JavaScript, asm.js, or WebAssembly with tools like Emscripten or GopherJS isn't enough to successfully run many programs client-side, as browsers present a non-traditional runtime environment that lacks OS functionality. Porting these applications to the web currently requires extensive rewriting or paying to host significant portions of code in the cloud.

Browsix is our answer to these challenges.

Neat.

Unix history repository, now on GitHub

The history and evolution of the Unix operating system is made available as a revision management repository, covering the period from its inception in 1970 as a 2.5 thousand line kernel and 26 commands, to 2016 as a widely-used 27 million line system. The 1.1GB repository contains about half a million commits and more than two thousand merges. The repository employs Git system for its storage and is hosted on GitHub. It has been created by synthesizing with custom software 24 snapshots of systems developed at Bell Labs, the University of California at Berkeley, and the 386BSD team, two legacy repositories, and the modern repository of the open source FreeBSD system. In total, about one thousand individual contributors are identified, the early ones through primary research. The data set can be used for empirical research in software engineering, information systems, and software archaeology.

The project aims to put in the repository as much metadata as possible, allowing the automated analysis of Unix history.

more, less, and a story of typical Unix fossilization

In a sane world, Unix vendors would have either replaced their version of more with the clearly superior less or at least updated their version of more to the 4.3 BSD version. Maybe less wouldn't have replaced more immediately, but certainly over say the next five years, when it kept on being better and most people kept preferring it when they had a choice. This would have been Unix evolving to pick a better alternative. In this world, basically neither happened. Unix fossilized around more; no one was willing to outright replace more and even updating it to the 4.3 BSD version was a slow thing (which of course drove more and more people to less). Eventually the Single Unix Specification came along and standardized more with more features than it originally had but still with a subset of less's features (which had kept growing).

This entire history has led to a series of vaguely absurd outcomes on various modern Unixes.

How the Atari ST almost had Real Unix

The latest problem I was working out was how to run Unix on the Atari ST. The Tramiels had somehow wrangled a license for AT&T's SVR-something-or-other version of Unix (might have been SVR3, but this was in the bad old days when AT&T was actively fucking up Unix, and it could have been just about any version, including SVR666). The license was for a mind boggling, nay, jaw-dropping ten bucks a seat. The problem was that the ST didn’t have any kind of memory management hardware, just a raw CPU flinging real addresses at naked DRAM, and the machine's cheap-ass vanilla 68000 was incapable of recovering from a fault unless you cheated.

On a related note, there's MiNT.

A history of modern init systems

The subject of process management, supervision and init(8) for Unix-like systems is one plagued by a large degree of ahistoricity and "pop culture" explanations. This leads to a lot of confusion and misunderstanding surrounding feature sets and how one formulates the problems surrounding reliable process management on Unix in general, making it a ripe topic for demagogues of all persuasions.

The purpose of this article is to set the record straight on the history of attempts to create "modern" init systems, where we define "modern" somewhat broadly as anything that tries to improve the classical BSD and System V styles of initialization and service management.

Coherent UNIX clone goes Open Source

We missed this earlier this year, but Coherent has been released as open source. Coherent is a UNIX clone originally developed for the PDP-11, but later ported to a number of other platforms, including the IBM PC. It was developed by the Mark Williams Company, and despite an official investigation by AT&T, no signs of copied code were ever found.

Mark Williams Company closed in 1995. In 2001, Bob Swartz asked me to archive the hard disks containing the Mark Williams source repository; the command and system sources here are from that repository. I have long intended to catalog and organize these sources, but in the meantime they are posted here as is. MWC's documentation guru Fred Butzen provided the MWC documentation sources.

“Have you ever kissed a girl?”

An old (2010) story from former Sun employee Jeremy Allison.

David Miller wrote (at the end of a long email explaining how Sparc Linux used cache optimizations to beat Solaris on performance):

"One final note. When you have to deal with SunSOFT to report a bug, how "important" do you have (ie. Fortune 500?) to be and how big of a customer do you have to be (multi million dollar purchases?) to get direct access to Sun's Engineers at Sun Quentin? With Linux, all you have to do is send me or one of the other SparcLinux hackers an email and we will attend to your bug in due time. We have too much pride in our system to ignore you and not fix the bug."

To which Bryan Cantrill replied with this amazing retort:

"Have you ever kissed a girl?"

Talk about missing the point and underestimating the competition.

The article offers an interesting look at why Sun eventually failed. I stumbled upon this story because Rob Landley mentioned the girl comment in his email about BSD/SysV.

The BSD/System V split

Rob Landley:

So gcc's library bindings strongly preferring System V system calls to BSD was due to the flood of Solaris end-users dominating the late 80's gcc development community to avoid paying Ed Zander extra for Sun's Solaris compiler. This combined with Linus reading Sun workstation manuals to get a system call list gave linux a very System V flavor.

Interesting - if opinionated - view on the whole situation. Not sure if all the dates check out, but it's a fun read nonetheless.

The history of grep, the 40 years old Unix command

grep is a Unix command line utility (well most Unix utilities are command line) that searches the input files for pattern and prints lines that contain the pattern. If you are reading this you, you are probably no stranger to grep.

grep was written by Ken Thompson, the same guy who wrote Unix. grep first appeared in Unix v4 with limited features as compared to today's grep.

I've used grep so much over the years. One of the countless little utilities that's the staple of all UNIX-like systems that you never really think about, but use all the time.

Explorations in Unix

"Few tools are more indispensable to my work than Unix. Manipulating data into different formats, performing transformations, and conducting exploratory data analysis (EDA) is the lingua franca of data science.1 The coffers of Unix hold many simple tools, which by themselves are powerful, but when chained together facilitate complex data manipulations. Unix's use of functional composition eliminates much of the tedious boilerplate of I/0 and text parsing found in scripting languages. This design creates a simple and succinct interface for manipulating data and a foundation upon which custom tools can be built. Although languages like R and Python are invaluable for data analysis, I find Unix to be superior in many scenarios for quick and simple data cleaning, idea prototyping, and understanding data. This post is about how I use Unix for EDA."

Unix doesn’t follow the Unix philosophy

James Hague: "But all the little bits of complexity, all those cases where indecision caused one option that probably wasn't even needed in the first place to be replaced by two options, all those bad choices that were never remedied for fear of someone somewhere having to change a line of code... They slowly accreted until it all got out of control, and we got comfortable with systems that were impossible to understand." Counterpoint by John Cook: "Some of the growth in complexity is understandable. It's a lot easier to maintain an orthogonal design when your software isn't being used. Software that gets used becomes less orthogonal and develops diagonal shortcuts." If there's ever been a system in dire need of a complete redesign, it's UNIX and its derivatives. A mess doesn't even begin to describe it (for those already frantically reaching for the comment button, note that this applies to other systems as well).

Understanding the /bin, /sbin, /usr/bin, /usr/sbin Split

Finally something really interesting to talk about. If you've used UNIX or any of its derivatives, you've probably wondered why there's /bin, /sbin, /usr/bin, /usr/sbin in the file system. You may even have a rationalisation for the existence of each and every one of these directories. The thing is, though - all these rationalisations were thought up after these directories were created. As it turns out, the real reasoning is pretty damn straightforward.

The /bin/true Command and Copyright

"One of the fun examples among all the copyright fuss is the extreme example of copyright claims made by AT&T some time in the 1980s. It's the /bin/true program. This is a dummy' library program whose main function is to make it easy to write infinite loops (while true do ...) in shells scripts. The 'true' program does nothing; it merely exits with a zero exit status. This can be done with an empty file that's marked executable, and that's what it was in the earliest unix system libraries. Such an empty file will be interpreted as a shell script that does nothing, and since it does this successfully, the shell exits with a zero exit status. But AT&T's lawyers decided that this was worthy of copyright protection." Three blank lines. Copyrighted. You can't make this stuff up.