posted by Brendan Younger on Fri 7th Feb 2003 03:51 UTC
IconFile systems need to change. Current file systems are horribly out-of-touch with the realities of what users need to effectively find, organize, and modify their vast quantities of files. Unfortunately, no major consumer OS vendor (Microsoft, Apple, various Linux distos, etc.) has had the foresight, the will, and most of all, the cajones to implement anything more elaborate than a small departure from the standard hierarchical name-space which we all grew up on and should rightfully deplore. Worst of all, the best suggestions for changing the current entrenched standard are incredibly toothless, incredibly feeble.

Not a single proposal I've read has ever started by considering the most important motivator of good file system design: how will the user interact with it? The tacit assumption that everyone seems to be making is that by making it easy for programmers to code for, the program can then present whatever "user-friendly" interface it wants to the hapless user. This is a damaging notion, and is born of the misplaced belief that any user-centered file system must necessarily be difficult to interface with for the programmer. False! Programmers must translate user intent into machine code. When the file system is created to support a user's intent, the programmer's job becomes trivial. This, then, is our task: design a file system sympathetic to the user's weaknesses which nonetheless fits all the needs of programmers. For this I propose the Brendan File System, or BFS.

The Lost Art of Collating

The traditional hierarchical structure was introduced to create structure where there was none, in order to aid both the user and the programmer in categorizing information. Unfortunately, it became the only way to organize structure on-disk, and so became much more restrictive than the flat file system's disorganized chaos. We've come to the point in OS design where over-simplification causes nothing but headaches and provides no measurable performance advantage. BFS throws hierarchy out the window. Instead, folders become "groups", and files can meander about and show up wherever they're needed, even in multiple places at once. Groups function much like folders, but a file is never tied to a particular group; files have an independent existence and can belong to any number of groups (or none!).

Let's say a user is starting a new programming project. She first creates the group for the new project, optionally names it (who said a group had to have a name?), and starts adding some of the relevant documentation to the group. Need that primer on C++ templates? Add it to the group, and never mind how many other "copies" you have floating around in other groups of yours. When you're done, you can just remove it from the group and you needn't worry about it disappearing or being deleted before you need it again. There is, of course, a separate delete command available if you never want to see the file again. Certainly, all this can be done with folders and symlinks and the like, but it's just so much cleaner this way.

Finding Simplicity in Complexity

Well, if you don't have any directory structure, how on earth do you find things? You can't use file names because there's no guarantee you'll remember the name (besides, who said your file had to have a name?), and you can't use the file's "location" because it doesn't really have one, so what to use? The best answer is a good search mechanism. The Be file system basically nailed this with extendable attributes and live queries, and the BFS embraces this system whole-heartedly. Issue a query for all mail messages from Bob Smith and you get a list of all of them, updated in real time as new ones come in. But where the Be file system made these queries persist at the application level, the BFS lets users and programmers create and extend groups by adding a search pattern to their metadata. The user could create a group for storing all her HTML documents which would automatically be updated as new ones are created; the user need never physically add an HTML file to the group. Of course, you can make groups which are mixtures of a live query and user-placed files. In the HTML group, you could also stick your favorite browser for easy previewing. The possibilities are limited only by the complexity of the search and the imagination of the user. This has tremendous potential for the programmer as well, and will be further elaborated on in the section on the System group.

Metadata: Man's Best Friend

Metadata is worth its weight in gold. There is absolutely nothing as important to a modern file system as metadata. Without it, you simply cannot find, catalog, or classify the vast amount of data present on even the lowliest consumer PC. To that end, there is absolutely no reason why file systems should in any way limit the type or quantity of metadata associated with a file. Once again, the Be file system shines in this regard. Storing small, keyed attributes, such as the file's type (probably as a MIME type), the default application for the file, the various dates associated with it, and the other, custom, types is the only sane way to organize and classify data in a modern filesystem. To this end, everything about the file, including it's name (and any localized names), should be stored in the auxiliary metadata file associated with every regular file on the system.

Of utmost importance is that there be no limit on the size of type of the metadata present. The metadata file is designed to be the way to identify a file. Keywords, document titles, authors, etc. should all be stored in the file's metadata, allowing the user to issue searches for "all documents with author 'Brendan' and contain the word 'money'". Nothing less will suffice as the gateway to the vast quantity of files a use must deal with. Metadata searching is the biggest aid to productivity since the whip, and BFS will have it.

Table of contents
  1. "Brendan File System, Part I"
  2. "Brendan File System, Part II"
e p (0)    84 Comment(s)

Technology White Papers

See More