This is my third attempt at writing this all down. The first was too detailed. The second was still too long. This time I've decided to just lay out my ideas, and try to keep them clear and brief. This is more or less just an overview. There are just too many details to try to squeeze them all in here. Many of the ideas are not really new, some have been talked about before. I'm just trying to figure out how to put them together into one system.
I've been working with computers for 30 years and they certainly haven't gotten any easier. In fact I'm pretty sure they have gotten harder. True, I'm doing things now that I never dreamed of doing then. But, it feels like, while they've become 1,000 times more powerful, it seems like they have also become 1,000 times more complex.
I've tried almost every major operating system out there and I always seem to run into difficulties. So I have been thinking: "What if we started over from scratch and re-thought how an operating system is put together?". One of the first operating systems I ever dealt with was Exec 8 on a Univac 1108. It contained most of the concepts used by all of today's major operating systems. Now you are probably thinking "it didn't have a GUI!". That's very true. What I'm talking about are the underlying concepts: processes, files, how programs run and interact, things like that.
I have been brainstorming for some time now, and I have some ideas that I would like to share. They are just ideas right now, some or all of them may never work. It is possible that I have failed to think of some aspect of an idea that makes implementation infeasible. But I feel the need to get them out there for several reasons. One, I am hoping that if they are published somewhere they will quit going around and around in my head and I can get some sleep. Two, I don't want a scene like the one with Microsoft and SenderID
One last thing I would like to say up front. I am going to mention some software that I use, as examples, such as Linux, OpenBSD, gcc, K3B, etc. Please don't take that as my disparaging that particular piece of software. I am just using what I am most familiar with. I still think it is excellent software, which I use everyday and will continue to use for the foreseeable future.
So without further ado, I guess I'll jump right in with both feet.
I have been reading the book Better, Faster, Lighter Java and although the author is talking specifically about Java, he might as well be talking about computing in general.
Chapter 2 is titled "Keep It Simple" and explains the value of simplicity. It even goes on to say that "Simplicity may be the core value". I wish I could reproduce or link to the first section of chapter 2 because I think it applies to all of computing.
So that is my primary goal is to keep the entire system as simple and elegant as possible. When complexity is inescapable, contain it in one place, and don't allow it to infect the rest of the system.
Flat Files and Relational Databases
The biggest thing that I can think of that adds complexity to computer programs is storing the data. It seems to me that any non-trivial program has a good portion of the code just to mess around with files. There are configuration files and data files, which most of the time cannot be used directly by the program, and have to be interpreted and converted into the internal format of the application.
Another alternative to storing data in flat files is to store them in a database, most commonly a relational database. While this has several advantages over the flat files, if using a relational database, the program has to do queries on the database and then usually reformat the result into the program's internal data structures. This is really messy when using Object Oriented programming. There are of course Object Oriented databases, but I am not aware of any that have gained wide acceptance, especially ones that are Open Source like MySQL and PostgreSQL.
My idea is to have all of the information stored on the disk in the native Object format of the program. That way instead of having to constantly convert and interpret data, the application can just access that object directly. And when that data is needed in a flat file format, you have a converter App (object) that can access the internal data and convert it to a flat file type of format.
For example, let's say you have some compressed (with Ogg Vorbis or whatever) audio objects on your computer. And you want to burn an audio CD which can be played in a normal audio CD player. You would create a playlist object and connect the output (more on this later) to the input of a Ogg Vorbis converter object and then into the Audio CD burning object.
Where Is It Stored?
Another problem that I would like to solve is the "where is that data?" or "what did I do with that file?" problem. Just yesterday I was looking for a spreadsheet that had some information I needed in it. At one time I had it stored on my desktop machine. But I've changed that machine several times in the past year and reloaded different operating systems several more times. So I finally put it on my file server. But when I looked for it, I didn't see it anywhere. Worse yet I couldn't remember exactly what it was named. I had to do many searches before I finally found the stupid thing. I probably wasted 20 minutes looking for it and as time went by I started worrying that I had lost it and wondering when (if?) I had backed it up. I just don't need this kind of stress!
My thought on that is to remove the distinction between different partitions, disk drives, file servers, CD and DVD media, etc. No more C:, D: and E: partitions. No more host7:/shared... No more mounting the CD drive. It's all just have one big object storage space. If it's out there the machine can find it. If it's public information it can even be obtained over the Internet. If it was stored on a tape, CD or DVD backup and that media is not in the drive, the OS will tell you on what media that object was stored and you can find that media and insert it and the App can continue without even knowing what happened. If you don't have access to that media at that time you should have the option of stopping that App right where it is and restarting it later at the same exact point when you have the media (more on this later).**It seems that Google, Apple, and others are moving toward making this a reality on today's operating systems, so that's good news.
Relating to Each Other
Another problem with Flat Files is the lack of information about the data in the file. The BeOS had a database built into the file system with was nice addition, and I understand that Microsoft's Longhorn was going to have some sort of database built in before it was scaled back. I also recently discovered that the Linux ext3 filesystem allows additional "attributes" to be added to a file.
But I would like to go beyond all of that. My idea is to take an object as defined in current languages like C++ and Eiffel and add some additional information to it. Information such as when the object was created, who created it, md5 checksum, some protection bits, the usual meta-data that a file would have in the OS now.
In addition to those, the most significant addition is the reference table. This is a table maintained by the OS which lists all of the objects which reference that object. That provides a couple of benefits. First it allows you to easily find any object that references a given object. Second it allows one to determine which objects are still in use on a system. If no other active object on a system refers to an object (other than the directory) it could be moved to back up storage and deleted from the active system.
For example, going back to the music example, all of the music objects would reference the artist that created them. All of the Beatles tracks would reference the Beatles Artist object. The Beatles Artist object would not specifically have references to songs because songs would be only one type of object that could refer to the Beatles. We wouldn't want to encumber an Artist object with explicit references to every possible type of object that could refer to it. We don't even know what every type of object that could reference it even is. And if we did that we would have to change the Artist class every time some other class was created that referred to an artist class. Instead every object will just have a generic list that can point to any type of object (the "ANY" class in Eiffel). This list will be automatically maintained by the OS and updated as the program runs. And when you want to find all of the songs you have by The Beatles, you simply get the Beatles object and search it's reference list for objects of the class "Track".