OS Development: Opening Streams

This article offers feature suggestions to budding OS developers looking for that neat edge. This osnews mini-series is a series to give people developing hobby operating systems – for fun – ideas that take their OS from a kernel to a ‘system’.In previous articles we have described using URIs [1] for file paths and the idea that streams of various types can be transparently converted between by the operating system [2]. This article describes how these might be unified into a single, consistent user interface.

Opening Streams

UNIX has the design principle that ‘everything is a file’ [3]. A file is, more properly, a stream. I have introduced, in the previous article, the idea that a stream, if we know its type, might be presented transparently to a program as another, compatible type. This means that conceptually a file is in fact many streams, with each stream marked by type [4].

Streams don’t just refer to physical files on your computer or remote computers. Streams can be generated by other things such as applications and device drivers. In UNIX, named pipes and UNIX-domain sockets provide this kind of facility [5].

Many UNIX programs are an excellent example of applications that provide streams programmatically. The ‘sort’ command takes one stream as input and outputs to another, as does the ‘grep’ command. In fact, all command line programs do this. As our hobby operating system is marking streams by type, you could expect all these command line applications to be reading and writing text/plain streams. If you want to ‘grep’ on a rich document, for example, the operating system would be quite capable of converting that to text/plain – if the virtual file system has the right plug-in – and ‘grep’ need not know [6].

We’ll make the jump that your hobby OS includes a GUI of sorts! How might applications provide streams for the user to interact with?

GUI widgets (windows, even individual controls such as text boxes) might report to the system that they provide/consume streams of certain mime-types. This would allow the window manager to provide drag-n-drop stream connections and work as a versatile clip-board. This is not so strange – this is pretty much what the windows clipboard is doing anyway!

So think of ‘open’ and ‘save’ as reading and writing streams, rather than files (a slightly thin distinction to make, I know). Applications conventionally use standard system dialogs for allowing the user to select files to open and names of files to save-as. This allows the hobby operating system author to unify application streams, remote streams and file streams and present them to the user in a sensible manner.

You’ll see from my dialog mockup that I allow the user to select from a recent streams history, from a location (much like a conventional file-open dialog) and by type. This later category is something akin to a search. Popular consensus in the human-computer-interaction camp at the moment seems to think that users don’t really remember where they put files or what they are called. Modern operating systems such as BeOS, Windows and the like are moving in the direction of letting the user search for files instead. This is the whole big idea behind the ‘file system is a database’ hype at the moment!

So the user doesn’t care where the stream comes from, they care only that they find their stream. The ‘by type’ presents all files of the appropriate type, regardless of location. In the example dialog an application-provided stream is shown highlighted, mixed in with streams provided by conventional files from the user’s home directory.

A ‘by type’ query that is constantly updating live is termed an ‘active query’ and is one of the many big features in BeOS. The idea is that, while you have a search window open, files that are added, modified or removed that fit the terms of the query are automatically updated [7].

The Save-As dialog might also present application streams as valid destinations. A user might ‘save’ a bitmap to their presentation software which is running at the moment, and that presentation software might insert that received image file into the actively-being-edited application.

Opening a stream is assisted by search agents. The idea is that various mime-types might register search agents that take words, phrases or even complex query expressions and filter a list based upon user-entered criteria. You might want to only documents that contain a reference to a certain activity, for example. Various search agents might use meta-data, a contextual search or an index to find matches; it depends upon the mime-type and the agent’s needs.

It could be taken further and integrate search agents that represent remote locations e.g. coalescing a Google search into a local files search.

[1] “Network Transparency and the Multi-Root File System
[2] “Files and Programmatic Streams
[3] Google results
[4] Some file systems allow a single file to consist of more than one actual physical stream. On the Mac, a file consisted of exactly two streams, one containing the ‘meta data’ (as more recent file systems placed in peer attributes). On Windows NTFS, files can contain many streams (although this is hardly a much-used feature) and so on. When your hobby operating system has to mount such file systems, this leads to stream ambiguity. One good sounding approach is to give each native stream a name (most such file systems expect this) and treat them as individual files with the actual multi-stream node being presented as a directory.
[5] This provides a good overview to UNIX inter-process-communication (IPC), which is about files, pipes and sockets.
[6] As an aside, it probably would make sense for a hobby version of ‘grep’ to work from a dedicated application/x-grep stream that might contain meta information such as logical locations; this would allow a viewer of the actual native stream type to understand the stream positions of the matching terms and highlight it appropriately. Such a system would be very useful for doing regular expression searches across a wide range of actual native stream types that contain text elements.
[7] One approach to adding active queries to a hobby operating system is described here.

If you would like to see your thoughts or experiences with technology published, please consider writing an article for OSNews.


  1. 2004-09-01 7:52 pm
  2. 2004-09-01 7:53 pm
  3. 2004-09-01 7:59 pm
  4. 2004-09-01 8:19 pm
  5. 2004-09-01 8:30 pm
  6. 2004-09-01 8:30 pm
  7. 2004-09-01 8:36 pm
  8. 2004-09-01 9:27 pm
  9. 2004-09-01 10:08 pm
  10. 2004-09-02 2:12 am
  11. 2004-09-02 6:06 am
  12. 2004-09-02 7:25 am
  13. 2004-09-02 11:19 am
  14. 2004-09-02 11:46 am
  15. 2004-09-02 12:23 pm
  16. 2004-09-02 1:43 pm
  17. 2004-09-02 7:29 pm