http://www.osnews.com/story/8158/OS_Development_Opening_Streams Exploring the Future of Computing en-us Copyright 2001-2009, David Adams adam+nospam@osnews.com Tue, 07 Jul 2009 01:15:13 GMT http://www.osnews.com/images/osnews.gif http://www.osnews.com http://osnews.com/thread? http://osnews.com/thread? ... This can all be done with userland file servers. You really need to stop writing articles and actually see what the operating system world beyond Linux has to offer. Wed, 01 Sep 2004 19:52:00 GMT donotreply@osnews.com (Anonymous) Comments http://osnews.com/thread? http://osnews.com/thread? Great article -- but best of all, Bach gets a mention! Huzzah for the ultimate master of counterpoint. Mmmm, baroque 'n roll... Wed, 01 Sep 2004 19:53:00 GMT donotreply@osnews.com (Anonymous) Comments http://osnews.com/thread? http://osnews.com/thread? Isn't it called "opening"?

- Simon Wed, 01 Sep 2004 19:55:00 GMT donotreply@osnews.com (Anonymous) Comments http://osnews.com/thread? http://osnews.com/thread? > This can all be done with userland file servers.

What exactly do you mean by "user land file servers"? Im not familiar with the term. Also, why is it a better method? Wed, 01 Sep 2004 19:59:00 GMT donotreply@osnews.com (Anonymous) Comments http://osnews.com/thread? http://osnews.com/thread? The article starts with stating that there is a *nix design philosophy that states that "everything is a file". This is a widely spread idea, but it is also an idea that is not very true. When you think about it, how much of a modern *nix is actually a file? It turns out that there are a lot of fundamental concepts that are not files:

* Processes - A process cannot be manipulated like a file. There are filesystem extentions that allow you to inspect the state of a process like it was a collection of files, but it is not possible to e.g. create a process like you would create a file.

* Network sockets - There is no way to create or manipulate network sockets like they were files. There is an old proposal called "portals" for BSD that actually makes network sockets appear in the file system, but this has not been adopted.

* Network interfaces - It is not possible to configure a network interface by using the file system - we are forced to use ifconfig (or the underlying kernel communication magic).

* Users - A user does not have any representation as a file. Sure, all users are stored in the passwd file, but that does not make a user a file.

* Un-named pipes (the ones that appear if you type the | character between commands) - they may be implemented using files, but that does not make them files

* Signals - You cannot send or receive signals using the file system.

Maybe it is time to finally drop the idea that "everything is a file" and acknowledge the fact that most things in modern *nixes are *not* files. Wed, 01 Sep 2004 20:19:00 GMT donotreply@osnews.com (Anonymous) Comments http://osnews.com/thread? http://osnews.com/thread? You should also read this enlightening works:
http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm
http://www.namesys.com/whitepaper.html

Instead of having a naspace identifying collection of bits in the computer memory the namespace should identify the conceptual data, the system should just handle representations of the concept. Wed, 01 Sep 2004 20:30:00 GMT donotreply@osnews.com (Anonymous) Comments http://osnews.com/thread? http://osnews.com/thread? I'm pretty sure the BeOS find dialog is not find-as-you-type. Wed, 01 Sep 2004 20:30:00 GMT donotreply@osnews.com (Anonymous) Comments http://osnews.com/thread? http://osnews.com/thread? See "Plan 9" for a more extreme implementation of "everything is a file".

* Un-named pipes

Even if the pipes isn't represented as files, I do belive that you could find a stdin/stdout/stderr for each process in the pipeline at /proc/[0-9]+/fd/*. Am I wrong here? Wed, 01 Sep 2004 20:36:00 GMT donotreply@osnews.com (Anonymous) Comments http://osnews.com/thread? http://osnews.com/thread? in case anyone has a morbid interest in these sorts of things in our beloved linux kernel, you might like to have a quick glance at this very heated debate that's been raging on the linux kernel mailing list for some time.

google groups says the thread has reached over 480 posts in just a few days!

http://groups.google.co.uk/groups?hl=en&lr=&ie=UTF-8&th...

a quick executive summary is that reiser4 allows some neat things with its idea of plugins (streams and meta-data are the focus of the discussion but the sky is the limit). unfortunately this is not compatible with POSIX, SVR4 or current linux VFS semantics. cue a major philosophical flame-fest. Wed, 01 Sep 2004 21:27:00 GMT donotreply@osnews.com (Anonymous) Comments http://osnews.com/thread? http://osnews.com/thread? Maybe it is time to finally drop the idea that "everything is a file" and acknowledge the fact that most things in modern *nixes are *not* files.

Yes. I don't like the way drivers are done on Unixes-- device files, with horrible hacks like ioctl to get around the fact that the file is not a suitable interface for most devices, and now all this UDev and DevFS palaver. Why not just expose a nice set of interface functions for the device class (with the option of hardware-specific functions). Ioctl is just a silly way of doing exactly this, but incurs more overhead and is harder to use.

The file system does provide one useful feature for device drivers, that is a way of accessing and listing by name. But you can do this some other way. Wed, 01 Sep 2004 22:08:00 GMT donotreply@osnews.com (Anonymous) Comments http://osnews.com/thread? http://osnews.com/thread? It would appear that the historical significance of "everything is a file" has been lost. When this philosophy was coined everything had a different interface. So when you wanted to write to a device you had to learn a brand new interface. What's more you had to learn how to refer to that interface. After looking at a lot of these interfaces it was found that most of them have a lot in common. It was decided that there needed to be a standard way of talking to stuff and the "file level" was the lowest common demoninator. When we say "everything is a file" we're saying "you can use the file interface to talk to everything". If you want to do something special with the device then you can extend the file interface (with ioctl). Userland libraries can provide more "pretty" interfaces if you really need, but by making everything conform to this standard interface it makes it easier to join programs together. The pipe is simply two such interfaces with a buffer in-between, and by using a lot of programs and devices which all share the same interface we can get a very powerful system from a small number of components. Remember, as system's power is measured by the number of components raised to the power of the number of possible interconnections between those components. Thu, 02 Sep 2004 02:12:00 GMT donotreply@osnews.com (Anonymous) Comments http://osnews.com/thread? http://osnews.com/thread? You are right that everything is not a file, but IMHO an operating system which gives uniform interface to access device is always going to be easier and liked by developers. How exactly do you think people writing code at high level manage difference between those devices?

Today i can use same WriteFile functions on a network device as well as a file system and it is much better than having 100s of different functions specific to each device and making it unusable when the device change.

May be what you want to propose is a more powerful interface than file but not non-uniform interface based on devices. Thu, 02 Sep 2004 06:06:00 GMT donotreply@osnews.com (Anonymous) Comments http://osnews.com/thread? http://osnews.com/thread? The purpose of having everything, from a user process standpoint, appear as a file descriptor, is that it gives you a simple and powerful way of cleaning up when the process exits.

ioctl() is indeed not a very elegant way of supporting interfaces, even though you could publish interfaces in the C++/component sense by marshalling/demarshalling and using
the file descriptor as a 'wire'.

However, it simplifies the OS and the applications a lot that everything supports open() and close().

See what happens when a process crashes in Syllable and it uses the "non file compliant" message ports and windows for example: they don't go away. Thu, 02 Sep 2004 07:25:00 GMT donotreply@osnews.com (Anonymous) Comments http://osnews.com/thread? http://osnews.com/thread? > However, it simplifies the OS and the applications
> a lot that everything supports open() and close().

Last time I looked into /dev and /proc, I wasn't terribly impressed by the "simplicity" of it, and I can't say I remember ever open()ing or close()ing anything in there.

I think while the "everything is a file" metaphor makes things easier for (a certain brand of) developers, it's more than a bit confusing for the rest, including end users.

Unfortunately, *nix isn't exactly about end users, anyway. Thu, 02 Sep 2004 11:19:00 GMT donotreply@osnews.com (Anonymous) Comments http://osnews.com/thread? http://osnews.com/thread? Well, in Linux (and I guess that in all other unixes) when I want to see if my usb mouse has been recognised I do "cat /dev/input/mice", and I bet that many, many other people do similar things. Thu, 02 Sep 2004 11:46:00 GMT donotreply@osnews.com (Anonymous) Comments http://osnews.com/thread? http://osnews.com/thread? When you open a terminal, the terminal open(s) /dev/console, /dev/ttyS0, etc. and this is passed over to child processes.

/dev is a way for user processes to open() stuff that isn't a physical file. In itself it's not a bad idea; what is a bad idea is that it's exposed to the end user (you and me); whether it's physical inodes or a filesystem like devfs doesn't really make any difference.

The Windows NT kernel works the same way (I think the hierarchy is called \device.. NT gurus please correct me :-) but you don't see it exposed directly to apps.

So, again, there is nothing wrong (and a lot of good) with wrapping the ugly semantics of the "everything looks like a file" using nice C++ or component-based interfaces.

By simplifying life for applications, I mean that you don't have to explicitly free and close everything when your application exits, and the system (or yourself) can safely slay a process without leaving things hanging. If you ever programmed for AmigaOS, you know what I mean.

That was my point :-) Thu, 02 Sep 2004 12:23:00 GMT donotreply@osnews.com (Anonymous) Comments http://osnews.com/thread? http://osnews.com/thread? One interface to rule them all, one place to find them, one call to open them all and in the darkness bind them. Thu, 02 Sep 2004 13:43:00 GMT donotreply@osnews.com (Anonymous) Comments http://osnews.com/thread? http://osnews.com/thread? If it scares you when you ls /dev then don't.

Or find another distro that doesn't make a scarey /dev.

I think everything should be a file. But I can deal with the fact that some things, like users, won't become files in the near future. Though I might treat them like files if all they do is complain. Thu, 02 Sep 2004 19:29:00 GMT donotreply@osnews.com (Anonymous) Comments