To view parent comment, click here.
To read all comments associated with this story, please click here.
I did it for you
.
Writing a fine grained kernel is hard, especially when you need to support some functionality that's not so friendly for large MP systems.
Maybe Dillon's approach is right. Instead of trying to coordinate actions between processors in a lock free or finely grained manner while trying to maintain tons of shared state, why not treat your large MP machine as many smaller ones that are coordinated at a higher level?
The Google cluster is a great example of this. They don't really need a scalable OS at all: a simple kernel that efficiently manages I/O and gets out of the way of the one executing task on a particular node is all that's needed. Everything else is coordinated from their aggregation servers and their distributed namespace/locking system.
. Writing a fine grained kernel is hard, especially when you need to support some functionality that's not so friendly for large MP systems.
Maybe Dillon's approach is right. Instead of trying to coordinate actions between processors in a lock free or finely grained manner while trying to maintain tons of shared state, why not treat your large MP machine as many smaller ones that are coordinated at a higher level?
This is the McVoy cache coherent cluster approach
http://www.bitmover.com/cc-pitch/
It has always seemed like hadwaving to me. The problem with this is: "what higher level?"
What does it buy you to program a multiprocessor system as a set of communicating cluster nodes, that you can't do as a monolithic kernel? As far as I can see, it only serves to place restrictions on the ways you can communicate and interact between CPUs.
So why do proponents of this approach think they can just assert that a system of communicating nodes -- that *must* still be synchronized and share data at some points -- can scale as well as a monolithic kernel? I don't think that is a given at all.
Well, they use Linux, and actually they are pushing some pretty complex functionality into the kernel (to do resource control, for example).
And I don't know what it's requirements are, but you can bet it's nothing like a regular UNIX filesystem / POSIX API.
Also, I don't see why you think Google is a great example of this. Google does not have a large MP machine. It has a big, non-cache-coherent cluster. So there is only one way to program it -- like a cluster.





Member since:
2005-07-21
It's a shame that one cannot moderate after having posted, for I would have moderated you up for such a well-written, informative post.