Most of us are probably aware of Singularity, a research operating system out of Microsoft Research which explored a number of new ideas, which is available as open source software. Singularity isn’t the only research OS out of Microsoft; they recently released the first snapshot of a new operating system, called Barrelfish. It introduces the concept of the multikernel, which treats a multicore system as a network of independent cores, using ideas from distributed systems.
The new concept operating system is a joint project byETH Zurich and Microsoft Research, Cambridge. The team argues that “the challenge of future multicore hardware is best met by embracing the networked nature of the machine, rethinking OS architecture using ideas from distributed systems.” “We investigate a new OS structure, the multikernel, that treats the machine as a network of independent cores, assumes no inter-core sharing at the lowest level, and moves traditional OS functionality to a distributed system of processes that communicate via message-passing,” they write in their article “The Multikernel: A new OS architecture for scalable multicore systems“.
They didn’t just investigate the concept, they also implemented it to show that it can work. They call their multikernel implementation ‘Barrelfish’, and according to them, it shows promising performance figures compared to other, more conventional operating systems. On top of that, the multikernel concept should be more scalable and adaptable for the future.
The team believes that challenges faced by traditional ccNUMA and SMP high performance computing machines are not the same as the challenges a general purpose operating system will face in the future. They argue that single computers increasingly resemble networked systems, “and should be programmed as such”.
This resulted in the multikernel model. “In a nutshell, we structure the OS as a distributed system of cores that communicate using messages and share no memory,” they write. It’s based on three principles:
- Make all inter-core communication explicit.
- Make OS structure hardware-neutral.
- View state as replicated instead of shared.
“These principles allow the OS to benefit from the distributed systems approach to gain improved performance, natural support for hardware heterogeneity, greater modularity, and the ability to reuse algorithms developed for distributed systems,” they claim.
The article goes into quite some detail about the motivations, implementation, and performance figures, but a lot of that simply goes way over my head, so I’ll let the more educated people on this subject take over from here (lovely to have a reader base filled with people who actually understand stuff like this). It all sounds very intriguing and interesting to me, in any case.
The team has published their first snapshot release, licensed under a BSD-like license.
Neat
Hopefully that’ll mean in the future one can create a truly robust operating system where if one kernel crashes on a massive multicore monster – the isolating will mean the rest will keep running without a hitch. I do, however, wonder what the overhead is, how complex it would be to use it for a general purpose operating system and whether such a concept can be retrofitted to an existing operating system.
IIRC, “Message Passing” is Latin for “Slower than the January Molasses”.
All that hardware bandwidth. All that potential for fast, low-latency IPC mechanisms. And it gets wasted, killed by latency, passing messages back and forth.
I always knew that the fantastically powerful computers of the future, running the software of the future, would perform significantly more poorly than what we have today. And this concept may just be a glimpse of how that future is to unfold.
You don’t think the hardware will adapt?
I doubt it, seems logical that new processors will be designed with pipelines facilitating nanosecond IPC.
Microsoft just provided very costly R&D to the IT community free of charge, and is signaling to their partners that theoretical technology is now practical to some extent ….
… and in essence communicating that they should plan accordingly!
Very costly R&D, or very cheap PR?
It certainly was very costly in terms of time, effort and the skilled labor to produce it.
It’s safe to say this end result (the down-loadable code) cost at least a million USD to produce.
It served both R&D and PR purposes … ‘cheap’ is a matter of perspective, though.
If there is a cadre of hobbyists who have released an alternative, I would love a link!
Most of the work was done by ETH, I doubt it would have cost even a fraction of that to Microsoft. It seems a mostly academic effort, but good to see MS involved.
Even at a million dollars, cheaper than a superbowl ad.
http://www.kernel.org 🙂
I see now that you can’t help diminishing the effort, and any unique value it has — seemingly because it is either from a commercial source, or Microsoft specifically
I can’t understand the point of the constant negativity, it doesn’t help anyone!
It’s silly to debate the overall cost, but let’s just say it was far more than zero, and no one did it first, to this extent, for free.
Well, free in the sense it’s not encumbered by GPL limitations, but MS limitations – free nonetheless.
edit: typed less instead of ‘more’
Edited 2009-09-26 06:58 UTC
Not at all! It is great that companies like Microsoft contribute to pure OS research. More companies should take their lead.
Can it, while remaining compatible with the existing architecture? Intel/AMD aren’t going to break the backwards compatibility of the x86 arch without a darn good reason. Keep in mind that backwards compatibility is largely why x86 has been so successful.
And if this were easy, wouldn’t it already have been done by now? After all, message-passing is also at the heart of the microkernel concept, an idea that has been around for decades but has gone absolutely nowhere because of, as the GP sarcastically pointed out, bad performance.
I suspect they’re going to need some hard proof of really dramatic improvements before Intel/AMD will pay attention to them.
I never mentioned x86, you did.
As you can see in the very graphic inlined in the article, this technology was developed for heterogeneous architectures.
Obviously, non-x86 only.
My point that some hardware could well evolve to take advantage of this technology stands unmolested.
It was mentioned down-thread that the dev platform for this was an AMD system.
I don’t know which inlined graphic you’re referring to, but now having read parts of this:
http://www.barrelfish.org/barrelfish_sosp09.pdf
the 64bit flavor of x86 is clearly a prime focus of their work (they also mention Sun’s “Niagra” system, but I don’t know what that is). They explicitly mention AMD Opteron and Intel Nehmalem(sp?). Barrelfish currently only works on x86-64 systems, though an ARM port is also in the works (they say).
I don’t think you understand correctly what they mean by “heterogeneous” here. They’re also referring to systems with cores of the same ISA, but just running at different speeds for example.
They seem to be claiming that the hardware is headed in this direction anyway, driven largely by the needs of multicore *server* systems. Section 2 of that pdf is a very interesting read. High-end server hardware is not something I’m familar with, so my original post may be wrong about the assumption that these changes would require incompatible changes to the x86 ISA.
hardware will never adapt. microsoft doesn’t make an OS that doesn’t require new hardware.
another point, microsoft will never use this in any of their products or they wouldn’t have given it away. being message based means its slow so they realize its useless and they’re trying to look good by giving something away they’ve appeared to work on for the last few years (but gave up on in reality because its useless)
FYI, their paper does argue that message passing on a multicode architecture is significantly faster than shared memory access on the same machine.
But then they explain they have made the “OS structure hardware-neutral” in 3.2.
So in other words: Let’s use message passing since it is fast on our AMD development machine, but if it is too slow on the next gen hardware, we will switch to something else.
Not exactly solving the problem, IMHO.
Edited 2009-09-26 05:11 UTC
That’s not really an actual portrayal of what they said.
Their basic conclusion is that as the number of cores increases, the cost of cache-coherency will increase such that updates that span multiple-cache lines will be slower than passing a message to each core and letting the update occur locally. There’s no real way around this problem so assuming that core counts continue to increase using a message passing approach like they took here, will make sense (it already does on large machines, there doesn’t seem to be much of an advantage on 4 core machines).
What is architecture specific is the most efficient message passing method. From what I gathered from the paper, a lot of this is handled by the system knowledge base, but even if a future piece of hardware requires a fundamentally different message passing mechanism (like the addition of a dedicated inter-core messaging) it won’t require a fundamental change in the organization of the OS.
You are right. I meant to refer to the message passing system, not messaging passing. That is fundamental and not going to change.
An Intel Engineer once said it best:
=> “What Intel Giveth, Microsoft Taketh Away”
That’s not completely fair – it’s more like “What Intel Giveth, Symantec Taketh Away”…
Uhm, You mean symantec the anti virus software that is absolutely necessary due to the historically crappy security on MS Windows? Who to blame, who to blame …
No… he means semantec, the super crappy anti-virus system that behaves more like spyware than anything and causes themachine to perform very poorly.
There are better AV suites out there.
All communication, basically, involves messages. It all depends on the sender and receiver. Memory can viewed as a service that handles read and write requests (messages).
In multi-core systems inter-core communication must go through memory, except for atomic operation coordination, which obviously has to be core-to-core. This results in multiple messages going back and forth for at simple exchange og infomation:
1. Sender: write data to memory (write msg)
2. Sender: inform receiver of new data (read/write, core-to-core msgs).
3. Receiver: read data from memory (read msg)
4. Receiver: inform sender of reception (read/write, core-to-core msgs).
I have left out all the nasty synchronization details in #2 and #4, but it usually involves atomic updates of a memory address, which can cause core-to-core sync messages, depending on cache state. Also, cache coherency in general might cause lots of messages.
It is easy to imagine that this could be done faster and in fewer steps if low-level core-to-core communication were provided. All the hardware is already point-to-point.
My point is, that it is not the message passing in u-kernels that gives the overhead. In fact, it is the extra protection (a long story).
Also, shared memory as a programming platform doesn’t scale if you code programs the obvious way. You have to know what’s going on underneath. It’s like cache optimization. You have to know the cache (Lx) and line sizes before you can do a good job. The non-uniform in NUMA does make things better.
I think we have to make memory a high-level abstraction and give OS and middleware programmers more control of what is communicated were.
Hi,
If you compare 16 separate single-core computers running 16 separate OSs communicating via. networking, to 16 separate CPUs (in a single computer) running 16 separate OSs communicating via. IPC, then I think you’ll find that IPC is extremely fast compared to any form of networking.
If you compare 16 CPUs (in a single computer) running 16 separate OSs using IPC, to 16 CPUs (in a single computer) running one OSs; then will the overhead of IPC be more or less than the overhead of mutexes, semaphores, “cache-line ping-pong”, scheduler efficiency, and other scalability problems? In this case, my guess is that IPC has less overhead (especially when there’s lots of CPUs) and is easier to get right (e.g. without subtle race conditions, etc); but the approach itself is going to have some major new scalability problems of it’s own (e.g. writing distributed applications that are capable of keeping all those OSs/CPUs busy will be a challenge).
-Brendan
Yes, good points. I would presume that this model would only take down applications that were currently running on the failed core. However, you would have to deal with messages in flight to the running core, so there would be unknown state to clean up. I bet you could easily cycle/reset the core into a known state. So, greater up-time in the long run.
As far as overhead is concerned, they say that native IPC was 420 cycles and the similar message passing implementation cost 757 cycles. That’s 151ns vs 270ns on the 2.8GHz chips they were testing on. However, by breaking the current synchronous approach and using a user RPC mechanism they dropped the message passing to 450cycles on die, and 532cycles one hop. With two hops only costing tens of cycles more. Which is really starting to become negligible. So, it does cost, but where they excelled was multi-core shared memory updates. But, to get back to your comments, that really is not general purpose computing as of today, as most applications on my Linux box are single threaded. Of the few apps that aren’t single threaded, ffmpeg and Id’s Doom3 engine, they are most likely aren’t synchronizing shared memory updates, rather I think they would isolate memory access to certain threads and pass commands around via a dispatcher thread. So, this is a pretty specific type of applications that excel on Barrelfish. I think they are targeting Google’s MapReduce and Microsoft’s Dryad.
Finally, it’s important to notice that HW is moving to a message passing type architecture as well. AMD had implemented HyperTransport and Intel now has the QuickPath Interconnect. So, in Barrelfish, the implementation of the message passing on AMD’s cpus is based on cache lines being routed via HT. In other words, hardware accelerated message passing. They isolated the transport mechanism from the message passing API, so I believe they could swap in different accelerated transport implementations depending on the architecture it’s currently running on.
I’d assume with the multiple kernels there would be a thin virtualisation layer sitting on top which makes the ‘cluster’ of kernels appear as a single image with the same sort of result when a single machine goes off line there is maybe a momentary stall as either there is a retry of the processing or issuing a failure notice to the end users – the preferable scenario being the former rather than the later.
Well, it is the same argument I remember having about Micro-kernels and the apparent slowness. Although every desires the absolute maximum performance I would sooner sacrifice some speed and have slightly more overhead if the net result is a more stable and secure operating system. Internet Explorer 8 for example has a higher over head because of process isolation and separation but it is a very small price to pay if the net result is a more stable and secure piece of software.
It therefore concerns me when I hear on osnews.com the number of people who decry an increase in specifications off the back of improved security and stability (with a slight performance penalty). Hopefully if such an idea were to get off the ground there wouldn’t be a similar backlash because the last thing I want to see is yet another technology that comes in half baked simply to keep the ricers happy that their system is ‘teh max speed’. They did it with Windows when moving the whole graphics layer into the kernel, I hope that the same compromise isn’t made when it comes to delivering this idea to the real world.
Great to see that MS is supporting OS research is such a big way, and nice to see the source shared with a BSD-style licence.
I applaud the efforts of these researchers, this looks really creative. Without real research, computers are never going to get to be anything more than they are now. Releasing code (no matter the license) furthers the validity of their work.
There is nothing interesting about this OS. It’s just a distributed, message-passing OS. Whup-dee-do. We’ve had those for decades. Microsoft, Intel, AMD, Nvidia and the others should be put to shame for coming out with a bunch of cretinous, non-innovative ideas on how to solve the parallel programming crisis. Multithreading is an abomination. The universe is a parallel system and it uses no threads. The only way to solve the crisis is by getting rid of threads altogether.
How to Solve the Parallel Programming Crisis:
http://rebelscience.blogspot.com/2008/07/how-to-solve-parallel-prog…
What’s the point in wasting money on research?
Microsoft know very well, and we know very well, that even in 2050 when they will release Windows 20, it will be a bloated monster which primary goal will be compatibility with Windows 3.1.
They had their chances to break with the past and introduce new and bright operating systems. And what have they done? They released Windows after Windows.
They even abandoned better OS-es like Xenix and OS/2 out of love for their “precious”.
*blink*
Wait a sec, I don’t use Windows anymore, so someone please tell me, can you actually run a Win3.1 app today on Win7? Back when I was using XP, I had a number of older Win’98 games that wouldn’t work no matter what I did, so it seems MS has been breaking compatibility all along whenever they wanted/needed to.
If Win20 is a bloated monster, I doubt it will be because of compatibility, more likely it’ll be due to the size and complexity of that monstrosity to be known as dotNET15.
Yes, as long as you are using 32 bit version. The 64 bit version cannot run 16 bit software due to processor architecture limitations.
Try runnung Java of the time, and you’ll see that .Net 15 is not that slow after all.
Yeah, who needs this stupid thing called “science” anyway?
[/q]Yeah, who needs this stupid thing called “science” anyway? [/q]
I think you mean technology – writing software isn’t science.
As I am studying “computer science” I will have to disagree with you. Writing software can be science since writing software includes creating algorithms, and algorithms are a fundamental part of computer science, no matter of the field is artificial intelligence, image analysis or database optimization.
Relax, this is most likely the view of one of purists of science definition as “Acquiring knowledge by applying Scientific Method”.
Too bad that definition leaves Math out of it, that’s why I refuse to accept such definition, but most people I know gladly accept it. So, it’s simply a matter of semantics, I guess.
Please note that I don’t mean the word “purist” in an offending way, more like describing a specific line of thought.
Programming doesn’t follow the basic tenets of science such as observation, creating and testing hypotheses or experimentation.
Algorithms are mathematics. Mathematics is not considered a science. Mathematics is a tool used in science.
It should be described as engineering but it is much closer to science than the social sciences which are pretty much left-wing politics pretending to be objective.
Sociologists for example actually take an active position in discouraging theories related to inequality that involve genetics. They’re openly biased and see nothing wrong with it.
The social sciences encourage a politically correct form of critical thinking which amounts to disregarding objectivity when it comes to sensitive subjects. They also still want to believe in tabula rasa or “blank slate” even though anyone with half a brain can see that it is wishful thinking.
Edited 2009-09-27 09:04 UTC
Have you ever coded?
Not trying to be rude, but… that’s exactly what you do when programming.
I beg to differ. Algorithmic science is very real and even if the science is more abstract than, for example, biology of physics it is a well established science with theories and observations in actual implementations.
You may not agree with me on the definition of science but I would still argue that computer science should be consider a science and hence writing software is a very fundamental part in exploring that science.
Edited 2009-09-27 16:05 UTC
64bit versions of 7 are not Win16 compatible anymore.
So about every 20 years they break compability.
Yes, but you can install Virtual XP in Windows 7, which will (probably) allow you to run your older 16 bit DOS and Win 3.1 software.
So, unfortunately they still did not completely break the compatiblity.
So they shouldn’t include virtualization? Why do you care when it doesn’t affect the performance of your system?
Windows 7 has security advantages over XP so I’m glad that MS is providing incentives for businesses to migrate.
That’s not unfortunate, that is the way it should have been done in the first place. If your OS is so bloated because it has to support older legacy applications, then using a virtual machine instead frees your resources since for the most part all you have to do is make sure you include a virtual machine to run the apps. Its what Apple did with OS9 and when OSX finally got to a useable state they dropped OS9. MS has to learn how to push 3rd party developers and customers to the latest technology without being afraid of stepping on some toes. There is no obligation for them to support somebody else old code, but yet they do.
Oeh! Oeh! I know this one!
They have an obligation to shareholders. Windows’ backwards compatibility is a major boon for a LOT of people. Ripping it out – as much as I want them to – is not a valid option, money-wise.
My comment included some dose of sarcasm. I’m not against backwards compatibility. It’s there for a good reason.
People turn on a computer to launch applications, not marvel at the OS or underlying architecture.
But I can see you are one of those people that would break Win32 compatibility and cause hundreds of billions in unneeded migration costs just to have everyone run a system that you think is neat-o.
sounds like it is a little bit similar to the gnu hurd project!
stallman, Microsoft is copying _you_ now and releasing it under an almost freesoftware license…
This is like the HURD? Are you crazy?
HURD uses a microkernel with a client-server architecture – Barrelfish is something completely different. It takes existing ideas – yes – but applies them in a completely different environment.
well, the concept is at least closely related to microkernels.
they split the kernel into this monitors.
That kind of reminds me of saying that Chevy copied Ford because they made a car, and Ford makes cars. Hurd isn’t the only microkernel you know.
Disclaimer: I don’t know much about Hurd so if there is a deeper similarity, I wouldn’t know it.
Edited 2009-09-27 04:32 UTC
Also, Barrelfish is actually usable in comparison.
“almost freesoftware license”
What do you mean with the almost? OSI and FSF call this license a (fully) free software license.
I think that they are striving to do pretty much exactly what HURD attempted many years ago. So I for one don’t think you are crazy. Their phraseology is differentm, but it is mostly word play.
Imagine a Beowulf Cluster of Hurds! (Not entirely a joke… on a couple of levels.)
Edited 2009-09-27 17:21 UTC
I only skimmed parts of it but they’re essentially saying that as your system looks more and more like a distributed system you should have an OS that acts accordingly.
They also say the existing mechanisms wont scale whereas message passing does. One you get to more than ~13 cores traditional IPC is actually slower.
> Most of us are probably aware of Singularity,
> a research operating system out of Microsoft
> Research which explored a number of new ideas,
> which is available as open source software.
The difference of the license-side between Singularity and Barrelfish is, that Singularity is open source
http://www.codeplex.com/singularity/license
but Barrelfish is OpenSource (like the OSI-definition, or free software like the FSF-definition):
http://www.barrelfish.org/release_20090914.html
From the technical point of view, it both OperatingSystems are very different and can not be compared to each other.
But from the license side, Barrelfish is a lot better! 🙂
Isn’t this in the same lineage as what DragonflyBSD tries to develop? Especially the message passing part.
Good to see that software and most of all OS R&D is still progressing. Nice collaboration between Microsoft and german students.
I find it strangely odd that Microsoft of companies is giving this away. I bet they never plan to use it in any of their products – likely they consider it a failure and are using it for PR.
Message based? slower than molasses in january – rendered on a 486 with 8mb of ram with 3DStudio Max
They’re giving it away because it’s crap. That’s why. The way to solve the parallel programming crisis is to get rid of multithreading. We must reinvent computers as we know them because the CPU was not designed with parallelism in mind. The CPU is a 150-year old idea that started with Babbage and Lovelace. It belongs in the Smithsonian, right next to the analytical engine, buggy whip and the slide rule. And the GPU is not the answer either since it’s not universal. Both CPU and GPU are doomed. It’s time to abandoned the failed computing paradigm of the baby-boomer century and forge a new computing future.
How to Solve the Parallel Programming Crisis:
http://rebelscience.blogspot.com/2008/07/how-to-solve-parallel-prog…
i’ve missed you dude, i really have
Man, what’s the deal with OSNews? They’ve been taken over by jackasses or what?
Well, since you aren’t happy with MS, you can always bitch a little about freedom of the software and pray to your God Stallman.
I guess that the site is called ‘Os news’, not ‘FSF News’.
Edited 2009-09-28 10:23 UTC
What a dumb thing to say. I hate Linux just as much as I hate every other OS out there. I hate them all, equally. There are all dinosaurs, ancient relics of a bygone age.
By the way, all of you jackasses who keep voting me down can kiss my ass. Moderation is a form of ad hominem argument, i.e., censorship by the majority opinion. I think it’s time that OSnews get rid of this moderation crap. It gives too much power to imbeciles.
To which I just had to give this a -1. It also gives the rest of us the power to mod down and disappear imbecile trolls, you know like yourself. Enjoy, have a nice day.
p.s. Was that ad hominem calling you an imbecile? How does jackass work?
Yeah, yeah. f–k you, jackass. How about that?
It helps if you think of Microsoft Research as our time’s Xerox Palo Alto Research. Lots of good ideas and interesting research that is being largely ignored by the company and not put into any of their products.
My goodness. This fits so well!
Microsoft is way better at making quality apps, programming toolkits, and computer languages. This company indeed has love-hate relationship with the OS division.
…have done this a long time ago.