Linked by Hadrien Grasland on Sat 5th Feb 2011 10:59 UTC
OSNews, Generic OSes So you have taken the test and you think you are ready to get started with OS development? At this point, many OS-deving hobbyists are tempted to go looking for a simple step-by-step tutorial which would guide them into making a binary boot, do some text I/O, and other "simple" stuff. The implicit plan is more or less as follow: any time they'll think about something which in their opinion would be cool to implement, they'll implement it. Gradually, feature after feature, their OS would supposedly build up, slowly getting superior to anything out there. This is, in my opinion, not the best way to get somewhere (if getting somewhere is your goal). In this article, I'll try to explain why, and what you should be doing at this stage instead in my opinion.
Thread beginning with comment 461121
To view parent comment, click here.
To read all comments associated with this story, please click here.
RE[3]: Not always rational
by Alfman on Sun 6th Feb 2011 11:17 UTC in reply to "RE[2]: Not always rational"
Alfman
Member since:
2011-01-28

"Microkernels offers stability at the expense of performance...There are lots of places where these characteristics are worth the loss performance, but the desktop isn't one of them."

I'd say the stability problems such as corruption and overflow stem more from the choice of highly "unsafe" languages rather than choice of micro-kernel/macro-kernel.


You're argument in favor of a macrokernel in order to achieve performance is somewhat dependent on the assumption that a microkernel cannot perform well. However I think there are various things to boast the performance of a microkernel design.


The microkernel does not have to imply expensive IPC. If modules are linked together at run or compile time, they could run together within a privileged cpu ring to boast performance.

As for stability and module isolation, there are a few things we can try:

1. Code could be written in a type safe language under a VM such as Java or Mono. The calls for IPC could be implemented by exchanging data pointers between VMs sharing a common heap or memory space without changing CPU rings. Individual models would never step on each other despite existing in the same memory space.

Not only is this approach plausible, I think it's realistic given the current performance and transparency of JIT compilers.

2. Segmentation has been declared a legacy feature in favor of flat memory models, but hypothetically memory segmentation could provide isolation among microkernel modules while eliminating the need for expensive IPC.

3. User mode CPU protections may not be necessary if the compiler can generate binary modules which are inherently isolated even though running in the same memory space. Therefor, the compiler rather than the CPU would be enforcing module isolation.




As much as people hated my opinion on up front performance analysis, I'd say this is an instance where the module inter-communications interface should be performance tested up front. Obviously, as more of the kernel modules get built, this will be very difficult to change later on when we notice an efficiency problem.

Reply Parent Score: 1

RE[4]: Not always rational
by Neolander on Sun 6th Feb 2011 14:11 in reply to "RE[3]: Not always rational"
Neolander Member since:
2010-03-08

I'd say the stability problems such as corruption and overflow stem more from the choice of highly "unsafe" languages rather than choice of micro-kernel/macro-kernel.

I've spent hours arguing on this precise subject with moondevil, I won't start over. In short, I'll believe that it's possible to write a decent desktop OS in a "safe" language when I see it.

In meantime, microkernels offer the advantage of reducing much the impact of failures and exploits., when there are some. A buggy process can only have the impact it's authorized to have.

You're argument in favor of a macrokernel in order to achieve performance is somewhat dependent on the assumption that a microkernel cannot perform well. However I think there are various things to boast the performance of a microkernel design.

That's not what I said. My take on the subject is that microkernels can obviously not have the same performance as a macrokernel (some optimization is only possible when kernel components share a common address space), but that they can have sufficient performance for desktop use.

The microkernel does not have to imply expensive IPC. If modules are linked together at run or compile time, they could run together within a privileged cpu ring to boast performance.

Then you do not have a microkernel, but a modular monolithic kernel. Putting components in separate processes is afaik a defining characteristic of microkernels.

As for stability and module isolation, there are a few things we can try:

1. Code could be written in a type safe language under a VM such as Java or Mono. The calls for IPC could be implemented by exchanging data pointers between VMs sharing a common heap or memory space without changing CPU rings. Individual models would never step on each other despite existing in the same memory space.

Not only is this approach plausible, I think it's realistic given the current performance and transparency of JIT compilers.

As said before, I'll believe it when I see it.

Note that microkernels are not incompatible with shared memory regions between processes, though. It's one of the niceties which paging permits. In fact, I believe that they are the key to fast IPC.

2. Segmentation has been declared a legacy feature in favor of flat memory models, but hypothetically memory segmentation could provide isolation among microkernel modules while eliminating the need for expensive IPC.

Segmentation is disabled in AMD64 and non-existent in most non-x86 architectures, so I'm not sure it has much of a future. Besides... How would you want to use it ? If you prevent each process from peeking in other process' address space, then they need IPC to communicate with each other. But perhaps you had something more subtle in mind ?

3. User mode CPU protections may not be necessary if the compiler can generate binary modules which are inherently isolated even though running in the same memory space. Therefor, the compiler rather than the CPU would be enforcing module isolation.

But then hand-crafted machine code and code from other compilers than yours could bypass system security... Unless you would forbid those ?

As much as people hated my opinion on up front performance analysis, I'd say this is an instance where the module inter-communications interface should be performance tested up front. Obviously, as more of the kernel modules get built, this will be very difficult to change later on when we notice an efficiency problem.

It is possible to stress-test inter-module/process communication after implementing it and before implementing modules, or even while implementing it. The problem is to determine what is good enough performance at this early stage. Better make code as flexible as possible.

Edited 2011-02-06 14:15 UTC

Reply Parent Score: 1

RE[5]: Not always rational
by Alfman on Sun 6th Feb 2011 23:10 in reply to "RE[4]: Not always rational"
Alfman Member since:
2011-01-28

"That's not what I said."

Sorry, I responded to your post quoting something which was from someone else.

"I've spent hours arguing on this precise subject with moondevil, I won't start over."

Fair enough, but it's not really adequate to dismiss my argument, there isn't even a citation.

"As said before, I'll believe it when I see it."

It doesn't exist yet, therefor you don't believe it could exist?

Neolander, I appreciate your view, but I cannot let you get away with that type of reasoning.

All of today's (major) kernels predate the advent of efficient VMs. With some original out of the box thinking, plus the benefit of the technological progress in the field in the past 15 years, a type safe efficient kernel is not far-fetched at all.

Per usual, the main impediments are political and financial rather than technological.


"Segmentation is disabled in AMD64 and non-existent in most non-x86 architectures, so I'm not sure it has much of a future."

That's exactly what I meant when I called it a legacy feature. However, conceivably the feature might not have been dropped if we had popular microkernels around using it.


"But then hand-crafted machine code and code from other compilers than yours could bypass system security... Unless you would forbid those ?"

You need to either trust your binaries are not malicious, or validate them for compliance somehow.
If we're running malicious kernel modules which are never the less "in spec", then there's not much any kernel can do. In any case, this is not a reason to dismiss a microkernel.

"It is possible to stress-test inter-module/process communication after implementing it and before implementing modules, or even while implementing it."

I am glad we agree here.

Reply Parent Score: 1

RE[5]: Not always rational
by Kochise on Mon 7th Feb 2011 19:42 in reply to "RE[4]: Not always rational"
Kochise Member since:
2006-03-03

"In short, I'll believe that it's possible to write a decent desktop OS in a "safe" language when I see it."

http://programatica.cs.pdx.edu/House/
http://web.cecs.pdx.edu/~kennyg/house/

Now bend down and praise the Lords...

Kochise

Reply Parent Score: 2

RE[4]: Not always rational
by Morin on Mon 7th Feb 2011 08:58 in reply to "RE[3]: Not always rational"
Morin Member since:
2005-12-31

1. Code could be written in a type safe language under a VM such as Java or Mono. The calls for IPC could be implemented by exchanging data pointers between VMs sharing a common heap or memory space without changing CPU rings.


I used to consider this a plausible approach, too. However, any shared-memory approach will make the RAM a bottleneck. It would also enforce a single shared RAM by definition.

This made me consider isolated processes and message passing again, with shared RAM to boost performance but avoiding excessive IPC whenever possible. One of the concepts I think is useful for that is uploading (bytecode) scripts into server processes. This avoids needless IPC round-trips and even allows server processes to handle events like keypresses in client-supplied scripts instead of IPC-ing to the client, avoiding round-trips and thus be more responsive.

The idea isn't new, though. SQL does this with complex expressions and stored procedures. X11 and OpenGL do this with display lists. Web sites do this with Javascript. Windows 7 does it to a certain extent with retained-mode draing in WPF. There just doesn't seem to be an OS that does it everywhere, presumably using some kind of configurable bytecode interopreter to enable client script support in server processes in a generalized way.

Example: a GUI server process would know about the widget tree of a process and has client scripts installed like "on key press: ignore if the key is (...). for TAB, cycle the GUI focus. On ESC, close window (window reference). On ENTER, run input validation (validation constraints), and send the client process an IPC message is successful. (...)"

There you have a lot of highly responsive application-specific code, running in the server process and sending the client an IPC message only if absolutely needed, while still being "safe" due to being interpreted and any action checked.

2. Segmentation has been declared a legacy feature in favor of flat memory models, but hypothetically memory segmentation could provide isolation among microkernel modules while eliminating the need for expensive IPC.


That would be a more elegant way to do the same as could be done with paging. On 64-bit CPUs the discussion becomes moot anyway. Those can emulate segments by using subranges of the address space; virtual address space is so abundant that you can afford it. The only thing you don't get with that is implicit bounds checking, but you still can't access memory locations which the process cannot access anyway.

3. User mode CPU protections may not be necessary if the compiler can generate binary modules which are inherently isolated even though running in the same memory space.


If used for "real" programs, this argument is the same as using a JVM or .NET runtime.

On the other hand, if you allow interpreted as well as compiled programs, and run them in the context of a server process, you get my scripting approach.

Reply Parent Score: 2

RE[5]: Not always rational
by Alfman on Mon 7th Feb 2011 16:17 in reply to "RE[4]: Not always rational"
Alfman Member since:
2011-01-28

Morin,

"I used to consider this a plausible approach, too. However, any shared-memory approach will make the RAM a bottleneck. It would also enforce a single shared RAM by definition."

That's a fair criticism - the shared ram and cache coherency model used by x86 systems is fundamentally unscalable. However, considering that shared memory is the only form of IPC possible on multicore x86 processors, we can't really view it as a weakness of the OS.

"This made me consider isolated processes and message passing again, with shared RAM to boost performance but avoiding excessive IPC whenever possible. One of the concepts I think is useful for that is uploading (bytecode) scripts into server processes."

I like that idea a lot, especially because it could be used across computers on a network without any shared memory.

Further still, if we had a language capability which could extract and submit the logic surrounding web service calls instead of submitting web service calls individually, that would be a killer feature of these "bytecodes".

"That would be a more elegant way to do the same as could be done with paging. On 64-bit CPUs the discussion becomes moot anyway."

See my other post as to why this isn't so if we're not using a VM for isolation, but your conclusion is correct.

Reply Parent Score: 1