Linked by Hadrien Grasland on Sun 29th May 2011 09:42 UTC
OSNews, Generic OSes It's funny how trying to have a consistent system design makes you constantly jump from one area of the designed OS to another. I initially just tried to implement interrupt handling, and now I'm cleaning up the design of an RPC-based daemon model, which will be used to implement interrupt handlers, along with most other system services. Anyway, now that I get to something I'm personally satisfied with, I wanted to ask everyone who's interested to check that design and tell me if anything in it sounds like a bad idea to them in the short or long run. That's because this is a core part of this OS' design, and I'm really not interested in core design mistakes emerging in a few years if I can fix them now. Many thanks in advance.
Thread beginning with comment 475103
To view parent comment, click here.
To read all comments associated with this story, please click here.
RE[9]: Comment by Kaj-de-Vos
by Neolander on Mon 30th May 2011 14:57 UTC in reply to "RE[8]: Comment by Kaj-de-Vos"
Neolander
Member since:
2010-03-08

Thanks you a lot, this makes it much easier to understand the concepts which you're invoking.

Some points remain obscure, though...

1/How does the type system help the switch from integer to float in the drawing system ?
2/More generally, is function overloading dealt with by the parser, or by the daemon doing the drawing work ?
3/Biggest issue I have : how is this so different from the kind of RPC which I advocate ? I mean, all in all it still looks a lot like a non-blocking RPC interface implemented on top of a messaging protocol. Even sending batches of RPC requests could be done in a "pure" RPC model, given an extra layer of abstraction that allows to send whole arrays of "RPC call" objects to the RPC subsystem.

Also...

Because we defined a command type, the data stream is self-synchronising: if an interface has consumed all the parameters it understands, it can simply skip forward to the next command.

I fail to see how letting client process send requests with an incorrect set of parameters could be a good idea.

Edited 2011-05-30 15:02 UTC

Reply Parent Score: 1

RE[10]: Comment by Kaj-de-Vos
by Kaj-de-Vos on Mon 30th May 2011 15:20 in reply to "RE[9]: Comment by Kaj-de-Vos"
Kaj-de-Vos Member since:
2010-06-09

A type system is needed if you want to support polymorphism in an interface. How else would you know what type an item is and what size it has in the encoding? With types, it's trivial for a drawing server to support both integer and floating point coordinates.

Skipping of unknown parameters and commands is useful to enable old interfaces to use some new ones. This is what web browsers do with HTML. If your browser doesn't support gradients, you'll get graphics without gradients. If you deem any interface upgrade to be incompatible, you just bump the wholesale interface version.

There are no functions, so there is no function overloading. You really have to let go of such terms. :-)

Regarding which side does what: each service has to implement a little binary parser to interpret messages sent to it.

This is really quite different from RPC, but from other posts I understand that you are confusing the concept of RPC. You're also conflating the semantic payload with the transport mechanism. I'm only concerned with the payload here. You're basically free to choose a complementary transport method.

Reply Parent Score: 1

RE[11]: Comment by Kaj-de-Vos
by Neolander on Mon 30th May 2011 15:50 in reply to "RE[10]: Comment by Kaj-de-Vos"
Neolander Member since:
2010-03-08

A type system is needed if you want to support polymorphism in an interface. How else would you know what type an item is and what size it has in the encoding? With types, it's trivial for a drawing server to support both integer and floating point coordinates.

I still don't understand. At first, the way you presented your messaging system made it sound like it had a killer feature that allowed you to fully switch from integer drawing coordinates to floating point drawing coordinates without having to recompile old code. Were you just advocating the ability to do polymorphism ?

Skipping of unknown parameters and commands is useful to enable old interfaces to use some new ones. This is what web browsers do with HTML. If your browser doesn't support gradients, you'll get graphics without gradients. If you deem any interface upgrade to be incompatible, you just bump the wholesale interface version.

Hold on. Skipping of unknown commands I can understand. On the other hand, skipping of unknown parameters isn't so easy to do, at least the way you presented your messaging system above.

Let's imagine you had a command for drawing colored lines in the spirit of "line color_code x y" where color_code, x, and y are integers.

Then later you decide that putting color support in the line drawing function is a mistake, and prefer to go with a more classical brush system. You hence define your new line-drawing command to be "line x y".

If you try to run legacy code this way, no warning will ever occur, but the color code will be understood as an x coordinate and the x coordinate will be understood as an y coordinate, so problems will occur unless the version number is bumped.

Only, only if the dropped parameter was at the end of the parameter list, will the command execute without problem.

Is it worth it ?

There are no functions, so there is no function overloading. You really have to let go of such terms. :-)

Meh ;) Commands are functions without parentheses and commas, the way they were done in early programming languages, why is this distinction so important ? It's all syntax, the concept is the same...

Regarding which side does what: each service has to implement a little binary parser to interpret messages sent to it.

This is really quite different from RPC, but from other posts I understand that you are confusing the concept of RPC.

Indeed. I call the mechanism which I advocate nonblocking/asynchronous RPC, but the ongoing discussion with Brendan implies that the name may be inappropriate. In case someone knows a better name for what I'm advocating, I'm all for it. Otherwise, I'll have to try to find a new name for it.

Again, the principle which I'm advocating is as follows :
1/Daemon provides the kernel with a prototype of a function which can be "remotely" run by client processes, corresponding to a service which it can provide.
2/Client provides the kernel with a prototype of a function which it wants to "remotely" run. Kernel checks that the service is available, and optionally does some black magic to prepare communication between different compatible versions of the same service if required, then says that everything is okay.
3/At some point, client wants daemon to perform the action it has publicly advertised to be up to. So it performs something similar to a procedure call, except that it's one of the daemon's functions that is run and that the client is not blocked while the code is executed : it has just sent a service request.
4/If an operation completion notice or results are required, they are sent to the client in the form of a callback : a function specified in advance by the client is run on the client, and results are transmitted through this function's parameters.

Again, if this IPC mechanism has a name, I'm interested.

You're also conflating the semantic payload with the transport mechanism. I'm only concerned with the payload here. You're basically free to choose a complementary transport method.

Where am I confusing those ?

Edited 2011-05-30 15:58 UTC

Reply Parent Score: 1

RE[10]: Comment by Kaj-de-Vos
by xiaokj on Mon 30th May 2011 17:27 in reply to "RE[9]: Comment by Kaj-de-Vos"
xiaokj Member since:
2005-06-30

Let me help here too!

First of all, let's deal with the earlier question. You asked something along the lines of "why bother with this when we can just design the RPC sensibly in the first place?" Well, the answer is that this *is* the sensible way out. It is inevitable that you will need to incorporate some fundamental changes somewhere down the road, why not do it properly in the first place? Also, you can simply make an optimising parser -- given that it would not change so often, the parser can run slow (this can be something like mkinitrd). If the filesystem supports notification, then that can be used to auto-invoke the parser per alteration. This ensures that we can actually not get much of a performance hit.

Now, for the specific questions,
1) There is no type system! Okay, it does look like one, but it actually is just regular data written in a specific way. The great thing is that it can be parsed by an simple program and the outcome can instantaneously migrate the system from integer to floating point calculations.

2) This depends on the choice of the implementer. If infraction is known rare, then it should be sensible to make a compromise -- the standard case is done by the drawing primitive, and the edge cases can be done by an external parser generated by the optimising parser of the data spec sheet. This ensures performance with no problems in compatibility.

3) This interface is a lot more flexible! Different OSes can just pass around the spec sheet and everybody can interoperate without difficulty (even understand each other's binary blobs; bad idea, I know, but still). Changes can be made at whim and most importantly, you are no longer hard-wiring code; you are able to just modify plain old data, which is a lot more malleable than code, surely!

Okay. Now to the last part. Maybe processes will have it less, but programs, in general, should not obnoxiously assume that they are free to mangle whatever they have been given. If there are parts they do not understand, barfing may actually destroy the critical debugging information. A program that keeps silent of the unknown (barfing only upon stuff it knows is bad) is actually desirable: it is capable of being combined with others!

Take the Troff package for example: The datastream actually includes more than just roff data, it includes eqn and tbl for example. When eqn does not understand the tbl input it receives, it just keeps quiet, knowing that something down the chain will understand it. Of course, it does barf when it is given nonsense in its own area of expertise.

Also, the example given above is only one part of the entire philosophy here. The ascii program's example is one of the more amazing ones I have seen: Instead of generating the entire program's output from scratch, the original author had realised that the whole table, precomputed, is actually better to work with.

Neolander, please try to read the Art of Unix Programming before we can actually continue with the discussion. There is a lot from there permeating this discussion.

Reply Parent Score: 2

RE[11]: Comment by Kaj-de-Vos
by Neolander on Mon 30th May 2011 18:59 in reply to "RE[10]: Comment by Kaj-de-Vos"
Neolander Member since:
2010-03-08

First of all, let's deal with the earlier question. You asked something along the lines of "why bother with this when we can just design the RPC sensibly in the first place?" Well, the answer is that this *is* the sensible way out. It is inevitable that you will need to incorporate some fundamental changes somewhere down the road, why not do it properly in the first place?

This sounds a bit religious. Why should there be a single sensible way out ? Isn't there supposed to be several sensible ways out depending on which design criteria you have, which things matter more or less to you ? I personally believe in a "the right tool for the right job" approach, and try to learn the benefits and drawbacks of each approach before deciding which one is best for my design criteria, while other people with other use criteria will probably make other choices.

Also, you can simply make an optimising parser -- given that it would not change so often, the parser can run slow (this can be something like mkinitrd). If the filesystem supports notification, then that can be used to auto-invoke the parser per alteration. This ensures that we can actually not get much of a performance hit.

Wait a minute, why are you talking about file systems already ? This is interprocess communication, were are in the early stages of a microkernel's boot and the filesystem is still far from being available yet.

Now, for the specific questions,
1) There is no type system! Okay, it does look like one, but it actually is just regular data written in a specific way. The great thing is that it can be parsed by an simple program and the outcome can instantaneously migrate the system from integer to floating point calculations.

Hmmm... You should check Kaj's post, he totally mentions a type system in there (that's why the first integer number in his command parameters is here for), that separates integer and floating point data.

3) This interface is a lot more flexible! Different OSes can just pass around the spec sheet and everybody can interoperate without difficulty (even understand each other's binary blobs; bad idea, I know, but still).

This is desirable for a number of scenarios, but I'd like to point out that I'm currently talking about a system API which does not have "being portable to other OSs without issue" as it goals. Besides, I don't see why basing my OS on a declarative data paradigm would offer better interoperability with anything but OSs based on declarative data paradigms (like Syllabe). How would such a paradigm make interoperability with, say, an UNIX system where everything is about text files and data pipes, easier ?

Changes can be made at whim

Define "changes". I don't think that you can suddenly decide tomorrow that's malloc's integer parameter should be a number of megabytes instead of a number of bytes and have all programs based on the old implementation work without recompiling them. Some changes may be easier, sure, but you'd have to precise which and why.

and most importantly, you are no longer hard-wiring code; you are able to just modify plain old data, which is a lot more malleable than code, surely!

That plain old data defines an interface to a chunk of code. If it does not include code, and if the code is not modified, then I can't see how anything but cosmetic changes can be made without breaking compatibility between the interface and the underlying code (which would sure be a bad idea).

Neolander, please try to read the Art of Unix Programming before we can actually continue with the discussion. There is a lot from there permeating this discussion.

I'm currently in the process of doing it. Didn't know that it was freely accessible on the web when it was mentioned before.

Edited 2011-05-30 19:00 UTC

Reply Parent Score: 1