Linked by Hadrien Grasland on Sun 29th May 2011 09:42 UTC
OSNews, Generic OSes It's funny how trying to have a consistent system design makes you constantly jump from one area of the designed OS to another. I initially just tried to implement interrupt handling, and now I'm cleaning up the design of an RPC-based daemon model, which will be used to implement interrupt handlers, along with most other system services. Anyway, now that I get to something I'm personally satisfied with, I wanted to ask everyone who's interested to check that design and tell me if anything in it sounds like a bad idea to them in the short or long run. That's because this is a core part of this OS' design, and I'm really not interested in core design mistakes emerging in a few years if I can fix them now. Many thanks in advance.
Thread beginning with comment 475152
To view parent comment, click here.
To read all comments associated with this story, please click here.
RE[12]: Comment by Kaj-de-Vos
by xiaokj on Mon 30th May 2011 19:52 UTC in reply to "RE[11]: Comment by Kaj-de-Vos"
xiaokj
Member since:
2005-06-30

This sounds a bit religious. Why should there be a single sensible way out ?

Yes it does sound so. Which is why I was thinking it would be rather heavy-handed to decide to do it at the systems-API level. But your own point about RPC is also as religious, no? ;-)

Wait a minute, why are you talking about file systems already ? This is interprocess communication, were are in the early stages of a microkernel's boot and the filesystem is still far from being available yet.

Read that again! This is clearly too early to do, but is part of the final design. The initrd created by mkinitrd is fixed and is loaded into the kernel at boot, while the rest of the filesystem is still dead. If you have no time to do this, you are certainly free to just leave it be.

Hmmm... You should check Kaj's post, he totally mentions a type system in there (that's why the first integer number in his command parameters is here for), that separates integer and floating point data.

This sounds Zen, but as you focus on the type system, I am focusing on the fact that this "type system" you are talking about lives as plain text arranged in readable and parse-able format. Of course, in the end, you will be implementing a type system, but the difference is huge! You will be able to reconfigure with impunity as it has been shown.

This is desirable for a number of scenarios, but I'd like to point out that I'm currently talking about a system API which does not have "being portable to other OSs without issue" as it goals. Besides, I don't see why basing my OS on a declarative data paradigm would offer better interoperability with anything but OSs based on declarative data paradigms (like Syllabe). How would such a paradigm make interoperability with, say, an UNIX system where everything is about text files and data pipes, easier ?

a) Portable also means portable in time. Within your own OS, if you had taken the time to write it in this manner, you may find it much easier to keep things sane (not breaking every few moments you want to change).

b) If you want to port against Unix, for example, you can make the parser generate an API translator, I suppose. Sadly, almost every API call will incur one translation overhead (or actually maybe less), but at least the code will work out of the box.

Define "changes". I don't think that you can suddenly decide tomorrow that's malloc's integer parameter should be a number of megabytes instead of a number of bytes and have all programs based on the old implementation work without recompiling them. Some changes may be easier, sure, but you'd have to precise which and why.

I actually myself advocate some compromise -- it is clear you won't just change malloc so easily, so there is little reason to do it as declarative. However, even there, you can see obvious improvements. If you had done it as declarative, then there is no reason why you cannot immediately change malloc to accepting size_t only, and have the parser convert old calls, using integer, into size_t in a seamless way. In this way, you can see how the information provided can be utilised to minimise your headache. Also, because of versioning, you can do abrupt breaks without trouble as long as you have seamless conversion between transitions. Also, once versioning is done, it also means you can provide function overloading (in versions, not parameters, this time), and then you can look back into your codebase and select functions still using the old version, and slowly eliminate them. This process tends to create multi-versioning disasters, but at least the system as a whole could continue working instead of dying right out. It also means that you can employ temporary fixes as you go along, which is definitely powerful.

That plain old data defines an interface to a chunk of code. If it does not include code, and if the code is not modified, then I can't see how anything but cosmetic changes can be made without breaking compatibility between the interface and the underlying code (which would sure be a bad idea).

It should not include code, and even if it does modify behaviour, it should not include code "proper". Data Tables yes, precomputed values of clear reason yes, but code, no. The aim is not to make it so general that you code an OS within an OS (bad idea always). The aim is to make machine parsing and human readability help you.

I kinda think of this as if you actually create the tools to manage your codebase a little like how you would use Wordpress with your blog. Clearly, it should not interfere with whatever genius you want to do, but it also should not be such that you find yourself hardcoding the API (html). Of course, if you find yourself spending a lot of time on the engine, that is bad too. It is a lot of balancing, of which I doubt there is anything other than raw experience that you can learn from.

I can only take comfort in the fact that I have successfully started another soul on the AoUP. That is a true jewel. Of course, there are others, like the opposite tale of Unix Hater's Handbook, and so on. Computing is HARD! So much balancing to do! :-)

Reply Parent Score: 2

RE[13]: Comment by Kaj-de-Vos
by Neolander on Tue 31st May 2011 06:59 in reply to "RE[12]: Comment by Kaj-de-Vos"
Neolander Member since:
2010-03-08

Yes it does sound so. Which is why I was thinking it would be rather heavy-handed to decide to do it at the systems-API level. But your own point about RPC is also as religious, no? ;-)

Yes and no.

I've noticed over the year that while I can, like everyone else, understand others' points of view, I'm bad at trying to embrace it. I tend, on the other hand, to be more reasonable after having been beaten in an argumentative fight. So I get around these limitations of my mind by using a "hit the wall" approach : I follow the idea I've initially had, and challenge proponents of other ideas to prove it wrong, or more precisely to prove theirs better.

This works well while I can find people who know the subject well and are good at arguing, of course, but at my current level of OSdeving mastery and for common topics like the ones which I study, it's not exactly hard.

This is not religious, because I do everything I can to stay on an intellectual field (my idea is good, because...). Anyone who catches me using insufficient argumentation, avoiding a vital point of the opponent's one, or worse using any statement of the "it's simply better" kind, is invited to whip me with nettles, as Alfman has done in a past when on a tiring day I had written that threads were "simply more scalable" than asynchronous approaches.

By the way, your point about this being maybe too heavy-handed for a system API is something which I could have said myself, and have actually said somewhere in this thread I think.

Since you have led me to read a book about UNIX, I think you can understand a will on my side to have system-wide abstractions of beautiful simplicity that have a large spectrum of applications. So an IPC model that would be suitable for all API layers would be quite attractive to me.

This sounds Zen, but as you focus on the type system, I am focusing on the fact that this "type system" you are talking about lives as plain text arranged in readable and parse-able format. Of course, in the end, you will be implementing a type system, but the difference is huge! You will be able to reconfigure with impunity as it has been shown.

Doesn't code qualify as plain text arranged in a readable and parse-able format ?

What I'm trying to say is that if external text config files are not used outside of development periods, maybe it's not worth it to bother keeping them.

a) Portable also means portable in time. Within your own OS, if you had taken the time to write it in this manner, you may find it much easier to keep things sane (not breaking every few moments you want to change).

I have already heard this, but can't understand why. You seem to have answered below, so I'll look there.

b) If you want to port against Unix, for example, you can make the parser generate an API translator, I suppose. Sadly, almost every API call will incur one translation overhead (or actually maybe less), but at least the code will work out of the box.

If you have to write extra code and suffer a translation overhead anyway, how is it different than writing wrapper libraries for other OSs and languages ?

I actually myself advocate some compromise -- it is clear you won't just change malloc so easily, so there is little reason to do it as declarative.

If declarative is not universal enough that I could write things like malloc with it and require extra abstractions, that's one of its weak points ;)

However, even there, you can see obvious improvements. If you had done it as declarative, then there is no reason why you cannot immediately change malloc to accepting size_t only, and have the parser convert old calls, using integer, into size_t in a seamless way. In this way, you can see how the information provided can be utilised to minimise your headache.

Okay, so you'd have the parser itself convert old calls. That looks like what I was investigating for my own model above, but I ran into issues when trying to push it further, so it'd be interesting to see if you also have them in your declarative data model or not.

Let's picture ourselves a function that takes a class as a parameter. Between version N and version N+1, this class gains new members. One of those members is a unique integer identifying each instance of the class, generated using static variables in the constructor. If the old code was recompiled, there would be no issue at all. But here we're trying to keep binary compatibility without recompiling, isn't it ?

Question is : can you, with declarative data, transform an old instance of the class into a new instance of the class without putting inappropriate data in the "identifier" class member ? My conclusion was that it is impossible in my "sort of like RPC" model, but maybe declarative data can do the trick.

Also, because of versioning, you can do abrupt breaks without trouble as long as you have seamless conversion between transitions.

The feature of versioning can definitely be added to an RPC mechanism : at the time where prototypes are broadcasted to the kernel, the client and server processes only have to also broadcast a version number. From this point, it works like function overloading : the kernel investigates whether a compatible version of what the client is looking for is available.

Also, once versioning is done, it also means you can provide function overloading (in versions, not parameters, this time), (...) It also means that you can employ temporary fixes as you go along, which is definitely powerful.

Not sure I understand this concept, can you give more details ?

I kinda think of this as if you actually create the tools to manage your codebase a little like how you would use Wordpress with your blog. Clearly, it should not interfere with whatever genius you want to do, but it also should not be such that you find yourself hardcoding the API (html). Of course, if you find yourself spending a lot of time on the engine, that is bad too. It is a lot of balancing, of which I doubt there is anything other than raw experience that you can learn from.

Yeah ;) The good old framework problem : people who don't spend enough time on the design phase of their product end up creating a gigantic-sized arithmetic operation framework supporting every single mathematic calculation known to man just to add two numbers. The only way to avoid this is to have precise knowledge of your needs, and avoid the temptation of excessive genericity.

I can only take comfort in the fact that I have successfully started another soul on the AoUP. That is a true jewel. Of course, there are others, like the opposite tale of Unix Hater's Handbook, and so on. Computing is HARD! So much balancing to do! :-)

Heh ;) Not all computing is like this, but development of software planned to last very long, have a wide range of uses, and be deployed on a relatively large scale, like an OS, definitely is.

Reply Parent Score: 1

RE[14]: Comment by Kaj-de-Vos
by xiaokj on Tue 31st May 2011 07:19 in reply to "RE[13]: Comment by Kaj-de-Vos"
xiaokj Member since:
2005-06-30

I see most of that as quite pragmatic, so I don't think I can argue much further than already had been. However:

Question is: can you, with declarative data, transform an old instance of the class into a new instance of the class without putting inappropriate data in the "identifier" class member? My conclusion was that it is impossible in my "sort of like RPC" model, but maybe declarative data can do the trick.

You may not be able to make the old code suddenly be new, but without recompiling, you can make the old code speak in the new slang. The parser can just export glue code (as thin as possible, hopefully).

"Also, once versioning is done, it also means you can provide function overloading (in versions, not parameters, this time), (...) It also means that you can employ temporary fixes as you go along, which is definitely powerful.


Not sure I understand this concept, can you give more details ?
"
Nah, it's simple stuff. For the moment, think of a design loop: Maybe to implement something important, you found that your own design phase had an infinite loop. To implement A, you had to first implement B, which requires A. Then, what you can do is to implement proto-A and proto-B and get it over with. The mechanism can take over from there, really.

Or, if you found yourself in a temporary crisis: Something important crashed in the middle of your computing. Your debug options are in peril. Then, you may find yourself implementing temporary fixes in your codebase that you intend to remove and reconstruct later. (Something you do to just keep temporary order at the fastest moment, so that you can still get some rest.) Something like the BKL (Big Kernel Lock) Linux had.

The feature of versioning can definitely be added to an RPC mechanism : at the time where prototypes are broadcasted to the kernel, the client and server processes only have to also broadcast a version number. From this point, it works like function overloading : the kernel investigates whether a compatible version of what the client is looking for is available.

If all you have is a version number, then it is really troublesome trying to keep the details intact. Having a complete spec sheet makes interoperability easier. With a version number, then you can guarantee that the functionality is provided in a later version. But you cannot guarantee the functionality works exactly as prescribed before. Also, it means that you absolutely have to provide said functionality in future versions of the library code -- you cannot do a turnabout and remove said functionality. With a spec sheet, you can guarantee that the client code can still run, as long as it does not use the removed functionality.

Reply Parent Score: 2