Linked by Hadrien Grasland on Sun 29th May 2011 09:42 UTC
OSNews, Generic OSes It's funny how trying to have a consistent system design makes you constantly jump from one area of the designed OS to another. I initially just tried to implement interrupt handling, and now I'm cleaning up the design of an RPC-based daemon model, which will be used to implement interrupt handlers, along with most other system services. Anyway, now that I get to something I'm personally satisfied with, I wanted to ask everyone who's interested to check that design and tell me if anything in it sounds like a bad idea to them in the short or long run. That's because this is a core part of this OS' design, and I'm really not interested in core design mistakes emerging in a few years if I can fix them now. Many thanks in advance.
Thread beginning with comment 475373
To view parent comment, click here.
To read all comments associated with this story, please click here.
RE[16]: Comment by Kaj-de-Vos
by xiaokj on Tue 31st May 2011 23:05 UTC in reply to "RE[15]: Comment by Kaj-de-Vos"
xiaokj
Member since:
2005-06-30

More generally, I have been for some time under the impression that in this topic, we are talking about very similar things with a very similar set of advantages and drawbacks even though we don't know it yet.

*laughs*

Same here. |0|
(in-joke, somebody read that as lol)

I broadcast a version number ALONG WITH a prototype, doesn't the whole thing qualify as a spec sheet ?

Still too crude, since you can have minor behavioural changes. Mathematica is one of such examples -- each update, it will tweak some commands a little bit, and even though the parameters are the exact same, the functionality may be drastically changed such that automated updating is impossible.

Version numbers do little, especially if you run a gap of a few version numbers, depending upon the scale of the problem (determined mainly by the coder, really).

I am really more interested in compatible breakage -- for example, a previously provided functionality A is now replaced by B and C whereby most cases go for B, and some go to C under some conditions. If automation can still be of use, I do not see why the original code needs to be recompiled -- the slight performance hit should be okay for most. Even after a few more rounds, I see no problem. It really should be the translator's job, to me (the translator will kill me, hehe).

Come to think of it, this is really abuse of translation. Some things just cannot be handled that way. For example, the old C string format had been changed drastically because of massive security holes. Such that, we realised, that one of the tokens is completely dangerous and it is no longer even allowed (let alone support)! Most new implementations will just "politely segfault" if you tried to use it. (I'm talking about the one that outputs the number of bytes written into a memory address thing). I don't know how the translator should handle this: Should it barf (as is currently done), or should it silently drop the message? Or something in between? This is a huge thing to judge, because of the myriad implications.

Sigh.

Reply Parent Score: 2

Neolander Member since:
2010-03-08

Still too crude, since you can have minor behavioural changes. Mathematica is one of such examples -- each update, it will tweak some commands a little bit, and even though the parameters are the exact same, the functionality may be drastically changed such that automated updating is impossible.

In that case, compatibility with the old code is broken, so the "breaking" version number should be incremented.

In fact, I'm not sure that a secondary "compatible" version number would be needed. If the new prototype is compatible with the old one, then one should just leave it be as if it was the old version.

Version numbers do little, especially if you run a gap of a few version numbers, depending upon the scale of the problem (determined mainly by the coder, really).

Why ? They help old code not to break as long as their API version is still supported.

I am really more interested in compatible breakage -- for example, a previously provided functionality A is now replaced by B and C whereby most cases go for B, and some go to C under some conditions. If automation can still be of use, I do not see why the original code needs to be recompiled -- the slight performance hit should be okay for most. Even after a few more rounds, I see no problem. It really should be the translator's job, to me (the translator will kill me, hehe).

Wow. Now this starts to make the translator very complicated.

Myself, I think that if the server process breaks compatibility in a way as extreme as changing the role of function parameters, it is responsible for it and it is its job, not the translator's one, to manage the transition to the new API version. If a recompile would not be sufficient to make old code work with the new API version, then we are outside of the translator's area of expertise.

As an example, one could imagine handling your scenario this way :

-Before, server process broadcasted A, version 1
-Now, server process broadcasts B and C, version x, and A version 1
-The implementation of A version 1 provided by the server is in fact just a placeholder one, that calls in turn B and C depending on the situation.

Come to think of it, this is really abuse of translation. Some things just cannot be handled that way. For example, the old C string format had been changed drastically because of massive security holes. Such that, we realised, that one of the tokens is completely dangerous and it is no longer even allowed (let alone support)! Most new implementations will just "politely segfault" if you tried to use it. (I'm talking about the one that outputs the number of bytes written into a memory address thing). I don't know how the translator should handle this: Should it barf (as is currently done), or should it silently drop the message? Or something in between? This is a huge thing to judge, because of the myriad implications.

Sigh.

What you seem to imply is that it is not the implementation of the function that was faulty and compromised system security, but that it was broken by design and that the API function should never have existed in the first place.

We agree that in such a case, keeping a compatible placeholder implementation is impossible, so when receiving such a call there are two options : either ignore it and return a false result or send some form of "illegal instruction" signal that may crash the software if it didn't handle it.

I think what should be done should be evaluated on a per-case basis.

In the case of a function that computes a string's length, as an example, a correct result is needed by the caller and it's impossible to simply ignore the call, so the server should report the function as not supported and let the client face the consequences if it tries to use it anyway.

On the other hand, if we consider, say, a function that changes brush size in a 2D drawing API, it's possible to keep the previous brush size. Broken visual appearance will result, but at least the software will be able to run.

The question is : is what the function does critical to the operation of software which use it ? And I think this should be decided on a per-function basis. Barfing has the advantage that you don't have to ask yourself the question and carefully consider every use case of the dropped API function : it's not supported anymore. That's it. Use an old release of the OS in a VM if you want to use it anyway.

Edited 2011-06-01 09:24 UTC

Reply Parent Score: 1

RE[18]: Comment by Kaj-de-Vos
by Alfman on Wed 1st Jun 2011 10:33 in reply to "RE[17]: Comment by Kaj-de-Vos"
Alfman Member since:
2011-01-28

Neolander,

"In the case of a function that computes a string's length, as an example"

But strlen is considered a "safe" function call.

Is the example just an oversight?


The functions which are considered unsafe are those like gets(), where data is written to unspecified memory bounds.

gets, fgets // never safe.
sprintf // may be safe in controlled contexts


In any case, would it even be possible to craft such functions in an RPC scenario? We're talking about separate address spaces after all, shouldn't the RPC mapping protect the address spaces?

Edited 2011-06-01 10:42 UTC

Reply Parent Score: 2

RE[18]: Comment by Kaj-de-Vos
by xiaokj on Wed 1st Jun 2011 11:23 in reply to "RE[17]: Comment by Kaj-de-Vos"
xiaokj Member since:
2005-06-30

As an example, one could imagine handling your scenario this way :

-Before, server process broadcasted A, version 1
-Now, server process broadcasts B and C, version x, and A version 1
-The implementation of A version 1 provided by the server is in fact just a placeholder one, that calls in turn B and C depending on the situation.

Hmm. Actually, my whole point is that this method of doing things make the server full of compatibility kludges. This is not a good idea in the long run. Compatibility causes the codebase to be slower and longer and more difficult to maintain. If you implemented it as part of the translator, you have stuff working, but incur a translation overhead. This is a good thing by design! This forms a part of the "inflation" that forces the entire ecosystem to contribute.

My idea is that, instead of piling up the precious codebase with fixes, each release should come with a small translator (it may be the same for each release, if you have time to code a general one, or many small wrappers, one for each release). This would help the code be maintainable.

"Still too crude, since you can have minor behavioural changes. Mathematica is one of such examples -- each update, it will tweak some commands a little bit, and even though the parameters are the exact same, the functionality may be drastically changed such that automated updating is impossible.

In that case, compatibility with the old code is broken, so the "breaking" version number should be incremented.

In fact, I'm not sure that a secondary "compatible" version number would be needed. If the new prototype is compatible with the old one, then one should just leave it be as if it was the old version.

Version numbers do little, especially if you run a gap of a few version numbers, depending upon the scale of the problem (determined mainly by the coder, really).

Why ? They help old code not to break as long as their API version is still supported.
"
Now that my intention is made clearer from the one above, you should catch that I think the granularity of version numbers is too coarse. I think that it is healthy for codebases to break compatibility, minor or major, once in a while. But, it also means that the minor breaking version number will increment like crazy. This, it may as well be. However, I doubt that is going to tell you anything more than "Oh, there is a breakage". This is going to be just like the original Unix codebase -- problem areas, potential or not, is just marked with a "?", without any explanations whatsoever, and the reader (hence maintainer/developer) needs to know exactly what is the problem from there. A much more sensible approach would be to have comments telling you what is wrong, and an even better one, transition plan. If you provided information about the transition in the form of spec sheet, then I think the transition would turn out to be a lot smoother than "needs recompile now", or "API is broken; get new version immediately". Not to mention that sometimes, you just want the older version for the removed features.

As per usual, I agree with the rest; case-by-case analysis tend to give better results too. I am just wary of making code be obnoxious to the users. I refer to the RPM4/5/6 ugliness that is just mind blowing. (intentional internal inconsistency is upheld? What nonsensical obnoxious behaviour!)

Reply Parent Score: 2