As a programmer and manager of embedded software products for a living, I think that operating system programming is so much fun that it will eventually be outlawed. I've previously published two articles on OSNews, So, you want to write
an operating system and Climbing
the kernel mountain, and tried to summarize my experience in designing operating system kernels as well as technical traps that can be easily avoided.
Post a Comment
The real reason why you shouldn't write your own kernel:
Because you don't like having to write your own drivers.
Yeah... write a kernel seems too much work, but always using the same one doesn't look good either so I guess new kernels are only needed once in a while don't you all think?
Also when writing a kernel you need to use low level languages like Assembly or C and they are really hard, so I guess a new wide used kernel will show up when we are able to code one with a high level language.
Also when writing a kernel you need to use low level languages like Assembly or C and they are really hard, so I guess a new wide used kernel will show up when we are able to code one with a high level language.
C and even Assembly are not hard, its just more detail that you have to pay attention to - especially assembly and the x86 with what registers can do what, etc...
Anyway, writing something like quake I back in the day that had to run acceptably on a 486 is much more elite than programming kernels - which this guy makes out to be the holy grail.
... i will continue on my OS - which is actually more than only a custom *kernel*. Why? I already get my hands dirty with day-to-day software development, so delving into kernel and OS land is something different(tm) and intrigueing.
I 've got no hankering to reach any market with it nor will I ever attract any so called team mates to my project.
stay safe folks.
@emmanuel marty: you are correct, there is no need for doing it. But to *understand* whats going on behind the veil, it's something essential. :-) having done such a thing f. ex. gives one a huuuge training in debugging and finding logical mistakes (which a compiler usually never finds).
ok, but how about writing your own compiler and/or language?
A solid and well reasoned article. Frankly, I don't think you need to be brain surgeon to realize that reinventing the wheel is stupid.
Take a look at all the successful operating systems to date. None of them are original. All of them stole ideas, code, concepts, even kernels and kernel utilities from already existing projects.
All the operating system that tried to be original, that reivented the wheel, died. This isn't rocket science, this is undeniable logic. Use what works today, and evolve from there. Don't, in an effort to be original, innovative, different, squander your time, effort, skills and money.
Kernel gurus will be better of playing with already existing free kernels and specializing in a component they find fascinating, rather starting yet another kernel/operating system project.
I think the article misses one rather important point. Don't write your own kernel if you don't know what you're doing. If you want to learn about systems/OS programming, maybe it'd be best to learn about it properly first (take a course or something). Then you will be better prepared to actually write a kernel, and probably skip a lot of the issues raised here...
What a great, coherent, well-argued article. There's still hope for Osnews. 
Thanks Emmanuel for the article and I hope that many will hear your wise words.
You must be god's gift to humanity. I can't wait for you to come and show us your brand spanking kernel with great driver support, because we all know that all of the Linux kernel developers did not go to school or learn how to do a kernel properly.
Many many OSes are hobby systems. Hobbies scratch itches.
For some OSxxx is about ruling the world; for others that is just excusing some itch scratching ;-)
On the other hand, most hobby operating systems seem to start (and end) with a kernel. There are very few that seem to be 'systems' out there.
It seems to make sense to borrow kernels for hobby operating systems too, and build the libraries that actually constitute the 'platform' upon which applications might stand instead. It might make results come quicker. Or does that not scratch the itch vigorously enough?
You must be god's gift to humanity. I can't wait for you to come and show us your brand spanking kernel with great driver support, because we all know that all of the Linux kernel developers did not go to school or learn how to do a kernel properly.
I can't quite tell if you're trying to troll or not. What I was trying to say is that many people try to write a kernel and know little or nothing about systems programming in general. Is that a good thing? Will that mean they're going to have an easy ride? Of course not. Maybe if they learnt a bit more about systems programming in general, they might find kernel programming a little better. Is that such a bad thing?
I think a lot of the Linux or whatever programmers would be well versed in kernel and systems programming. Otherwise would they be able to commit bad code to the kernel?
it hasn't all been done before... and anyway the linux path seems to have narrowed... its branching factors limited...
there is is a need for new ideas and new ways of doing things ... but we are running into the limits of the inflexible i386 architecture...
i would even suggest amateur kernel hackers to develop thiers within a sandbox... like a JVM or similar.. to reduce the amount of less-interesting work that is aside from the issue they want to experiment with...
t
The other point completely missed:
The number of people who are capable of writing a "functional kernel", and interested in doing the same is so damn small. Most computer science grads wouldn't have a clue where to begin.
A lot of nice talk, but I think that it boils down to "it's more fun but less efficient to write new code than to modify existing one", which can often mean "it's more fun but less efficient to write new bugs than to fix existing ones".
"I think the article misses one rather important point. Don't write your own kernel if you don't know what you're doing."
Did you even read the article? I think not.
The point of the article is that if you are qualified to do kernel hacking, do not write a new kernel. There is little point in doing so and the author cogently explains why. It is not a question of whether or not you know what you are doing. Read the article, instead of just commenting nilly willy.
Me thinks you must first RFA and then speak your wisdom about
Yes, I RFA, thanks so much for asking.
I did not get that same impression you had, unfortunaely. I got the impression that if you're trying to implement some great new thing, don't reinvent the wheel just trying to achieve it (http://www.osnews.com/story.php?news_id=8162&page=3, "filesystem thingie").
I wish Linus Torvald had read that article, than there wouldn't be some much fuss about Linux these days. Sorry couldn't help it.
Why does everything in the world needs to be efficient, if someone likes to write kernels for fun, so what! A lot of inventions are made by people who were considered wasting their time, history proves that most critics where wrong in the end.
If you tie everything to an economy model, sure it will be considered inefficient but that's the problem in general of society nowadays only profit counts.
Good ideas die because of economical reasons and that's a pitty. I don't wanna bring up the Betamax vs VHS discussion here but facts prove that free market models not always favor the technical most advanced technology.
Only when a technology is being developped by someone not under pressure to generate profit will flourish.
Point is that if someone wants to write his/her own kernel and has fun doing so, he/she should do so and not think about economical impact. If Linus had done so we wouldn't have Linux now.
"Point is that if someone wants to write his/her own kernel and has fun doing so, he/she should do so and not think about economical impact. If Linus had done so we wouldn't have Linux now."
I thought I recalled that Linus developed Linux directly for the economical impact? Wasn't he upset that minix wasn't free nor free?
As an aside, I rather imagined my 'filesystem thingy' would be extending an existing OS, not an excuse to write a hobby OS to encapsulate.
When you have an OS, you make a choice in file system schematics: UNIX-like, VMS-like, MS-DOS-like, etc etc. Putting forward other ideas (Internet-like) is just scratching my itch ;-)
We do still need an good and working POSIX micro-kernel with HAL and real Drivers Model (whitch could use codebase of the linux/bsd device-modules)
Generally one could "steal" a lot of the Linux kernel (like Scheduler, Semaphores etc.)
I would like this kernel be able replacing 70'monolithic-Linux-arch for good. I wanna run my system for more then fiew weeks (until a new kernel-version is out fixing 10000 bugs ans sec. issues, it sucks really!)
When Linus began writing Linux, there wasn't a Free kernel available of the quality of Linux 2.6.
When Linus began writing Linux, the expectation of what a a kernel and the OS that sits on top were supposed to do was very different from what we expect of both today.
I think technical merit continues to drive the Linux kernel development more than any concern with profitability.
@ stew:
> The real reason why you shouldn't write your own
> kernel: Because you don't like having to write your
> own drivers.
Repetition #{I don't really know anymore}:
There are cross-platform driver architectures readily available, like SNAP or UDI. Every single OS out there could work with a shared pool of device drivers. No reinventing the wheel, no hardware compatibility lists.
But Microsoft doesn't embrace them because they would be hilariously stupid to give away one of their core advantages - drivers.
The FSF doesn't embrace them because those architectures don't fit their idea of "freedom" (or rather, because it would give away one of *their* core advantages - drivers).
@ Mystilleef:
> All the operating system that tried to be original,
> that reivented the wheel, died.
And many of them left behind new, original things that others picked up. Preemptive multitasking wasn't an invention of MS-DOS, Linux / BSD, or AppleOS.
And much of what was left behind that died has yet to be picked up by someone. Much of this has fallen into forgetfulness, with only a few people still remembering the ease of navigating a file structure that could be understood by mere mortals, adding a .catalog in your native language the application programmer didn't even know about, or backing up your OS to a ZIP drive with a simple copy operation and booting from that copy without further ado.
Windows, Linux, and MacOS suck in more than one regard, and that's the raison d'etre for many OS projects out there. But the big three are very happy with their oligarchy, and don't consider for a second of evening the playing grounds (see my driver comment above).
Either one of the big three finally gets the act together and produces an OS that's actually easy, fun, and painless to use, or alternative OS projects are here to stay.
And, like all things, OS needs to evolve.
Yes, you don't need to reinvent wheel. But if you just get the idea of plane and rocket. Please do it ! Even if all people say they prefer boat to go over water...
Do it for your own pleasure. If you think you GET an idea. Try it. That's how evolution take place.
If we stick to what we have now, if we all wait that "some thing that will come for us", then, in the end, OS dev will die. Like all things, OS dev needs evolution. Even microsoft knows this.
I think the real problem here is whether or not some project NEEDS to create a kernel.
If it is just a new FileSystem, why would you rewrite a kernel ?
You can write it on Linux, BSD, even Windows or Hurd if you prefer...
The same for UI. If you get an idea for a really new enjoying way of dealing with human-machine interaction, do it in front of GDI, GTK or QT...
Then, if you get a really new scheduler algorithm. If you know you think you can do a better I/O manager,
TRY ! TRY ! TRY !
Please help a smart project.
http://www.blueeyedos.com/
Face it the author missed a rather large point when he said lets build a new OS around this filesystem and then someone ports it to Linux with great sucess this is called INNOVATION and this is how new ideas are formed. the same feature may never become apparent or desirable in Linux until it is seen somewhere else--hobbist are how things progress after all corps and people trying to keep the enterprise customers happy waste all day making small tweaks to existing thing hobbist go out and invent something new. So I would have to say interesting read but 100% WRONG.
"Any domain of human knowledge gets commoditized over time, this is good for everyone, and the experts have to learn to adapt their expertise to a higher-level problem set"
_____________________
Nope, mathematics and physics don't fit in that commoditized knowledge, they are always evolving. I am sorry for software researches if they cannot get out of the market world.
the article reads more like
"i thought i could make millions selling a halfbaked toy os with no drivers. we had no customers. now we sell on existing stuff as a reseller with out little custom whotzit included."
the author must have been the only hobbyist os developer with his eyes closed to the reality of creating/selling an operating system.
"Either one of the big three finally gets the act together and produces an OS that's actually easy, fun, and painless to use, or alternative OS projects are here to stay."
I think most alternative projects (smaller than BSD, Linux) are about either trying less common internals like exokernels or just earning bragging rights--among other technical bits--rather than creating a friendly desktop, but I see your point. I also don't agree with it.
If you have an idea that you want to come shipped with the distribution you use (like the friendly desktop you say you want), it's best to contribue to an existing project than start a new one. It's the most practical way of getting it done.
If you have an idea that would require the kind of overhaul that a project doesn't want to go through, fork it.
If what you want is some small research OS to play with different kinds of internals (the kind of thing people don't usually put on their desktop), it might be too small a project to direct anything current to its goals. Just steal as much as you can. Reinvent as little as possible.
If you want something so different that you think you should just start from scratch, really look into an evolutionary approach. If you still decide to go anew, realize you will likely be a martar--your project will die, only to pass on its itch-scratcher in 2-15 years to projects that do take the evolutionary path.
Back in the 80's, every home computer came with it's own unique built-in OS. I used to enjoy low-level assembly coding then. Then came the PC, and DOS was everywhere. Then Windows came, and it's on about any PC now. Linux is growing, and I think it might overtake Windows (some day). And there's the BSD's, Solaris, Mac OS X etc. But: in terms of basic concepts, there's not much variation, and: all of these OS'es have their own problems and imperfections.
I search the web regularly, and look for something better. There's dozens of hobby OS'es, but all coded in C, C++ or x86 assembly, for PC-compatibles only, and roughly re-doing things already done. Projects that are REALLY innovative, are few, and these are the ones that are interesting. Many of these are semi-completed university research projects. But what I want, either doesn't exist yet, or I didn't find it yet.
Therefore I decided to start my own OS project anyway, and wrote a sort of "manifest" recently. If what I want, turns up tomorrow: great, that'll save me a lot of time. If not: maybe I really am starting something that might live inside next decade's computers.
Time will tell, and: actual coding is the LAST thing I'm planning to do. Explore concepts & ideas first, then design, re-design, re-think, and re-design again. When it's finally time to implement stuff, use existing code wherever possible. We'll see what comes of it. You're welcome to check it out:
http://www.alwinh.dds.nl/tops/
JBQ summarized the article quite sharply. Wrt. other people's comments :
1. This is OSNews, not GameNews.
2. There was no freely modifiable and redistributable, general purpose kernel when Linus started developing his own. I can't speak for Linus, however if BSD had fully cleared up license issues 3 years earlier than it did, I believe that Linux would never have seen the light of day.
3. You can become an expert in memory management, scheduling issues and kernel hacker extraordinaire without writing a kernel from scratch that will end up being inferior in almost every single category. I believe that talented people like Ingo Molnar and Matt Dillon fall in that category. Your good idea, that nobody else has thought of, might reach people and help them in their daily lives a lot quicker.
4. I'm nobody to tell anyone what to do. People are free to work on a kernel if they don't ambition to improve the state of affairs. That's my personal goal; it's not directly related to making money.
5. Mathematicians do not research the best way to do additions and multiplications these days, if that would be the "kernel" of math. The best minds work on higher-level concepts on top, usually way on top, of these basics.
6. @ Ceaser, although you're free to formulate an opinion on a product that you don't know about, Consumer Electronics products don't use standard hardware for the most part, it's designed for cost and custom assembled, therefore the customers write their own drivers. The OS we shipped is robust and has a much higher feature set (vs. footprint) than what our customers typically use, and that wasn't the issue. The issue was to port millions of lines of code.
7. @ "de Selby": martyr is exactly the right term.
Keep the comments coming (good and bad) :-)
Solar,
I have nothing against alternative operating systems. It seems every alternative operating system is trying to achieve what the big three already have, perhaps with slight alterations and/or motivations.
My qualm with these alternative operating system projects is that they are eager to rewrite everything more stable, more tested and more trusted operating systems have had for years. In other words, they reinvent the wheel and don't do a remotely impressive job at that.
Name one alternative operating system that performs one function better than, say the big three, and I will name a million the big three does better than the alternative. Forgive my hyperbole.
I see, for example, many alternative operating systems trying to recreate BeOS. So what do these hackers do? They write a kernel from scratch to mimick good ol BeOS. Yes, BeOS was solid for its time, but heavens sake, Linux and BSD have a better kernel than BeOS could ever dream of, at least today.
Why not just use those already robust, stable and tested kernels? You get an upsetting amount of drivers for free. You get to pick and tweak which scheduler suits your needs. You get pick and tweak which filesystem better purports BeFS. You get chances of easily porting must-have apps to your new project among other benefits.
But what do our friends do? They do what every geek worthy of his esteem will do. They rewrite the kernel. They magically expect drivers to fall from heaven. The kernel they write hasn't seem any form of real life testing. The kernel is epileptic under high load. Aha, they forgot the kernel needs tools, applications, and a window system.
Instead of tweaking, modifying stealing already existing, well tested, well (ab)used, stable "free" tools, but God forbid! they rewrite those too.
Many of these so-called alternative system won't boot from a floppy, don't recognize your USB keyboard, wouldn't operate above 1600x1200 resolution, wouldn't recognize your printer, don't have an office suite, and on and on and on. The operating system they are trying to recreate had all that and more.
These are functionalities users have come to expect in an operating system. So we say alternative operating systems need to exist because of innovation and new ways of doing things. I say name one alternative operating system that is doing something I can't do on the big three.
Yes, it used to be that I indulged in alternative operating system just because they did something uber cool that wasn't done anywhere else. Today, that is all but gone. By all means right your kernel from scratch, I just hope its an intellectual exercise and nothing more.
When the alternatives do something I can't do on the big three, perhaps then it might raise some heads. Just don't tell me you can't recreate AmigaOS, BeOS, OS2, or whatever you were infatuated with back then with freely available tools and frameworks today. And don't tell me to keep your new innovative idea untainted, you have to rewrite everything from scratch. But geeks will always be geeks. Real men rewrite it from scratch, right?
Meh, I'm rambling.
Mystilleef: "BeOS. So what do these hackers do? They write a kernel from scratch to mimick good ol BeOS. [...] Why not just use those already robust, stable and tested kernels?"
* Haiku are using NewOS kernel IIRC (still, Eugenia has a point http://www.osnews.com/story.php?news_id=8114)
* Cosmoe http://www.cosmoe.com/ (goes so far as to port stuff from those other BeOS wannabies who are starting lower and lower)
Just being pedantic, sorry.
Emmanuel, think we could use a clarification about who you are addressing with this: strict hobbyists or next-big-thing-hopers?
This article should be titled "why you shouldn't write your own kernel anymore __unless__ you innovate and develop the next generation". Current kernels are far from embodying next-generation principles, and anyone who knows what they are doing should be encouraged to develop what goes next: e.g. amoeba, etc.
It would be nice if these so called 'smart' people could fix some of the problems with the current Kernel.
But I guess that will never happen, because they are to smart to relize it does NOT work right to begin with.
Another area that needs SEVERE work is laptop drivers, they don't exist and until then you can forget it...
Why you shouldn't write your own
Word processor
Spreadsheet
Text editor
web rendering engine
etc...
You can build or modify perfectly good components already.
But, please write your own
Mind reading user interface,
Symphonic arrangement toolkit
and so on
About BeOS recreated using a linux or bsdkernel. If this project was a commercial project I would say yes. Apple has done this and this must have saved tons of time and costs.
But if you have the choice to use a kernel which is more suited for the purpose it's better to create it. In this case Haiku based it on newos and a such it's not even from scratch ;-)
Linux is a good kernel I give you that but if I need to compile my kernel to add support for another filesystem than it's not suited for a new BeOS.
Having to hack and patch an existing kernel to make it behave the way you want doesn't seem a clean way of doing stuff.
Look at car-engines, basically they are the same as 100 years ago only more and more finetuned during the years. But it is still old technology. It works, we know how it works and it generates money. This attitude however is holding back the introduction of more modern, environment friendly engines. Same applies to kernels.
"Projects that are REALLY innovative, are few, and these are the ones that are interesting. Many of these are semi-completed university research projects."
I've noticed this. The last one to grab my attention was <a href="http://www.eros-os.org/">EROS, but I haven't looked around since. What good ideas have you found?
If we didnt re-invent the wheel once and a while, we wouldnt have linux...
I appreciate the author's comments, but the article is definitely looking at the commercial aspect. From that point of view I'd agree that it makes very little sense to write a kernel from scratch.
However there are plenty of folks out there just writing an OS for the hell of it. Not for fame, not for delusions of being the next Gates, just because they find it a fun way to spend their free-time. If we're going to allow the philatelists to pursue their hobby then why deride the efforts of the hobby OS-devers?
Actually, "mind-reading" user interfaces, or actually, Brain-Machine Interfaces (BMI) are an amazing topic of research at the moment. There have been recent amazing breakthroughs in understanding our brain's decision process, especially for moving our limbs.
http://www.cnn.com/2004/TECH/science/07/09/monkeymind/index.html
This is something that has REALLY new possibilites. Commercial ones (such as robotic limbs duplicating human limb movement in a human-hostile environment such as nuclear power plants). And more importantly, humanitary ones (allowing paralyzed people to use synthetic limbs).
I think that if I was given the opportunity to write software for this kind of project, I'd do it almost for free.
Linus didn't have the luxury many of us have today when he was developing Linux. Developing a kernel approximately a decade ago is quite different from writing one now.
Linus' excuse for reinventing the wheel is valid. He couldn't find any free kernel to play with. He decided to clone one with available public documentation and by trial and error.
From the CNN link: "The monkeys used in the study had sets of fine wires, about the size of human hairs, surgically implanted in their brains."
Shame. Wonder how far a hobbyist could get with less intrusive sampling and a lego mindstorms (argh) set..
As you know, the "claim to fame" of EROS is orthogonal persistence, i.e. the ability to pull the plug on your computer, power it back on, and find your environment exactly as it was, modulo a few minutes work lost, inbetween the last "checkpoint" and the time you pulled the plug.
On paper, this is a wonderful concept. There are two ways of doing it, (1) applications need to support it and you need to rewrite the universe, or (2) applications need not bother with it.
EROS claims to be in category (2) but in practice, applications need to be written to take advantage of all sorts of EROS-specific implementations of capabilities and whatnot, so in essence you have to rewrite the universe, or as they did, do a clever hack so as to support UNIX applications which in essence equates to saying that you could do modify a UNIXy kernel in the first place.
In practice, the world has scratched that itch, starting with laptops (the most exposed to the problem of power going out) by implementing a "freeze execution, store physical memory to disk, power down" strategy. This is not as elegant, but it works, and by not rewriting the universe, you can channel efforts on making that technique quick, use less disk space, etc.
On top of this, as it turns out, the EROS way is not that clean, even if they did manage to rewrite the universe better than the staggering amount of developers for existing platforms. For instance, some things need to be re-started when the system boots up, such as network connections.
Don't take the latter point from me, but do take it from Shapiro himself:
http://osnews.com/story.php?news_id=5316
Hello Emmanuel,
Nice article, well done! ... Say, weren't you guy called
SunTech back in 2000 ?
Jean-Louis
You're correct. It's the same company, going on 7 years of existence.
Nice to see you guys are still around, altought you kinda focus-shifted, but for the best as you state in your article.
Cheers,
jean-louis
@ de Selby:
> Just steal as much as you can. Reinvent as little
> as possible.
You blissfully assume that I consider the GPL a viable license for my own project, which I don't due to its restrictiveness.
> If you want something so different that you think
> you should just start from scratch, really look
> into an evolutionary approach.
Care to elaborate what you consider an "evolutionary approach"?
@ Mystilleef:
> It seems every alternative operating system is
> trying to achieve what the big three already have,
> perhaps with slight alterations and/or motivations.
Depends on what you call "slight".
> Name one alternative operating system that performs
> one function better than, say the big three, and I
> will name a million the big three does better than
> the alternative. Forgive my hyperbole.
That comes naturally with being one of the big three, now does it? If I knew an alternative that could stand up to the big three, we'd have big four, now would we? And perhaps I'd even be happy with #4 and not try to reinvent the wheel...
> ...but heavens sake, Linux and BSD have a better
> kernel than BeOS could ever dream of, at least
> today.
Again, depends. I consider the Windows kernel to be much superior for the end user since it doesn't force you to compile in support for XYZ and then compiling all drivers anew because the kernel ABI changed.
> Why not just use those already robust, stable and
> tested kernels?
Because several of the things I consider broken in Linux are directly related to the kernel - like, not allowing for C++ code, not allowing for binary drivers, and having to be configured and compiled by the user?
> You get an upsetting amount of drivers for free.
*If* you are willing to put up with the Linux way of drivers, since the guys at the helm so eloquently deny any development into the direction of platform-independent drivers, which they - heralding "freedom of choice!" - could push like nobody else...
> You get chances of easily porting must-have apps
> to your new project among other benefits.
*If* I am willing to put up with a POSIX API, *if* you consider the sh**load of yet-another-way-to-do-it Linux-ish tools and their broken command line syntax a benefit...
> Instead of tweaking, modifying stealing already
> existing, well tested, well (ab)used, stable "free"
> tools, but God forbid! they rewrite those too.
Not few of them because they consider the GPL to be a non-option - there goes the wonderful kernel code base you're referring to.
> I say name one alternative operating system that
> is doing something I can't do on the big three.
I don't really know where to begin. Proper addressing of removable storage devices, for instance (AmigaOS). System file names and directory structure understandable to mere mortals (again, AmigaOS). A system installation that is protected against rogue applications, properly seperates applications while allowing shared ressources, and allows clean uninstallation. (A concept in my own drawer.)
> By all means right your kernel from scratch, I
> just hope its an intellectual exercise and nothing
> more.
Actually, I re-focussed on writing tools and libs for OS developers that aren't so damn arrogant about assuming everything has to be POSIX, Linux, and / or GPL. My own OS project was scrapped because it was so very easy to find dozens of smartasses chanting "great ideas!" which fled once it got to doing real work.
> When the alternatives do something I can't do on
> the big three, perhaps then it might raise some
> heads.
I don't count on it, since Linux successfully squashed the tolerance for anything that isn't "free" and already does more than my-favourite-distro, regardless of what it might do *different*.
> Just don't tell me you can't recreate AmigaOS, BeOS,
> OS2, or whatever you were infatuated with back then
> with freely available tools and frameworks today.
AROS worked hard on trying to recreate AmigaOS with freely available tools and frameworks.
You are blissfully unaware of how *different* an OS can be. Linux - meaning the kernel - doesn't fit all flavours, period.
> Real men rewrite it from scratch, right?
I would very much have preferred some way that would have allowed me code reuse. You don't happen to know some framework that doesn't just assume you are comfortable with the GPL, Unix-style processes, Unix-style file structures, and Unix-style everything?
> Meh, I'm rambling.
Yes, you are. Then again, so am I. 
"You blissfully assume that I consider the GPL a viable license for my own project, which I don't due to its restrictiveness."
*BSD?
The author is mostly concerned with the financial/market-driven impact of deciding to write a kernel. Technologically, that's actually a pretty poor reason to compromise on a potential kernel design and go with Linux. Linux is a monolithic kernel. There have been voluminous debates over whether Linux should have embraced a micro-kernel design. I don't want to turn this thread into a debate between monolithic and micro-kernel design. That isn't my intention. But it is my intention to point out that there really isn't any one-size-fits-all kernel. Nor should there be. Also, it's more than a little arrogant to say that we have found nirvana in kernel design -- and we're sticking to it. Linux has its place. But it shouldn't be the only place.
Thinking evolves. Technology evolves. OSes evolve. Hardware evolves. Etc. I don't care how few kernel writers there are on this planet. Not long ago, it was somewhat of a joke to insist that a college student could write a serious operating system -- and we all know the result. Linux is not the ultimate evolution of the kernel. There will be others. I say great. Bring it on.
If you have plans for writing a full, workable OS that others will actually use, then I'd agree with the author. But I'm puttering around with my own OS -- http://www.karig.net/ -- and I really don't care if no one ever uses it. I'm writing this purely for my own enjoyment. I write code when I get inspired and post the results on the website. It's simply a hobby. And even if I never really have a complete system for anybody to download and use, I can see from the webstats that others are at least looking over my site. :-) The page on unreal mode has apparently proved useful to some people; somebody on Mega-Tokyo recommended that page to somebody else in response to a question.
I imagine that there are others who have thought that writing an OS would make an interesting and challenging hobby -- the kind of people (like me) who'd pay for a PDF on the innards of MMURTL (a real-time OS for the 386) and read it for fun. On the other hand, I occasionally see some breathless announcement somewhere that someone is in the pre-alpha stages of writing an OS that will (say) run Windows, Linux, and Mac software all on the same platform, or some such thing. You KNOW such people are going to be disappointed and give up.
That's my secret, I guess -- I keep my expectations low. :-) My little OS isn't going to revolutionize ANYTHING.
@ Will:
>> "You blissfully assume that I consider the GPL a
>> viable license for my own project, which I don't
>> due to its restrictiveness."
>
> *BSD?
That's still hunting a supersonic moving target with a blowgun (the kernel doesn't stand *still* while I focus on other things, so I have to merge all over again when I return). I still would have to put up with POSIX-ish everywhere, drivers compiled into kernel proper and everybody just looking at how well he can recompile KDE on it.
And I still haven't really found myself at ease with the BSD license. In its way, it's even more confusing than the GPL (which is at least *clear* on what it wants from you), and while I know how it's applied *today*, that doesn't tell me how an attorney might tweak it *tomorrow*.
No, thanks.
Why write your own kernel, when there are ready-made kernels to use? Writing your own code is a pain.
Why build your own birdhouse, when you can just buy one at the store? Woodworking tools are expensive and a pain to use.
Why cook your family a meal, when you can just stop by the fast food joint on the way home? It's such a hassle cleaning up the kitchen.
Why sing, when you can play a CD? Heck, even Rob Zombie has a better singing voice than you.
Why write yet another book, when there are already more books than anyone could read in a lifetime?
Why paint yet another picture, when their are already more paintings in the world than anyone could possibly fit on all their walls?
As soon as the last artist in the world puts away is paintbrush for good, the last novelist retires, people stop singing in the shower, and only designated professionals cook or build things, I promise then I will stop working on my kernel. But not a minute sooner than that...
Basically the author says "don't write your own kernel, 'cause nobody is going to port their stuff over to your kernel". Well, I'm happy I can still continue my work on ReactOS then (which aims to run unmodified Win32 binaries, both apps and drivers). Feel free to join us Emmanuel :-)
http://www.forbes.com/technology/enterprisetech/2004/08/31/cz_dl_08...
and
http://www.eweek.com/article2/0,1759,1640521,00.asp?kc=ewnws083004d...
The Linux KERNEL is the least of its problems....
Excellent links from UN-biased sources.
`
Depends on what you call "slight".
Slight as in how Windows is different from OS X and is different from Linux even though they all do the same thing at a fundamental level?
That comes naturally with being one of the big three, now does it? If I knew an alternative that could stand up to the big three, we'd have big four, now would we? And perhaps I'd even be happy with #4 and not try to reinvent the wheel
They are not the big three because they are God's chosen son, you know.
Again, depends. I consider the Windows kernel to be much superior for the end user since it doesn't force you to compile in support for XYZ and then compiling all drivers anew because the kernel ABI changed.
Yeah, it depends. Because I consider a kernel that permits strangers somewhere on the Internet access to key calls in its kernel, via IE6, which is also ingrained in the kernel fundamentally flawed. I also abhor having to reboot my system after installing software. There goes your superiority. Oh look, the windowing system is embedded in the kernel sweet! Now I can write apps that crash the system! Yuppeee! And you wondering why your system is rebooting randomingly. Duh, Lavasoft is telling me you have 300, of course poorly code, spyware running behind the scenes.
I'm sorry, but I will eat compiling support for XYZ for dinner than have to deal with the repugnant rubbish I go through on Windows. Tell me why I need to reboot the damned machine because I installed software? Or why I have to stop all running processes to install software? Don't get me started on security, viruses, trojans, Active X, proprietary formats, ugh...I'll pretend I didn't you didn't read that statement.
Because several of the things I consider broken in Linux are directly related to the kernel - like, not allowing for C++ code, not allowing for binary drivers, and having to be configured and compiled by the user?
That's it?!? Are you for real?
*If* you are willing to put up with the Linux way of drivers, since the guys at the helm so eloquently deny any development into the direction of platform-independent drivers, which they - heralding "freedom of choice!" - could push like nobody else
I have no clue what you trying to brew here.
*If* I am willing to put up with a POSIX API, *if* you consider the sh**load of yet-another-way-to-do-it Linux-ish tools and their broken command line syntax a benefit...
I don't know any operating system that have better command line tools than POSIX and UNIX. Where is this command line syntax utopia you talk about?
Not few of them because they consider the GPL to be a non-option - there goes the wonderful kernel code base you're referring to.
Fine, go license a proprietary OS if that serves you better. We are given you an OS on a platter gold, but you hate gold, you prefer wood instead.
I don't really know where to begin. Proper addressing of removable storage devices, for instance (AmigaOS). System file names and directory structure understandable to mere mortals (again, AmigaOS). A system installation that is protected against rogue applications, properly seperates applications while allowing shared ressources, and allows clean uninstallation. (A concept in my own drawer.)
And where exactly do the big three fail in this regard?
Actually, I re-focussed on writing tools and libs for OS developers that aren't so damn arrogant about assuming everything has to be POSIX, Linux, and / or GPL. My own OS project was scrapped because it was so very easy to find dozens of smartasses chanting "great ideas!" which fled once it got to doing real work.
I figured. And I'm sure there probably free tools out there that already serve the purpose of your rewrites.
I don't count on it, since Linux successfully squashed the tolerance for anything that isn't "free" and already does more than my-favourite-distro, regardless of what it might do *different*.
You actually wanted me to pay for your favorite distro that does less than Linux? I get a car to take me to school for free. And here you are suggesting I pay for a bicycle ride to school. What, I'm I nuts?
You are blissfully unaware of how *different* an OS can be. Linux - meaning the kernel - doesn't fit all flavours, peI would very much have preferred some way that would have allowed me code reuse. You don't happen to know some framework that doesn't just assume you are comfortable with the GPL, Unix-style processes, Unix-style file structures, and Unix-style everything? riod.
How? If you are smart enough to write a kernel from scratch, how difficult will it be to modify already existing free, stable, mature, robust, scalable, customizable kernels to meet your needs.
I would very much have preferred some way that would have allowed me code reuse. You don't happen to know some framework that doesn't just assume you are comfortable with the GPL, Unix-style processes, Unix-style file structures, and Unix-style everything?
I do, unfortunately, many of them are dead. The last successful one remaining is an embarrassment to computer science and software engineering in general.
For some reason many long term linux users don't really care, Mr. Unbiased Sources. After years of using linux I use it because it works for me, and it works for me on commodity hardware. I used it before it had 1% market share, and I'll continue to use it. The true joy of linux, and GPLed code, is that it can live forever. There is no Be Inc. to go under, there is no target of a hostile takeover. . . There is just something that works for me. I mean sheesh, why can't people just be pro-whatevertheylike without resorting to ati-theotherguy tactics.
The article refered to the state of affairs (Linux and Windows) in which it was 'un-biased' in content.
It was not For or Against anything just stating facts backed up with stats and numbers.
http://www.forbes.com/technology/enterprisetech/2004/08/31/cz_dl_08...
The link above just covers what the current state is thats all and nothing else.
MAC
There's no reason not to write the kernel, too, if you're writing your own OS--particularly, if it is an exokernel, which is will lend greater flexibility than monolithic kernels or microkernels. Yes, even greater flexibility to drivers (sure, you might still find yourself writing them). With an exokernel, one could utilize far greater things from other OSes--even have other OSes run on top of it. Some, quite unmodified. Depends on what one wishes to do.
I see no real reason in this article not to write a kernel, save for one based on experiences that might have simply created a personal bias against them. There will always be someone, somewhere, writing an OS kernel. The prospect and the interest is too great for some to avoid it.
I do, however, appreciate the one movement started by some of trying to implement a kernel for all. But, some like me would rather avoid even the general licenses of some things. Open source is here to stay, but it too is not ubiquitous.
--EyeAm (author of NOVIO)
http://s87767106.onlinehome.us/mes/NovioSite/index.html
if they feel like wasteing their time then let them. as long as it does not harm you then just let them go their own way. maybe they will come up with something, maybe not, but there should never be anyone other then yourself that tells you what you do with your own time. that is the essence of freedom.
why should we care what os people use these days? or hardware platform forthat matter? with the net and good standards one can create files one place and read them everywhere else. this is why stuff like xml was created. a human readable filetype that can be used to create anything, includeing vector graphic files (svg).
the main point is freedom, freedom to choose whatever os or program that you want to use to do the job that you set out to do. this may make marketing people in every it company on the planet get gray hair but i dont care. freedom is the essence of a free market, freedom for both the seller and the buyer...
If IE exposes system calls how the hell can that be a kernel problem? it's obviously an IE problem don't you think??
Also the IE isn't ingrained in the kernel and the gdi isn't part of the kernel, it just runs on kernel space (different things).
None of things you say make up for the fact that linux lacks in terms of hardware abstraction, the fact that is lacks modularization, the fact that it doesn't have a decent driver model, and the fact that any future development is bounded by aging posix standards.
"None of things you say make up for the fact that linux lacks in terms of hardware abstraction"
Then start with NetBSD.
"the fact that is lacks modularization"
Fixing it's still easier than starting over.
"the fact that it doesn't have a decent driver model"
But it has drivers. We can argue about how hard changing the driver model would be (both Linux and BSD have gone through changes), but without Linux or BSD you wouldn't have drivers in spite of your driver model.
Even if changing from a monolithic kernel to a microkernel, I'd grab ahold of these drivers.
"and the fact that any future development is bounded by aging posix standards."
Changing that is still easier than starting over. Isn't that what some BeOS clones are doing?
Back in the cretaceous period the world was dominated by two kinds of creatures: lizard-hipped dinosaurs and bird-hipped dinosaurs. It was obvious that there was no need for a new type of creature, since those two were so successful. In fact, each type of dinosaur had analogous features with the other. File-based data storage, windowed GUIs, big honking teeth to rend the flesh of herbivores, they were going to dominate the gene pool for all time.
Of course, useless other creatures kept evolving. Flying things with feathers, marsupials with 4-chambered hearts, placental creatures who bore their young live and noursihed them from special glands. It was a waste of time for these genotypes to emerge, since the dinosaurs were all that you needed.
Bang. Who's digging up whose bones 60 millions years later? Is emulating Linux or Windows on one of the OSes being written right now the only way a museum will be able to demonstrate how we used computers at the start of the 21st century?
The world only really needs one or two computers anyways.
I'm not saying that changing a current kernel or OS is sooo easy that anyone could do it on a weekend. I'm just saying it takes a pretty extreme case (such cases exist) for rewriting the whole thing (boot, kernel and its algorithms, drivers, userland) to be a time-saver. If that's you, bunch of luck.
No, making Linux a microkernel and so modular that it can compete with Windows in that SPECIFIC area would be a rewrite.
Especially funny the guys that want to change from monolithic to a microkernel without touching the drivers. Very interesting concept, does not work however! As you'll have to write code to manage the hardware access of the drivers in user mode, where you might at least need to adapt the drivers, and in most times you may want to run them in a real userspace environment, too.
And POSIX is aged enough to say, it should be done differently in nearly all areas. Device interfacing (replace device files (I saw /dev/null being specified by posix) with a component model like CORBA), kernel interfacing (replace fork() and a lot of other functions) etc.
CORBA does then fulfill a similiar purpose, however, it will do the job better, and on the way we can get rid of ioctl on the way, too, or at least least hide it in stub/skeleton code.
And how do you want to create a truely different OS on top of Linux? You might ship a different shell, a different gui system, but your base will be automatically posix like.
From what is seen on this thread, Linux has weaknesses (I guess I'll get flamed for saying that holy Linux has any weakness) and Windows has some.
The Linux Kernel can't really use binary drivers.
Windows as a whole has great potential for trouble (at least IE and Outlook, which are mostly not kernel problems, which I saw correctly stated out here).
However, changing the very basis at which the Linux kernel was designed will *NOT* be easier then writing a new kernel.
If you change what everything is build on, you need to change everything that is built on top of it, too, or you add an emulation layer, like Cygwin is for POSIX for Windows. However, would you feel comfortable with that in your OS something like Cygwin was designed on purpose although everything that you create for your OS was intended to be *native* for your OS?
"Especially funny the guys that want to change from monolithic to a microkernel without touching the drivers. Very interesting concept, does not work however!"
Of course you have to modify the drivers, but do you want to figure out how to interface with a few thousand devices, most of which I assume you don't have? Linux and BSD, even in the most extreme case, serve as a good information repository for drivers.
If IE exposes system calls how the hell can that be a kernel problem? it's obviously an IE problem don't you think??
Ugh...the kernel shouldn't allow that. Don't you get it, IE is running as a priviledge process!!! Isn't that crazy? Whether or not it is IE's problem is irrelevant. Who wrote IE? Who wrote the kernel?
Also the IE isn't ingrained in the kernel and the gdi isn't part of the kernel, it just runs on kernel space (different things).
It is. That's why you need to reboot the bloody system after you upgrade applications or even IE. Another clear and gaping design flaw. That's why faulty applications can crash Windows. That's why you need to close all running processes to install applications and then reboot after that.
None of things you say make up for the fact that linux lacks in terms of hardware abstraction, the fact that is lacks modularization, the fact that it doesn't have a decent driver model, and the fact that any future development is bounded by aging posix standards.
Okay prof, questions:
1). How is windows hardware abstraction better than Linux'?
2). How is windows more modular than Linux?
3). What are the newer better standards Linux should follow aside from posix and why?
4). How does posix suck and why?
How should the kernel stop IE from executing another program? IE should not accept commands like this from a JavaScript in a HTML file.
1. Compare DirectX and messing around at the same level for example with a sound card under Linux. Enough said.
2. Instead of recompiling the whole kernel with a lot of drivers, I install one driver.
3. Those have to be created, but nobody does as they are all on the "lets-improve-linux" trip.
4. Look at fork() one good example. Look at /dev/ in your Linux installation (is with other *NIX systems the same). LOTS OF EXAMPLES!
How should the kernel stop IE from executing another program? IE should not accept commands like this from a JavaScript in a HTML file.
No, friend, this has nothing to do with HTML and JavaScript. Ever heard of ActiveX?
1). You didn't say jack!
2). Who says you have to recompile the whole kernel with drivers?
3). You didn't answer the question.
4). What's wrong with fork() and /dev?
Look I'm getting tired of arguing. Go write a new kernel if you want, who cares!
"Still I doubt that you can test them all anyway!"
I know, but I think I made my point. In my opinion, the drivers are the single most important bit of free code you can leverage. It might be as bad as porting to another OS, but by god I would do anything I could to steal these drivers!
I also think a complete rethinking of the API and so on is something will eventually have to be done. (Love BeOS!--could go ever further.) Maybe this is a case where rewriting is best. I'm not a kernel hacker, but I can't help but imagine you could cannibalize boot code, scheduling, or something to save some time...
In either case, you have to admit that most projects try to be POSIX compliant to leverage compatibility and because they aren't about rethinking everything.
And I think I've said "even in the most extreme case" one too many times.
Then deactivate it. It doesn't bug normal pages anyway, as no one uses it.
1. I did, simply experience these two different APIs which have the purpose to give an interface, an abstraction to the hardware, and you'll see that DirectX is the better one. And no FMOD under Linux is above the level that DirectX operates on.
2. How should it really work without reliable? At best with binary modules that I need for the right kernel version then.
3. I did, innovation has already been stalled too much.
4. fork in Linux is only a hack to make a bad system working at reasonable speed, it had to be extended to clone. The exec call replaces your process with a new one. Sense, Usefullness? No idea, but with fork() and exec() you can create a process that runs seperately. However why not give one single command that does not rely on Copy On Write?
And threading, where the fork relative clone comes into play. Having a process-only scheduling model, at the beginning you had to fork to simulate threads. Now with clone you get at least rid of the address space switches involved with fork, however, why not a clean task/thread model from the beginning?
/dev/ - all those device files are flawed abstractions.
Some devices have no meaning on reads, writes, and the rest is all done via messy looking ioctl code.
The first thing I would split is kernel and drivers.
Then I can try to port those drivers to my OS, however, they may get major changes in interfacing.
"/dev/ - all those device files are flawed abstractions."
You deny that Unix is God? You corrupt the children! (Wonder what you think of Plan 9...)
You don't want to know!
No, after reading a bit in a Plan 9 I don't know what is really up with that. The Unix concept taken a step further. So what?
Nice article, dude. Enjoyed reading it...
So this is what you've been up to on those long flights.
Catch ya later.
PW
The article made several good points, in reasonable, well articulated language. There's nothing new in the advice, it's all been said before, and most people will ignore or misunderstand it. But I appreciate the attempt.
Projects must have a goal, if they are accomplish anything worthwhile. A narrowly defined goal is likely to lead to a focused project with useful results. A broad, ambigious goal is likely to lead to a project galloping off in several directions at once, with little to show for the effort.
If your goal is to build a better desktop, focus on the desktop. Don't get distracted into building an OS when you can use or adapt an existing one. Even a desktop is too large to bite off as a single project; perhaps it would be better to build an application or game as a demonstration of your UI ideas. If you insist on creating a new language, compiler, kernel, drivers, window manager, and UI toolkit, you aren't likely to have a better desktop to show.
If you want to learn about kernels, then go ahead and write one. But adapting an existing one may teach you more. Reading other people's code may teach you more than writing your own, although it's less fun. Learning as a goal is reachable; creating a kernel that anyone wants to use is less so.
Do one thing well. Then do the next thing. You will soon have something to be proud of.
Try to do everything, and you will have nothing.
Okay prof, questions:
1). How is windows hardware abstraction better than Linux'?
2). How is windows more modular than Linux?
3). What are the newer better standards Linux should follow aside from posix and why?
4). How does posix suck and why?
1) Uh? because linux doesn't have an hardware abstraction layer, each driver has to take care of its own stuff (like interrupts etc), the drivers themselves are the abstraction layer, the linux drivers have to deal with interrupts, locking etc, while windows drivers don't have to
2) Because instead of being a big monolithic blob it's divided in multiple components.
http://tutorials.findtutorials.com/read/category/97/id/379
3) and 4) because i don't like how the posix standard deals with things like threading and asynchronous i/o, and all the lame hacks posix has picked up over time to remain usable. new standard are needed
Article: I hope I catch you before you burn any proof that you ever downloaded the Intel System Programming Manuals, as you look nervously through the window for the Code Police to come and send you to coder rehab.
Well I have downloaded them... heh.
Article: Most kernel projects start with an idea such as, let's create an OS around this new filesystem thingie. Then, night after night, a small, expanding team of programmers duplicates scheduling, virtual memory, a posixish API, the windows registry, device drivers for ISA network cards and an USB stack...producing nothing