This bit of interview with DragonflyBSD’s core from ONLamp’s BSD advocacy page will give you a nice introduction into what is DragonflyBSD, how it evolved from FreeBSD’s 4.x series.
DragonflyBSD, the Innovator
About The Author
Eugenia Loli
Ex-programmer, ex-editor in chief at OSNews.com, now a visual artist/filmmaker.
Follow me on Twitter @EugeniaLoli
Those who say open source does not innovate are wrong.
Perhaps I’m feeding a troll, but what innovation? I read the article. I didn’t see anything indicating DragonflyBSD is innovating. It seems they are just adding stuff other things already has all awhile adding one BSD to the overcrowded BSD world.
If you read the article you should have noticed that among other things they focus on a radical reconstruction of the internal process/threading model and scheduling which should result in much better performance/scalability and maintainability.
In general, DragonFly seems to be much more open to redesign things, and this was probably the best (and most enlightening) interview I’ve ever read.
When I find the time I’ll install QEMU on my iBook and run that beast!
I hope Theo considers using LWKT in the (distant?) future. It seems like a great technology, and makes more sense to me. But I’m not a developer…
I’ll definitely be trying out DragonflyBSD when an official release comes out. Maybe by then I’ll have time to do some fun stuff.
The review was very interesting. It didn’t ask extensive questions about Linux, which is in a way, could be its biggest competitor.
I have a couple of questions. They feel their SMP model is superior to Linux and FreeBSD’s, and refer to it as “outdated” and “Our networking stack is designed to scale to large number of processors, more than the 2 or 4 that SMP hardware can effectively run on today”, while ignoring the fact that Linux’s scalability has rapidly taken off, and is probably up there with, if not surpassing Solaris, while still being probably still the fastest modern, general purpose operating system for most single threaded tasks too.
While some of their SMP and multiprocessor ideas seem very nice and clean and have potential, I think they by no means outdate the current mechanisms yet. And it is by no means a given that their systems will ever be superior.
Nice review.. I can’t wait to try it when it hits 1.0..
Hiten Pandya is one smart 17 year old. Dang.
from here [http://www.dragonflybsd.org/main/] i read:
>It is our belief that(…)
>We also believe that(…)
>Finally, we believe that(…)
God bless you and your “multi-year project”.
I hope they model their ports system after Gentoo’s portage. It’s a great and very capable tool. While some say that it’s not suitable for binary packages, the capability is there, and it is of excellent quality. There are just not many binary packages available, because people use Gentoo because they want to build their system from source.
And if you look at their MP work, it seems god has so far been very generous to the believers.
I hate the word “innovation”. It’s already lost it’s meaning, Microsoft uses it all the time for little things that are nothing new. Try google searching microsoft.com for “innovation” and “innovative”, over 30 000 hits
But well, on the technology side, DragonflyBSD tries to be different from other BSD’s, and on the BSD side it’s “innovating”, but not really inventing anything new (in technology sense) that isn’t already in other os’es.
But good luck for Dragonfly, nice to see more BSD forks to compete together!
Can anybody explain to me what a SSI cluster will look like from a user perspective?
For example, if I would connect 5 PCs via switch – will there be a ‘head’ node where the user logs in? Will there be only one directory structure (visible to the user)? Will processes/(threads?) be scheduled transparently across all nodes without requiring the user to do this? Can nodes dynamically be added?
I know this isn’t implemented yet. But what will it look like when it’s ready?
While some of their SMP and multiprocessor ideas seem very nice and clean and have potential, I think they by no means outdate the current mechanisms yet. And it is by no means a given that their systems will ever be superior.
Missing the point I think. DragonFly’s concepts in and of themselves aren’t what’s making the Linux and FreeBSD models obsolete, it’s modern hardware that’s doing the outdating.
Multiprocessor support in these older systems was built upon the idea that all processors have equal access to memory, and that all memory is shared in a global pool, which can lead to inefficiency when physical distances between processors and memory are taken into considderation. Keep in mind that NUMA support is still new in Linux, and it is based on support for the older way of doing things.
Further, mutex centric models entail a lot of overhead with regard to the number of steps reuired to perform an operation on a shared resource, and the semantics of doing so can get quickly convoluted, and additionally become an absolute nightmare to debug and maintain.
Wherever possible (and it’s not always possible to be sure), DragonFly forgoes locking of any sort, often simplifying a given procedure just by having done so, reducing the number of steps required to get the particular task done, saving cpu cycles in the process. That helps (slightly) with performance. Also, by not having duplicated data in processor caches, they system is made more efficient. Not having to worry about preemptive thread migration also reduces both scheduler and processing overhead.
To be clear, I am not claiming that DragonFly is going to outperform Linux in the next little while. However, it is becoming more and more clear the the architecture of Linux as it is today, just isn’t ideal with regard to scalability, debuggability, and robustness required by enterprise systems (hence why only highly patched kernels run on those 512 cpu boxes, and why Red Hat and SuSE can only support their official patched kernels for high end work), and if it doesn’t adopt a newer architecture like the one that DragonFly is evolving into, then in a few years time, it will be a much simpler task to get the much numbler DragonFly kernel to run on rediculously large machines than it will to do the same with Linux.
Coupled with the fact that DragonFly’s abstractions are designed with simplicity in mind (meaning that ordinary experienced developers will be able to maintain it, as opposed to requiring god-like mutex debugging junky developers), DragonFly just seems like the way to go once they’ve got their new architecture fully in place.
Of course there are a few other things they are doing to better fit more modern systems, but IMO, those are some of the more important points that you seem to be not fully appreciating.
There is a new BSD development company Crescent Anchor using parts of DragonFly already for a new operating system they have wrote. From the site a large number of companies already have interest in the new BSD developments going on. Their website address is ( http://www.crescentanchor.com )
There is a new BSD development company Crescent Anchor using parts of DragonFly already for a new operating system they have wrote. From the site a large number of companies already have interest in the new BSD developments going on. Their website address is ( http://www.crescentanchor.com )
Best I can tell, SilverOS is DragonFly, with perhaps only a few thousand lines of different code. I guess you can think of it as the Slackware of DragonFly BSD.
Seeing as the current set of DragonFly third party packages at http://www.GoBSD.com is incomplete with no complete working desktop software (GNOME or KDE ATM), if you want to run a stable version of DragonFly with useful applications, then SilverOS is the way to go.
Add to the fact that support is readilly available (so they claim), and we have our very first DragonFly distribution well on it’s way to taking some mindshare away from Linux ;^)
Ok, I am going to feed the various trolls about innovation.
1) caching
2) iomodel
3) messaging
4) packages
5) threads
6) userapi
7) vfsmodel
Yes, all this is listed on their main page. Please give it a look and then talk about innovation. It will be interesting to see how this project grows.
As for the statement: “It seems they are just adding stuff other things already has all awhile adding one BSD to the overcrowded BSD world”
Lets examine the number of *bsd OS (not including embedded systems).
1) FreeBSD
2) OpenBSD
3) NetBSD
4) DragonflyBSD
5) MirOS
6) ClosedBSD (firewall).
7) And their are a few others that I am forgetting.
These projects are an incredible source of energy and innovation. Various commercial OS’ have their roots in *BSD’s. A few examples would be:
1) SunOS which was eventually merged into Solaris (Sys V)
2) Junos (Juniper routers)
3) Darwin (Apple)
4) Nokia’s friewalls
5) Various OS’ use the *BSD TCP/IP stack (even Microsoft)
6) Microsofts Unix utilities are a product based on OpenBSD via Interix.
And the list goes on and on. Think about the contributions that have been given back to the community via some corporations.
1) Sun released NFS and NIS to the world. Perhaps, Java and maybe even Solaris, dTrace and more.
2) Apple makes Darwins code available to the public (less the graphical interface).
The *BSD’s have placed their footprint in OS history and have continued to do so. I look forward to seeing what DragonflyBSD does with their model.
1) Sun released NFS and NIS to the world.
Modern Math libraries as well.
“Joerg Sonnenberger: I’m a 20-year-old student of mathematics and spare-time programmer. I have investigated different fields of CS in the past years and got a bit bored last year. That’s why I’m here.”
What if the system is considered for a production machine and someone in charge happens to read this interview? I would instantly ask: “Gee, what do you reckon will happen if this young chap finds a new romantic interest?”
“Gee, what do you reckon will happen if this young chap finds a new romantic interest?”
..then another developer would take over, that’s the good thing about OSS.
The same can’t be said when a company loses the interest on their closed-source product..then you’re in real trouble.
“…considered for a production machine…”
The developers have made it quite clear that this project is in the R&D phase, and in no way ready for production use.
Instead of talking about the infinite number of things that DragonflyBSD isn’t, how about we discuss what actually is there?
The thread and message architecture looks pretty interesting to me. As others have pointed out, it’s clear that NUMA hardware will become inexpensive and common. It looks like DragonflyBSD is designed to run very well on that hardware architecture. This could turn out to be a very interesting project.
ekkobsd http://www.ekkobsd.org/
microbsd http://www.microbsd.net/
i’m very impressed:
* the youth of these developers is staggering. and brings shame upon many supposedly older and wiser veterans
* i am glad that at least one BSD is designing for simplicity in its kernel architecture. there is a need and a space for it.
re: comment on romantic interest – he may well improve his output!!
http://www.livebsd.com/dfly/
It’s going into the final release which will be EARLY next week. Install now and test, then join us in #dfinstaller on EFNet!
GG
“But well, on the technology side, DragonflyBSD tries to be different from other BSD’s, and on the BSD side it’s “innovating”, but not really inventing anything new (in technology sense) that isn’t already in other os’es.”
Well their MP support seems quite innovative in that it differs from all other open source implementations. I have no idea how it compares with Solaris or Irix…
But I do have tremendous faith in Matt Dillon. He’s an incredibly bright guy who apparently has some time on his hands. I was quite sad to see him leave the FreeBSD project.
Missing the point I think. DragonFly’s concepts in and of themselves aren’t what’s making the Linux and FreeBSD models obsolete, it’s modern hardware that’s doing the outdating.
Well, modern hardware would be outdating – a newer method (ie. dragonfly) would be obsoleting
But semantics aside, Linux’s multiprocessor model hasn’t proven to be outdated or obsolete yet… I reckon you should see SGI’s Altix with 1024 CPUs running Linux 2.6, maybe even 2048 if they use dual core Itaniums. They already do have Linux running on 512 CPU systems.
Multiprocessor support in these older systems was built upon the idea that all processors have equal access to memory, and that all memory is shared in a global pool, which can lead to inefficiency when physical distances between processors and memory are taken into considderation. Keep in mind that NUMA support is still new in Linux, and it is based on support for the older way of doing things.
Yes, initial multiprocessor support wasn’t NUMA aware. But you know, I doubt DragonflyBSD is NUMA aware either yet.
What’s more, NUMA support appeared in Linux in 2.4, and is now ubiquitous throughout the kernel in 2.6, and by no means new or ineffient. Witness a 512 CPU Altix’s 256 distinct memory nodes, and about 8 different memory distances when viewed from any one CPU.
Further, mutex centric models entail a lot of overhead with regard to the number of steps reuired to perform an operation on a shared resource, and the semantics of doing so can get quickly convoluted, and additionally become an absolute nightmare to debug and maintain.
Mutex/lock operations are very lightweight in Linux, and locking hasn’t become convoluted (see FreeBSD 5 for that). This is mainly through per-object locking, good design decisions, etc. What’s more, Linux 2.6 uses many lockless algorithms where appropriate: RCU, per-cpu data, etc.
Wherever possible (and it’s not always possible to be sure), DragonFly forgoes locking of any sort, often simplifying a given procedure just by having done so, reducing the number of steps required to get the particular task done, saving cpu cycles in the process. That helps (slightly) with performance. Also, by not having duplicated data in processor caches, they system is made more efficient. Not having to worry about preemptive thread migration also reduces both scheduler and processing overhead.
Some workloads even on highly NUMA architectures absolutely need to migrate threads between CPUs in vast amounts and very quickly (in order to reduce idle time and get CPUs doing useful work). I am speaking from experience. I wonder how DFBSD is going to cope with that.
To be clear, I am not claiming that DragonFly is going to outperform Linux in the next little while.
And it might never outperform Linux. Did you consider that?
However, it is becoming more and more clear the the architecture of Linux as it is today, just isn’t ideal with regard to scalability, debuggability, and robustness required by enterprise systems (hence why only highly patched kernels run on those 512 cpu boxes, and why Red Hat and SuSE can only support their official patched kernels for high end work), and if it doesn’t adopt a newer architecture like the one that DragonFly is evolving into, then in a few years time, it will be a much simpler task to get the much numbler DragonFly kernel to run on rediculously large machines than it will to do the same with Linux.
You know, most of those patches are GPL, and many are backports from 2.6 (like the O(1) scheduler). The last major scalability problem area in 2.6 I think was a last global lock in the block layer which had been removed a few releases ago. I think SGI is aiming for 1024 CPUs with 2.6, and without requiring patches.
Coupled with the fact that DragonFly’s abstractions are designed with simplicity in mind (meaning that ordinary experienced developers will be able to maintain it, as opposed to requiring god-like mutex debugging junky developers), DragonFly just seems like the way to go once they’ve got their new architecture fully in place.
Don’t kid yourself that kernel programming will ever become easy, but any “experienced developer” can get into it *provided* they put in the hard yards and learn.
Of course there are a few other things they are doing to better fit more modern systems, but IMO, those are some of the more important points that you seem to be not fully appreciating.
But how do you know they’re a better fit? That is my point. What’s more, there are some serious problems with DFBSD’s inflexible approach to scheduling.
But semantics aside, Linux’s multiprocessor model hasn’t proven to be outdated or obsolete yet… I reckon you should see SGI’s Altix with 1024 CPUs running Linux 2.6, maybe even 2048 if they use dual core Itaniums. They already do have Linux running on 512 CPU systems.
I don’t believe Linux’s model to have reached it’s limit yet either, nor do I think itwill ultimately compare without (the inevitable) further evolution. I’d love to see either one run on a 1024 dual core Itanium box and not trip up 🙂
Yes, initial multiprocessor support wasn’t NUMA aware. But you know, I doubt DragonflyBSD is NUMA aware either yet.
DragonFly isn’t fully NUMA aware yet, but it is being redesigned specifically with both NUMA and clustering in mind. Linux is ahead with NUMA ATM, and will be for at least another year or two.
Mutex/lock operations are very lightweight in Linux, and locking hasn’t become convoluted (see FreeBSD 5 for that). This is mainly through per-object locking, good design decisions, etc. What’s more, Linux 2.6 uses many lockless algorithms where appropriate: RCU, per-cpu data, etc.
Mutexes perhaps, threads are not nearly as light-weight. It’s been a while, but I do seem to recall Linux’s context switching times to be acceptable, if not exactly stellar, so my argument may well be pretty weak here. I’m definately going to look into it.
Some workloads even on highly NUMA architectures absolutely need to migrate threads between CPUs in vast amounts and very quickly (in order to reduce idle time and get CPUs doing useful work). I am speaking from experience. I wonder how DFBSD is going to cope with that.
I have not worked extensively with large NUMA machines and certainly not yet with DragonFly, so I’ll not be able to say too much more on this until I do.
And it might never outperform Linux. Did you consider that?
Most certainly. I do not believe that it won’t, but I do not make decisions based on belief alone, and am quite happy to learn through experience if I am right or wrong. I have been wrong before, and I will be wrong again. One of life’s great joys 😉
The last major scalability problem area in 2.6 I think was a last global lock in the block layer which had been removed a few releases ago. I think SGI is aiming for 1024 CPUs with 2.6, and without requiring patches.
It would be pretty cool for a vanilla kernel to be able to do that. I can’t wait to see them both running on such hardware a few years from now, as I really am curious to see how well each performs (a moot point really, as developers from both camps will take steps to eliminate bottlenecks and other inefficiencies discovered).
Don’t kid yourself that kernel programming will ever become easy, but any “experienced developer” can get into it *provided* they put in the hard yards and learn.
Simpler APIs that provide the same capabilities tend to lower the ‘barriers to entry.’ I look forward to see if DragonFly’s simplification results in easier mainainence or not. Time will tell.
But how do you know they’re a better fit? That is my point.
I don’t “know.” From the limited testing I’ve done, and from reading the results of other people’s more extensive testing, I am begining to get an idea that it is the right direction. The results aren’t all in yet, and it is entirely possible that DragonFly’s approach will not be as wonderful as it seems right now. Either way, there are lessons to be learned, and with each one, a little more enlightenment.
What’s more, there are some serious problems with DFBSD’s inflexible approach to scheduling.
Could you please explain what the problems are? Scheduling is not my area, but from my (admittedly limited) testing I can’t say that I find DragonFly to be underperforming at any load I can throw at it. Please enlighten me 🙂
I must say, its nice to have an intelligent conversation here. Unfortunately the discussion facilities aren’t great, so I’ll have to convert it to email style.
Anonymous (IP: 168.143.113.—) wrote:
>Anonymous (IP: —.dyn.iinet.net.au) wrote:
>> Yes, initial multiprocessor support wasn’t NUMA aware.
>> But you know, I doubt DragonflyBSD is NUMA aware either
>> yet.
>
> DragonFly isn’t fully NUMA aware yet, but it is being
> redesigned specifically with both NUMA and clustering in
> mind. Linux is ahead with NUMA ATM, and will be for at
> least another year or two.
>
Watch Linux on the clustering front too. There has recently been a push for full clustering filesystems, which is, arguably, the most difficult aspect of an SSI cluster. There are a number of GPLed implementations out there now, so don’t be surprised if Linux 2.8 can do SSI clustering.
>> Mutex/lock operations are very lightweight in Linux, and
>> locking hasn’t become convoluted (see FreeBSD 5 for
>> that). This is mainly through per-object locking, good
>> design decisions, etc. What’s more, Linux 2.6 uses many
>> lockless algorithms where appropriate: RCU, per-cpu data,
>> etc.
>
> Mutexes perhaps, threads are not nearly as light-weight.
> It’s been a while, but I do seem to recall Linux’s context
> switching times to be acceptable, if not exactly stellar,
> so my argument may well be pretty weak here. I’m
> definately going to look into it.
>
Heh, I recall Linux to have the fastest context switching times out of any unix OS I have seen. IIRC I saw one comparison where it was edged out by Windows 2000 or 2003, although that could have been a function of the mechanism used to do the measurements.
I should be getting my hands on a new computer soonish though. I’d love to install some free BSDs and do some testing of things like this. I think I’d be too scared to publish anything on the internet though
>> Some workloads even on highly NUMA architectures
>> absolutely need to migrate threads between CPUs in vast
>> amounts and very quickly (in order to reduce idle time
>> and get CPUs doing useful work). I am speaking from
>> experience. I wonder how DFBSD is going to cope with
>> that.
>
> I have not worked extensively with large NUMA machines and
> certainly not yet with DragonFly, so I’ll not be able to
> say too much more on this until I do.
>
Fair enough.
>> And it might never outperform Linux. Did you consider
>> that?
>
> Most certainly. I do not believe that it won’t, but I do
> not make decisions based on belief alone, and am quite
> happy to learn through experience if I am right or wrong.
> I have been wrong before, and I will be wrong again. One
> of life’s great joys 😉
>
That’s the spirit
I don’t come with any preconceived as to what DF will be able to achieve (and it’ll inevitibly beat everything at *something*). Hands down beating Linux at the performance game is a pretty high bar to set these days though.
>> The last major scalability problem area in 2.6 I think
>> was a last global lock in the block layer which had been
>> removed a few releases ago. I think SGI is aiming for
>> 1024 CPUs with 2.6, and without requiring patches.
>
> It would be pretty cool for a vanilla kernel to be able to
> do that. I can’t wait to see them both running on such
> hardware a few years from now, as I really am curious to
> see how well each performs (a moot point really, as
> developers from both camps will take steps to eliminate
> bottlenecks and other inefficiencies discovered).
>
Well, word on the grape vine is that 2.6 unpatched does pretty well on their 512 CPU systems without them doing much tuning of it… 2.4 unpatched obviously wouldn’t never work: even the scheduler has a single global lock.
>> What’s more, there are some serious problems with DFBSD’s
>> inflexible approach to scheduling.
>
> Could you please explain what the problems are? Scheduling
> is not my area, but from my (admittedly limited) testing I
> can’t say that I find DragonFly to be underperforming at
> any load I can throw at it. Please enlighten me 🙂
One I recently ran into on a 8 way non NUMA was on a database workload, where it turns out that CPUs quite easily become idle, and really need to pull tasks off other CPUs in order to keep throughput up. *However* I never fully got to the bottom of this one, so it could, for example, be an artifact coming from poor program design, or semaphores in the kernel.
Just as an FYI, the Gentoo portage system borrows a lot from the FreeBSD ports systems.
That means DragonFly’s package management system/packaging is probably not exactly borrowing from Gentoo as much as it is borrowing from FreeBSD and re-vamping it.
I would suggest anyone that goes in to #dfinstaller on efnet be careful — do not forget that IRC brings the worst out of some people.
For the most part everyone’s cool (in #dfinstaller), but there are definitely a few jaded souls in there — who kicked me last night when I tried to explain the importance of UI design (relevant to the discussion at the time), and why certain OSS projects made the decisions they did.
I spent a considerable amount of time downloading their ISO’s over dialup to help test, while trying to give useful feedback — after being kicked for stupid reasons I kind of feel it was a waste of my time.
You have been warned.
> Gee, what do you reckon will happen if this young chap finds a new romantic interest?
Hm. It slows down development speed a bit, just like university or TV are killing development time. No big deal, it is a question of managing ressources.
[quote]Can anybody explain to me what a SSI cluster will look like from a user perspective?[/quote]
It’ll look like a single system, hence the name single system image. Similar to how Mosix works. You start a program, the program migrates from CPU to CPU as needed, and the user doesn’t know if they are running on a single CPU system, an SMP system, an MP system, or a cluster of a dozen systems. As far as the user is concerned, they are working on a single computer.
For the most part everyone’s cool (in #dfinstaller), but there are definitely a few jaded souls in there — who kicked me last night when I tried to explain the importance of UI design (relevant to the discussion at the time), and why certain OSS projects made the decisions they did.
But of course what you won’t say is when you were told politely that your view was respected but not agreed with and asked to end the discussion repeatedly you continued to argue and insist on being right. You were asked nicely to end the discussion and did not, that is why you were kicked.