Linked by Thom Holwerda on Tue 30th Jan 2007 21:36 UTC, submitted by David Dengg
BSD and Darwin derivatives DragonFly 1.8.0 has been released. The biggest kernel change in this release is the addition of virtual kernel support and a virtual kernel build target. The biggest user-visible changes include updates to third party applications included in the base system, a major rewrite of NULLFS which removes all directory recursion restrictions from mount_null and removes nearly all the kernel resource overhead when using such mounts, and a multi-ip feature for jails.
Thread beginning with comment 207609
To view parent comment, click here.
To read all comments associated with this story, please click here.
Don T. Bothers
Member since:
2006-03-15

"so matt was right all along about freebsd 5.x. I like DF because it's based of freebsd 4.8., i have been a huge fan of freebsd's 4.x, but now that freebsd 4.11 is about to be retired, i might switch over to df 1.8.x. and add df along the other os i run."


From my understanding of the situation, the answer to whether Matt was right or wrong is yes and no and quite a bit more complex than that. ;)

Theoretically speaking, the FreeBSD project chose the correct solution but ran into several major problems. 1) It is extremely difficult to implement, the code becomes extremely bloated, and there just isn't enough resources to maintain it. 2) There is little to no application that could take advantage of the advanced threading model. From what I understood of what I read, FreeBSD was basically implementing a very complex M:N scheduler where all the applications were treating things as M=N=1. So in almost all cases, a native 1:1 threading model scaled much better because it did not have any overhead. 3) In the spirit of biting more than you can chew, the FreeBSD team decided to also implement some kind of fair queue scheme for processes. However, the fair queue code was a place holder waiting for someone to properly implement it. This place holder code had the nice benefit of slowing things to a crawl while causing all sorts of instabilities. A lot of these problems have been fixed for FreeBSD 7.

In addition, it is very hard to fault the FreeBSD team for making the decision to which way to go. What Matt wanted to do is rather revolutionary and is rather risky for a product lots of companies use for production purposes. While there is the potential of a huge payoff, there is a certain amount of risk involved with it. The FreeBSD project would be rather self-centered and careless to try such a revolutionary path without some alternate "safe" path to fall back to if it fails. In a way, the way Solaris, HPUX, AIX, and Linux chose to scale was the only safe and reasonable choice for such a large project.

What happened was probably the best thing for both projects and for BSD in general. Now, the FreeBSD project is free to provide a safe, proven path for scalability, performance, and stability for companies dependent on it while Matt is free to explore an alternative path that might prove to be superior.

Edited 2007-01-31 04:19

Reply Parent Score: 5

CaptainPinko Member since:
2005-07-21

So did FreeBSD abandon the M:N threading model? That would be a shame after all the work that went into it. Then again: if you must then you must. Where can I find more information about this? Wikipedia doesn't seem to mention this more does the KSE homepage. Did I misread your comment?

Reply Parent Score: 2

kaiwai Member since:
2005-07-06

It was mentioned on a FreeBSD quartly report a while back in regards to KSE and slow performance on non-x86 architectures.

One has to ask how much is due to KSE tuning beant towards a particular architecture, or are the architecturers were there is bad performance, just plain well crap architecture.

Reply Parent Score: 3

butters Member since:
2005-07-08

So did FreeBSD abandon the M:N threading model? That would be a shame after all the work that went into it.

Everybody is moving away from M:N. It's a bad idea that looks good on paper. Solaris and AIX both support M:N, and Linux can use it with NGPT (not the same at NPTL), but it is not the default on any of these systems.

Basically, threading only works efficiently if the kernel "knows" about all of the user threads. For example, in M:N (and M:1), it is possible for one blocking thread to block one or more of its peer threads, even if they are runnable. With increases in physical memory and addressable memory (with 64-bit systems), there is no excuse for the kernel to delegate thread management to userspace.

Reply Parent Score: 3

Yoke Member since:
2005-08-28

In addition, it is very hard to fault the FreeBSD team for making the decision to which way to go.

Back when FreeBSD decided to go with an M:N implementation, Solaris had already switched from M:N to 1:1 and put out a white paper (http://www.sun.com/software/whitepapers/solaris9/multithread.pdf) explaining why, and Linux had decided to go with NPTL instead of NGPT, IBM's M:N implementation for Linux. That was a very uncontroversial decision in the Linux camp (http://thread.gmane.org/gmane.linux.kernel/54620), because _they actually understood the ramifications_ of the different implementations.

Reply Parent Score: 2

rayiner Member since:
2005-07-06

That's the fun thing about "theoretically better" solutions. In practice, you're so often limited by developer resources, that a simple, but well-implemented and highly tuned solution will end up being better than a more sophisticated design that hasn't had the same amount of tuning. Take, for example, GCC's register allocator. It's a theoretically unsophisticated design with an enormously complex (and convoluted) implementation. We're talking about something that wouldn't have been state of the art even 20 years ago. Yet, it creates surprisingly good allocations, because it's had so much tuning. Or, take the x86 code generator in Ocaml. It doesn't do any fancy SSA-based optimizations, but it generates very good code.The generated code is very straightforward and small, and it turns out that modern x86's care very much about code size and very little about fancy scheduling.

Now, of course, implementation tuning being equal, a fancy design is almost always going to be better. However, developers rarely have the luxury of unbounded time, and given that constraint, a more sophisticated design often implies a less well-tuned implementation. That's why there is no alternative to experimentation when writing performance-sensitive code. You'll be surprised just how often theoretical arguments fail to fully model all the relevant details.

Reply Parent Score: 2

butters Member since:
2005-07-08

In addition, it is very hard to fault the FreeBSD team for making the decision to which way to go. What Matt wanted to do is rather revolutionary and is rather risky for a product lots of companies use for production purposes. While there is the potential of a huge payoff, there is a certain amount of risk involved with it. The FreeBSD project would be rather self-centered and careless to try such a revolutionary path without some alternate "safe" path to fall back to if it fails. In a way, the way Solaris, HPUX, AIX, and Linux chose to scale was the only safe and reasonable choice for such a large project.

FreeBSD took pretty much the same approach as Linux to address SMP, at about the same time. They both introduced big kernel locks and planned on phasing in finer-grained locking in the future. Linux also experimented with M:N threading at about the same time as FreeBSD implemented it. The difference is that FreeBSD just doesn't have the same resources. Linux was able to implement fine-grained locking, per-cpu runqueues, and a vast array of threading and process scheduling experiments while FreeBSD seemed stuck in the mud.

Everybody and their dog is prototyping some new theory on Linux, and often it pays off. The kernel maintainers honestly thought IBM's NGPT model would be the next-gen thread model for 2.6, but Ingo Molnar did some quick hacks. Whoa! That simple, unsophisticated 1:1 model rocks IBM's fancy-shmancy implementation. Nicksched or staircase, anticipatory or CFQ... Linux doesn't need to pick a path to pursue, it just picks whatever proves most successful.

Reply Parent Score: 2