This release is slightly late: 3 days behind <a href=”http://www.freebsd.org/releases/5.3R/schedule.html“>schedule<… but since I believe the last release was 2 days late this is fairly on tract. After all these years ladies and gentlemen FreeBSD 5 should be here.
I’m betting on a Halloween release. Anyone else wanna guess?
What I can’t wait for is when all this new code gets imported into Darwin and therefore a Mac OS X release. I don’t think it will happen until 10.5 however.
Well, the cool stuff in FreeBSD 5.x (the theading model), won’t be in future MacOS X releases. Darwin uses the Mach threading code, not the FreeBSD threading code.
Well, the cool stuff in FreeBSD 5.x (the theading model), won’t be in future MacOS X releases. Darwin uses the Mach threading code, not the FreeBSD threading code.
You’re thinking of kernel threads, and in that case you are indeed correct – XNU uses Mach threads, and not KSE. However, I was talking about the fine grained locking work in the network stack, which the Apple folks *will* be adopting.
It’s one of the reasons they are staying in sync with FreeBSD 5.x, so that a large amount of the hard work gets done for them, which is really a smart business move. Open source software is quite empowering that way.
Threading and locking are two completely different things.
I was wondering how this affected OS X. What will be the benefit of moving this code? I mean is it performance, security, etc? Also you said it would be around 10.5…why? How far behind is 10.3/10.4?
I believe that the Tiger release 10.4 will be using FreeBSD 5.x stable branch code for the BSD Subsystem in that release. Atleast that’s what their developer pages and Tiger preview pages say about the BSD subsystem.
I believe that the current release of OS X 10.3 uses code from FreeBSD 4.8 (can anyone correct me on this?) So the BSD subsystem is based on the latest branch of code from the FreeBSD 4.x branch. It’s doesn’t really put OS X behind in anyway as it is uses the latest Stable branch of code. Since the BSD subsystem is not directly linked into the Darwin kernel (aka: xnu) it does not affect the maturity of the kernel nor affect the drivers or hardware support of the OS.
FreeBSD subsystem provides Command Line tools, and network features such as Firewall services, as well as development libraries.
Apple’s developers need time to port FreeBSD code over to Darwin, as the core kernels are quite different. While it’s true that most of the networking and VFS stuff (among many other things) is almost identical to the same code in FreeBSD (at the time of import), a lot of the low level stuff (as mentioned earlier WRT Mach threads etc) are completely different, and the imported code must be tweaked to use it.
I say that this newest code won’t likely be in until version 10.5, as version 10.4 is stablizing now, and they are largely working with earlier 5.x code, and now is not the time for them to start pulling in massive changes like all the work that has gone into the fine-grained locking of the network stack – it’s too close to when they want to release 10.4 for it to be sane to do otherwise.
At any rate, there are anumber of Darwin CVS and source code mirrors that can let you know exactly which code came from FreeBSD and other sources, and from when. I personally like the source mirror at http://www.GoBSD.com, but the CVS repository at http://www.OpenDarwin.org might be the best place to go look and see for yourself.
looks like a bit of Darwin code is going back to FreeBSD…
Merge of Darwin msdosfs, other fixes
Not done
Apple’s Darwin operating system has fairly extensive improvements to msdosfs and other kernel services; these fixes must be reviewed and merged to the FreeBSD tree.
Apple’s Darwin operating system has fairly extensive improvements to msdosfs and other kernel services; these fixes must be reviewed and merged to the FreeBSD tree.
Actually, there are quite a few really impressive things that the Apple guys have done with their core OS, like the fact that their metadata journalling layer is theoretically accessible from all supported filesystems, as opposed to being funtionality that’s been built into just one or two. DragonFly is evolving to do things similarly.
looks like a bit of Darwin code is going back to FreeBSD…
Merge of Darwin msdosfs, other fixes
That note has been there since at least before 5.0 was released (perhaps even earlier as that is when I noticed it). And it is still not done. Not too high on the priority list I guess. It will most likely get done quickly when the more extensive changes for 5-STABLE is rock solid. The work of R. Watson on netperf looks soo sweet though and well worth the wait.
Actually, there are quite a few really impressive things that the Apple guys have done with their core OS, like the fact that their metadata journalling layer is theoretically accessible from all supported filesystems, as opposed to being funtionality that’s been built into just one or two. DragonFly is evolving to do things similarly.
What operating system has journalling built into one or two filesystems but not the VFS?
The actual underlying filesystem of course still has to support journalling no matter what, right? It isn’t like they just put a journalling layer in the VFS which makes all filesystems journalling filesystems.
The actual underlying filesystem of course still has to support journalling no matter what, right? It isn’t like they just put a journalling layer in the VFS which makes all filesystems journalling filesystems.
The journal is simply like a file, so it would be possible to put pseudo-journaling on any filesystem you want.
The journal is simply like a file, so it would be possible to put pseudo-journaling on any filesystem you want.
The VFS doesn’t know about the underlying structure of the filesystem.
About the closest you could get is to have the VFS call into the filesystem to tell it to write a structured log of metadata to the specific file. At that point you have a *journalling filesystem*, with a hack in the VFS that prevents filesystems that don’t use journal files (ie Reiser4).
With soft updates and background fsck, you have all the benefits of a journaling filesystem with none of the drawbacks, such as losing files in write cache, but also have the added benefits of metadata caching. Given that, who cares about journaling?
With soft updates and background fsck, you have all the benefits of a journaling filesystem with none of the drawbacks, such as losing files in write cache, but also have the added benefits of metadata caching. Given that, who cares about journaling?
What? Softupdates aren’t robust against writeback cache on disk drives or controllers any more than journalling filesystems. You need to turn off the cache or have write barrier support in the block layer (or trust a battery backed cache, which is a fair thing to do in many cases).
Also, softupdates, like today’s journalling filesystems don’t provide complete data integrity by any means. They ensure the filesystem metadata can will be in a sane state afterwards (or to the point where fsck can make it sane).
However tomorrows journalling filesystems (eg. Reiser4) can provide fully atomic filesystem transactions from a userspace point of view. Something that softupdates of course, will never be able to do by definition.
Journalling filesystems can be less complicated than softupdates, they can be more free to reorder write operations (giving higher performance).
Journalling filesystems don’t require fsck at all. Background fsck is a step forward, but still saps the IO system of a lot of IOPs which is a big negative in a lot of production scenarios.
It’s likely that I’ve read most the same papers on the topic as you have, and perhaps I’ve even been able to comprehend a large percentage of it. After all that reading, I’ve been more or less convinced that softupdates really were the way to go.
However there seems to be one group of people that have a big problem with softupdates, and that group is anyone who has worked on the softupdates code who is not also Marshall Kirk McKusick. While it’s certainly true that there are quite a number of people who understand how the system works, and can do minor maintenance on the code, but apparently there are very few people who are truely qualified to dig too deeply into the guts of this beast.
Surely journalling (specifically the all-to common metadata journalling) are suboptimal solutions in comparison, but they do have the distinct advantage of being more easily tested and debugged by mere mortals. From what I’ve read. I’m no expert of course, and I always find your comments enlightening, so please fill me in on anything obvious that you think I may have missed. 🙂
and also i want to know whether there is any utility in FreeBSD 5.3 to configure or remove hard device configuration automatically whenever a new hardware is added ??? and ask for a question to add or remove or something likethat, so that it will be easy to configure for desktop users..
This is possible now with FreeBSD 5.x for any hardware that it supports. The device nodes in the /dev directory are created / removed as hardware is added / removed from the system.
Plug in a USB harddrive, and the /dev/da0 node is created for you. Just mount that to a directory and off you go.
Add in a soundcard, boot the system, and /dev/snd* and its brethren are created for you.
Add in a CD-ROM drive, boot the system, and /dev/acd0 will appear.
And so on. All that is required is that you have the device drivers compiled into the kernel, or that you manually load the required kernel modules. Most modules are auto-loaded for you in certain circumstances (mount a DOS floppy, and the msdosfs.ko is loaded, plug in a USB mouse and the ums.ko is loaded, and so on).
It’s so much nicer than having to run MAKEDEV manually.
Perhaps you are thinking this is a Linux thread, since you are known supporter. Yes, we know it doesn’t work well in Linux. Or perhaps you’re just pretending to be a FreeBSD user, a typical Linux propoganda technique. Likely.
This release is slightly late: 3 days behind <a href=”http://www.freebsd.org/releases/5.3R/schedule.html“>schedule<… but since I believe the last release was 2 days late this is fairly on tract. After all these years ladies and gentlemen FreeBSD 5 should be here.
I’m betting on a Halloween release. Anyone else wanna guess?
What I can’t wait for is when all this new code gets imported into Darwin and therefore a Mac OS X release. I don’t think it will happen until 10.5 however.
is this a branch 6 under development yet?
Well, the cool stuff in FreeBSD 5.x (the theading model), won’t be in future MacOS X releases. Darwin uses the Mach threading code, not the FreeBSD threading code.
Yes there is a 6. It is current and was tagged as 6 around when the code freeze for 5.3beta was started.
Well, the cool stuff in FreeBSD 5.x (the theading model), won’t be in future MacOS X releases. Darwin uses the Mach threading code, not the FreeBSD threading code.
You’re thinking of kernel threads, and in that case you are indeed correct – XNU uses Mach threads, and not KSE. However, I was talking about the fine grained locking work in the network stack, which the Apple folks *will* be adopting.
It’s one of the reasons they are staying in sync with FreeBSD 5.x, so that a large amount of the hard work gets done for them, which is really a smart business move. Open source software is quite empowering that way.
Threading and locking are two completely different things.
I was wondering how this affected OS X. What will be the benefit of moving this code? I mean is it performance, security, etc? Also you said it would be around 10.5…why? How far behind is 10.3/10.4?
Thanks,
Matt (I’m not trolling)
I’m pleased to see that the X server configuration has been removed from sysinstall. It served no purpose beyond catching unwitting noobs off-guard.
I believe that the Tiger release 10.4 will be using FreeBSD 5.x stable branch code for the BSD Subsystem in that release. Atleast that’s what their developer pages and Tiger preview pages say about the BSD subsystem.
I believe that the current release of OS X 10.3 uses code from FreeBSD 4.8 (can anyone correct me on this?) So the BSD subsystem is based on the latest branch of code from the FreeBSD 4.x branch. It’s doesn’t really put OS X behind in anyway as it is uses the latest Stable branch of code. Since the BSD subsystem is not directly linked into the Darwin kernel (aka: xnu) it does not affect the maturity of the kernel nor affect the drivers or hardware support of the OS.
FreeBSD subsystem provides Command Line tools, and network features such as Firewall services, as well as development libraries.
Apple’s developers need time to port FreeBSD code over to Darwin, as the core kernels are quite different. While it’s true that most of the networking and VFS stuff (among many other things) is almost identical to the same code in FreeBSD (at the time of import), a lot of the low level stuff (as mentioned earlier WRT Mach threads etc) are completely different, and the imported code must be tweaked to use it.
I say that this newest code won’t likely be in until version 10.5, as version 10.4 is stablizing now, and they are largely working with earlier 5.x code, and now is not the time for them to start pulling in massive changes like all the work that has gone into the fine-grained locking of the network stack – it’s too close to when they want to release 10.4 for it to be sane to do otherwise.
At any rate, there are anumber of Darwin CVS and source code mirrors that can let you know exactly which code came from FreeBSD and other sources, and from when. I personally like the source mirror at http://www.GoBSD.com, but the CVS repository at http://www.OpenDarwin.org might be the best place to go look and see for yourself.
Anyway, hope that was helpful.
from:
http://www.freebsd.org/releases/5.3R/todo.html
looks like a bit of Darwin code is going back to FreeBSD…
Merge of Darwin msdosfs, other fixes
Not done
Apple’s Darwin operating system has fairly extensive improvements to msdosfs and other kernel services; these fixes must be reviewed and merged to the FreeBSD tree.
Apple’s Darwin operating system has fairly extensive improvements to msdosfs and other kernel services; these fixes must be reviewed and merged to the FreeBSD tree.
Actually, there are quite a few really impressive things that the Apple guys have done with their core OS, like the fact that their metadata journalling layer is theoretically accessible from all supported filesystems, as opposed to being funtionality that’s been built into just one or two. DragonFly is evolving to do things similarly.
looks like a bit of Darwin code is going back to FreeBSD…
Merge of Darwin msdosfs, other fixes
That note has been there since at least before 5.0 was released (perhaps even earlier as that is when I noticed it). And it is still not done. Not too high on the priority list I guess. It will most likely get done quickly when the more extensive changes for 5-STABLE is rock solid. The work of R. Watson on netperf looks soo sweet though and well worth the wait.
Actually, there are quite a few really impressive things that the Apple guys have done with their core OS, like the fact that their metadata journalling layer is theoretically accessible from all supported filesystems, as opposed to being funtionality that’s been built into just one or two. DragonFly is evolving to do things similarly.
What operating system has journalling built into one or two filesystems but not the VFS?
The actual underlying filesystem of course still has to support journalling no matter what, right? It isn’t like they just put a journalling layer in the VFS which makes all filesystems journalling filesystems.
The actual underlying filesystem of course still has to support journalling no matter what, right? It isn’t like they just put a journalling layer in the VFS which makes all filesystems journalling filesystems.
The journal is simply like a file, so it would be possible to put pseudo-journaling on any filesystem you want.
The journal is simply like a file, so it would be possible to put pseudo-journaling on any filesystem you want.
The VFS doesn’t know about the underlying structure of the filesystem.
About the closest you could get is to have the VFS call into the filesystem to tell it to write a structured log of metadata to the specific file. At that point you have a *journalling filesystem*, with a hack in the VFS that prevents filesystems that don’t use journal files (ie Reiser4).
With soft updates and background fsck, you have all the benefits of a journaling filesystem with none of the drawbacks, such as losing files in write cache, but also have the added benefits of metadata caching. Given that, who cares about journaling?
With soft updates and background fsck, you have all the benefits of a journaling filesystem with none of the drawbacks, such as losing files in write cache, but also have the added benefits of metadata caching. Given that, who cares about journaling?
What? Softupdates aren’t robust against writeback cache on disk drives or controllers any more than journalling filesystems. You need to turn off the cache or have write barrier support in the block layer (or trust a battery backed cache, which is a fair thing to do in many cases).
Also, softupdates, like today’s journalling filesystems don’t provide complete data integrity by any means. They ensure the filesystem metadata can will be in a sane state afterwards (or to the point where fsck can make it sane).
However tomorrows journalling filesystems (eg. Reiser4) can provide fully atomic filesystem transactions from a userspace point of view. Something that softupdates of course, will never be able to do by definition.
Journalling filesystems can be less complicated than softupdates, they can be more free to reorder write operations (giving higher performance).
Journalling filesystems don’t require fsck at all. Background fsck is a step forward, but still saps the IO system of a lot of IOPs which is a big negative in a lot of production scenarios.
But each to his own.
Given that, who cares about journaling?
It’s likely that I’ve read most the same papers on the topic as you have, and perhaps I’ve even been able to comprehend a large percentage of it. After all that reading, I’ve been more or less convinced that softupdates really were the way to go.
However there seems to be one group of people that have a big problem with softupdates, and that group is anyone who has worked on the softupdates code who is not also Marshall Kirk McKusick. While it’s certainly true that there are quite a number of people who understand how the system works, and can do minor maintenance on the code, but apparently there are very few people who are truely qualified to dig too deeply into the guts of this beast.
Surely journalling (specifically the all-to common metadata journalling) are suboptimal solutions in comparison, but they do have the distinct advantage of being more easily tested and debugged by mere mortals. From what I’ve read. I’m no expert of course, and I always find your comments enlightening, so please fill me in on anything obvious that you think I may have missed. 🙂
and also i want to know whether there is any utility in FreeBSD 5.3 to configure or remove hard device configuration automatically whenever a new hardware is added ??? and ask for a question to add or remove or something likethat, so that it will be easy to configure for desktop users..
-sathish
You mean hotplugging. it doesnt work well
Anonymous (IP: —.home.cgocable.net)
jbett (IP: —.cg.shawcable.net)
Thanks to both, I’ve got a better understanding now. I’ll check out the CVS too, because I hear about it a lot.
Matt
Never had any problem hot swapping a piece of hardware that is physically hot swappable here.
This is possible now with FreeBSD 5.x for any hardware that it supports. The device nodes in the /dev directory are created / removed as hardware is added / removed from the system.
Plug in a USB harddrive, and the /dev/da0 node is created for you. Just mount that to a directory and off you go.
Add in a soundcard, boot the system, and /dev/snd* and its brethren are created for you.
Add in a CD-ROM drive, boot the system, and /dev/acd0 will appear.
And so on. All that is required is that you have the device drivers compiled into the kernel, or that you manually load the required kernel modules. Most modules are auto-loaded for you in certain circumstances (mount a DOS floppy, and the msdosfs.ko is loaded, plug in a USB mouse and the ums.ko is loaded, and so on).
It’s so much nicer than having to run MAKEDEV manually.
Perhaps you are thinking this is a Linux thread, since you are known supporter. Yes, we know it doesn’t work well in Linux. Or perhaps you’re just pretending to be a FreeBSD user, a typical Linux propoganda technique. Likely.