Today, support for metadata journaling has been added to NetBSD’s implementation of the Berkeley Fast File System (FFS), eliminating the need for lengthy file system checks after a crash or power failure. Support for converting existing file systems to use the new journaling capabilities is provided as well. See the official announcement
for all the details.
So, NetBSD just got journaling? Did they not have soft-updates previously? How can a server OS survive in today’s large-disk world without some way to get around its fscking problems?
Soft updates were available and documented here: http://www.netbsd.org/docs/misc/#softdeps
Yes…but not in the same way other people do journaling. Apparently they’ve metadata journaling – just like XFS, JFS or ext3 with the “data=writeback” mount option.
But if you only journal the metadata, you’re only sure that the metadata is safe – not the data. IOW, you don’t get metadata corruption (you don’t need to fsck after a hard reboot), but the data of the file you were writting at the time of the reboot can be lost/corrupted. In XFS for example a file can get completely zeroed after a hard reboot.
There’re two main ways to workaround this. One, journal the data (like in ext3 with the data=journal option) – but it’s too slow. Second, write the data of a file before you journal its metadata, so that even in the event of a hard reboot, you are sure that the content of the file are not corrupted/lost – they may be outdated (the data was in ram and was not written to disk when the reboot happened), but not lost/corrupte (you may get a “corrupted” file in the sense that the process which was writting to it doesn’t finishes writting what it was doing, but certainly you don’t get rubbish). This is what ext3 does by default – it harms performance, but it’s worth of it, specially for desktops.
You can also avoid completely the journaling, like zfs/btrfs do, but that’s not really a “journaling mode”
it says in the announcement to remove “softdep” if presnet.
Edited 2008-08-01 17:25 UTC
Soft-updates and the log-structured filesystem (LFS). But soft-updates are fairly complex, and LFS does not have the same proven reliability as FFS. So, the addition of journaling to FFS is great, and a very nice contribution from Wasabi!
I am really glad to see Wasabi Systems contributing code back into NetBSD. Wasabi has done a lot of cool things with NetBSD, I hope more of them make it into the 5.0 release.
And it only took them five years to open-source it. A big round of applause for Wasabi!
</sarcasm>
The whole debate of soft updates vs journaling, and the bragging of BSD dudes about soft updates being superior, is now enlightened with a whole new light.
Of course, I applaud NetBSD and Wasibi for offering this, and it’s a Good Thing that the user can /choose/ between soft updates and journaling.
But what will be the default? Journaling or soft updates? And if it will be journaling, are the fs devs who advised journaling still to be taken serious?
My understanding has been that soft-updates was a fairly complex feature to implement. More so than journaling. It may very well be that the maintenance overhead outweighed any possible technical advantages it may have had. The BSDs have always been a bit out in right field, in my opinion, on this issue.
The default is neither logging nor softdep. The NetBSD installer sysinst will let you enable one or the other, or you can enable it after the fact by editing /etc/fstab.
There should be a default. Quivering in limbo is not proper installer behavior.
Edited 2008-08-02 02:17 UTC
It’s not “quivering in limbo,” but rather the filesystem’s default state is to have neither softdep nor journaling enabled and, depending on certain partition setups, this actually does make sense. Take, for example, a secure partition for applications, intended to be mounted read-only, and the partition archive/image is already created. The only thing that you intend to do with that partition is mount it rw, extract the archive, then mount it read-only from then on. In this case, you’d want to leave both softdep and journaling turned off. Sure, it takes about five seconds to go into /etc/fstab and get rid of the option, but if you know you’re going to use that partition for a specific use where you know you won’t need them, why have them enabled at all to start? Also, NetBSD is not a hand-holding system, period. It assumes you know what you’re doing, it’s defaults are barebones. The idea is that if you need something, you know you need it and will enable it yourself, either during the installation process or afterwards. Given this, it only makes sense that neither softdeps nor journaling is used by default, as it’s not the filesystem’s default state.
sbergman27 was talking about a default. The default is currently to have no journaling AND no soft updates. IMHO, that’s a bad choice. Having a “safe” default protects data. The scenarios you explained are valid, but users who know what they are doing will know how to turn it of for those special cases, and since it’s only a few seconds editing fstab…
But the installer should default to something that protects data. Period.
I can assure you that your data is protected on FFS with neither logging nor softdeps enabled. What could possibly make you think otherwise?
If you are right, and the data is protected without soft updates and journaling, why the effort of adding them?
I’m not convinced yet.
Edit: OK, maybe you mean that those additions are only meant to shorten filesystem checks. I would agree with that. Still, fast filesystem recovery, when possible, is normally a default option for both desktops and servers. I remember Windows 98 blue start screens, scanning the FAT32 filesystem for 15 minutes. Windows 2000, with NTFS (journalled), booted fine, even after an unclean shutdown. Same goes for Linux with Ext3, it’s mostly faster. All OS-es I know of default to enable the journaling. I don’t see a compelling argument not to do it.
Edited 2008-08-02 11:46 UTC
To check off a buzzword? They listened to user feedback? I’m sure there are people who need those features, and adding journalling keeps the people who do need it from jumping ship.
NetBSD fills the portable embedded system niche in the BSD universe. It’s going to be installed on systems that don’t necessarily need journalling or soft-updates on the system, or the systems don’t have the resources to support them. Journalling takes up disk space and disk IO throughput takes a hit.
That was the kind of argument I was waiting for 🙂
Thanks, now I understand the reason not to enable one of them by default.
But how concerned are embedded developers with the standard NetBSD installer? Do they use it to install the OS on each toaster? This discussion should be about regular users of NetBSD on the PCs with which we are all familiar.
Very true. But even desktop users care about performance. Only installing the bare minimum (by default) ensures that you have a fast computing experience. It’s like Slackware: quite fast, but some things are not installed by default and it is marketed to experienced users. (although slack has ext3/reiser support per default) I think I understand the reasoning. Personally, I really care about fast filesystem recovery – both for desktops and servers – and I would choose a distro wich defaults to journalling myself, but that is about choice I guess.
Edited 2008-08-03 07:36 UTC
If I recall correctly (and I welcome any corrections from people better informed than I am upon this matter), the *BSDs have traditionally used synchronous metadata updates in FFS. I remember the FreeBSD guys used to criticize us ext2 users for playing so fast and loose with our metadata before journaling filesystems came to Linux. Those synchronous updates have to extract a significant performance penalty. Metadata journaling is likely faster. And I suspect that soft updates would also be faster.
Edited 2008-08-03 15:08 UTC
That’s correct, UFS itself is synchronous. This is much more secure, but also much slower. The idea of FFS and meta-journaling was to get asynchronous like performance, but still maintain synchronous data security.