OSNews:

OSNews: http://www.osnews.com/story/20137/NetBSD_Metadata_Journaling_Support_Added_to_FFS Exploring the Future of Computing en-us Copyright 2001-2009, David Adams adam+nospam@osnews.com Mon, 09 Nov 2009 04:43:20 GMT http://www.osnews.com/images/osnews.gif OSNews.com http://www.osnews.com NetBSD just got journaling? http://osnews.com/thread?325294 http://osnews.com/thread?325294 So, NetBSD just got journaling? Did they not have soft-updates previously? How can a server OS survive in today's large-disk world without some way to get around its fscking problems? Fri, 01 Aug 2008 16:05:00 GMT donotreply@osnews.com (sbergman27) Comments nice http://osnews.com/thread?325296 http://osnews.com/thread?325296 I am really glad to see Wasabi Systems contributing code back into NetBSD. Wasabi has done a lot of cool things with NetBSD, I hope more of them make it into the 5.0 release. Fri, 01 Aug 2008 16:09:00 GMT donotreply@osnews.com (poundsmack) Comments soft updates http://osnews.com/thread?325298 http://osnews.com/thread?325298 The whole debate of soft updates vs journaling, and the bragging of BSD dudes about soft updates being superior, is now enlightened with a whole new light.

Of course, I applaud NetBSD and Wasibi for offering this, and it's a Good Thing that the user can /choose/ between soft updates and journaling.

But what will be the default? Journaling or soft updates? And if it will be journaling, are the fs devs who advised journaling still to be taken serious? Fri, 01 Aug 2008 16:23:00 GMT donotreply@osnews.com (evert) Comments RE: NetBSD just got journaling? http://osnews.com/thread?325299 http://osnews.com/thread?325299

Did they not have soft-updates previously?

Soft updates were available and documented here: http://www.netbsd.org/docs/misc/#softdeps Fri, 01 Aug 2008 16:27:00 GMT donotreply@osnews.com (anomie) Comments RE: soft updates http://osnews.com/thread?325300 http://osnews.com/thread?325300 My understanding has been that soft-updates was a fairly complex feature to implement. More so than journaling. It may very well be that the maintenance overhead outweighed any possible technical advantages it may have had. The BSDs have always been a bit out in right field, in my opinion, on this issue. Fri, 01 Aug 2008 16:43:00 GMT donotreply@osnews.com (sbergman27) Comments RE: NetBSD just got journaling? http://osnews.com/thread?325301 http://osnews.com/thread?325301

So, NetBSD just got journaling?

Yes...but not in the same way other people do journaling. Apparently they've metadata journaling - just like XFS, JFS or ext3 with the "data=writeback" mount option.

But if you only journal the metadata, you're only sure that the metadata is safe - not the data. IOW, you don't get metadata corruption (you don't need to fsck after a hard reboot), but the data of the file you were writting at the time of the reboot can be lost/corrupted. In XFS for example a file can get completely zeroed after a hard reboot.

There're two main ways to workaround this. One, journal the data (like in ext3 with the data=journal option) - but it's too slow. Second, write the data of a file before you journal its metadata, so that even in the event of a hard reboot, you are sure that the content of the file are not corrupted/lost - they may be outdated (the data was in ram and was not written to disk when the reboot happened), but not lost/corrupte (you may get a "corrupted" file in the sense that the process which was writting to it doesn't finishes writting what it was doing, but certainly you don't get rubbish). This is what ext3 does by default - it harms performance, but it's worth of it, specially for desktops.

You can also avoid completely the journaling, like zfs/btrfs do, but that's not really a "journaling mode" Fri, 01 Aug 2008 16:43:00 GMT donotreply@osnews.com (diegocg) Comments RE: NetBSD just got journaling? http://osnews.com/thread?325307 http://osnews.com/thread?325307

Did they not have soft-updates previously?

it says in the announcement to remove "softdep" if presnet.

Note that WAPBL is not compatible with soft-dependencies, so please ensure that you first remove the softdep option if present. See the wapbl(4) manual page for more information.

Edited 2008-08-01 17:25 UTC Fri, 01 Aug 2008 17:24:00 GMT donotreply@osnews.com (happycamper) Comments RE: soft updates http://osnews.com/thread?325309 http://osnews.com/thread?325309 The default is neither logging nor softdep. The NetBSD installer sysinst will let you enable one or the other, or you can enable it after the fact by editing /etc/fstab. Fri, 01 Aug 2008 17:47:00 GMT donotreply@osnews.com (jmcneill) Comments RE: nice http://osnews.com/thread?325339 http://osnews.com/thread?325339 And it only took them five years to open-source it. A big round of applause for Wasabi! Sat, 02 Aug 2008 01:28:00 GMT donotreply@osnews.com (fuzzyping) Comments RE[2]: soft updates http://osnews.com/thread?325341 http://osnews.com/thread?325341

The default is neither logging nor softdep. The NetBSD installer sysinst will let you enable one or the other, or you can enable it after the fact by editing /etc/fstab.

There should be a default. Quivering in limbo is not proper installer behavior.Edited 2008-08-02 02:17 UTC Sat, 02 Aug 2008 02:16:00 GMT donotreply@osnews.com (sbergman27) Comments RE[3]: soft updates http://osnews.com/thread?325352 http://osnews.com/thread?325352 It's not "quivering in limbo," but rather the filesystem's default state is to have neither softdep nor journaling enabled and, depending on certain partition setups, this actually does make sense. Take, for example, a secure partition for applications, intended to be mounted read-only, and the partition archive/image is already created. The only thing that you intend to do with that partition is mount it rw, extract the archive, then mount it read-only from then on. In this case, you'd want to leave both softdep and journaling turned off. Sure, it takes about five seconds to go into /etc/fstab and get rid of the option, but if you know you're going to use that partition for a specific use where you know you won't need them, why have them enabled at all to start? Also, NetBSD is not a hand-holding system, period. It assumes you know what you're doing, it's defaults are barebones. The idea is that if you need something, you know you need it and will enable it yourself, either during the installation process or afterwards. Given this, it only makes sense that neither softdeps nor journaling is used by default, as it's not the filesystem's default state. Sat, 02 Aug 2008 08:05:00 GMT donotreply@osnews.com (darknexus) Comments RE[4]: soft updates http://osnews.com/thread?325359 http://osnews.com/thread?325359 sbergman27 was talking about a default. The default is currently to have no journaling AND no soft updates. IMHO, that's a bad choice. Having a "safe" default protects data. The scenarios you explained are valid, but users who know what they are doing will know how to turn it of for those special cases, and since it's only a few seconds editing fstab...

But the installer should default to something that protects data. Period. Sat, 02 Aug 2008 10:27:00 GMT donotreply@osnews.com (evert) Comments RE: NetBSD just got journaling? http://osnews.com/thread?325360 http://osnews.com/thread?325360

So, NetBSD just got journaling? Did they not have soft-updates previously?

Soft-updates and the log-structured filesystem (LFS). But soft-updates are fairly complex, and LFS does not have the same proven reliability as FFS. So, the addition of journaling to FFS is great, and a very nice contribution from Wasabi! Sat, 02 Aug 2008 10:28:00 GMT donotreply@osnews.com (danieldk) Comments RE[5]: soft updates http://osnews.com/thread?325362 http://osnews.com/thread?325362

But the installer should default to something that protects data. Period.

I can assure you that your data is protected on FFS with neither logging nor softdeps enabled. What could possibly make you think otherwise? Sat, 02 Aug 2008 11:34:00 GMT donotreply@osnews.com (jmcneill) Comments RE[6]: soft updates http://osnews.com/thread?325363 http://osnews.com/thread?325363 If you are right, and the data is protected without soft updates and journaling, why the effort of adding them?

I'm not convinced yet.

Edit: OK, maybe you mean that those additions are only meant to shorten filesystem checks. I would agree with that. Still, fast filesystem recovery, when possible, is normally a default option for both desktops and servers. I remember Windows 98 blue start screens, scanning the FAT32 filesystem for 15 minutes. Windows 2000, with NTFS (journalled), booted fine, even after an unclean shutdown. Same goes for Linux with Ext3, it's mostly faster. All OS-es I know of default to enable the journaling. I don't see a compelling argument not to do it.Edited 2008-08-02 11:46 UTC Sat, 02 Aug 2008 11:41:00 GMT donotreply@osnews.com (evert) Comments RE[7]: soft updates http://osnews.com/thread?325396 http://osnews.com/thread?325396

If you are right, and the data is protected without soft updates and journalling, why the effort of adding them?

To check off a buzzword? They listened to user feedback? I'm sure there are people who need those features, and adding journalling keeps the people who do need it from jumping ship.

I don't see a compelling argument not to do it.

NetBSD fills the portable embedded system niche in the BSD universe. It's going to be installed on systems that don't necessarily need journalling or soft-updates on the system, or the systems don't have the resources to support them. Journalling takes up disk space and disk IO throughput takes a hit. Sun, 03 Aug 2008 03:43:00 GMT donotreply@osnews.com (Flatland_Spider) Comments RE[8]: soft updates http://osnews.com/thread?325407 http://osnews.com/thread?325407

That was the kind of argument I was waiting for :-)
Thanks, now I understand the reason not to enable one of them by default. Sun, 03 Aug 2008 06:31:00 GMT donotreply@osnews.com (evert) Comments RE[9]: soft updates http://osnews.com/thread?325409 http://osnews.com/thread?325409

That was the kind of argument I was waiting for :-)

But how concerned are embedded developers with the standard NetBSD installer? Do they use it to install the OS on each toaster? This discussion should be about regular users of NetBSD on the PCs with which we are all familiar. Sun, 03 Aug 2008 06:55:00 GMT donotreply@osnews.com (sbergman27) Comments RE[10]: soft updates http://osnews.com/thread?325412 http://osnews.com/thread?325412

"That was the kind of argument I was waiting for :-)

But how concerned are embedded developers with the standard NetBSD installer? Do they use it to install the OS on each toaster? This discussion should be about regular users of NetBSD on the PCs with which we are all familiar. "

Very true. But even desktop users care about performance. Only installing the bare minimum (by default) ensures that you have a fast computing experience. It's like Slackware: quite fast, but some things are not installed by default and it is marketed to experienced users. (although slack has ext3/reiser support per default) I think I understand the reasoning. Personally, I really care about fast filesystem recovery - both for desktops and servers - and I would choose a distro wich defaults to journalling myself, but that is about choice I guess.Edited 2008-08-03 07:36 UTC Sun, 03 Aug 2008 07:35:00 GMT donotreply@osnews.com (evert) Comments RE[11]: soft updates http://osnews.com/thread?325419 http://osnews.com/thread?325419

But even desktop users care about performance.

If I recall correctly (and I welcome any corrections from people better informed than I am upon this matter), the *BSDs have traditionally used synchronous metadata updates in FFS. I remember the FreeBSD guys used to criticize us ext2 users for playing so fast and loose with our metadata before journaling filesystems came to Linux. Those synchronous updates have to extract a significant performance penalty. Metadata journaling is likely faster. And I suspect that soft updates would also be faster.Edited 2008-08-03 15:08 UTC Sun, 03 Aug 2008 15:07:00 GMT donotreply@osnews.com (sbergman27) Comments RE[12]: soft updates http://osnews.com/thread?325422 http://osnews.com/thread?325422 That's correct, UFS itself is synchronous. This is much more secure, but also much slower. The idea of FFS and meta-journaling was to get asynchronous like performance, but still maintain synchronous data security. Sun, 03 Aug 2008 15:37:00 GMT donotreply@osnews.com (itanic) Comments