Linked by Amjith Ramanujam on Fri 1st Aug 2008 15:32 UTC, submitted by Daniel Sieger
NetBSD Today, support for metadata journaling has been added to NetBSD's implementation of the Berkeley Fast File System (FFS), eliminating the need for lengthy file system checks after a crash or power failure. Support for converting existing file systems to use the new journaling capabilities is provided as well. See the official announcement for all the details.
Order by: Score:
NetBSD just got journaling?
by sbergman27 on Fri 1st Aug 2008 16:05 UTC
sbergman27
Member since:
2005-07-24

So, NetBSD just got journaling? Did they not have soft-updates previously? How can a server OS survive in today's large-disk world without some way to get around its fscking problems?

Reply Score: 2

RE: NetBSD just got journaling?
by anomie on Fri 1st Aug 2008 16:27 UTC in reply to "NetBSD just got journaling?"
anomie Member since:
2007-02-26

Did they not have soft-updates previously?


Soft updates were available and documented here: http://www.netbsd.org/docs/misc/#softdeps

Reply Score: 2

RE: NetBSD just got journaling?
by diegocg on Fri 1st Aug 2008 16:43 UTC in reply to "NetBSD just got journaling?"
diegocg Member since:
2005-07-08

So, NetBSD just got journaling?



Yes...but not in the same way other people do journaling. Apparently they've metadata journaling - just like XFS, JFS or ext3 with the "data=writeback" mount option.

But if you only journal the metadata, you're only sure that the metadata is safe - not the data. IOW, you don't get metadata corruption (you don't need to fsck after a hard reboot), but the data of the file you were writting at the time of the reboot can be lost/corrupted. In XFS for example a file can get completely zeroed after a hard reboot.

There're two main ways to workaround this. One, journal the data (like in ext3 with the data=journal option) - but it's too slow. Second, write the data of a file before you journal its metadata, so that even in the event of a hard reboot, you are sure that the content of the file are not corrupted/lost - they may be outdated (the data was in ram and was not written to disk when the reboot happened), but not lost/corrupte (you may get a "corrupted" file in the sense that the process which was writting to it doesn't finishes writting what it was doing, but certainly you don't get rubbish). This is what ext3 does by default - it harms performance, but it's worth of it, specially for desktops.

You can also avoid completely the journaling, like zfs/btrfs do, but that's not really a "journaling mode"

Reply Score: 3

RE: NetBSD just got journaling?
by happycamper on Fri 1st Aug 2008 17:24 UTC in reply to "NetBSD just got journaling?"
happycamper Member since:
2006-01-01

Did they not have soft-updates previously?



it says in the announcement to remove "softdep" if presnet.

Note that WAPBL is not compatible with soft-dependencies, so please ensure that you first remove the softdep option if present. See the wapbl(4) manual page for more information.


Edited 2008-08-01 17:25 UTC

Reply Score: 3

RE: NetBSD just got journaling?
by danieldk on Sat 2nd Aug 2008 10:28 UTC in reply to "NetBSD just got journaling?"
danieldk Member since:
2005-11-18

So, NetBSD just got journaling? Did they not have soft-updates previously?


Soft-updates and the log-structured filesystem (LFS). But soft-updates are fairly complex, and LFS does not have the same proven reliability as FFS. So, the addition of journaling to FFS is great, and a very nice contribution from Wasabi!

Reply Score: 3

nice
by poundsmack on Fri 1st Aug 2008 16:09 UTC
poundsmack
Member since:
2005-07-13

I am really glad to see Wasabi Systems contributing code back into NetBSD. Wasabi has done a lot of cool things with NetBSD, I hope more of them make it into the 5.0 release.

Reply Score: 4

v RE: nice
by fuzzyping on Sat 2nd Aug 2008 01:28 UTC in reply to "nice"
soft updates
by evert on Fri 1st Aug 2008 16:23 UTC
evert
Member since:
2005-07-06

The whole debate of soft updates vs journaling, and the bragging of BSD dudes about soft updates being superior, is now enlightened with a whole new light.

Of course, I applaud NetBSD and Wasibi for offering this, and it's a Good Thing that the user can /choose/ between soft updates and journaling.

But what will be the default? Journaling or soft updates? And if it will be journaling, are the fs devs who advised journaling still to be taken serious?

Reply Score: 3

RE: soft updates
by sbergman27 on Fri 1st Aug 2008 16:43 UTC in reply to "soft updates"
sbergman27 Member since:
2005-07-24

My understanding has been that soft-updates was a fairly complex feature to implement. More so than journaling. It may very well be that the maintenance overhead outweighed any possible technical advantages it may have had. The BSDs have always been a bit out in right field, in my opinion, on this issue.

Reply Score: 5

RE: soft updates
by jmcneill on Fri 1st Aug 2008 17:47 UTC in reply to "soft updates"
jmcneill Member since:
2008-08-01

The default is neither logging nor softdep. The NetBSD installer sysinst will let you enable one or the other, or you can enable it after the fact by editing /etc/fstab.

Reply Score: 3

RE[2]: soft updates
by sbergman27 on Sat 2nd Aug 2008 02:16 UTC in reply to "RE: soft updates"
sbergman27 Member since:
2005-07-24

The default is neither logging nor softdep. The NetBSD installer sysinst will let you enable one or the other, or you can enable it after the fact by editing /etc/fstab.


There should be a default. Quivering in limbo is not proper installer behavior.

Edited 2008-08-02 02:17 UTC

Reply Score: 1

RE[3]: soft updates
by darknexus on Sat 2nd Aug 2008 08:05 UTC in reply to "RE[2]: soft updates"
darknexus Member since:
2008-07-15

It's not "quivering in limbo," but rather the filesystem's default state is to have neither softdep nor journaling enabled and, depending on certain partition setups, this actually does make sense. Take, for example, a secure partition for applications, intended to be mounted read-only, and the partition archive/image is already created. The only thing that you intend to do with that partition is mount it rw, extract the archive, then mount it read-only from then on. In this case, you'd want to leave both softdep and journaling turned off. Sure, it takes about five seconds to go into /etc/fstab and get rid of the option, but if you know you're going to use that partition for a specific use where you know you won't need them, why have them enabled at all to start? Also, NetBSD is not a hand-holding system, period. It assumes you know what you're doing, it's defaults are barebones. The idea is that if you need something, you know you need it and will enable it yourself, either during the installation process or afterwards. Given this, it only makes sense that neither softdeps nor journaling is used by default, as it's not the filesystem's default state.

Reply Score: 3

RE[4]: soft updates
by evert on Sat 2nd Aug 2008 10:27 UTC in reply to "RE[3]: soft updates"
evert Member since:
2005-07-06

sbergman27 was talking about a default. The default is currently to have no journaling AND no soft updates. IMHO, that's a bad choice. Having a "safe" default protects data. The scenarios you explained are valid, but users who know what they are doing will know how to turn it of for those special cases, and since it's only a few seconds editing fstab...

But the installer should default to something that protects data. Period.

Reply Score: 3

RE[5]: soft updates
by jmcneill on Sat 2nd Aug 2008 11:34 UTC in reply to "RE[4]: soft updates"
jmcneill Member since:
2008-08-01

But the installer should default to something that protects data. Period.


I can assure you that your data is protected on FFS with neither logging nor softdeps enabled. What could possibly make you think otherwise?

Reply Score: 2

RE[6]: soft updates
by evert on Sat 2nd Aug 2008 11:41 UTC in reply to "RE[5]: soft updates"
evert Member since:
2005-07-06

If you are right, and the data is protected without soft updates and journaling, why the effort of adding them?

I'm not convinced yet.

Edit: OK, maybe you mean that those additions are only meant to shorten filesystem checks. I would agree with that. Still, fast filesystem recovery, when possible, is normally a default option for both desktops and servers. I remember Windows 98 blue start screens, scanning the FAT32 filesystem for 15 minutes. Windows 2000, with NTFS (journalled), booted fine, even after an unclean shutdown. Same goes for Linux with Ext3, it's mostly faster. All OS-es I know of default to enable the journaling. I don't see a compelling argument not to do it.

Edited 2008-08-02 11:46 UTC

Reply Score: 2

RE[7]: soft updates
by Flatland_Spider on Sun 3rd Aug 2008 03:43 UTC in reply to "RE[6]: soft updates"
Flatland_Spider Member since:
2006-09-01

If you are right, and the data is protected without soft updates and journalling, why the effort of adding them?


To check off a buzzword? They listened to user feedback? I'm sure there are people who need those features, and adding journalling keeps the people who do need it from jumping ship.


I don't see a compelling argument not to do it.


NetBSD fills the portable embedded system niche in the BSD universe. It's going to be installed on systems that don't necessarily need journalling or soft-updates on the system, or the systems don't have the resources to support them. Journalling takes up disk space and disk IO throughput takes a hit.

Reply Score: 3