Linked by moondevil on Wed 11th Jan 2012 00:10 UTC
Windows The latest blog entry from Steven Sinofsky about Windows 8 describes the Storage Spaces functionality . From the blog entry it seems Windows 8 is getting something ZFS-like. The Storage Spaces can be created in the command line via Powershell, or in the Control Panel for the ones that prefer a more mouse-friendly interface.
Thread beginning with comment 502950
To read all comments associated with this story, please click here.
OSNews RTFA and comments
by hechacker1 on Wed 11th Jan 2012 23:37 UTC
hechacker1
Member since:
2005-08-01

I think this is a very positive improvement in Windows 8. Especially for the server environment, or for users with lots of storage needs.

1. It's not just a reimplementation of existing windows logical and dynamic disks. It isn't even RAID 1, 0, or 5. It's a system that allows you to specify redundancy like RAID 1, or parity with RAID 5 or 6 like redundancy.

2. It uses 256MB chunks distributed around all disks equally, unless you specify certain conditions. For example, you could allocate speed critical sections to your fastest drive.

3. Over-provisioning is a nice feature because it means you can "set it and forget it." No need to do lengthy NTFS resizes and RAID rebuilds in the future when you need to add space. Simply pop in a new disk. You really reduce the risk of doing resizes and rebuilds too.

4. It can intelligently repopulate the 256MB chunks as needed, instead of rebuild the whole array.

5. Windows 8 includes things like background NTFS scans (scrubbing), online chkdsk (for most problems), and really fast offline chkdsk for serious problems.

6. Yes it doesn't include checksumming built into the storage pool (AFAIK so far), but it does provide an API to checksum the data you have on the storage pool. This allows you to have a program choose the correct copy of the data from the pool, and restore from that (assuming you have redundancy).

7. It lets you specify a backing device for the journal (similar to Linux), so you could put the journal on a SSD for the storage pool, and really speed up the slowest parts of calculating parity and keeping a consistent state.

All in all, it isn't ZFS, or even provide nearly as much flexibility as some of Linux's solutions, but it is a major improvement to Windows when dealing with lots of data.

Even for the average user, the background scrubbing and online NTFS checks will help catch errors before they get worse.

Edited 2012-01-11 23:42 UTC

Reply Score: 2

RE: OSNews RTFA and comments
by Neolander on Thu 12th Jan 2012 08:41 in reply to "OSNews RTFA and comments"
Neolander Member since:
2010-03-08

3. Over-provisioning is a nice feature because it means you can "set it and forget it." No need to do lengthy NTFS resizes and RAID rebuilds in the future when you need to add space. Simply pop in a new disk. You really reduce the risk of doing resizes and rebuilds too.

I don't understand this part. When you have virtual volumes made of several physical drives, isn't it possible to use, say, NTFS on every physical drive, and a much simpler virtual filesystem for the virtual drive that can be resized and remapped at will and transmits every "advanced" command (metadata, etc...) to the underlying physical FS ?

7. It lets you specify a backing device for the journal (similar to Linux), so you could put the journal on a SSD for the storage pool, and really speed up the slowest parts of calculating parity and keeping a consistent state.

This leads me to ask something I've regularly wondered about on Linux : what happens if the journal is corrupted on a journalized FS ? SSDs often brutally fail without warning and with full loss of data, could the filesystem survive such an event if it was only its journal that was on a SSD ?

Edited 2012-01-12 08:42 UTC

Reply Parent Score: 1

RE[2]: OSNews RTFA and comments
by Alfman on Thu 12th Jan 2012 16:56 in reply to "RE: OSNews RTFA and comments"
Alfman Member since:
2011-01-28

Neolander,

"isn't it possible to use, say, NTFS on every physical drive, and a much simpler virtual filesystem for the virtual drive that can be resized and remapped at will and transmits every 'advanced' command (metadata, etc...) to the underlying physical FS ?"

You're thinking something like a unionfs distributed across disks? That could work, but in my experience with AUFS, I think there are some performance issues with that approach, for example: the virtual fs has to check every union to see whether a file exists. It's not an insurmountable problem - one could create a fast index to locate files immediately, but I don't know why one wouldn't just use a dynamically growable file system directly over a dynamically growable container like an LVM.

"This leads me to ask something I've regularly wondered about on Linux : what happens if the journal is corrupted on a journalized FS ? SSDs often brutally fail without warning and with full loss of data, could the filesystem survive such an event if it was only its journal that was on a SSD ?"

I don't believe the journal is ever consulted if the file system is clean, so in that regard it's "optional". It is only intended to protect against a single type of failure mode: incomplete writes to the file system structures caused by power failures/crashes. I would think that a complete loss of the journal could only cause data loss if there is a simultaneous failure in the rest of the file system... however I don't have any experience with that kind of setup.

In any case I can't see any advantage to keeping the journal on an SSD as opposed to another hard disk, assuming the SSD has worse failure characteristics. The very property which SSD excels at, random access/seek time, gives no benefit for purely sequential journal data. The journal itself is extremely unlikely to be a bottleneck on a second disk.

Reply Parent Score: 2

hechacker1 Member since:
2005-08-01

I don't understand this part. When you have virtual volumes made of several physical drives, isn't it possible to use, say, NTFS on every physical drive, and a much simpler virtual filesystem for the virtual drive that can be resized and remapped at will and transmits every "advanced" command (metadata, etc...) to the underlying physical FS ?


The difference you are talking about is only in implementation. You can either have an underlying NTFS partition with some type of file-system on top to combine it, or an underlying storage pool (like LVM), with a NTFS partition on top.

Does it really matter which way it's done? Not really.

What matters in this case is that when you do need to resize, it's a simple operation of adding another disk. There's no need to resize the filesystem, or rebuild the entire array to distribute parity. Each of those operations carries the risk of failure; while having over-provisioning from the start means that the storage pool has already considered those cases in its design.

7. It lets you specify a backing device for the journal (similar to Linux), so you could put the journal on a SSD for the storage pool, and really speed up the slowest parts of calculating parity and keeping a consistent state.

This leads me to ask something I've regularly wondered about on Linux : what happens if the journal is corrupted on a journalized FS ? SSDs often brutally fail without warning and with full loss of data, could the filesystem survive such an event if it was only its journal that was on a SSD ?


With ext4, if the journal is corrupted, then you can just discard it. You can fallback to ext4 mode without journaling (similar to ext2). The data itself will be fine, but anything in the journal could be lost. Then it's possible to rebuild the journal when you reactivate it on a new device.

NTFS has a similar structure. The journal can be disabled when required. With Windows 8, it can also be allocated to a faster device (finally!).

http://msdn.microsoft.com/en-us/library/windows/desktop/aa363877(v=...).aspx

Edited 2012-01-12 19:06 UTC

Reply Parent Score: 2

RE[2]: OSNews RTFA and comments
by phoenix on Mon 16th Jan 2012 18:18 in reply to "RE: OSNews RTFA and comments"
phoenix Member since:
2005-07-11

"3. Over-provisioning is a nice feature because it means you can "set it and forget it." No need to do lengthy NTFS resizes and RAID rebuilds in the future when you need to add space. Simply pop in a new disk. You really reduce the risk of doing resizes and rebuilds too.

I don't understand this part. When you have virtual volumes made of several physical drives, isn't it possible to use, say, NTFS on every physical drive, and a much simpler virtual filesystem for the virtual drive that can be resized and remapped at will and transmits every "advanced" command (metadata, etc...) to the underlying physical FS ?
"

You're thinking of things backwards. You don't want to layer filesystems like that.

You want to aggregate your physical storage into a "virtual volume" or "storage pool". Then split that pool up into smaller chunks, whether that be "logical volumes" or "partitions" or whatever. And, then, finally, at the very top, the "filesystem".

Linux does this via LVM (physical storage -> physical volumes -> volume groups -> logical volume -> filesystem).

Solaris/OSol/FreeBSD does this via ZFS (physical storage -> virtual devices -> pool -> filesystems).

And now Windows has this (physical storage -> pool -> spaces -> filesystems).

This leads me to ask something I've regularly wondered about on Linux : what happens if the journal is corrupted on a journalized FS ? SSDs often brutally fail without warning and with full loss of data, could the filesystem survive such an event if it was only its journal that was on a SSD ?


In most filesystems/storage systems, the journal is write-only. Data is written to the journal, then later, the data is also written to the filesystem.

The only time the journal is read is at boot, and only if the filesystem is "dirty".

If the journal disk dies, and the filesystem is "clean", then no big deal, all the data in the journal is also in the filesystem.

If the journal disk dies and the filesystem is "dirty", then you will lose whatever data is in the journal and not yet written to the disk.

Reply Parent Score: 2