Linked by malxau on Tue 17th Jan 2012 13:34 UTC
Windows Along with Storage Spaces coming in Windows 8, ReFS forms the foundation of storage on Windows for the next decade or more. Key features of the two when used together include: metadata integrity with checksums; integrity streams providing optional user data integrity; allocate on write transactional model for robust disk updates; large volume, file and directory sizes; storage pooling and virtualization making file system creation and management easy; data striping for performance and redundancy for fault tolerance; disk scrubbing for protection against latent disk errors; resiliency to corruptions with "salvage" for maximum volume availability in all cases; and shared storage pools across machines for additional failure tolerance and load balancing.
Order by: Score:
Looks good
by kragil on Tue 17th Jan 2012 14:05 UTC
kragil
Member since:
2006-01-04

Although some nice features of NTFS will be missing it uses some current FS "best practices" like B-trees, Copy on write, integrity checks and extends (like ZFS and BtrFS)

MS has gone for complexity reduction (reduced feature set and NTFS code sharing) because otherwise you cannot ship a new production ready file system.

That it shares a lot of code with NTFS could be nice for a reimplementation in alternative operating systems.

Reply Score: 5

RE: Looks good
by B. Janssen on Thu 19th Jan 2012 09:45 UTC in reply to "Looks good"
B. Janssen Member since:
2006-10-11

Although some nice features of NTFS will be missing it uses some current FS "best practices" like B-trees, Copy on write, integrity checks and extends (like ZFS and BtrFS)

NTFS has all of these, but copy-on-write. SIS and VSC use something similiar, though. And unlike ZFS and BtrFS NTFS also has a working undelete function ;)

Other than that I agree, ReFS looks good. Let's wait and see where it is going.

Reply Score: 1

RE[2]: Looks good
by kragil on Fri 20th Jan 2012 08:09 UTC in reply to "RE: Looks good"
kragil Member since:
2006-01-04

BS.
NTFS is not B-tree based, has no file integrity checks and no extends and it misses a lot more features ZFS and BtrFS have.
Read the Wikipedia or something.

Edited 2012-01-20 08:15 UTC

Reply Score: 2

Don't worry
by Sodki on Tue 17th Jan 2012 14:51 UTC
Sodki
Member since:
2005-11-10

Don't worry, ReFS is most likely to be scrapped before the Windows 8 Server release. Don't mod me down, you know it's true.

Reply Score: 2

RE: Don't worry
by silviucc on Tue 17th Jan 2012 14:57 UTC in reply to "Don't worry"
silviucc Member since:
2009-12-05

Actually I know they're going with it. Linux will have btrfs (Oracle will make sure of it), Solaris and FreeBSD have ZFS so Windows Server is left waaaaaay behind and MS does not want that.

Aint competition a great thing ?

Reply Score: 2

RE: Don't worry
by dragossh on Tue 17th Jan 2012 15:42 UTC in reply to "Don't worry"
dragossh Member since:
2008-12-16

After Vista launched with lots of features removed, Microsoft brought the Office team to work on Windows. Ever since then, whenever they announce and demo a feature, you can bet it's going to pop up in the operating system. So, yes, it is going to be in Server.

Edited 2012-01-17 15:42 UTC

Reply Score: 5

What Will Be Removed
by Pro-Competition on Tue 17th Jan 2012 15:33 UTC
Pro-Competition
Member since:
2007-08-20

FTA:

Q) What semantics or features of NTFS are no longer supported on ReFS?

The NTFS features we have chosen to not support in ReFS are: named streams, object IDs, short names, compression, file level encryption (EFS), user data transactions, sparse, hard-links, extended attributes, and quotas.


Some of those are useful features, even if they are underused.

Reply Score: 4

RE: What Will Be Removed
by cmchittom on Tue 17th Jan 2012 16:01 UTC in reply to "What Will Be Removed"
cmchittom Member since:
2011-03-18

Some of those are useful features, even if they are underused.


True. But didn't NTFS go through several iterations? Wikipedia says five versions:

The NTFS on-disk format has five released versions:

[ . . . ]

V1.0 and V1.1 (and newer) are incompatible: that is, volumes written by NT 3.5x cannot be read by NT 3.1 until an update on the NT 3.5x CD is applied to NT 3.1, which also adds FAT long file name support.[9] V1.2 supports compressed files, named streams, ACL-based security, etc.[2] V3.0 added disk quotas, encryption, sparse files, reparse points, update sequence number (USN) journaling, the $Extend folder and its files, and reorganized security descriptors so that multiple files which use the same security setting can share the same descriptor.[2] V3.1 expanded the Master File Table (MFT) entries with redundant MFT record number (useful for recovering damaged MFT files).


Nothing's stopping Microsoft from doing something similar again.

Reply Score: 2

RE[2]: What Will Be Removed
by hechacker1 on Tue 17th Jan 2012 16:53 UTC in reply to "RE: What Will Be Removed"
hechacker1 Member since:
2005-08-01

That's my thought as well. As they implement ReFS onto the client (perhaps with Windows 9, perhaps with a service pack to Windows 8) they'll have to add important features that clients use.

For the server environment, the features they removed won't be missed considering Storage Pools and ReFS (with copy on write semantics) supersede them. Server 8 will have deduplication, which takes care of the need for Sparse files, hard links, and compression. It's probably needlessly complex to support all the dropped features considering the new file system and storage pool.

But for the client, Sparse files are critical because large files would take forever to allocate on typical HDDs. Hard links I'm not so sure about, because they are rarely used in Windows, but they are still nice to have for the client without dedup.

Finally, removing those other features is actually a good thing considering malware. Often all those obscure NTFS features are used to hide malware so deep in NTFS only specialized tools can find them.

Edited 2012-01-17 16:54 UTC

Reply Score: 2

RE[3]: What Will Be Removed
by kragil on Tue 17th Jan 2012 17:02 UTC in reply to "RE[2]: What Will Be Removed"
kragil Member since:
2006-01-04

Server 8 will have deduplication

Citation needed. I've just read exactly the opposite for ReFS(at least out of the box, it seems to be avail via third party addons)

Edited 2012-01-17 17:06 UTC

Reply Score: 2

RE[4]: What Will Be Removed
by hechacker1 on Tue 17th Jan 2012 17:05 UTC in reply to "RE[3]: What Will Be Removed"
hechacker1 Member since:
2005-08-01

http://social.msdn.microsoft.com/Forums/zh/windowsdeveloperpreviewg...

It's implemented as a scrub job that runs in the background to deduplicate data after the fact.

Edited 2012-01-17 17:19 UTC

Reply Score: 1

RE[5]: What Will Be Removed
by kragil on Tue 17th Jan 2012 17:16 UTC in reply to "RE[4]: What Will Be Removed"
kragil Member since:
2006-01-04

RTFA:

Q) How come ReFS does not have deduplication, second level caching between DRAM & storage, and writable snapshots?

ReFS does not itself offer deduplication. One side effect of its familiar, pluggable, file system architecture is that other deduplication products will be able to plug into ReFS the same way they do with NTFS.

Reply Score: 2

RE[6]: What Will Be Removed
by hechacker1 on Tue 17th Jan 2012 17:20 UTC in reply to "RE[5]: What Will Be Removed"
hechacker1 Member since:
2005-08-01

Hmm, a tradeoff I suppose.

NTFS supports deduplicatation in server 8.

I take back what I said about ReFS then. It doesn't supercede all the needed features.

EDIT:

Thinking about it for a minute, it does seem kind of a weird omission considering the support is already there for NTFS for dedup.

But I guess the reason they decided to not support it is because it goes against having redundancy in the system. They specifically mention you only get checksum repairing when using mirrored storage pools, not parity. Though with parity errors are still logged with checksums.

Still, it leaves an interesting opening for dedup while using ReFS. I know there is software that allows you to layer one file system on top of another. Presumably, you could layer an NTFS volume on top of ReFS to get advantages of both, but probably suffer a performance hit.

Or just wait for Microsoft to implement it in future versions.

Edited 2012-01-17 17:40 UTC

Reply Score: 1

RE[7]: What Will Be Removed
by phoenix on Tue 17th Jan 2012 19:49 UTC in reply to "RE[6]: What Will Be Removed"
phoenix Member since:
2005-07-11

Dedupe is not built into NTFS on Windows 8. It's a separate "thing" that runs on top of NTFS.

http://blog.fosketts.net/2012/01/03/microsoft-adds-data-deduplicati...

Edited 2012-01-17 19:52 UTC

Reply Score: 2

Comment by Luminair
by Luminair on Tue 17th Jan 2012 17:59 UTC
Luminair
Member since:
2007-03-30

more reason to predict the enthusiasts will be using windows 8 server!

Reply Score: 3

Windows N
by fran on Tue 17th Jan 2012 18:30 UTC
fran
Member since:
2010-08-06

I just have to:^)

One year back..

http://www.osnews.com/permalink?456042

http://www.osnews.com/thread?456047

Excuse the assinine attitude in the second comment.

Edited 2012-01-17 18:34 UTC

Reply Score: 3

RE: Windows N
by Tuishimi on Thu 19th Jan 2012 04:18 UTC in reply to "Windows N"
Tuishimi Member since:
2005-07-06

Heh!

Reply Score: 2

liking the direction
by REM2000 on Thu 19th Jan 2012 12:49 UTC
REM2000
Member since:
2006-07-25

I really like the work going into bit rot protection through checksums, this is the number one feature of ZFS for me. Yes de-dup and superior RAIDZ capability are really useful as well as snapshots. However bit rot has to be my number one worry as it's so easy to reproduce, through bad RAM, faulty controllers on the RAID Card, HDD controller etc.. so i think these checks are a minimum requirement of our information society.

Slightly disappointed not to see de-dup, but this is a version 1 and ZFS went through quite a few versions to get where it is today.

Ill have to reserve judgement on the storage spaces concept until i see how it works in the real world.

My only worry is that each new feature added to ReFS will require a new version of Windows, i really hope that Microsoft can backport the features through updates and Service Packs, as i wouldn't want to keep going through each server to get fundamental file system features. Generally in today's Windows word you can work on the other principle, Windows 2003 skip 2008 2008R2, for some features it might compel me / us to go for 2012.

Im also a little worried how this will work with Windows 7 / 2008R2, i personally think Microsoft could get a lot of good will by backporting this to Windows 7/2008R2.

Obviously in an ideal world, the licence would be free and allow other OS's such as MacOSX and Linux to take advantage of it aswell, but i don't see it happening.

Reply Score: 2

RE: liking the direction
by Kebabbert on Fri 20th Jan 2012 11:54 UTC in reply to "liking the direction"
Kebabbert Member since:
2007-07-27

I really like the work going into bit rot protection through checksums, this is the number one feature of ZFS for me. Yes de-dup and superior RAIDZ capability are really useful as well as snapshots. However bit rot has to be my number one worry as it's so easy to reproduce, through bad RAM, faulty controllers on the RAID Card, HDD controller etc.. so i think these checks are a minimum requirement of our information society.

It seems that MS ReFS only has checksums for metadata, so the data itself might still be corrupted. ReFS exposes an API so a vendor can use checksums on data, but that is not default behavior.

Thus, ReFS seems safer than NTFS, but will it be safe enough?

Reply Score: 2

RE[2]: liking the direction
by malxau on Fri 20th Jan 2012 15:23 UTC in reply to "RE: liking the direction"
malxau Member since:
2005-12-04

It seems that MS ReFS only has checksums for metadata, so the data itself might still be corrupted. ReFS exposes an API so a vendor can use checksums on data, but that is not default behavior.


From the article:
In addition, we have added an option where the contents of a file are check-summed as well. When this option, known as “integrity streams,” is enabled, ReFS always writes the file changes to a location different from the original one. This allocate-on-write technique ensures that pre-existing data is not lost due to the new write. The checksum update is done atomically with the data write, so that if power is lost during the write, we always have a consistently verifiable version of the file available whereby corruptions can be detected authoritatively...

By default, when the /i switch is not specified, the behavior that the system chooses depends on whether the volume resides on a mirrored space. On a mirrored space, integrity is enabled because we expect the benefits to significantly outweigh the costs. Applications can always override this programmatically for individual files.


It's the default behavior when redundancy on spaces is present. When that is true, we can use the data checksum to find a good copy if one exists and another is bad. Without redundancy, all we can do with the checksum is prevent bad data going to applications - ie., start failing requests. Since applications will not always deal with that gracefully, the benefit without redundancy is much more limited - where failure really is better than incorrect data.

I realize many people might want to get religious on this point, but seriously, watch what happens when reads are failed under applications first. Particularly consider what happens when a read for a block of code is failed, for example. Then consider the fraction of that code that will actually be executed.

- M // ReFS co-author

Reply Score: 2

Patents is nowadays the name of the game
by Nth_Man on Thu 19th Jan 2012 18:07 UTC
Nth_Man
Member since:
2010-05-16

If you've read the news lately, you'll know that "software patents" are a main key there.

Hey, getting free money from other people, like LG, even if LG invented what they are using. :-(

If you want to write a reader for that new filesystem, you know you'll have to...

Edited 2012-01-19 18:17 UTC

Reply Score: 2

features dropped
by TomF on Thu 19th Jan 2012 20:22 UTC
TomF
Member since:
2010-01-22

they are dropping support for (among others) hardlinks, quota, encryption, compression, sparse files....

I don't get it ?? rather very usefull features which all/most FS have... and MS drops them ?

companies need quota
efficient backups need hardlinks
cheapos need compression
government does not want encryption (ah..?)
sparse files: very useful for databases

nop... I don't and won't get it in this form :/

Reply Score: 2