Linked by Rahul on Mon 13th Oct 2008 21:19 UTC
Linux Linux Foundation is organizing a end user collaboration summit this week. A major topic will be a presentation on the new upcoming filesystems - Ext4 and Btrfs. Ted Tso, who is a Linux kernel filesystem developer on a sabbatical from IBM working for Linux Foundation for a year, has talked about the two-pronged approach for the Linux kernel, taking a incremental approach with Ext4 while simultaneously working on the next generation filesystem called btrfs. Read more for details.
Thread beginning with comment 333555
To read all comments associated with this story, please click here.
mistakes, assumptions, extrapolations
by Phobos on Mon 13th Oct 2008 23:53 UTC
Phobos
Member since:
2008-04-30

1) file systems do not follow Moore's Law...
2) btrfs is designed as a FS for DBs... ZFS is designed as a multipurpose FS... Of course it can "leapfrog ZFS on several fronts", surely, DB related fronts... that comment from Ts'o couldn't fall further on the irrelevancy world

Truth is, everybody wants ZFS... at all cost

Reply Score: 1

jwwf Member since:
2006-01-19

Yeah, hate to be a hater, but that article contained a lot of nonsense. Why would Microsoft adopt ext4? Aside from lack of obvious need, NTFS isn't even bad. Also, it doesn't give any specifics on how btrfs will "leapfrog" ZFS, but if enterprise deployment is targeted for 2012 (I read this as inclusion and endorsement in RHEL), it sure as hell better (btr?) be better than ZFS, which was released and promoted in a production Solaris release in mid-2006.

Oh, and COW filesystems and databases are not a natural fit. Doesn't mean it doesn't work, but database is not the most obvious strength of a COW FS, which by design fragments files on modification. The article seems to imply otherwise, probably just because Mason works for Oracle. Oracle has their preferred storage solution already: ASM. Unless btrfs grows cluster capability, I don't see why Oracle would promote it for databases at all.

Reply Parent Score: 5

Pfeifer Member since:
2006-02-20

ORACLE is also working on CRFS, built upon BTRFS. CRFS uses the same disk format as BTRFS, but adds cluster capabilities.

Is it getting interesting for you?

Reply Parent Score: 4

segedunum Member since:
2005-07-06

You've wildly misinterpreted the article.

Why would Microsoft adopt ext4?

The article didn't say that Microsoft would adopt ext4 or btrfs. They said that licensing issues would prevent them providing any sort of support for them. No shock there.

Aside from lack of obvious need, NTFS isn't even bad.

Wow, really? Where's that 500 terabyte array running NTFS as its principle filesystem? That's what the article is about and we're they're heading with these - if you'd read it properly. Not that many Linux filesystems can't do that today, but some things could use improvement on that scale. Many things are 'nice to haves' for everyone else.

However, perversely, with that statement you have touched on why people aren't as excited about ZFS as some people think that they should be.

it sure as hell better (btr?) be better than ZFS, which was released and promoted in a production Solaris release in mid-2006.

ZFS is still largely a very unproven filesystem, regardless of when it was put into production. btrfs will probably remain so for many years as well, but it depends on how fast it develops. Additionally, there are some very large question marks over ZFS's ability to be used as a filesystem from very small ARM NAS devices right up to the large systems ZFS is restricted to today. At the very least, btrfs is in the right testing environment for those sorts of things to be tried and tested.

Oh, and COW filesystems and databases are not a natural fit.......database is not the most obvious strength of a COW FS.......The article seems to imply otherwise, probably just because Mason works for Oracle......Unless btrfs grows cluster capability, I don't see why Oracle would promote it for databases at all.

The article mentioned the word 'databases' once, and didn't mention copy-on-write at all (if that is indeed what COW is an acronym for). I'm not entirely sure how you've managed to extrapolate all that. The article implied nothing about copy-on-write and databases in the case of btrfs because it was never talked about. You can guess that large databases are one of the things they'll look at though. EDIT: After the comment above I'd forgot about CRFS.

I'm also not entirely sure why you are wandering off and questioning an Oracle engineer's involvement in a fledgling filesystem, based on what you think Oracle's overall strategy is, when it will probably be maintained and developed by lots of other interested parties as well.

Edited 2008-10-14 11:18 UTC

Reply Parent Score: 6

segedunum Member since:
2005-07-06

1) file systems do not follow Moore's Law...

The article didn't say that it did, but Moore's Law has allowed people to do things with their storage devices where filesystems and storage containers like LVM and RAID haven't haven't quite kept up. That was the point the article was making.

2) btrfs is designed as a FS for DBs... ZFS is designed as a multipurpose FS...

The article didn't say what direction btrfs would take, but database usage is probably just one of their use cases. It sounds like you're already carving out a niche for ZFS.................

Truth is, everybody wants ZFS... at all cost

Hmmmmmmm, no. Some people want to believe that, but it isn't true I'm afraid. For the vast majority of storage uses in the world today, and when you look at reviews of OpenSolaris, no one is the slightest bit interested in ZFS or even aware that it exists. It makes certain things somewhat better, but Sun unfortunately don't have the userland tools that would expose ZFS as something remotely useful for the majority.

The biggest drawback we have with storage today is the storage devices. To maintain the large amounts of storage that many people are using Linux filesystems like XFS for today, and keep it reliable, we need to ditch disk drives with lots of mechanical moving parts and make data integrity a better part of the hardware. ZFS hasn't changed that fact at all and neither will btrfs.

In summary, no one is going to rush off their existing platform to get ZFS and btrfs (a select few might). Filesystems tend to change slowly, and adoption happens in the normal course of other things, normal iterative development and the price of upheaval. However, when compared with ZFS in Solaris, btrfs already has a head start even now in testing and development in that it is developed within a a kernel that runs on small ARM NAS boxes to very large arrays. Its code will also be scrutinised as such. ZFS isn't going to have that kind of free testing environment until people start doing the things with OpenSolaris and its source code that are currently done with Linux.

Reply Parent Score: 3

kernpanic Member since:
2008-03-15

ZFS testing/debugging is a little wider than that as it is also being used on FreeBSD.

EDIT: It will be used on Mac OS too so the testing and use of ZFS is not even limited to Solaris/FreeBSD and you can bet the GUI/tools used to manage ZFS on Mac OS will more intuitive than that on Solaris (though there's nothing wrong with command line ZFS usage, its ridiculously easy).

Edited 2008-10-14 15:40 UTC

Reply Parent Score: 1

Arun Member since:
2005-07-07


The biggest drawback we have with storage today is the storage devices. To maintain the large amounts of storage that many people are using Linux filesystems like XFS for today, and keep it reliable, we need to ditch disk drives with lots of mechanical moving parts and make data integrity a better part of the hardware. ZFS hasn't changed that fact at all and neither will btrfs.


Yes it has. XFS can't detect bad hardware like ZFS. Silent bit-rot and data corruption are common issues with hardware and most linux filesystems are piss poor at detecting those.

It does not matter how much you cram into hardware there will always be bugs and errata that can cause all sorts of nastiness. Claiming anything else is silly really.


In summary, no one is going to rush off their existing platform to get ZFS and btrfs (a select few might). Filesystems tend to change slowly, and adoption happens in the normal course of other things, normal iterative development and the price of upheaval. However, when compared with ZFS in Solaris, btrfs already has a head start even now in testing and development in that it is developed within a a kernel that runs on small ARM NAS boxes to very large arrays. Its code will also be scrutinised as such. ZFS isn't going to have that kind of free testing environment until people start doing the things with OpenSolaris and its source code that are currently done with Linux.


Again not True. ZFS has testing on a much wider platforms than you give it credit for, the BSDs and Apple are testing them so are many many people out side of Sun .

Edited 2008-10-14 16:01 UTC

Reply Parent Score: 1

Phobos Member since:
2008-04-30

"1) file systems do not follow Moore's Law...

The article didn't say that it did, but Moore's Law has allowed people to do things with their storage devices where filesystems and storage containers like LVM and RAID haven't haven't quite kept up. That was the point the article was making.
"

the article said:
The problem with contemporary file systems, Ts'o said, is that -- following Moore's Law -- file sizes have grown bigger, and disk drives have doubled in capacity every couple of years.


so, indeed, it did say FS followed Moore's Law... try proof reading next time.

Moore's Law was about the growth of the number of transistor on a CPU die... people mistakenly extrapolated that to everything, being a wrong assumption at it's root.

Hmmmmmmm, no. Some people want to believe that, but it isn't true I'm afraid. For the vast majority of storage uses in the world today, and when you look at reviews of OpenSolaris, no one is the slightest bit interested in ZFS or even aware that it exists.


So, you need proof, eh?

- Linus Torvalds:
And yes, maybe ZFS is worthwhile enough that I'm willing to go to the effort of trying to relicense the kernel. But quite frankly, I can almost guarantee that Sun won't release ZFS under the GPLv3 even if they release other parts.

http://lwn.net/Articles/237905/

- Zemlin's desperate and groundless attacks like:
With capabilities such as ZFS and DTrace, Sun is trying to compete based on minor features


while later stating:
Zemlin suggests that it should make ZFS and DTrace available under a Linux-compatible license.


(So minor he wants it on Linux so badly?... A bit contradictory, isn't it?)
http://www.nytimes.com/idg/IDG_852573C400693880002574CE00371FE1.htm...

- Apple: http://www.apple.com/server/macosx/technology/filesystem.html

- FreeBSD: http://lists.freebsd.org/pipermail/freebsd-current/2007-April/07054...

- ZFS-on-FUSE: http://zfs-on-fuse.blogspot.com/

Not only that everybody wants it, it also has inspired other works like Mathew Dillon's HAMMER: http://kerneltrap.org/DragonFlyBSD/HAMMER_Filesystem_Design

While desktop users don't know/don't care about FSs, whenever they buy a Mac or install Solaris or FreeBSD, they will get ZFS, and they don't ever need to know about their existence... that's the point of it, FSs should be transparent to end users...



It makes certain things somewhat better, but Sun unfortunately don't have the userland tools that would expose ZFS as something remotely useful for the majority.


Heh... investigate a little before doing such assertions: http://blogs.sun.com/erwann/entry/zfs_on_the_desktop_zfs


In summary, no one is going to rush off their existing platform to get ZFS and btrfs (a select few might). Filesystems tend to change slowly, and adoption happens in the normal course of other things, normal iterative development and the price of upheaval.


I think my previous list can prove you wrong.


Sorry, but no it wouldn't. It has a great deal of useful features, but for a filesystem, it consumes far too much memory and CPU without some pretty damn serious tuning. It certainly isn't a general purpose filesystem you can throw any workload at.


You see, ZFS was designed taking in account the vast percentage of wasted CPU cycles on modern servers and computers in general... same thing that has motivated virtualization on all platforms, in modern times there is an excess of processing power, not lack of it like 20 years ago. The whole idea of hardware disk controllers to make disks arrays has become obsolete in many (save some very specialized) cases, thanks to ZFS... I invite you to read some papers and the ideas behind the Thumper X4500... a storage solution using ZFS

Even so, ZFS is no so massively resource intensive as you seem to imply, it just need a decent (not even great) configuration based on modern modern standards and it works like a charm...

Reply Parent Score: 0