Linked by Rahul on Mon 13th Oct 2008 21:19 UTC
Linux Linux Foundation is organizing a end user collaboration summit this week. A major topic will be a presentation on the new upcoming filesystems - Ext4 and Btrfs. Ted Tso, who is a Linux kernel filesystem developer on a sabbatical from IBM working for Linux Foundation for a year, has talked about the two-pronged approach for the Linux kernel, taking a incremental approach with Ext4 while simultaneously working on the next generation filesystem called btrfs. Read more for details.
Order by: Score:
v ZFS?
by atehrani on Mon 13th Oct 2008 23:02 UTC
RE: ZFS?
by shiny on Mon 13th Oct 2008 23:13 UTC in reply to "ZFS?"
shiny Member since:
2005-08-09

Boo, I hate GPL.


Not again...

Reply Score: 4

RE: ZFS?
by poundsmack on Mon 13th Oct 2008 23:17 UTC in reply to "ZFS?"
poundsmack Member since:
2005-07-13

even if Linux never gets ZFS I am still thrilled ZFS exists and is being developed and adopted by the BSD's and Apple. After all, if it wernt for ZFS Oracle likely wouldn't have said "me too" and started development on Btrfs. well, they liekly would have, though i don't think it would have been made as easily avalible. could have easily become a good piece of Oracle IP. glad its open though.

Reply Score: 7

RE: ZFS?
by stabbyjones on Mon 13th Oct 2008 23:32 UTC in reply to "ZFS?"
stabbyjones Member since:
2008-04-15

Shame that license issues has blocked ZFS adoption in Linux distros. Typical that the Linux community is all about opensource as long as it came from them.


That must be why i'm willing to stick to ext3/4 while i watch development of btfs.

All my zealous love for the gpl is blinding me to the truth that i truly truly want ZFS for my servers.

Reply Score: 6

RE: ZFS?
by ba1l on Tue 14th Oct 2008 10:28 UTC in reply to "ZFS?"
ba1l Member since:
2007-09-08

The license issue with ZFS is Sun's fault. They deliberately chose a license that's incompatible with the GPL. That means that any combination of ZFS and the Linux kernel is impossible to distribute without violating both the GPL (on the Linux code) and the CDDL (on the ZFS code).

Did you really think that the Linux kernel developers were going to stop work, and devote all their time to tracking down all of the previous contributors to the kernel, ask them for permission to change the license to something else, and then relicense the entire thing under another license just so we can use ZFS? Most of the kernel contributors were one-off contributors who left no contact details, and a few of them are even dead.

Besides, ZFS would never be acceptable for the mainstream Linux kernel anyway. Like Reiser4, it re-implements far too many other filesystem layers, like the block cache, and has rampant layering violations. Most of the improvements from ZFS should be implemented into Linux itself, so that all filesystems can benefit from them. Of course, doing it that way isn't nearly as marketable - you can't just slap a single name on the whole thing and sell it.

Reply Score: 3

RE: ZFS?
by segedunum on Tue 14th Oct 2008 10:31 UTC in reply to "ZFS?"
segedunum Member since:
2005-07-06

Shame that license issues has blocked ZFS adoption in Linux distros.

Hmmmmmm. Funny. My recollection is that Linux's usage of the GPL pre-dates both ZFS and the CDDL, and Sun knew fine well what it was doing when it started using the CDDL for selected bits of software.

Reply Score: 3

RE[2]: ZFS?
by Arun on Tue 14th Oct 2008 15:45 UTC in reply to "RE: ZFS?"
Arun Member since:
2005-07-07

"Shame that license issues has blocked ZFS adoption in Linux distros.

Hmmmmmm. Funny. My recollection is that Linux's usage of the GPL pre-dates both ZFS and the CDDL, and Sun knew fine well what it was doing when it started using the CDDL for selected bits of software.
"

If Sun had used GPL, other OSes like MacOS, *BSD and QNX couldn't have ported ZFS or other Solaris tech. Sun chose a more open license to spread the tech around. Too bad Linux users are so selfish.

Edited 2008-10-14 15:53 UTC

Reply Score: 1

RE[3]: ZFS?
by FunkyELF on Tue 14th Oct 2008 17:49 UTC in reply to "RE[2]: ZFS?"
FunkyELF Member since:
2006-07-26

BSD License...done

Reply Score: 2

RE[3]: ZFS?
by segedunum on Tue 14th Oct 2008 20:50 UTC in reply to "RE[2]: ZFS?"
segedunum Member since:
2005-07-06

If Sun had used GPL, other OSes like MacOS, *BSD and QNX couldn't have ported ZFS or other Solaris tech.Sun chose a more open license to spread the tech around.

So what's the problem with what they did with Java and dual licensing then? Sorry, but that doesn't really wash.

Too bad Linux users are so selfish.

No, they just want to keep code flowing into their project rather than out of it so they can keep the code open ;-).

Reply Score: 3

v RE[4]: ZFS?
by Arun on Tue 14th Oct 2008 22:22 UTC in reply to "RE[3]: ZFS?"
RE[5]: ZFS?
by hobgoblin on Wed 15th Oct 2008 02:50 UTC in reply to "RE[4]: ZFS?"
hobgoblin Member since:
2005-07-06

Why doesn't the Linux community dual license if it needs Solaris tech?


need it? when was it ever said that linux needs anything solaris?

Reply Score: 2

RE[6]: ZFS?
by Arun on Wed 15th Oct 2008 03:53 UTC in reply to "RE[5]: ZFS?"
Arun Member since:
2005-07-07


need it? when was it ever said that linux needs anything solaris?


Linux has implemented a lot of Solaris technology in the past. The slab allocator for one. Such blind zealotry is bad form.

Edited 2008-10-15 03:53 UTC

Reply Score: 0

RE[7]: ZFS?
by hobgoblin on Wed 15th Oct 2008 04:18 UTC in reply to "RE[6]: ZFS?"
hobgoblin Member since:
2005-07-06

implemented, yes. but i suspect it was more a case of "path of least effort within the bounds of the gpl" then "we cant do without it, and only solaris can provide it" that the word "need" implies.

Reply Score: 2

RE[5]: ZFS?
by smitty on Wed 15th Oct 2008 03:34 UTC in reply to "RE[4]: ZFS?"
smitty Member since:
2005-10-13

Come on, quit trolling. Just come out and say you hate the GPL and think Linux should switch away from it, because that's clearly what you are arguing for.

As for why Linux isn't dual licensed, there are a bunch of reasons. Some developers are opposed to non-GPL licenses. Some of them are dead. Some can't be contacted. All the code they wrote would have to be rewritten just to change the license, and no one thinks that would be worth the effort when they can spend that same effort on improving what they have and writing alternatives. One of the benefits that Sun has with it's code is that is the only copyright owner, which means it is much simpler to change their licensing however they want.

Reply Score: 2

RE[6]: ZFS?
by Arun on Wed 15th Oct 2008 03:51 UTC in reply to "RE[5]: ZFS?"
Arun Member since:
2005-07-07

Come on, quit trolling. Just come out and say you hate the GPL and think Linux should switch away from it, because that's clearly what you are arguing for.


I don't give a rats ass what Linux is licensed under. Nor do I hate the GPL. I do have a problem with all the Linux zealots that think that everything must be licensed under the GPL so Linux can get the code.

Sun chose what it chose, other Open Source projects are benefiting from it just fine. How about the whining about licensing stops?

Reply Score: 0

RE[7]: ZFS?
by sbergman27 on Wed 15th Oct 2008 04:02 UTC in reply to "RE[6]: ZFS?"
sbergman27 Member since:
2005-07-24

I don't give a rats ass what Linux is licensed under. Nor do I hate the GPL.

Then why have you made 9 posts in the last 12 hours on those themes?

This thread is nonconstructive and an annoyance to other readers.

Reply Score: 3

RE[7]: ZFS?
by smitty on Wed 15th Oct 2008 04:18 UTC in reply to "RE[6]: ZFS?"
smitty Member since:
2005-10-13

Please read the post that started this whole discussion. It's the same one that comes up every single time there is an article about ZFS...

Shame that license issues has blocked ZFS adoption in Linux distros. Typical that the Linux community is all about opensource as long as it came from them.


The only people whining about licensing are the pro-ZFS, Linux sux because it doesn't have ZFS, linux needs ZFS crowd. They never seem to say much more than that, I guess they just assume that the necessity of having ZFS support is obvious. But it isn't to most people.

Most people would agree that it would be a nice feature for Linux to have, but it's pretty clearly something that isn't hurting Linux adoption. The vast majority of Linux users are very happy with the current and upcoming Linux file systems, like BTRFS and Tux3, which plan to compete against ZFS. There are a number of legal and technical reasons it is hard to put ZFS into linux. Sun could solve most of these themselves, if they wished to. They apparently don't, and that is fine.

So, let's make this clear: I think ZFS needs Linux much more than the other way around. If Sun ever decides to make ZFS compatible, then good for them. Maybe ZFS will become more than a fringe FS. But I'm not counting on it, and I don't really care.

Edited 2008-10-15 04:21 UTC

Reply Score: 3

RE[5]: ZFS?
by segedunum on Wed 15th Oct 2008 17:03 UTC in reply to "RE[4]: ZFS?"
segedunum Member since:
2005-07-06

Why doesn't the Linux community dual license if it needs Solaris tech?

You haven't answered the question. Why did Sun think that GPL dual licensing Java was the correct option there, maintaining compatibility with lots of existing software with exceptions for where they needed it, and that a completely new license in the CDDL was required elsewhere?

How did you come to that illogical conclusion? Real data shows Solaris IP is flowing very well to other projects.

Ha, ha, ha, ha, ha. I love the usage of 'IP', which is meaningless. Not the point. I'm not talking about code flowing from Sun but others' willingness to commit code to Sun and Solaris. There's no evidence that Sun is even accepting that if it is happening, and things such as the PowerPC port of Solaris tell us that an awful lot is being kept in Sun's four walls.

The license is far more open than the GPL.

Based on what? There is zero code flowing into Solaris from outside Sun. That's when you know you don't have an open source community. If anything, Sun is strangling it.

Reply Score: 2

RE[6]: ZFS?
by Arun on Thu 16th Oct 2008 04:10 UTC in reply to "RE[5]: ZFS?"
Arun Member since:
2005-07-07


You haven't answered the question. Why did Sun think that GPL dual licensing Java was the correct option there, maintaining compatibility with lots of existing software with exceptions for where they needed it, and that a completely new license in the CDDL was required elsewhere?


You didn't answer my question either. Why can't linux dual license when most projects out there do?

How did you come to that illogical conclusion? Real data shows Solaris IP is flowing very well to other projects.

Ha, ha, ha, ha, ha. I love the usage of 'IP', which is meaningless. Not the point. I'm not talking about code flowing from Sun but others' willingness to commit code to Sun and Solaris. There's no evidence that Sun is even accepting that if it is happening, and things such as the PowerPC port of Solaris tell us that an awful lot is being kept in Sun's four walls. [/q]


You mean to say Linus accepts every single line of code someone wrote in to the main line tree, Really?

The license is far more open than the GPL.

Based on what? There is zero code flowing into Solaris from outside Sun. That's when you know you don't have an open source community. If anything, Sun is strangling it. [/q]

That has nothing to do with the license. Do you even understand what your are saying your self. That response was the most illogical thing I have ever heard.

Reply Score: 1

RE[3]: ZFS?
by muszek on Wed 15th Oct 2008 23:55 UTC in reply to "RE[2]: ZFS?"
muszek Member since:
2007-04-25

Yeah, it's a real shame you can't http://en.wikipedia.org/wiki/Dual-licensing">dual the code.

Reply Score: 1

mistakes, assumptions, extrapolations
by Phobos on Mon 13th Oct 2008 23:53 UTC
Phobos
Member since:
2008-04-30

1) file systems do not follow Moore's Law...
2) btrfs is designed as a FS for DBs... ZFS is designed as a multipurpose FS... Of course it can "leapfrog ZFS on several fronts", surely, DB related fronts... that comment from Ts'o couldn't fall further on the irrelevancy world

Truth is, everybody wants ZFS... at all cost

Reply Score: 1

jwwf Member since:
2006-01-19

Yeah, hate to be a hater, but that article contained a lot of nonsense. Why would Microsoft adopt ext4? Aside from lack of obvious need, NTFS isn't even bad. Also, it doesn't give any specifics on how btrfs will "leapfrog" ZFS, but if enterprise deployment is targeted for 2012 (I read this as inclusion and endorsement in RHEL), it sure as hell better (btr?) be better than ZFS, which was released and promoted in a production Solaris release in mid-2006.

Oh, and COW filesystems and databases are not a natural fit. Doesn't mean it doesn't work, but database is not the most obvious strength of a COW FS, which by design fragments files on modification. The article seems to imply otherwise, probably just because Mason works for Oracle. Oracle has their preferred storage solution already: ASM. Unless btrfs grows cluster capability, I don't see why Oracle would promote it for databases at all.

Reply Score: 5

Pfeifer Member since:
2006-02-20

ORACLE is also working on CRFS, built upon BTRFS. CRFS uses the same disk format as BTRFS, but adds cluster capabilities.

Is it getting interesting for you?

Reply Score: 4

segedunum Member since:
2005-07-06

You've wildly misinterpreted the article.

Why would Microsoft adopt ext4?

The article didn't say that Microsoft would adopt ext4 or btrfs. They said that licensing issues would prevent them providing any sort of support for them. No shock there.

Aside from lack of obvious need, NTFS isn't even bad.

Wow, really? Where's that 500 terabyte array running NTFS as its principle filesystem? That's what the article is about and we're they're heading with these - if you'd read it properly. Not that many Linux filesystems can't do that today, but some things could use improvement on that scale. Many things are 'nice to haves' for everyone else.

However, perversely, with that statement you have touched on why people aren't as excited about ZFS as some people think that they should be.

it sure as hell better (btr?) be better than ZFS, which was released and promoted in a production Solaris release in mid-2006.

ZFS is still largely a very unproven filesystem, regardless of when it was put into production. btrfs will probably remain so for many years as well, but it depends on how fast it develops. Additionally, there are some very large question marks over ZFS's ability to be used as a filesystem from very small ARM NAS devices right up to the large systems ZFS is restricted to today. At the very least, btrfs is in the right testing environment for those sorts of things to be tried and tested.

Oh, and COW filesystems and databases are not a natural fit.......database is not the most obvious strength of a COW FS.......The article seems to imply otherwise, probably just because Mason works for Oracle......Unless btrfs grows cluster capability, I don't see why Oracle would promote it for databases at all.

The article mentioned the word 'databases' once, and didn't mention copy-on-write at all (if that is indeed what COW is an acronym for). I'm not entirely sure how you've managed to extrapolate all that. The article implied nothing about copy-on-write and databases in the case of btrfs because it was never talked about. You can guess that large databases are one of the things they'll look at though. EDIT: After the comment above I'd forgot about CRFS.

I'm also not entirely sure why you are wandering off and questioning an Oracle engineer's involvement in a fledgling filesystem, based on what you think Oracle's overall strategy is, when it will probably be maintained and developed by lots of other interested parties as well.

Edited 2008-10-14 11:18 UTC

Reply Score: 6

Arun Member since:
2005-07-07

ZFS is still largely a very unproven filesystem, regardless of when it was put into production. btrfs will probably remain so for many years as well, but it depends on how fast it develops. Additionally, there are some very large question marks over ZFS's ability to be used as a filesystem from very small ARM NAS devices right up to the large systems ZFS is restricted to today. At the very least, btrfs is in the right testing environment for those sorts of things to be tried and tested.


Is there a point in there? What gave you the impression that ZFS can't run on ARM? ZFS is endian neutral and can be tuned to use less memory. If you really want a NAS device performance is not a key requirement so the ARC can be tiny and you still get all of ZFS' protection. There is nothing in ZFS' design that prevents it from being implemented in a small NAS box.

Reply Score: 1

segedunum Member since:
2005-07-06

Is there a point in there? What gave you the impression that ZFS can't run on ARM?

Errrrrrrr:

1. It doesn't today in any shape or form.

2. The chances of Solaris running on ARM are practically non-existant.

3. There just simply isn't enough memory or resources on such devices.

4. No one is even contemplating it despite some source code kicking around, and the lessons learned from the PowerPC port should tell you that it will never happen in the usual open source fashion of picking up the code and compiling it.

ZFS is endian neutral and can be tuned to use less memory.

How much less? Whatever way you cut it, ZFS needs at the very least two gigabytes of memory. That's the bare minimum, and I really don't care that certain people have been able to run it on a laptop for a day with less.

If you really want a NAS device performance is not a key requirement so the ARC can be tiny and you still get all of ZFS' protection.

I doubt very much whether you could cut down ZFS memory usage and limits to way less than 256 or 128 megabytes and leave it running. ZFS has not historically been a happy bunny when it reaches various memory and other limits. I/O wise it is pretty CPU intensive, which is understandable for what it does, but it really strikes it out here.

In a NAS box with perhaps one or two disk drives you also don't get any protection from ZFS. Try to understand that the reliability problems we have today need to be solved by our storage media and not by filesystems trying to jump through an awful lot of hoops. We've reached the end of the road on that front and I hope Btrfs understands that, although Btrfs just starting at the right time with SSDs in their infancy.

Reply Score: 3

Arun Member since:
2005-07-07


Errrrrrrr:

1. It doesn't today in any shape or form.


Doesn't prove that it can't.

2. The chances of Solaris running on ARM are practically non-existant.


Why?

3. There just simply isn't enough memory or resources on such devices.


Memory is cheap. You can easliy build a mini-itx box with 2GB of memory for dirt cheap. Most commercial Linux based NAS boxes are overpriced.

http://blog.flowbuzz.com/search/label/NAS


4. No one is even contemplating it despite some source code kicking around, and the lessons learned from the PowerPC port should tell you that it will never happen in the usual open source fashion of picking up the code and compiling it.


Irrelevant, again. You don't need ARM in a NAS box. you can build an intel based one cheaper than buying a commercial one.

How much less? Whatever way you cut it, ZFS needs at the very least two gigabytes of memory. That's the bare minimum, and I really don't care that certain people have been able to run it on a laptop for a day with less.


There is no reason to make ZFS smaller. 1 GB is recommended but memory is cheap.


I doubt very much whether you could cut down ZFS memory usage and limits to way less than 256 or 128 megabytes and leave it running. ZFS has not historically been a happy bunny when it reaches various memory and other limits. I/O wise it is pretty CPU intensive, which is understandable for what it does, but it really strikes it out here.


Memory is cheap.

In a NAS box with perhaps one or two disk drives you also don't get any protection from ZFS. Try to understand that the reliability problems we have today need to be solved by our storage media and not by filesystems trying to jump through an awful lot of hoops. We've reached the end of the road on that front and I hope Btrfs understands that, although Btrfs just starting at the right time with SSDs in their infancy.


Wrong. Silent data corruption is bad making excuses for poor software is wrong.

Snapshots are easy backup for most home users. Checksums detect failing drives. Mirroring is easy as pie on ZFS using 2 drives. ZFS can use SSDs.

Edited 2008-10-14 22:56 UTC

Reply Score: 0

segedunum Member since:
2005-07-06

1) file systems do not follow Moore's Law...

The article didn't say that it did, but Moore's Law has allowed people to do things with their storage devices where filesystems and storage containers like LVM and RAID haven't haven't quite kept up. That was the point the article was making.

2) btrfs is designed as a FS for DBs... ZFS is designed as a multipurpose FS...

The article didn't say what direction btrfs would take, but database usage is probably just one of their use cases. It sounds like you're already carving out a niche for ZFS.................

Truth is, everybody wants ZFS... at all cost

Hmmmmmmm, no. Some people want to believe that, but it isn't true I'm afraid. For the vast majority of storage uses in the world today, and when you look at reviews of OpenSolaris, no one is the slightest bit interested in ZFS or even aware that it exists. It makes certain things somewhat better, but Sun unfortunately don't have the userland tools that would expose ZFS as something remotely useful for the majority.

The biggest drawback we have with storage today is the storage devices. To maintain the large amounts of storage that many people are using Linux filesystems like XFS for today, and keep it reliable, we need to ditch disk drives with lots of mechanical moving parts and make data integrity a better part of the hardware. ZFS hasn't changed that fact at all and neither will btrfs.

In summary, no one is going to rush off their existing platform to get ZFS and btrfs (a select few might). Filesystems tend to change slowly, and adoption happens in the normal course of other things, normal iterative development and the price of upheaval. However, when compared with ZFS in Solaris, btrfs already has a head start even now in testing and development in that it is developed within a a kernel that runs on small ARM NAS boxes to very large arrays. Its code will also be scrutinised as such. ZFS isn't going to have that kind of free testing environment until people start doing the things with OpenSolaris and its source code that are currently done with Linux.

Reply Score: 3

kernpanic Member since:
2008-03-15

ZFS testing/debugging is a little wider than that as it is also being used on FreeBSD.

EDIT: It will be used on Mac OS too so the testing and use of ZFS is not even limited to Solaris/FreeBSD and you can bet the GUI/tools used to manage ZFS on Mac OS will more intuitive than that on Solaris (though there's nothing wrong with command line ZFS usage, its ridiculously easy).

Edited 2008-10-14 15:40 UTC

Reply Score: 1

Arun Member since:
2005-07-07


The biggest drawback we have with storage today is the storage devices. To maintain the large amounts of storage that many people are using Linux filesystems like XFS for today, and keep it reliable, we need to ditch disk drives with lots of mechanical moving parts and make data integrity a better part of the hardware. ZFS hasn't changed that fact at all and neither will btrfs.


Yes it has. XFS can't detect bad hardware like ZFS. Silent bit-rot and data corruption are common issues with hardware and most linux filesystems are piss poor at detecting those.

It does not matter how much you cram into hardware there will always be bugs and errata that can cause all sorts of nastiness. Claiming anything else is silly really.


In summary, no one is going to rush off their existing platform to get ZFS and btrfs (a select few might). Filesystems tend to change slowly, and adoption happens in the normal course of other things, normal iterative development and the price of upheaval. However, when compared with ZFS in Solaris, btrfs already has a head start even now in testing and development in that it is developed within a a kernel that runs on small ARM NAS boxes to very large arrays. Its code will also be scrutinised as such. ZFS isn't going to have that kind of free testing environment until people start doing the things with OpenSolaris and its source code that are currently done with Linux.


Again not True. ZFS has testing on a much wider platforms than you give it credit for, the BSDs and Apple are testing them so are many many people out side of Sun .

Edited 2008-10-14 16:01 UTC

Reply Score: 1

segedunum Member since:
2005-07-06

Yes it has. XFS can't detect bad hardware like ZFS. Silent bit-rot and data corruption are common issues with hardware and most linux filesystems are piss poor at detecting those.

You miss the point sweetheart.............again. Detecting bad hardware is a problem of hardware and the current state of drive technology. Using ZFS isn't going to change that situation, and all anyone is finding out as a result of using ZFS are Solaris driver problems ;-).

Again not True. ZFS has testing on a much wider platforms than you give it credit for

Feel free to furnish me with a list.

the BSDs and Apple are testing them so are many many people out side of Sun .

Currently, that is x86 only and preferably 64-bit if you value your data. The problem for Sun though is that little if any code is flowing back in Sun's direction to help them maintain ZFS, so the quality control and shared development just isn't there. If you ever wanted to know why Linux uses the GPL, that's it.

Reply Score: 3

kernpanic Member since:
2008-03-15

You are far too dismissive of ZFS; I wonder if they simply licensed it under the GPL, without making any other changes to it, your attitude would change. The constant attempt to prove ZFS is irrelevant or worthless invariably seem to come from hardcore Linux/GPL advocates. If ZFS, precisely as it exists now, came from the GNU/Linux community all the other non GPL open source projects and particularly the Microsofties would not hear the end of it.

Reply Score: 0

sbergman27 Member since:
2005-07-24

I wonder if they simply licensed it under the GPL, without making any other changes to it, your attitude would change.

License compatibility would make an in-kernel implementation of ZFS *possible*. But there would be many other problems. ZFS clashes rather violently with Linux Kernel design philosophy. ("Rampant layering violation" was the term Andrew Morton chose for it while *praising* its feature set.) It's hard to see ZFS making it into mainline in a recognizable form, licensing issues asside. Far more likely, ZFS concepts will make it into Linux via btrfs.

More generally, I've never felt that the GPL/CDDL licensing issues were that significant. The Linux and Solaris kernel internals are so different that the sharing and cross-pollination of *concepts* is more useful than the sharing of actual code.

Edited 2008-10-14 21:34 UTC

Reply Score: 2

segedunum Member since:
2005-07-06

You are far too dismissive of ZFS; I wonder if they simply licensed it under the GPL, without making any other changes to it, your attitude would change.

Sorry, but no it wouldn't. It has a great deal of useful features, but for a filesystem, it consumes far too much memory and CPU without some pretty damn serious tuning. It certainly isn't a general purpose filesystem you can throw any workload at.

The constant attempt to prove ZFS is irrelevant or worthless invariably seem to come from hardcore Linux/GPL advocates.

This is an article about possible future Linux filesystems (apart from ext4). Take a look at the first post ;-).

If ZFS, precisely as it exists now, came from the GNU/Linux community all the other non GPL open source projects and particularly the Microsofties would not hear the end of it.

No I'm sorry, but people just don't get that excited about filesystems. ext4 and Btrfs are attempts at trying to move some shortcomings on, but new filesystems are not big bang events and don't generate a lot of default excitement that some people think that ZFS automatically should.

Linux had some excitement over Reiser4 and what might be possible, but the vast majority simply could not stomach yet another completely incompatible filesystem change and another reformat no matter how great the new features were. Microsoft even had to make NTFS upgradeable from FAT, which just shows you. It's the main reason why ext has become the pre-eminent filesystem in the Linux world, despite some shortcomings. It remains to be seen how Btrfs fairs on that front and how long it will take to get a reasonable installed base, regardless of how good it might be in the future. Inertia is deep when it comes to storage.

Reply Score: 2

Arun Member since:
2005-07-07

[


This is an article about possible future Linux filesystems (apart from ext4). Take a look at the first post ;-).


May be the next time you post on Solaris articles you should too.


No I'm sorry, but people just don't get that excited about filesystems. ext4 and Btrfs are attempts at trying to move some shortcomings on, but new filesystems are not big bang events and don't generate a lot of default excitement that some people think that ZFS automatically should.


Yes they do.

http://blogs.smugmug.com/don/2008/10/10/success-with-opensolaris-zf...

Edited 2008-10-14 23:00 UTC

Reply Score: 1

Arun Member since:
2005-07-07


You miss the point sweetheart.............again. Detecting bad hardware is a problem of hardware and the current state of drive technology. Using ZFS isn't going to change that situation, and all anyone is finding out as a result of using ZFS are Solaris driver problems ;-).


No Miss Daisy, you are. Your statement is like saying "I have a cold and I can't smell the catastrophic gas leak. Since I can't smell it there is no problem". Pretty foolish.


Feel free to furnish me with a list.

Err.. I really don't care to.


Currently, that is x86 only and preferably 64-bit if you value your data. The problem for Sun though is that little if any code is flowing back in Sun's direction to help them maintain ZFS, so the quality control and shared development just isn't there. If you ever wanted to know why Linux uses the GPL, that's it.


You really need to provide some data to back this claim?

Reply Score: 0

segedunum Member since:
2005-07-06

Your statement is like saying "I have a cold and I can't smell the catastrophic gas leak. Since I can't smell it there is no problem". Pretty foolish.

You're not going to be able to smell gas all the time sweetheart, and smelling the gas does not fix the problem. Fixing your gas system might be a far better option ;-).

He, he, he, he, he. Seriously. You don't get that?

Err.. I really don't care to.

Hmmmmm, so ZFS is not tested on a wide list of hardware than I gave it credit for?

You really need to provide some data to back this claim?

Give me a list of outside contributors contributing source code to a central code repository, and that is going into Solaris today. I can't provide you with such a list, and I don't think you will care to either ;-).

Reply Score: 2

Arun Member since:
2005-07-07

"Your statement is like saying "I have a cold and I can't smell the catastrophic gas leak. Since I can't smell it there is no problem". Pretty foolish.

You're not going to be able to smell gas all the time sweetheart, and smelling the gas does not fix the problem. Fixing your gas system might be a far better option ;-).

He, he, he, he, he. Seriously. You don't get that?
"

Errr... not smelling the gas gets you killed... data corruption gets you fired or makes lose money/business/customers. Duh! I hope you are not responsible for anyone's data but your own. Your ineptitude could cause someone serious harm.


Err.. I really don't care to.
Hmmmmm, so ZFS is not tested on a wide list of hardware than I gave it credit for?


No. I don't care to indulge your every whim.


Give me a list of outside contributors contributing source code to a central code repository, and that is going into Solaris today. I can't provide you with such a list, and I don't think you will care to either ;-).


See above.

Edited 2008-10-16 03:58 UTC

Reply Score: 1

sbergman27 Member since:
2005-07-24

Hmmmmm, so ZFS is not tested on a wide list of hardware than I gave it credit for?

No. I don't care to indulge your every whim.


While I still consider most of this thread to be a silly waste of everyone's time... I do think the question is a valid one. IIRC, you were claiming that ZFS would be suitable for a small ARM-based NAS appliance. Yet you seem quite evasive regarding Segedunum's question about what hardware ZFS has actually been shown to scale down to.

I do quite a lot of work with such devices running Linux, and so am familiar with the rather stringent hardware limitations. I don't think there is a chance in hell of getting ZFS running on such a platform.

Reply Score: 2

segedunum Member since:
2005-07-06

Errr... not smelling the gas gets you killed... data corruption gets you fired or makes lose money/business/customers. Duh! I hope you are not responsible for anyone's data but your own..

F***. You mean to tell me that you rely on your own nose to tell you that you have a gas problem, and you rely on that to keep the system going rather than finding and fixing the problem or getting yourself a decent gas system to start off with? How many problems that ZFS has caught over time have resulted in fixes and patches to Solaris?

In this case, Solaris's drivers need fixing, because the vast majority of what ZFS picks up are invariably device driver problems when you get to the heart of it. Oh, if only Solaris had Linux's device drivers and the wide range of testing that goes on in a true open source project where people get the code, compile it on lots of hardware and find problems.

I'd check yourself into a clinic before you have a very large explosion.

No. I don't care to indulge your every whim.

So Solaris runs on very, very few platforms, as does ZFS, and is highly unlikely to ever run on a ARM or PowerPC device? Glad we've established that you have no data and no evidence of a counter claim whatsoever. You're mighty glad to start demanding evidence from others when people start pointing out issues and your paranoia kicks in.

See above.

You have produced no such list, lying about it isn't going to help you and I can give you an answer. No one is committing into Solaris's source tree because there is no open source repository and, as such, no open source Solaris project.

Difficult to face with someone of your obvious issues, but true nonetheless.

Edited 2008-10-17 10:02 UTC

Reply Score: 2

Phobos Member since:
2008-04-30

"1) file systems do not follow Moore's Law...

The article didn't say that it did, but Moore's Law has allowed people to do things with their storage devices where filesystems and storage containers like LVM and RAID haven't haven't quite kept up. That was the point the article was making.
"

the article said:
The problem with contemporary file systems, Ts'o said, is that -- following Moore's Law -- file sizes have grown bigger, and disk drives have doubled in capacity every couple of years.


so, indeed, it did say FS followed Moore's Law... try proof reading next time.

Moore's Law was about the growth of the number of transistor on a CPU die... people mistakenly extrapolated that to everything, being a wrong assumption at it's root.

Hmmmmmmm, no. Some people want to believe that, but it isn't true I'm afraid. For the vast majority of storage uses in the world today, and when you look at reviews of OpenSolaris, no one is the slightest bit interested in ZFS or even aware that it exists.


So, you need proof, eh?

- Linus Torvalds:
And yes, maybe ZFS is worthwhile enough that I'm willing to go to the effort of trying to relicense the kernel. But quite frankly, I can almost guarantee that Sun won't release ZFS under the GPLv3 even if they release other parts.

http://lwn.net/Articles/237905/

- Zemlin's desperate and groundless attacks like:
With capabilities such as ZFS and DTrace, Sun is trying to compete based on minor features


while later stating:
Zemlin suggests that it should make ZFS and DTrace available under a Linux-compatible license.


(So minor he wants it on Linux so badly?... A bit contradictory, isn't it?)
http://www.nytimes.com/idg/IDG_852573C400693880002574CE00371FE1.htm...

- Apple: http://www.apple.com/server/macosx/technology/filesystem.html

- FreeBSD: http://lists.freebsd.org/pipermail/freebsd-current/2007-April/07054...

- ZFS-on-FUSE: http://zfs-on-fuse.blogspot.com/

Not only that everybody wants it, it also has inspired other works like Mathew Dillon's HAMMER: http://kerneltrap.org/DragonFlyBSD/HAMMER_Filesystem_Design

While desktop users don't know/don't care about FSs, whenever they buy a Mac or install Solaris or FreeBSD, they will get ZFS, and they don't ever need to know about their existence... that's the point of it, FSs should be transparent to end users...



It makes certain things somewhat better, but Sun unfortunately don't have the userland tools that would expose ZFS as something remotely useful for the majority.


Heh... investigate a little before doing such assertions: http://blogs.sun.com/erwann/entry/zfs_on_the_desktop_zfs


In summary, no one is going to rush off their existing platform to get ZFS and btrfs (a select few might). Filesystems tend to change slowly, and adoption happens in the normal course of other things, normal iterative development and the price of upheaval.


I think my previous list can prove you wrong.


Sorry, but no it wouldn't. It has a great deal of useful features, but for a filesystem, it consumes far too much memory and CPU without some pretty damn serious tuning. It certainly isn't a general purpose filesystem you can throw any workload at.


You see, ZFS was designed taking in account the vast percentage of wasted CPU cycles on modern servers and computers in general... same thing that has motivated virtualization on all platforms, in modern times there is an excess of processing power, not lack of it like 20 years ago. The whole idea of hardware disk controllers to make disks arrays has become obsolete in many (save some very specialized) cases, thanks to ZFS... I invite you to read some papers and the ideas behind the Thumper X4500... a storage solution using ZFS

Even so, ZFS is no so massively resource intensive as you seem to imply, it just need a decent (not even great) configuration based on modern modern standards and it works like a charm...

Reply Score: 0

Comment by braddock
by braddock on Tue 14th Oct 2008 00:14 UTC
braddock
Member since:
2005-07-08

<em>"They agreed on btrfs, which was written from the ground up by Oracle's Mason based on his prior Novell work with the Linux-based reiserfs file system"</em>

I'm glad to see something come from Reiser's work. It would have been a shame for the technology itself to have been stigmatized.

Work with both high-res satellite imagery and large computational linguistics datasets have proven to me that more than a conventional filesystem is necessary for many fields.

Reply Score: 2

Btrfs not purely from parent's basement?
by tyrione on Tue 14th Oct 2008 02:03 UTC
tyrione
Member since:
2005-11-21

Btrfs is now a multi-vendor effort from Red Hat, HP, IBM and Oracle allowing a common pool to save costs of development and will provide a number of additional features which requires a fundamental redesign.


I've made this point before, but the heavy lifting of Linux, for nearly the past decade has come from Corporations and their funding of engineers to put in the code and time.

You may proclaim it superior to closed source, but the reason they are doing it is Time to Market. If they could have built a solution faster and equal in a proprietary vein you betcha they'd do it. I'm just glad they can't justify the cost and tow it alone so they've brough it into the open for us all to draw upon and add into, however minutely, as time goes on.

Reply Score: 2

MamiyaOtaru Member since:
2005-11-11

I've made this point before, but the heavy lifting of Linux, for nearly the past decade has come from Corporations and their funding of engineers to put in the code and time.

Two responses:
1: duh
2: so?

You may proclaim it superior to closed source, but the reason they are doing it is Time to Market. If they could have built a solution faster and equal in a proprietary vein you betcha they'd do it.

Paraphrase: "you might say it's superior, but get this, they're only doing it because it's actually superior." Why thank you for that insight. That's sort of what we were saying.

Reply Score: 8

Liquidator Member since:
2007-03-04

Btrfs... I thought is was BitTorrent File System ;)

Reply Score: 5

meh...
by hobgoblin on Tue 14th Oct 2008 04:33 UTC
hobgoblin
Member since:
2005-07-06

im more interested in tux3, or the one being developed for dragonflybsd (hammerfs?)...

as for corps sponsoring development or not, i have no worries as long as its under gpl or equivalent.

if so, its less likely for any one corp to hijack the prooject or dictate its use (see microsoft silliness, past and present)...

Edited 2008-10-14 04:33 UTC

Reply Score: 4

RE: meh...
by abraxas on Wed 15th Oct 2008 12:42 UTC in reply to "meh..."
abraxas Member since:
2005-07-07

im more interested in tux3, or the one being developed for dragonflybsd (hammerfs?)...

as for corps sponsoring development or not, i have no worries as long as its under gpl or equivalent.

if so, its less likely for any one corp to hijack the prooject or dictate its use (see microsoft silliness, past and present)...


Draonfly's filesystem is meant for clusters and won't really do much for a single system.

Reply Score: 2

Funny
by Kebabbert on Tue 14th Oct 2008 18:41 UTC
Kebabbert
Member since:
2007-07-27

How many people are negative to ZFS. If they would have tried ZFS themselves, they wouldnt say those things. Yes, ZFS is that good. It is not over hyped. I run it at home as my file server. Before I always backed up important files to other computers. Now I am nervous until I get it on my ZFS raid. Then I can calm down. ZFS doesnt like hardware raid card. It wants to manage all the drives it self.

Here is an article of a Linux guy tries out ZFS as his home file server and he loves it. As everyone else that has tried it. He investigates different solutions and chooses ZFS at the end. Very interesting motivation he has:

http://breden.org.uk/2008/03/02/home-fileserver-existing-products/

Reply Score: 3

FAA switches to ZFS:
by Kebabbert on Tue 14th Oct 2008 18:46 UTC
Kebabbert
Member since:
2007-07-27

After several embarassing air plane crashes, they switch to ZFS.

http://www.eweek.com/c/a/IT-Infrastructure/How-the-FAA-Is-Bringing-...

Reply Score: 0

RE: FAA switches to ZFS:
by sbergman27 on Tue 14th Oct 2008 19:25 UTC in reply to "FAA switches to ZFS:"
sbergman27 Member since:
2005-07-24

After several embarassing air plane crashes, they switch to ZFS.

Uhhh, read your own link. After some embarrassing crashes of their *flight plan filing system* they are upgrading some very old *internal business servers* to new equipment. And some of it happens to use ZFS.

The distinction between a "plane crash" and a "flight plan filing system crash" may seem subtle to you. But some people, including users of the respective hardware involved, do care.

Edited 2008-10-14 19:26 UTC

Reply Score: 3

ZFS fine with 1GB RAM
by Kebabbert on Wed 15th Oct 2008 08:57 UTC
Kebabbert
Member since:
2007-07-27

SEGEDUNUM,

I have run ZFS for over a year with 1 GB RAM intel P4@2.4 GHz at home. It worked fine. The thing is ZFS likes to cache a lot, as it is a Server Enterprise file system. It will use all the memory it can grab for it's cache several GB if it RAM is idle. And if some app wants memory, ZFS will release some memory. But that does not mean it REQUIRES a lot of memory to run. Several people have explained this several times, but you just ignore those posts.

Come on, ZFS is targeted to Enterprises, do they have 256MB RAM? No, they have several GB. If SUN would have designed ZFS without using a cache, so it would run fine with 128MB RAM, then you would have complained about it: "ZFS not using a cache sucks because ZFS is not usable in Enterprise environments that have lots of RAM. ZFS doesnt use the available idle memory. A computer should use all the idle memory for something. Otherwise it is a waste. Therefore ZFS sucks badly".

The most logical thing, when designing a new filesystem, is to allow it to use all idle RAM for performance reasons. But you state that is a bad design choice. You doesnt like the idea of a cache to speed things up. That is less optimal and goes against all computer knowledge. You SHOULD use a cache, right? (I bet if SEGEDUNUM would run an Enterprise, all computers with 512GB RAM would have their RAM unused. "Turn of all caches! Never cache anything! That is bad!").

Lets face it, nothing SUN does will please you. Damned if SUN does add a cache to ZFS, damned if SUN doesnt add a cache. Sometime in the past, some SUN affiliated guy must have done something terrible to you.





And for ZFS "rampant layering violation", the main architect behind ZFS explains why that is wrong:

http://blogs.sun.com/bonwick/entry/rampant_layering_violation

Reply Score: 2

Anatomy of an OSNews discussion
by abraxas on Wed 15th Oct 2008 13:13 UTC
abraxas
Member since:
2005-07-07

An article about Linux filesystems becomes a discussion about the superiority of Solaris over Linux.

An article about Mono becomes a discussion about the superiority of KDE over GNOME.

An article about Apple becomes a discussion about how there are lower priced, better alternatives for everything they create.

An article about Vista or XP becomes an article about the superiority of every other OS over anything Microsoft creates.

Unfortunately this place has become a haven for vendor trolls. Why can't we talk about the technology anymore?

Reply Score: 2

Great SEGEDUNUM.
by Kebabbert on Wed 15th Oct 2008 18:16 UTC
Kebabbert
Member since:
2007-07-27

Is it only me that finds it difficult to try to communicate with SEGEDUNUM? I have told him for the umpteenth time that Ive run ZFS on 1GB RAM with 32 bit CPU for over a year without problems. Other people have told him similar stories. And still he states that ZFS requires several GB of RAM to function? And probably he will continue to ignore our posts, because we tell things he doesnt like to hear.



And his implicit claim that ZFS' habit of trying to use all unused RAM as a cache is stupid - not many people agree with that? ZFS as a Enterprise filesystem can expect to run on machines with for instance, 64 GB RAM or more, and it is bad to try to use all idle RAM as a cache??? It is absolutely vital that SUN makes sure that an Enterprise filesystem runs well on 256MB RAM machines???



Is it just me that thinks this reasoning is... a bit strange?



As Ron writes:
"Funnily enough, most of the hardened ZFS critics I know who I endlessly debated with changed their tune after 5 minutes of actually using ZFS, the proof of the pudding is in the eating :-)"

Reply Score: 2

RE: Great SEGEDUNUM.
by jwwf on Thu 16th Oct 2008 01:23 UTC in reply to "Great SEGEDUNUM."
jwwf Member since:
2006-01-19

Is it only me that finds it difficult to try to communicate with SEGEDUNUM?


Maybe Jonathan Schwartz used that pony-tailed sensitive guy look to effect and stole his girl? If not, the vehemence seems a little, um, odd.

Reply Score: 2