OSNews

First of all a disclaimer: This is not an attempt to bash the parent post, but a genuine question.

How would it be that building a FUSE driver would solve the patents issue ?

Because a FUSE driver wouldn't have to link to the GPLed kernel and would consequently not present a license conflict. Also, there was already work in progress on a FUSE ZFS impl for last year's GSoC, though I have no idea how far that progressed in the end.

Update: Err, missed the actual point. The FUSE driver could, due to the lack of license issues, use the actual ZFS code, which would prevent the patent issue from cropping up.

Edited 2007-04-20 19:31 UTC

*and* will be available for multiple OSes including Linux, Mac OS X and BSD UNIX.

It will be available on MacOS X (will be part of Leopard), and is already available on FreeBSD -CURRENT (will be part of 7.0).

Linux is the only one not in the ZFS party.

Hehe, owned by their licence.

"""
Hehe, owned by their licence.
"""

Well. I'm a Linux advocate. And I am partial to the GPL, despite what a pain in the ass it can, at times, be.

BUT... I can agree that it is a good thing for GPL advocates to occasionally get hoist by their own petard.

There is a lot of unfairness going on in the form of "we can take your code but you can't take ours" and it is mostly in the direction of BSD Project -> GPL Project.

You can actually substitute many permissive licenses for "BSD" in the statement above.

It's good for GPL advocates to, at least occasionally, see that the GPL has thorns as well as teeth.

It is the inflexible nature of the GPL, more than the terms of the CDDL, that are denying Linux ZFS.

Then again, by Linux standards, ZFS is, in Andrew Morton's words, a "rampant layering violation".

Beyond the licensing issues, I think we'd be more likely to see Reiser4 merged than ZFS, without some major philosophical reworking of the design.

Edited 2007-04-20 22:18

How is it that Mac OS X and FreeBSD overcame the patents problem ? Wouldn't it be possible for Linux to do things the same way ?

Do they use binary compatible reimplementations of the FS ? If so I guess they're all in the same ship with those patents problems.

(I really doesn't know much about ZFS history and implementations, so please excuse if the answers to those questions are obvious).

They both use Sun's code. Presumably Apple worked something out with them, and the 2-clause BSD license is not incompatible with the CDDL, so there's no problem with FreeBSD. The issue with Linux is that Sun's license and Linux's are both restrictive, but in different ways. Mixing Sun's work with more permissively licensed code is not a problem.

The CDDL that ZFS is under has an explicit patent grant, so if you can keep the CDDL license, then you don't have a patent problem.

As mentioned, BSD doesn't really have this issue at all. CDDL is also, like MPL, a file based licensed. So, Apple is obligated to only return any changes they made to ZFS files, but aren't under any obligation to return their entire OS simply because they integrate ZFS.

Beyond perhaps some behind the scenes contracting between Apple and Sun for technical support, there need be otherwise no formal relationship between the companies.

Ha, already under way:
http://www.wizy.org/wiki/ZFS_on_FUSE

I see a huge problem, there's no way that I'm going to run a userspace filesystem on a machine in production.

And when it comes to my desktop, I couldn't care less about the filesystem, ext3 and lvm is more than flexible enough.

I would hope FreeBSD supports ZFS, in matters of legality, seeing as how FreeBSD isn't underwritten with the GPL as it's licensing scheme.

FreeBSD even has support for Sun's DTrace, although it can't be included with the GENERIC kernel by default due to licensing issues with a few bits (mainly headers, I believe).

That would defeat the "all software must be free -- by which we mean GPL'd" mantra.

The FSF want to hold the reigns. If there's going to be any licence compatibility, they want to be the ones who decide, not the actual project owners; and the GPL3 is even worse. I suspect when the first GPL3 violation occurs, it'll be laughed out of court for it's almost EULA-like unenforceableness (yes that is a real word; that I just made up).

Backup your claims, please.

I've said before - the FSF trying to step above the patent system is as bad as Microsoft using it. That is my stance on it and I shall refrain from detailing further to save a three-hundred long thread of disagreements.

Your first post claimed the FSF wants to decide upon license compatibilities, not step above patent systems.

First, the FSF doesn't decide at any stage which licenses are compatible. The license "decides". If a project takes license X, it made this decision itself!

Second, the GPLv3 isn't in any case more restrictive about compatibility than the GPLv2.


The kernel devs could have chosen LGPL as their license. They could also have chosen then, that parts of the kernel won't be licensed under GPL, but some other license, like CDDL. I doubt the kernel devs would ever want to do this step. So the only possibility for the kernel devs - self chosen! - to get code into their kernel, is to
a) rewrite it
b) having a license on it (like BSD) which allows to re-publish under GPL

Whereas, a) could not be legal, as it would violate SUN's patents.


Again, the FSF has to do _nothing_ with this issue. It's just the kernel developers who don't want to incorporate CDDL code. Which is totally sane to me.

Edited 2007-04-21 00:14

b) having a license on it (like BSD) which allows to re-publish under GPL

THAT'S a STRICT VIOLATION OF BSDL!

Read here: http://www.opensource.org/licenses/bsd-license.php

All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
* Neither the name of the <ORGANIZATION> nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.


Who was that idiot who thought that he can relicense BSDL as GPL?

You can take BSDL code and give it to others within another license. This is what Microsoft does, Apple does, and some open source projects do, too.

The other license just has to include the BSD licence terms in that case.

Thank you for the response. The presence of the link gave me a good laugh.

Right, and this is why the FSF puts a list of free software licenses and their compatibility with the GPL here: http://www.gnu.org/licenses/license-list.html#SoftwareLicenses

All of these licenses are what they consider free. Not all are copyleft, of course.

Especially the copyleft licenses lke the GPL should include a compatibility clause so that they allow to merge for example GPL code with any other Free license explicitly approved in the GPL. That way, the GPL code would remain GPL'ed, while the code under the other (now incompatible) license would remain under its own license.

Go to http://www.gnu.org to find out why this is a terrible idea. Basically you are giving no incentive for GPL software to be developed. No company working on Linux is going to release their code under the GPL if they don't have to that's why the GPL forces companies and everyone else to.

Why do you think ZFS comes under a license that's incompatible with the Linux kernel? Sun could have chosen any license they wanted, and they deliberately picked one that was incompatible with the Linux kernel.

It's Sun's right to do that, just understand why: they want to replace the Linux kernel with a kernel whose development they control.

And, no, they are not all "Free and Open Source projects". BSD are open source projects but are not Free Software. ZFS (and Solaris) may be shipping under an open source license, but they are not run like open source projects.

There are reasons all these projects have difference licenses, and they will not be made compatible because lots of people don't want it (and that's a good thing as far as I'm concerned).

Care to back up those claims with evidence? Sorry, but those are just lies. Flat out lies.



Conspiracy theorists unite!

Seriously, you don't have anything factual to back up these claims. You have nothing but wild speculation at best and the lies of another individual.

lol, it's exactly the contrary.
The more linux will get Solaris features, the less they will have potential clients for Solaris.

Fortunately for Sun, Linux can not get Solaris features.

Maybe. Sun offer end-to-end solutions for systems, from the hardware, software, testing, development, support and training. I know redhat offers this too, but they don't control the hardware like Sun can. Plus, in my view, Sun are more reliable than RedHat.

True, and if you have a look at the latest acquisitions - StorageTek which gets them into the IBM houses which rely on large tape libraries. Tarantella which brings together Windows, UNIX's of various flavours and mainframe access, Sun can offer a complete solution, like you said.

It isn't just about the control, but to make a loss off the hardware and make it up somewhere else. Rather than worrying about making a profit on each component sold, they make a profit on the whole kit sold at the same time. When you go for a Red Hat solution, everyone is gotta take their cut.

Dell sells you the server, they want to take a cut. EMC for the storage machine, then you'll need to grab some storage software, so you'll grab it from them. Then you'll need a company to deploy it, so you bring in someone like IBM global services or EDS.

With each cut taken by each company along the process of getting what you need, each must take a cut in terms of a profit; and what do you gain out of it? a frankenstein network from various companies who will simultaneous point the finger at each other when the shit hits the fan.

Fortunately for Linux, many Solaris features are probably a bad idea anyway. For example, volume management and the file system probably shouldn't be merged.

Care to explain why? I am starting to suspect sour grapes. Every pro linux person seems to regurgitate the same argument but with no reasoning about why.

Edited 2007-04-22 18:40

It's nice sun has this code there and available, but I think for the linux community the long term bang for the buck will be from new thinking and refactoring from what ZFS brings.

Also known as "Not Invented Here" syndrome, which is what the vast majority of Linux kernel developers have. Everyone thinks they can do it better, so everything gets redone, over and over and over again. Doesn't matter that some other OS has ready, reliable code. The Linux devs can do it better.

Sun's DTrace and ZFS are just two examples of this, although the GPL plays a large part in these.

Would you think that if Linux could implement DTrace and ZFS that they wouldn't do it their way?

FreeBSD for instance (and yes I understand the license differences please spare me) seems to not want to duplicate work that already works, so they just integrate it into their OS, which I think is the right way to go about it. I believe that Linux would do the same if they could, legally. But then again it seems to be a per-distro thing these days (Eg: Ubuntu's upstart... i understand that they want to improve things, but why make something completely different from the standard working implementation, but that's not related really...)

Overall from what I hear (haven't had a chance to test it out myself) but ZFS is pretty impressive. I wish I had some links to some numbers, but google is friend .

AFAIK there is no NIH-syndrome among Linux devs. It's a lie spread by opponents. Linux devs have more than once included technology from outside. More often than not it has been rewritten but this is due to techical reasons and not because of NIH-syndrome.

As someone who is a Linux developer, let me disabuse you of that notion.

Like many social groups, the Linux developer community has a social pecking order. Some of the people at the top, most notably Linus himself, do not suffer from NIH syndrome, but there are certain parts of the kernel in which there are very bad cases of NIH.

Examine the history of power management, file systems, usb, scsi, kernel objects, and a few other things and you'll easily find the NIH.

"""
AFAIK there is no NIH-syndrome among Linux devs. It's a lie spread by opponents.
"""

I think it more a misperception by people who didn't get their favorite feature included.

As users, we have the luxury of seeing features, say Reiser4, saying "Oooo! Shiny!" and petitioning for the code's inclusion. The people who actually have to integrate it and maintain it must take a more critical look.

Is the code maintainable? Does it contain an ugly compatibility layer like XFS? Does it try to replace whole layers of Linux, like VFS, with its own version running in parallel, as Reiser4 does?

If they say "no" to a set of patches, the fans of the excluded features immediately conclude that NIH is at work.

Earlier on, Linux had few enough features that some patches got a free ride. The kernel devs are free to be more critical in their evaluations today.

For example, I suspect that XFS, if presented for inclusion today, would have to live out of tree. The impedance mismatch would simply be too great to accept now that journaling Linux FSes are plentiful, and at least some of the aura of mystique that surrounds XFS has worn off, due to the passage of time, and the realization by users of just how fragile XFS really is.

One of the most important requirements of a maintainer of a large and popular project is the ability to say "no". But saying "no" is also a guarantee that some people are going to accuse him of having an NIH attitude.

Edited 2007-04-21 21:08

As users, we have the luxury of seeing features, say Reiser4, saying "Oooo! Shiny!" and petitioning for the code's inclusion. The people who actually have to integrate it and maintain it must take a more critical look.

Which is why no ZFS implementation would actually get in. It crosses so many layers that already exist in the kernel itself, it's even worse than Reiser4.

Also known as "Not Invented Here" syndrome, which is what the vast majority of Linux kernel developers have.

Go and ask Sun about that, because they explicitly put the whole thing under a license that was deliberately GPL incompatible. There has been a lot of code that has made it into the Linux kernel from outside, as long as licenses permit.

I'm intrigued by the whole patent thing though, because I wonder how this tallies with Sun's new found GPL love.

I don't know whether its ignorance on your part or if you have an agenda but please stop repeating the mistruth that CDDL was deliberately made incompatible with the GPL. Sun examines a lot of licenses for OpenSolaris, including the GPL. Please see Simon Phipps blog postings on this subject as well as the mailing post entry

http://www.opensolaris.org/jive/message.jspa?messageID=55008#55008

"""
please stop repeating the mistruth that CDDL was deliberately made incompatible with the GPL.
"""

CDDL was deliberately made incompatible with Linux. That is a slightly different statement. And the post you point to actually supports that viewpoint by stating that they could not wait for GPLv3 (which would have met the criterion of being Linux incompatible).

At any rate, if it comes down to trusting Danese Cooper or Simon Phipps, I'd pick Cooper any day of the week.

Edited 2007-04-21 23:01

Err, you apparently don't have a clue about what you're saying..ZFS is a lot more than a volume manager or a plain filesystem. First of all, it's a completely self-healing atomic system, it's consisted of pools, etc etc etc.. Does LVM protect you from silent corruption of filesystems? Nope, it doesn't. Why? Because it's just a volume manager..So that's already more than enough of a reason to use ZFS IMHO. I suggest you read a little bit more before you spread FUD

And I take it you have never used Veritas Volume Manager or Veritas File System (VxVm and VxFS). There is a lot more to managing large storage arrays than laying down a filesystem on a large hardware based RAID 5 array.

Does HP's or Linux LVM support snapshot capabilities? That alone is enough for me to use ZFS over a stock UFS filesystem. And the ability to add storage to a pool dynamically for immediate use rocks. And this is not limited to hard disks, check out this video of some Sun engineers using ZFS on USB memory sticks:

http://video.google.com/videoplay?docid=8100808442979626078

I somehow don't think that either HP's or Linux LVM solutions can reproduce what Sun does with ZFS.

Edited 2007-04-20 22:25

And I take it you have never used Veritas Volume Manager or Veritas File System (VxVm and VxFS). There is a lot more to managing large storage arrays than laying down a filesystem on a large hardware based RAID 5 array.

Does HP's or Linux LVM support snapshot capabilities? That alone is enough for me to use ZFS over a stock UFS filesystem. And the ability to add storage to a pool dynamically for immediate use rocks. And this is not limited to hard disks, check out this video of some Sun engineers using ZFS on USB memory sticks:

ZFS has many qualities that LVM (or Veritas) does not provide. Yet what you've mentioned can already be done.

* LVM2 does support snapshots (and we're actively using them for backup purposes)
* LVM2 does allow dynamically adding (or removing) storage items to the pool (and enlarging ext3 filesystems on the fly, but don't know about online shrinking).
* LVM2 does indeed support any block device (since it's based on device-mapper), yes it does USB disks too.

But as I said there are many areas (like online self healing) that ZFS can but LVM cannot, however your choice of features is already done.

Edited 2007-04-21 07:32

So far as I know, online shrinking if ext3 filesystems is not supported.

And you are correct that the Linux stack does not have checksumming and self-healing.

For the rest of it, Linux could really steal Sun's thunder simply by improving the admin tools.

dm/lvm/ext3 have all these capabilities, but the admin has to explicitly worry about partitioning the drives, marking them as physical volumes, adding them to the pool, resizing the logical volume, and resizing the filesystem.

It's even more complicated to remove storage from the pool.

If we could just get our act together with regards to admin tools, ZFS would not get all the Ooos! and Ahhhs! that it does.

Arguably, Sun's decision to dump all the capabilities into the filesystem layer itself made writing the admin tools easier for them. But there is no reason that things could not be just as simple and straightforward for Linux admins.

And I take it you have never used Veritas Volume Manager or Veritas File System (VxVm and VxFS). There is a lot more to managing large storage arrays than laying down a filesystem on a large hardware based RAID 5 array.

You've answered your own question really. ZFS is certainly a step forward for those running Sun systems and wanting large scale storage options, because let's face it, what went before was pretty non-existant and quite woeful. That's where ZFS' home really is - large storage arrays.

For those in the Linux and other areas of the Unix worlds, they have had the option of keeping their existing filesystems and using LVM or EVMS with a combination of RAID, ATA-over-ethernet, iSCSI (has its problems) etc. or products like those from Veritas. Those doing it are...........already doing it, and ZFS has slipped their consciousness. All Sun is doing is trying to stop existing Solaris using people from jumping ship.

Does HP's or Linux LVM support snapshot capabilities?

LVM certainly does, and has been widely known about for years. You do know this, right?

That alone is enough for me to use ZFS over a stock UFS filesystem.

Snapshots is enough on its own for you to use ZFS? Wow. Welcome to 2007.

I also don't see what UFS has got to do with it, because volume management is logically removed from the filesystem itself - which is why Andrew Morton complained about ZFS as a layering violation. You could reasonably install a UFS filesystem on a LVM volume and make it work. There's no reason to create a new uber filesystem just to do that and pretend it's something new and cool.

I somehow don't think that either HP's or Linux LVM solutions can reproduce what Sun does with ZFS.

Wow. Chaining a bunch of separate storage volumes together to create more storage space - a bunch of USB sticks in this case. Never been done before.

You can use Linux LVM, EVMS and/or RAID (whether that be software or hardware) and your normal filesystem to do exactly that. You can do LVM over a network for crying out loud. At a higher level, that's what GFS is about. Goodness, you can even do ATA-over-ethernet. If they'd done that over a network hub I might have found that moderately interesting. A nice and usable GUI representation, overview and interface would have been nice as well. Lame. I also don't get the scrambling of USB sticks into a random order at the end of the video, because...........that's what a RAID array pieces together.

It's a good example of why Andrew Morton thinks that ZFS is a layering violation, because much of what ZFS does has absolutely nothing to do with file storage itself or a filesystem. Sun pretending that it is just daft.

You're seriously telling me you don't think any of that can be done on a Linux system?

Personally, I think the most interesting part of that video was using USB flash drives as storage, because I think that's where the next advance in storage will come from. Widespread, cheap and large flash drive arrays. That's the only way reliability and availability will truly increase - better devices themselves.

Edited 2007-04-21 10:33

Actually, no I didn't. The vast majority of the Linux servers we use do not have large storage arrays attached to them, mostly local storage (DL385/585 internal disks), and they are set up using the hardware RAID controllers. So we have never really looked at using LVM or LVM snapshots. And based on this discussion, I really don't know whether I would want to:

http://lwn.net/Articles/226796/

and they are set up using the hardware RAID controllers. So we have never really looked at using LVM or LVM snapshots. And based on this discussion, I really don't know whether I would want to

*shrugs shoulders*

I don't really see why you wouldn't use LVM, even with hardware RAID. They're different layers in the system and have different and complementary uses.

That's an awful lot of trouble to go through for a web, MySQL or a Snort server. And I really wouldn't derive much benefit from the effort. If the machine was connected to a couple of terabytes of storage things would be different.

"""
That's an awful lot of trouble to go through for a web, MySQL or a Snort server. And I really wouldn't derive much benefit from the effort.
"""

What distribution are you using on these machines?

Modern server distributions give you LVM for free during the standard install.

It's one of those things that you may not always need, but if you do end up needing it, you're glad its already in place.

If your server distro does not give you standard server features by default, then I can begin to understand why you think Linux is so limited.

RedHat Enterprise Linux 3 and 4. I watched my boss do some of the LVM stuff one day and it reminded me a little too much of managing disks with AIX (which I never liked). The majority of the machines we have are Solaris boxes with about six RHEL machines. All of them are backed up with NetBackup so short of a complete disaster I can recover pretty quickly.

I am using ZFS in production and LVM in what it takes to set it up and use it just doesn't compare.

As a former HP Storage Consultant I am extremely impressed with ZFS capabilities. LVM on Linux and HP-UX can do some of the things, however ZFS really bridge some of the gaps between enterpise storage arrays and low cost jbods. look at some of the potential of the x4500 with the Solaris native iscsi target implimentation and ZFS.

Also managing thousands of luns from an enterprise array is pain, ZFS simplifies that trmendously. Restriping on the fly isn't an option if you need to add spindlesluns wth HP-UX LVM unless you add luns in the same number of the original stripe. ZFS does restriping on write which is a really nice feature when your environment changes.

I worked extensively on the HP Enterprise Virtual Arrays which impliments much of the same concepts of ZFS in the hardwarelun level. Those concepts implimented on the file system are an extremely powerful feature and would be glad to debate them. Needless to say the ability to not tie physical to logical is the way we are going. I just got back from Storage Networking World and virtualize everything is the mantra of the storage world, ZFS is a natural extension.

I also don't see what UFS has got to do with it, because volume management is logically removed from the filesystem itself - which is why Andrew Morton complained about ZFS as a layering violation. You could reasonably install a UFS filesystem on a LVM volume and make it work. There's no reason to create a new uber filesystem just to do that and pretend it's something new and cool.

Linux developers calling anything a layering violation is laughable at best. Drivers directly call into kernel functions and all sorts of ugliness exists in the Linux kernel. Solaris is more layered than linux will ever be.

Volume managment need not be logically removed and there is no logical reason it needs to be either. All it does is make managing storage messy and needlessly painful. ZFS' simplicity is testament to exactly why LVM and filesystems should be logically one.

. Goodness, you can even do ATA-over-ethernet. If they'd done that over a network hub I might have found that moderately interesting. A nice and usable GUI representation, overview and interface would have been nice as well. Lame. I also don't get the scrambling of USB sticks into a random order at the end of the video, because...........that's what a RAID array pieces together.

What does ATA over Ethernet have to do with LVM? If your can present a AoE device to ZFS it can user it just like LInux's LVM, The only limitation is using it in a clustered environment.


It's a good example of why Andrew Morton thinks that ZFS is a layering violation, because much of what ZFS does has absolutely nothing to do with file storage itself or a filesystem. Sun pretending that it is just daft.

What is daft is you bringing up Andrew Morton as an authority on storage and file systems. According to his own site he has no experience designing storage systems what so ever. Any one can have an opinion. If you intend on making a big deal of that opinion then the person better be an expert on the subject.

You're seriously telling me you don't think any of that can be done on a Linux system?

If it could there would be no discussion or requests for porting ZFS to linux and this entire article would be moot, no?

Edited 2007-04-22 06:06

Actually, Solaris has had an excellent albeit traditional volume manager for *years*. Until Solaris 9, it was an unbundled product but starting in 2002 it was bundled with Solaris.

Your characterization of ZFS has simply a LVM combined with a file system and some glue is sadly mistaken. You might want to visit the ZFS community at OpenSolaris

http://opensolaris.org/os/community/zfs/

to learn why its completely different than a simple combination of those two components.

In particular, look at the Source Tour.

Can you put the / partition on ZFS using FUSE?

If not, seems pretty useless to me.

You could always create a smaller partition - EXT3 or any other supported file system - to host /boot and leave the rest of / to ZFS Fuse... That ought to do the trick

That is completely uncalled for, and in very bad taste. You just invalidated your entire post. Nice way to shoot down your own argument...

"""
Schwartz has posted before about using the GPL for OpenSolaris as well as Java. If OpenSolaris goes GPL, the code will be immediately portable into Linux.
"""

Incorrect. Java *might* go GPLv2 first, with the intent of moving to GPLv3.

There has been no talk about GPLv2'ing Solaris, although they have suggested that they might GPLv3 it.

Sun would never allow Solaris' source to be license compatible with Linux.

Remember that GPL is not even compatible with different version levels of itself.

Edited 2007-04-21 23:43