Don’t use ZFS. It’s that simple. It was always more of a buzzword than anything else, I feel, and the licensing issues just make it a non-starter for me.
The benchmarks I’ve seen do not make ZFS look all that great. And as far as I can tell, it has no real maintenance behind it either any more, so from a long-term stability standpoint, why would you ever want to use it in the first place?
ZFS could be the fastest file system in the world and randomly disperse kittens and I still wouldn’t touch it with a ten metre pole if I were Linus. Oracle is a colony of snakes led by the biggest snake of them all, and adding their code – even through shims or interfaces – should be a complete non-starter for any project.
Strange; I thought ZoL has active maintenance.
Also the only alternative, btrfs (kinda same features), is hardly better than ZFS.
it’s also still an Oracle product! if the hate is about Oracle and what they might do, you’re no safer with btrfs.
The real test will be who comes out on top in oracle vs google. If that goes to google, then the open source license and that ruling should protect Linux with btrfs and all the other systems using ZFS.
a) Yes it’s in active development.
b) Except for the license and the original source, OpenZFS/ZFS on Linux has very little to do with Oracle.
Laughs in FreeBSD.
There’s two separate versions of ZFS: Oracle ZFS and OpenZFS. They share a common ancestor (ZFSv28 from SUN), but they are no longer compatible, and should not be confused with each other.
Oracle ZFS is highly proprietary, only runs on Oracle Solaris, and requires big payments from Oracle to access. They actively maintain it, and there’s even some new features that have been added over the years (like native encryption).
OpenZFS is actively maintained via the OpenZFS repo, and is used by dozens of hardware and software companies around the world. Recently, the OpenZFS repo was rebased on the ZFS-on-Linux repo (previously, it used the Illumos repo). There’s a lot of development happening with OpenZFS, and a lot of features have been added since the split with ZFSv28. Yes, it’s licensed under the CDDL which means it will never be integrated directly into the Linux kernel source tree, but it’s part of the source trees for various other OSes (FreeBSD, Illumos, OpenSolaris, Delphix, TruOS) and hardware storage appliance vendors. And the ZFS-on-Linux version is actively maintained and works great on multiple Linux distros.
So, in this instance, Linus is talking out of his ass, and really needs to shut up, lock himself in a room, and actually research what, exactly, ZFS, OpenZFS, and ZFS-on-Linux really are. He’s shot himself in the foot before, but this is beyond sad to see, and so very, very, very wrong that’s it’s embarrassing to the whole Linux dev community.
Maybe he should sit in on one of the monthly OpenZFS Dev Meetings. He’d learn a lot!
I agree discounting ZFS is what Linus shouldnt do given its stability elsewhere. However, I would definitely say the way oracle sued google over java interfaces that its no unreasonable to think Oracle wouldn’t sue the OpenZFS maintainers down the road if it benefited them
Oracle vs Google is still in the courts though. If google wins, we’re OK because of the licensing. If not, then we have a problem. However, other things like NIS could also be at risk and even some system calls.
NIS?
phoenix,
You’re right that linus is only speaking against ZFS because he can’t have it. If he could he would embrace it in a heartbeat. Although Thom chose to only quote that part of his post, it was not the bulk of his argument, which is about license incompatibilities and oracle litigation. IMHO he hit the nail on the head with regards to why he cannot merge it.
And you should lock yourself in a room until you at least spend 60 seconds reading the original post. The comment about lack of support was a single sentence and his very last sentence.
Everything else revolved around licensing issues and how that is simply a no go for the kernel and therefore he won’t care about ZFS in the role of a kernel maintainer. And this probably explains also why his knowledge about ZFS’s current state wasn’t… well… up to date. He knows he cannot use it and hence it isn’t something he keeps an eye on.
Time to consider FreeBSD (including SJW CoC) or OmniOS for advanced filesystem needs. Maybe DragonflyBSD with HAMMER2.
Linus is downplaying the value of ZFS, obviously it is more than a buzzword and there would be lots of value in having it in linux. There’s plenty of demand for ZFS and many operating systems want it…
https:// http://www.osnews.com/story/29103/ubuntu-may-ship-zfs-as-a-module-or-not/#comments
https:// http://www.osnews.com/story/29256/zfs-apples-new-filesystem-that-wasnt/#comments
Obviously the license is at odds with the GPL. There might have been a chance to rectify the situation when sun owned ZFS, but that ship has sailed and oracle is much more litigious than sun was, a point that torvalds brings up if you read his full quote. And I agree with torvalds that he cannot merge ZFS into linux.
Something he could do (but won’t) is adopt stable ABIs. That would at least resolve the issue for 3rd party modules including zfs. Alas it is common knowledge that the linux community is averse to stable ABIs.
The main alternative to ZFS feature-wise is btrfs, but it doesn’t have a history of being robust, although I don’t know if that’s a fair assessment today. Raid 5/6 are definite no-nos. Suse made it a default FS whereas redhat deprecated it.
https://www.phoronix.com/scan.php?page=news_item&px=Btrfs-RAID-56-Is-Bad
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/7.4_Release_Notes/chap-https://lwn.net/Articles/729488/
What’s everyone’s experience with ZFS & btrfs? I mostly use block level snapshotting, but LVM2 has it’s own caveats and it would be useful to do at the file-system layer. I’m in the “it’s already automated” and “why change what works” boat, but I might give these a shot to see whether they can improve my workflows.
This would only benefit those who want to use proprietary modules and not contribute for the greater good of the community. There’s no need, really. ZFS is in this regard a unicorn.
Bullllshit.
sj87, bullshit how?
I use btrfs on both of my desktop machines. I’ve had my share of trouble with it. The multi device support has been broken for ages, the write hole problem still exists.
ZFS is superior, but i understand the reasons for not merging it.
The argument that Linus makes here is mostly along the lines of “Oracle is evil, and they would screw everyone over”, to which i fully agree.
Having said that, i have used ZFS on solaris, both on sparc and x86, and while solaris is … solaris, ZFS is pretty damn awesome.
btrfs started as an Oracle project. It’s just as vulnerable as ZFS.
No, because btrfs was always a Linux targeted GPL-2 licensed project.
ZFS is not under GPL. It’s code is under CDDL and that license was intentionally aimed at preventing GPL compatibility. CDDL is a trap.
Zan Lynx,
You’re right about the btrfs being ok to use in linux because it’s GPL2.
However your statement about CDDL being a trap is just an opinion.
Some prefer weak-copyleft licenses including CDDL over strong-copyleft licenses like GPL, which can be considered more onerous for some developers. Furthermore CDDL grants not only copyrights, but patent rights as well whereas GPLv2 does not. This means linux contributions can technically be submitted by a patent holder who can then sue linux users for patent infringement. I’m not sure if this has ever happened or been tested in court. Regardless it’s one of the reasons some of us wish linux were licensed under GPL2+ since GPL3 protects us from potential patent infringement arising from GPL3 code.
Theoretically if oracle held patents on btrfs, they could sue for btrfs patent infringement under GPLv2 whereas they could not with ZFS due to the CDDL. I’m not suggesting they will, but it’s something I hadn’t really even thought about before right now.
sj87,
I find this choice of words odd because although the use of “unicorn” like that makes perfect sense here, I’m not sure the way it reads accurately reflects your thoughts about ZFS. Unicorns are often used as a term to describe things that are very rare and desirable. I never though I’d be posting this sort of thing here…
https://www.urbandictionary.com/define.php?term=Unicorn
https://en.m.wikipedia.org/wiki/Unicorn_(finance)
So while I’m sure you meant something else, the urban intepretation of a “unicorn file system” would mean a very successful file system. I don’t mean redirect the discussion, but it made me laugh and hopefully we can both have a chuckle about it 🙂
Anyways, I completely understand your opinion of not wanting to work with proprietary code in linux (it’s a damn shame GPLv3 isn’t allowed, but that’s a different matter). However we need to be fair and recognize the fact that ZFS is open source and CDDL is an open source lisence.
https://en.wikipedia.org/wiki/Common_Development_and_Distribution_License
I also think it’s fair to say torvalds speaking ill of ZFS’s technical merits in actuality stems from his prejudice against its ownership & incompatible license rather than because ZFS doesn’t have merit for linux. It clearly does and I think if he were being honest he would have to concede this. Ultimately I agree with his decision not to merge it, but it would have been nice if it could be merged.
Well I thought my post was fair, people really were loosing data due to btrfs bugs and my understanding is that some are even still there if you fail to avoid the raid configurations that trigger them.
https://www.mail-archive.com/[email protected]/msg55179.html
I really wanted to hear from everyone’s experiences and was hoping for more than “Bullllshit.” responses…or is that too much to ask? If you’ve never had a problem, great, that’s the sort of thing I wanted to hear.
I could be wrong, but I read ZFS being a Unicorn as being one of the rare pieces of desirable software (good file system + open source) that would benefit from a stable internal kernel ABI, while not bringing the bad side of a stable internal ABI. As in, too bad for ZFS, but were better off with an unstable ABI because it keeps excessive blobs at bay.
r_a_trip,
You could be right. Without clarification I can’t tell if sj87 meant it that way or not, haha.
> What’s everyone’s experience with ZFS & btrfs? I mostly use block level snapshotting, but LVM2 has it’s own caveats and it would be useful to do at the file-system layer. I’m in the “it’s already automated” and “why change what works” boat, but I might give these a shot to see whether they can improve my workflows.
I can’t really comment on ZFS, but BTRFS is fine provided you’re careful and don’t try to use the erasure coding modes (raid5 and raid6). Yes there are issues, but that’s always the case with relatively new software, and a decent percentage of the issues people post about are cases of either stupid distro defaults or PEBCAK.
ahferroin7,
I do use raid 5 & 6 (and 1 and 10 for that matter) at the block layer. File system raid is clearly more efficient at syncing, but beyond this does anyone have an opinion about using BTRFS on top of block raid?
Nearly all the systems I provision use LVM2, it’s just become indispensable. ZFS would make this completely redundant. but I’m not sure that I can get away without LVM2 using BTRFS. In particular I mount virtual machines using LVM thin volumes and make use of snapshots. ZFS has support for logical volumes, but with BTRFS I think it would be necessary to store virtual filesystems as ordinary files, is this right? I’d rather not have to do that. I don’t know what the performance impact would be and I don’t even know if functionality like “trim” would carry over up the chain the way it works on block devices.
I don’t know if I’ll get around to trying these things, but if you’ve done any of this I’d like to hear more 🙂
“File system raid is clearly more efficient at syncing, but beyond this does anyone have an opinion about using BTRFS on top of block raid?”
Don’t bother using block RAID. Why would you? One of btrfs’s primary features is being able to repair corrupted files. That relies on being in control of all the drives.
On top of MD or LVM all that btrfs can do is report file corruption. Or if it happens to the metadata at that point to recover you have to restore the entire filesystem.
ZFS uses its own RAID system as well and for the same reasons.
Zan Lynx,
I understand all of that and agree with it.
That would be a fair question, but it doesn’t look like you read the needs & concerns that I already brought in the post you just responded to.
1) Raid5/6 is known to add corruption risks in btrfs, and I’ve read that some synchronization barriers are missing and in any case it’s marked as not production ready on the official status page. So that’s a very good reason to avoid btrfs at least in these cases.
2) I run a lot of virtual machines, which mount the file system in the VM. The virtual machines currently run off of LVM volumes, which run on top of raid (either hardware or software). There’s very little running on the host side of things file system wise since it’s dedicated to running virtual machines.
Running btrfs inside the guests might be useful at times for snapshotting, but obviously the guest does not do it’s own raid unless a convoluted setup were used on the host to pass through many volumes. Suffice it to say, I find it most appropriate to keep raid on the host and not have each guest doing it.
Now the question is running btrfs on the host, my understanding is that ZFS supports logical volume creation for guests, but btrfs does not. First of all, is this correct? This goes against my preference, but it is possible to use files instead of volumes for storing disk images. So what is the performance impact of having guest disk images on top of a (btrfs) file system mounted on the host? I imagine that btrfs add more memory and CPU overhead on the host to handle virtual disk requests compared to LVM’s granular block mapping. The performance impact could be severe especially if every block is being checksumed, but I haven’t actually tried any of this.
I’m genuinely curious how others approach these things in their configurations.
BTRFS is over a decade old. People shouldn’t be making statements like “provided you’re careful” and “don’t try to use X” at this point.
It looked like a promising next gen FS, but it’s over at this point. It’s time to put the resources into something that’s going to work.
This is the number one problem with linux. When file systems start getting mature, they’re ditched for the next shiny. Meanwhile, ZFS continues to mature and get better (at least OpenZFS).
It’s not like the dark times when the power would go out and you’d lose your superblock (redhat 5 era with ext2) but it’s still pretty bad.
“The main alternative to ZFS feature-wise is btrfs, but it doesn’t have a history of being robust, although I don’t know if that’s a fair assessment today.”
I’ve used btrfs on my NAS since 2012 and on a series of three Linux laptops. RAID 1 and 10 and single on the laptops.
Yes there have been issues but I never lost data. In the same time period I’ve had to run full fsck on ext4 backup drives, and had lost+found unlinked files. I have no idea what was lost since they were all encrypted deja-dup file chunks. Luckily I didn’t need whatever it was.
So in my experience btrfs has been at least as good as ext4.
I’m using ZFS in anger (around 100Tbytes of storage across two workstations and a log server (100Gbytes of logs a day – on the fly compression is your friend)). No issues with ZFS so far.
My experience with btrfs is limited to a previous log server (so btrfs was old), but at that time it was a bit flaky – last thing you want in a filesystem.
Averse to stable kernel ABIs. The Linux kernel is a shining example of not breaking userspace compatibility without a very good reason.
The whole point of the conversation being quoted is Linus explaining why he won’t retire
sched_yield()just because he thinks nobody should ever use it. (I’ve been following that discussion since near the beginning, when the only topic was why spinlocks shouldn’t be used in userspace.)ssokolow,
Yes, I should have said that to be clearer 🙂
I poke around his posts now and then when they come up in relation to something I’m looking up, but normally I don’t really follow linus much. Anyways, reading more of the thread I see what you are referring to.
https://www.realworldtech.com/forum/?threadid=189711&curpostid=189723
It’s an interesting debate, and although he sits at a position of authority, he’s not always right, haha. Last year I was working on a realtime application and linux’s kernel locks just wouldn’t cut it. Maybe a real time kernel would help get timing right on the dot, but the jitter of the linux scheduler made it non-viable for my realtime control tasks. Even setting realtime priority was inadequate. Now some people would say linux is the wrong platform for realtime control, and that’s a fair point, but linux running on a raspberry pi is cheap and brings a lot of sophisticated functionality to the table that is extremely difficult/impossible to duplicate on microcontrollers. Using userspace spinlocks, one can get perfect timing while still getting the benefits of linux in general.
Obviously spinlocking is reminiscent of old DOS programs. They should be avoided if possible because they don’t permit a multitasking OS to timeshare the CPU as effectively with other processes. However for linus to say spinlocks should never be used clearly tells me he’s exaggerating his point, which he does often, or he lacks experience in areas where spinlocking is actually suitable. In dedicated applications where low jitter/latency is much less important than timesharing, spinlocks are both effective and I dare say “easy” solutions to overcome the limitations of the linux scheduler.
https://www.realworldtech.com/forum/?threadid=189711&curpostid=189747
I like the way malte puts it. Our preconceptions about whether things should be this way or that way aren’t as important as what the data shows. In theory, linux could one day offer real time constructs to make userspace spinlocking is unnecessary. However my immediate impression is that the amount of work needed to transform it into a RTOS would be exceedingly high because the linux kernel is not built on a good foundation for realtime processing.
https://www.realworldtech.com/forum/?threadid=189711&curpostid=189752
I agree with him here regarding sched_yield. At best the programmers using it are relying on it’s side effects rather than well defined semantics.
I disagree with him that locking has to be hard for application developers, we’re really dancing around the problem that linux lacks the realtime primitives that are needed…and rebuilding linux to be a RTOS is probably not something that linus would be up for. Maybe linux could offer a hybrid mode where only some cores are running in realtime mode while the main kernel is blocked from running on those cores – this is sort of similar to what happens when we use spinlocks with realtime processes. So even if linux never becomes a fully RTOS, dedicating some cores to realtime applications could satisfy the needs of programmers like me who require it.
I missed the edit window…
I wonder how a conversation between Linus and I would go, we’re both stubborn people, haha.
Off-topic, but can you please explain how you get those nice and shiny quote paragraphs into your comments? Since there is no WYSIWYG button bar here, I’m really wondering…
Gargyle,
Osnews should really post this on a help page somewhere (I checked the faq and it’s not there).
Anyways the old osnews was replaced with wordpress and the comments can be formatted as follows…
<blockquote>quoted text <b>bold</b> <i>italics</i></blockquote>
“I like the way malte puts it. Our preconceptions about whether things should be this way or that way aren’t as important as what the data shows.”
Linus is saying: in the tests people do they look pretty good, but when the code gets in the real world and are used by people running all kinds of hardware and software configurations, etc. the tests really didn’t mean much.
So the data shows: something, but that something isn’t as useful as the people who produce the data think it is.
Lennie,
Having experience with this myself, I know for a fact that spinlocks can help and it’s not just an imaginary benefit that goes away in real applications. It literally makes low jitter applications possible that the linux kernel does not otherwise handle well. As a general rule, I wouldn’t want to encourage too much spinlocking/etc in a multitasking OS when it’s not necessary or beneficial, however in specialized cases where minimal latency is crucial it absolutely brings real-world benefits to the table and it’s not as difficult as linus claims either.
The big drawcard for ZFS was meant to be RAID-Z. The big issue I found however was that you couldn’t simply couldn’t expand the raid’s. If you had a NAS with 4HDD’s in RAID 5 for instance, you couldn’t turn it into a 5HDD RAID5. I guess it makes sense for datacentres and servers, but impractical for home NAS’s and such.
The copy on write functionality is also available now on other filesystems too, so its not as unique as it used to be. It’s a pity, because if you could expand the RAID’s within it (and Oracle didn’t buy Sun). , I think it would have easily have dominated the competition.
RAID-Z expansion works now, and it’s being adopted left and right. I believe freenas now offers it by default.
Also, one killer ZFS feature for virtualization … zvols.
Personally, i don’t mind btrfs at all. I’ve had my share of troubles with it in the past, but it has gotten much better,
However … there is a lot of functionality that ZFS has that just isn’t present in btrfs. The main ones for me are … raid is broken (that is kind of a big deal!) and no raw device (zvol-like) functionality.
Raid-Z expansion does not work. You can add new vdevs but not expand existing raidz vdevs perhaps that is what you are thinking of. Nobody has commited any work on raidz expansion in months leaving it roughly in alpha state.
I use ZFS only on servers for backup storage, etc.
RAID-Z expansion not great, but is OK, choose your vdevs wisely. 😉
As I understand it someone is working on improving it in part.
It will not be great either, but improved.
Linus is unfortunately dead wrong in his comments about ZFS being technically inferior and unmaintained, but he’s also unfortunately spot on with his warning about the legal dangers.
As a Linux user, I *want* ZFS. I’m sure that BtrFS works fine for tons of users, but lamentably it has also burned tons of users. Brand reputation is incredibly important for a file system, and it only gets one chance to establish mindshare. In that sense, BtrFS blew it, whether or not it has improved or can be improved technically.
@rahim123
I believe Linus is correct here as I believe he’s commenting about the original ZFS code from SUN. I disagree if he counted the OpenZFS project, ZOL, etc. ALl of this, in my opinion, is nothing more than fluff to get the kiddies scrambling, just like the US political news.
Buzzword? He feels?? Sorry, lowering your credibility here. The licensing issues with regards to GPL I understand, and the risk of litigation (although not too worried about that myself). I am going to use ZFS extensively, especially now that native encryption is here.
Native encryption is indeed a key feature. I’m also going to use it, but I still consider OmniOS with native ZFS. In my test, OmniOSce in a KVM virtual machine uses around 5% CPU while idle and I don’t like that; although I like the integrated SMB and NFS functionality and the stability. I’ll probably assign a separate disk controller to the virtual machine (pci passthrough) so it has direct access to the disks.
evert,
If you ever do this, I’d love to see your before & after benchmarks!
Good idea; I can probably do a simple benchmark this weekeind. If I forget to post here, email me at [email protected]
however it is a simple old hobby server for personal fun, with a realy cheap pcie sata controller (the better maxtor based one suffered from an error under virtualization)
I believe Torvolds is correct, from a licensing point of view as all forms of ZFS is incompatible with GPL-type licensing and so shouldn’t be included in the Linux kernel. I’m viewing this from a liability standpoint only.
Yeah agreed and his comments about ZFS technical properties and maintenance should be ignored.
It’s not his field of expertise.
Well, looking at all the comments, let’s hope that bcachefs would solve the need of not having both ZFS and btrfs.
“So, in this instance, Linus is talking out of his ass, and really needs to shut up, lock himself in a room, and actually research what, exactly, ZFS, OpenZFS, and ZFS-on-Linux really are. He’s shot himself in the foot before, but this is beyond sad to see, and so very, very, very wrong that’s it’s embarrassing to the whole Linux dev community.”
>
>
Linus has every right to say linux should have nothing to do with ZFS and quite frankly do whatever to prevent it from becoming a part of it if that’s how he feels about it.
You on the other hand have every right to bugger off and create your own OS featuring ZFS….