Submitted by ‘Why Reiser4 is not merged’ is a widespread question around forums, Slashdot, OSNews, and wherever else Linux-related new appears on the web. The flame-wars on this topic have gotten to the ‘mine is bigger than yours’ level. Hence, it’s not easy to see where the real problem lies. This document tries to make clear the ‘official’ point of view of Linux developers on this matter.
some test on the web show than reiser4 is not the top for performance
Although it might be true about reiser4. The test I saw about reiser4 where it was slower was very bad. They were running a 500mhz machine with a 10000 rpm hdd (iirc). We all know MOST machines are IO bound and NOT cpu bound.
So reiser4 I guess uses more CPU attempting to increase IO peformance. Whether it does that is debatable!
Either way, it seems rather pointless without the extended metadata support… it was included in the design, and the docs (such as they are) talk about it, but there’s no code there because it’s not pure POSIX functionality, and the ReiserFS folks wouldn’t change their implementation to make it more compatible with normal filesystems.
– chrish
This is a nice position document but it would be better if it referenced the actual bugs.
Hi, I started that document – I wouldn’t know to tell you what the real bugs are, you need to have a deep knowledge about filesystems, VFS, etc to be able to talk at that level (personally I don’t know a lot about that). The reiser4 people (ie: who really cares about them) is already aware of them, you can search the lkml archives if you’re really curious though.
Edited 2006-07-19 19:18
“Since the initial request to get Reiser4 merged, it hasn’t been declared ready by people who know.”
“It must meet with the design tenets and quality specifications laid out by the existing maintainers.”
“Reiser4 is growing near to inclusion in the main tree. It could possibly be ready as soon as 2.6.19.”
“So that’s it: Things need to have a decent shape before being included, and Reiser4 is not there yet.”
Don’t get me wrong, I am sure Reiser4 is great… But it seems that there are some political friction between the 2 teams that sound like young childrens arguing about a slice of pizza!
My 2 Cents.
Didn’t you read the article? Both parties are a pissed at the way they’ve been treated by the other, but the underlying rationale for the decision not to merge has always been predominantly technical.
You might be sure Reiser4 is great, but many people aren’t. I have personally used Reiser4 for everything including the root partition on at least a half dozen machines for periods of more than 18 months. My conclusion? Reiser4 is noticeably faster at certain things and disastrously slow at others, and it doesn’t even come close to being as stable or coherent as any other journalling filesystem I’ve used on Linux (ReiserFS, XFS, and ext3). I’ve lost data on Reiser4 filesystems on at least a half dozen occasions. I don’t run huge file servers or anything, these are normal desktops and laptops that I use for personal and development applications.
Almost everyone here will agree that just because some software “works for me” doesn’t mean that it will work for most people. But the converse is much more valid: if Reiser4 doesn’t satisfy my basic filesystem litmus test (consistent performance and rock-solid stability/coherency), then it’s likely, but not certainly, to fall short of many other users’ expectations.
I’ve since migrated to ext3 with the ‘dir_index’ option on most of my machines, although I think XFS and ReiserFS are both excellent filesystems (all three pass my litmus test). When I see mainstream distributions shipping with Reiser4 by default (which probably won’t happen until after it’s merged into the mainline kernel), or when there’s a promising relational database plugin for Reiser4, I’ll give it another shot.
I read the article before… It’s an old debate… The technical reason for not merging is entirely valid.
But what the article made me aware is the tension between Reiser Dev Team and the Kernel Maintainers Team… I can’t help but think it’s a bit sade…
2 Team of grown-up of men/women having those kinds of problems… Now we know why we can’t have world-peace right?
As a general observation, it seem that those kind of ‘tension’ are more frequent by the day! Hopefully it will not become a trend…
I can’t help but feel a bit sad about it… But it’s only software… Imagine how I feel about real important stuff!
My 2 cents
Using reiserfs right now. Happy with it. Used reiserfs4 some time ago. Was not so happy. I think it is worth to wait than to get a bugbag in your pc.
they get all high and mighty over the slightest thing. I recall reading a thread of a promising young developer who was fixing bugs and trying to get a foot in the door, eventually the snotty mails from the kernel dev’s drove him away.
this guy tries to get too big for his boots by answering:
Q4. “I’ve been using it for years and it rocks, how can’t it be ready?”
with:
You are not a kernel programmer, aren’t you?
Now, I’m no expert, but his english is shite. Would he enjoy it if people sent him nasty mails on a mailing list intended for people to improve their english?
People will tell you that Reiser4 is not in there for technical reasons, but I think that’s mostly BS. The Kernel developers hate Hans Reiser and they make that hate no secret. Sure there are some technical aspects that are making it easier for them to say go fix this and we’ll think about including it, but it’s no coincidence that every time he returns requesting inclusion, a bitch fest that rivals satans depatcher from heaven insues. Yes, Hans is partly responcible for that, but the kernel has been significantly more liberal with other code getting inclusion in the past compared to Reiser4 and anyone who’s followed the kernel mailing list will admit this to you. Hell, the development kernels were created for software projects just like Reiser4. Andrew Mortan has spoken very briefly on the subject and Linus won’t even touch this subject for the most part, because while they will back their ardent testers for the most part, they see the bitchfest and know that this debate has gotten out of hand for the most part.Hans openly chalenged the current VFS implementation and that didn’t make him any friends. I think he was on to somthing big at the time, but they quickly crushed his implementation and proclamed technical superiority. Needles to say, he hasn’t taken that lying down and the battle forges on. Don’t be fooled into thinking that this battle is any less fearce today as when it began. Resier4 is a telling example of how even though all of the code is open sourse, the gate keepers are very particular about how open they are to other smart people challenging their ideas.
Reiser4 is not slow. It is slow when used with databases, becuase it has not been optimized for database use, but if you were to install it on your garden flavor distro, you’ll discover that it is more than a match for XFS, Resiser 3, and any of the ext’s.
“People will tell you that Reiser4 is not in there for technical reasons, but I think that’s mostly BS. The Kernel developers hate Hans Reiser and they make that hate no secret. Sure there are some technical aspects that are making it easier for them to say go fix this and we’ll think about including it, but it’s no coincidence that every time he returns requesting inclusion, a bitch fest that rivals satans depatcher from heaven insues.”
This would be what FAQ #3 is talking about, I suppose.
“Yes, Hans is partly responcible for that, but the kernel has been significantly more liberal with other code getting inclusion in the past compared to Reiser4 and anyone who’s followed the kernel mailing list will admit this to you.”
Is this in comparison to other filesystems? ‘Cause it makes sense to hold filesystems in general to a higher standard.
I’ve been using ReiserFS v3 for over a year and I haven’t had the slightest problem with it. But then I havent’ noticed the slightest benefit either.
Unless you have a side-by-side comparison, it’s hard to notice when one file system gives you a marginal improvement over another. Also ReiserFS has one tiny negative–you can’t use ‘dump’ or ‘restore’.
Therefore, I intend to use ext3 on all future installs just because I want the drop-dead most stable choice and one that is most likely to flex into whatever odd situation I might be in, like using some unknown recovery disk.
it’s reiserfs v4 that is being discussed here, or is that what you meant? I used to use XFS but now I just stick with ext3 and turn on dir_index.
No, I meant that not only am I NOT transitioning to v4, I’m backing away from v3 and sticking with ext3–just like you.
First the author decries the amount of politics and emotion pervading this issue. He then uses the terms “shocking” and “disturbing” (like Sprockets?) to describe his feelings about the discourse. For an outsider hearing about this controversy for the first time, this really does seem very funny. I think the kids need to be sent to their Time Out Corner.
Latest comments from Hans (slashdot) seem to indicate high CPU usage was a known problem and has been improved. How much is a matter for future benchmarking. Also, the compress by default behavior isn’t on (stable) yet so that would be something important to bench. There are still several important tweaks going into the reiser patches and I think we should wait to see what those do.
I know this is offtopic, but is XFS part of the kernel now, or just as an addon? It seems to be mature and pretty fast for me.
XFS is definitely supported by the kernel and has been for some time. As best as I understand it, having support for the file system is not the same thing as creating the file system. That has to be done using different tools. It is probably more complicated than that, though.
GP
It’s funny to here one side claim that isn’t politics, because that’s obviously a red herring that it _is_! Politics always brings out the bratty kid in all of us, at least at the House of Commons they still do it with a certain measure of eloquence.
I am fully in favor of Reiser4 getting a very thorough review prior to consideration for inclusion in the mainline kernel.
The thread linked below destroyed what confidence I might otherwise have had in taking Namesys’ statements at face value. The issue of deception involving their published benchmarks described in the thread has still not been addressed on the Namesys site… two years later.
http://lkml.org/lkml/2004/8/27/70
Edited 2006-07-19 22:38
Yeah, the quite well-known benchmark fiasco is just one of the missteps Namesys made which all but sabotaged their efforts to get Reiser4 merged. After everything that’s gone down, it’s a miracle of extraordinary patience that the Linux kernel developers are still responding to Hans’ emails. Certainly the Linux kernel developers communicate with independent software vendors quite a bit differently than your average CTO would. But they’re also offering a service that your average software vendor would even consider offering, even on their own terms.
Think about it this way: Namesys needs Linux. Their whole business model relies on selling commercial plugins and support for a standard Linux filesystem. If they brought their filesystem to Microsoft or Apple and said, “we got this working on your platform in our lab; you guys should merge this into your kernel and help us fix bugs so that we can start selling products and services based around it”… they’d get laughed out of the building.
Getting your code accepted into the mainline Linux kernel is no trivial agreement. Essentially, the kernel devs are saying, “this is great code, it fits nicely in the context of the rest of the kernel, and if you stop maintaining it, we’ll do our best to continue to support it.” Namesys has invested a lot of money in the development of Reiser4 to find that they really should have starting working with the kernel development community long before the first usable releases were submitted. Once rejected, Hans couldn’t believe it, and Namesys started question Hans’ original pitch that Reiser4 was almost ready for inclusion in the Linux kernel. But instead of realizing that getting Reiser4 in the kernel was critical to their business, and that they should do everything they can to address the concerns of the kernel developers, they decided to drag their feet and continue to argue politics and semantics on the mailing lists.
If you’re wondering who’s responsible for successfully attempting to turn this issue into a politcal flamefest, look no further than Hans Reiser himself. The kernel developers responded in kind, of course, but only after Hans took the rejection personally, as if the kernel developers were calling him incompetent, which they had no intention of doing. Instead of working with Linux VFS like every other Linux filesystem, Hans told the Linux developers that VFS is a piece of crap. Does this sound like the appropriate response of a guy who’s continued employment at Namesys depends on his ability to get Reiser4 merged into the Linux kernel?
In the proprietary software world, as well as in some commercial OSS projects, this same stuff goes on behind closed doors. When you approach another company and ask for special access to their resources, they always say “my way or the highway,” if they accept the possibility of such an arrangement at all. Linux doesn’t need Reiser4. Reiser4 needs Linux. If Hans and Co. think that the concerns of the kernel development community are unreasonable, than let them find another OS on which to offer their filesystem, and I wish them good luck at that.
Not that Namesys has the resources to do this, but what stops them from making a Reiser fork of linux? Or distributing a their own patched kernel?
Uless it’s gets accepted by default by RHEL/SLED combo, it’s really of no use.
“Hans told the Linux developers that VFS is a piece of crap. Does this sound like the appropriate response of a guy who’s continued employment at Namesys depends on his ability to get Reiser4 merged into the Linux kernel?”
It’s not really that simple. You are correct that on the one hand his income depends on working with the krenel developers on this. But on the other hand Hans has a few ideas on his own on why filesystem semantics of today sucks and how to fix it.[1] So when he says that the current VFS is a pice of crap, he actually has a point, allthough he isn’t the best person in the world to communicate what he means.
Now, Linus has argued that the VFS isn’t a pice of crap becaus it’ll actually support Hans ideas in it’s current form. But other thant that I haven’t seen any arguments against Hans stance that’s on the same level.
[1]http://www.namesys.com/whitepaper.html
Here’s one of Reiser’s “ideas” from the paper:
At these meetings you start to understand that most people who go into filesystem design are persons who didn’t have the guts to pursue a more interesting field in CS.
and yes, that’s typical of the level of discourse you get from Hans.
He has, in fact, made no compelling argument against VFS, and there is no need for anyone other than Linus to have explained that, since Linus’ explanation is sufficient to refute Hans.
Reiser is making the same class of mistake that the CMU researchers made when they designed the original Andrew file system: He is concentrating on one problem and trying to optimize for it. It’s not the only problem that file systems are required to solve. It’s not even the most common problem. And he doesn’t really understand it all that well.
You can see this from the ‘white paper’, which contains no clear exposition of principles, only muddled examples and ad hoc solutions.
One issue that wasn’t addressed anywhere is whether VFS is really inferior to Namesys architecture.
I guess this is the real question to be answered, unless the meritocracy is no longer in charge on kernel developement.
Not really.
It’s a cost-benefit analysis. Every change has a cost and that must be weighed against it’s benefit. Look at it in your own life. Suppose you discover that if you change careers you could get double your current salary, the catch is that you have to go back to university for 4 years and then do 4 more years in grad school. Do you make the switch, even though you have a family to support and will have to use up all your savings and retirement money, and put your life on hold? Some people would, but most wouldn’t. The cost is just too great.
Getting back to Reiser’s new architecture. I don’t know the current state of the debate, but I did look at it when it was first proposed some years back. It was drastically different than POSIX and would break a number of apps that were not converted. It also duplicated a lot of functionality that existed in other parts of the kernel, so it would add a boat load of bloat that kernel developers would have to maintain for years to come or would have required the old functionality to be ditched in favour of the new one. If that happened, all other file systems would need to be rewritten. The such a big change would have likely increased the instability of kernel releases. Reiser4 isn’t 2 times better than its nearest competitors, and it’s not some critical new feature that you just can’t do without (like HAL/DBUS integration). It’s just a file system and it’s simply not worth the changes originally proposed.
I imagine things have changed quite a bit since then, since it appears that consensus is a bit closer.
butters writes:
> Hans told the Linux developers that VFS is a piece of
> crap.
Hm. What I’ve read of Hans’s so far seems to be pretty level-headed. Can you please provide a link to where he says stuff like you say?
I remember a while back reading some stuff Hans had written about filesystems. I don’t recall the details, but I remember thinking, “wow, this is one smart guy — I’m keeping an eye on this filesystem of his.”.
> When I see mainstream distributions shipping with
> Reiser4 by default [snip]
Any non-mainstream distros shipping with reiser 4 yet?
“””Hm. What I’ve read of Hans’s so far seems to be pretty level-headed. Can you please provide a link to where he says stuff like you say?”””
Well, here’s a quote from lkml:
“””
snip
What makes you think kernel developers have a deep understanding of the
value of connectivity in the OS? They don’t. The average kernel developer
is not particularly bright. Just ask Ted why htrees are slower than
reiser4, or ext2 tail combining is slower, and, well, he has no clue. He is
happy to explain how architects don’t do real work and should not attend
the Linux Kernel Summit, and then when reiser4 blows htrees away he
undoubtedly still thinks I just take the credit away from the programmers
who do the real work.
snip
“””
And this is *after* the benchmarks had been publicly exposed as having been intentionally manipulated, at Hans’ direction, to remove all phases in which ext3 beat reiser4 from the published results. The guy has gall, I’ll give him that much.
Edited 2006-07-20 04:41
Ask any room full of Linux users you will get at least 2-3% of them that have a story about how a version of reiserfs has caused them to loose data. The old adage goes.
“screw me once, shame on you. Screw me twice shame on me”
the Reiserfs team has a habbit of releasing crappy, insuffiently tested code, I have seen comments in their code saying “fix this, tihs is badly broken”. Sure it may be better now, but hell if I’m going to trust my data to it.
Some say Reiserfs can’t get better untill its tested better, sorry but this is all wrong. You Test your filesystem constantly, you construct benchmarks, corner and worst case access patterns and test the hell out of the filesystem. You loose one byte of data. All work stops, untill the bug is found and fixed. Not untill it passes weeks of testing can it be considered stable enough to be in the kernel.
Filesystems are just not another part of the system where its okay if they crash or do the wrong thin once ever million accesses. Filesystems are the #1 part of the filesystem. Perhaps 5 years ago you could produce unsafe code, but this is no longer accecptable Linux has grown up and people and companies trust there data to be safe. Your filesystem fails and you loose data, companies fortunes and employess will loose jobs. Sorry Reiserfs has a lot of damage to repair to its reputation. And quite possibly its code base.
Yes, that’s why reiserfs 3 is in the kernel…because they hate Hans Reiser. It’s so obvious!
“Hans is partly responsible for that, but the kernel has been significantly more liberal with other code getting inclusion in the past compared to Reiser4 and anyone who’s followed the kernel mailing list will admit this to you.”
The past is not now. the kernel devs being more stringent on what code to include is a good thing.
“Needles to say, he hasn’t taken that lying down and the battle forges on.”
Yes, clearly this is ALL the kernel devs fault.
“Resier4 is a telling example of how even though all of the code is open sourse”
If it sucks it doesn’t matter if it’s OSS or not.
The past is not now. the kernel devs being more stringent on what code to include is a good thing.
But at the same time, look how long it took to finally get ReiserFS v3 merged in; and guess what? the only thing the developers were worried about was so-called ‘file corruption’ and when it was hunted down, the only people who were affected were those running buggy chipsets, for everyone else, they received a boost in performance without any stability or dataloss issues.
ReiserFS v4 is a big step forward, but if there are issues, and the group told to go back to the lab and test more; they need to write out a white paper rationalising the decision they’ve made, the reasons, and the parts of the said component that needs correcting. No use giving open ended statements like, “needs more work with the VFS team’ without actually laying out what needs to be done.
“Not that Namesys has the resources to do this, but what stops them from making a Reiser fork of linux? Or distributing a their own patched kernel?”
nothing, but who would use it? they need/want to be mainstream.
on a different note: I would be delighted if Hans and the devs sorted out this mess and got R4 into the 2.6.19 kernel.
Sometime ago Reiser say about VFS that it is not really “generic code” but just part of ext2/3 FS. Redhat force to use ext2/3 and sure thay have reason and power. Kernel developers require Reiser to use VFS but they know it is nearly impossible w/o VFS changes or bad reiser4 performance. And we all loose. May be Hans must stop this one-way talk and try to make reiser4 patches for vanilla stable tree ASAP?
Well, I used for some time on a Kanotix laptop a year or so ago and had problems. My system froze for some unrelated X11 reason at one time and I ran out of battery on another :-p, and I had to restart the laptop. At both occations I think I was writing heavily to the disk and I ended up with a hosed non-recoverable file system and lost data. I’m sure it was fast and all (didn’t notice much difference) but it was certainly not very crash tolerant back then.
A file system that makes recovering lost files easy would be nice… What would you guys recommend for this? Ext2/3?
Edited 2006-07-20 09:45
Probably JFS or XFS. Both are well tested, well thought out journalling filesystems. I’ve currently got a 750 GB XFS filesystem serving up NFS in the corner that’s quite happy.
That said, JFS is an IBM thing, and XFS is an SGI thing. IBM isn’t in danger of dissapearing any time soon (unlike SGI), and Grub works with JFS, so I’d pick JFS.
XFS is still the most fragile of the journaling filesystems, due to its policy of nulling out the data before doing the write. I know that the FAQ says it is fixed, but the reality is that it is just not as bad since v1.1. It’s probably the worst fs you could choose for a laptop.
I would go with ext3. In its default data=ordered mode it gives greater integrity guarantees than JFS.
Reiserfs v3 also has an ordered mode, but in my opinion ext3 is:
1. The most stable of the Linux fs’s.
2. A solid performer, despite being unfairly maligned by some.
Edited 2006-07-20 21:49