“The past few Linux kernel releases have brought a number of new file-systems to the Linux world, such as with EXT4 having been stabilized in the Linux 2.6.28 kernel, Btrfs being merged into Linux 2.6.29, and most recently the NILFS2 file-system premiering with the Linux 2.6.30 kernel. Other file-systems have been introduced too during the past few Linux kernel release cycles, but these three have been the most talked about and are often looked at as being the next-generation Linux file-systems. Being the benchmarking junkies that we are, we have set out to compare the file-system performance of EXT4, Btrfs, and NILFS2 under Ubuntu using the Linux 2.6.30 kernel. We also looked at how these file-systems compared to EXT3 and XFS.”
NILFS2 looks particularly interesting; the only downside is you can’t boot off it yet. I have heard rumours, however, that this will be addressed in the future. I assume that it is licensed under GPL; its kind of disappointing that it didn’t adopt a file based licence as to allow porting to alternative platforms.
It is a Linux kernel filesystem that wants to be developed upstream by everyone in Linus’ Kernel.
If that is what you want then there is no other choice but GPL2(+).
(I am not saying it is good or bad. It is just the way it is and complaining about it is pointless.)
Not true, they could have licensed it as BSD or even double licensed it with a different GPL-(in)compatible free software license.
However, Linux devs(and the corporations controllig them) like GPLv2 and will try to contribute modifications that make the Linux version of the code GPL-bound. There’s no point trying to contribute anything in other licenses if you expect any kind of backflow to your original multi-license project.
Not if they don’t want major trouble with random contributors agreeing to the second license etc.
Show me code in Linus’ tree that does what you say can be done (It has to be developed upstream in his tree.)
I am waiting.
…dictates that one ought to stick with true and tried file systems.
Ext3 is often regarded as inferior, but (depending on interpretion) this benchmark seems to indicate that it’s the overall better choice – of the file systems included.
Perhaps it’d be more correct to use the term “file system implementations” rather than “file systems”.
Mhm… beer
It seems like BTRFS is the answer to everything. Will it solve issues like having a ton of small files taking up too much disk space? Will it have compression?
I tar’ed up my root directory once for a backup and I wanna say it went from 30Gb to 14Gb, then I gzipped it and it went down to 9Gb.
From what I’ve heard, built in compression also makes your system perform better, unless you’re always doing stuff that is processor bound.
If you have lots of small files I recommend Reiser(3)FS. ReiserFs is the fastest and tightest for small files.
The downside of ReiserFS is high CPU usage. On single core systems this lets your applications start slower, because simultaneously to reading files the computer also has to execute the startup procedure of a the application.
My experience indicates, that XFS is best for large files like they occur with number crunching applications (FEM, CFD, MBS, …). XFS is also quite easy on processing power.
My system layout is: XFS for the scratch partition where I do my FEM calculations. EXT3 for everything else. My opinion towards EXT3 is, it is very good overall speed wise, and truly rock solid.
Some relevant history, as I recall, was that a change was made to ext3 a while back which exhibited the regression of destroying sqlite performance on ext3, due to some interaction with the cfq io scheduler. That change was backed out. Then when they went to do all that optimization of ext3 during the 2.6.30 release cycle, changing the default to data=writeback, btw, the change got put back in. Jen’s Axboe complained. And Linus told him that if he couldn’t fix his scheduler then the default io scheduler would be changed to something else. Jens fixed the problem, and everyone was happy.
I’m not sure that any of that is involved in the ext4 result we see here. But I figured I’d mention, in case anyone else is interested.
Edit: Duh! I completely misinterpreted the graph. I should be asking what’s the deal with ext4, XFS, and BTRFS.
Edited 2009-06-29 16:41 UTC
Yeah, that’s why I wrote ext3 looked like the overall wiser choice. OTOH there’s a difference between a file system and an implementation of said file system (e.g. fs-driver).
That said, the better file system is the one which is fastest and most stable in exactly the areas you need it to be.
I’m a big ext3 fan, and have been for a long time. My new servers are running ext4. But ext3 has probably been the most under-rated filesystem in recent history… except maybe for JFS, which never even seems to get noticed.
Ext3 always does well in benchmarks… and yet has a reputation for being slow. Ext2 has never suffered from that undeserved stigma.
Ext3 has been probably the most rock solid filesystem ever developed and put into general use.
I think Matthew Garrett hit the nail on the head when, during the Great Ext4 Unreliability Debate, he said to Ted T’So:
“The majority of [application developers] I know
felt that ext3 embodied the pony that they’d
always dreamed of as a five year old. Stephen
gave them that pony almost a decade ago and
now you’re trying to take it to the glue factory.â€
I should say that as it worked out, I’m now an ext4 fan. But all that bluster I heard from Ted about the importance of filesystem performance as balanced against reliability sure hasn’t seemed to have done Sqlite performance any good in ext4.
Though politically, he did manage to reduce ext3 reliability somewhat, thus making ext4 look better by comparison.
Edited 2009-06-29 20:07 UTC
Ext3 is the only choice.
Things going for ext3:
-Reliable
-Fast
-Backwards compatible(Readable from Systems that only support ext2)
Things going for new gen FSs:
-Hype
-Data loss
-Appalling performance
Well, I certainly don’t mind seeing ext3 get some praise. It helps make up for all those years it got unfairly trashed by people.
However, a couple of comments are in order. Whatever is going on that is making ext3 so fast in the sqlite benchmark is likely going to make it into ext4. The major problems with ext4, in the event of an unclean shutdown have now been addressed. Although it is still not as resilient as the pre-2.6.30 ext3. Which brings me to the third thing. Ext3 was substantially overhauled during the 2.6.30 development cycle, improving its speed, reducing latencies, and reducing its resilience after a crash to the same level as current ext4. To extend Matthew Garrett’s analogy, Ted succeeded not only in taking the pony to the glue factory… but it’s mother, as well. Of course, I should concede that the level of reliability of current ext3 and ext4 are pretty good. Just not as good as the old ext3.
All my points above assume use of the default mount options.
Edited 2009-06-30 12:39 UTC
SQLite creates a journal file per database transaction. A performance test that has thousands of transactions is going to be file system-dependent.
10,000 inserts where each insert is a transaction takes on the order of minutes on my NTFS/Windows machine. 10,000 inserts in a single transaction takes on the order of milliseconds.
Who cares about speed if the integrity of the data can not be guaranteed? btrfs & nilfs2 can checksum your data, ext4 can not. This makes ext4 a no go on any valuable data. Since all my data is valuable ext4 is not an option for me. (Yes, i’m a happy zfs user).
[edit]
oh, and for those who think they are save with their mirrored raid stuff : http://storagemojo.com/2007/09/19/cerns-data-corruption-research/
Edited 2009-06-29 21:11 UTC