Filesystems have really intrigued me; journaling filesystems in particular. Some time ago the industry criticized Linux because it lacked one, the community responded with an overkill of 4 filesystems: Ext3, ReiserFS, XFS and JFS. When searching the Internet for information about these filesystems, you will find websites explaining how wonderful each of these is. There are also many Bonnie++ benchmarks to be found. Yet these are all theoretical, and their aim is to judge overall performance. Concrete benchmarks are yet to be found…
Many computer enthusiasts will want to run Linux for an FTP server on an old abandoned Classic Pentium system. As did I. Therefore I installed Mandrake GNU/Linux 9.0 on a system with the following specifications:
- Tulip VisionLine de5/200
- Intel Pentium 200 (non-MMX)
- 128MB EDO-RAM
- Davicom 9102A based 100mbit NIC
- 3.2GB Western Digital 5400rpm HD
- 8.4GB Seagate 5400rpm HD
I chose to install Mandrake simply because it supports an extensive range of filesystems in its default kernel. This way I wouldn’t have to bother recompiling my kernel. I installed ProFTPD to function as server, as it should be pretty fast and easy to configure, while remaining reasonably secure. On the client side I took a Windows machine* with LeechFTP installed.
The Western Digital drive was used as root, while the Seagate drive was mounted on /var/ftp with a different filesystem each time. My test consisted of rebooting, uploading a file, rebooting once more and then downloading that very same file. To neutralize buffering I took a large file of about 300MB in size. The reason why, is because my server will only be transfering large files, i.e. game updates for LAN-parties.
Ext3
FTP Upload: average 2.5 MB/s (fluctuates between 2.1-3.1 MB/s)
FTP Download: 4.6 MB/s (practically no fluctuation)
ReiserFS
FTP Upload: average 2.7 MB/s (fluctuaties between 2.5-2.8 MB/s)
FTP Download: 4.7 MB/s (practically no fluctuation)
XFS
FTP Upload: average 3.0 MB/s (fluctuates between 2.5-3.5 MB/s)
FTP Download: 4.5 MB/s (practically no fluctuation)
As you might have noticed the differences for downloads are quite small. But ReiserFS would be best suited for my own ends, because my server only has anonymous access without an upload area. All uploads are done through a ftpadmin account which only I have access to, and I only use when I have enough time anyway…
* The Windows client machine could easily take a full 100mbit load if necessary, so the benchmarks were not dependent on the Windows machine.
—
Pascal de Bruijn lives in Sittard (The Netherlands) and has been toying around with Linux (various distros
Slack,MDK,RH,Debian,Gentoo), for about a half a year now, just as a hobby.
Recompiling a kernel is not that hard, I would have gone with a lighter distro on such a very-low-endish system
Interesting he decided to go with a distro kernel. His speed is very heavily dependent on what set of patches are used by the Mandrake folks.
I’d have to say that using the stock kernel and trying to get performance numbers off it is faulty. Most places interested in any kind of performance and stability (including where I work) will go download a standard kernel and apply the necessary patches themselves. I’ve personally been burned way too much in the past by vendors putting out unstable kernels (in my experience both RedHat and Mandrake are guilty of this).
I seriously doubt that we’ll be able to get any kind of conclusive results until well after the linux 2.6 kernel is released and all the filesystem code is well merged, tested and beat on.
why are there no ranges for the download speeds? it seems that the ranges would be similar to the upload ranges.
if this is the case, this comparison is inconclusive and useless because there is no statistically significant difference between the performance in the three cases.
There’s too many factors to consider these test solid. In addition it does not take into account robustness.
For instance, with XFS there has been numerous issues leading to widespread filesystem corruption problems, and is now discouraged by the Gentoo install documentation.
I’m going to stay with good old reliable ext3 until I see what Resierfs 4 is like.
It’s nice to see some benchmarks. However, I can’t see how using the stock kernel was a problem. The main point here is to see how the filesystems compare against each other, not to see what their maximum speeds are. Correct me if I’m wrong.
The most interesting thing for me is to see how ReiserFS is the most consistent in maintaining throughput. Based on this I’d choose ReiserFS, and SuSE already did, so I already have too..
Would the download variation be similar to the upload? Still I agree, it would have been nice to see variation in the download too. Otherwise I thank the author for his work, it’s an interesting read.
How does a upload/download test constitute the performance of the file system in question?
Since I have very experience with Linux, I can’t really comment too much. But I noticed that it takes a whole lot longer to transfer/move/copy a large number of files of small sizes that makes up 300MB than that of a single large file of 300MB
The little difference as shown in the test is expected. Moreover, IMHO the files system used on one’s machine has, technically, little or zero effect on the rate of uploading/downloading of a file since they all use a common ftp protocol. The speed is more likely affected by the hardware in use.
Just my 2 cents.
XFS is a killer file-system (no, it does not kill you files), if Gentoo writes this, they have missed something.
if this is the case, this comparison is inconclusive and useless because there is no statistically significant difference between the performance in the three cases.
The only reason I’d say you couldn’t be conclusive here is because the author mentioned nothing of his methods and the accuracy in his figures. Otherwise I’d disagree and say the figures are significant, I wouldn’t expect too much variation between well-coded filesystem average transfer speeds.
Certainly the deviation from the mean is the most significant thing to take away here.
IMHO the files system used on one’s machine has, technically, little or zero effect on the rate of uploading/downloading of a file since they all use a common ftp protocol. The speed is more likely affected by the hardware in use.
I think you missed the point. If you use the same hardware and software only changing the filesystem then you can perform a comparison on the filesystems. Of course the hardware will be the biggest factor, but the author was trying to see how the major filesystems compared.
notice that this dude is talking about ftp rates.. not the filesystem in specific but which filesystem performs best in ftp inviroment (or however you write that)
I like this kind of comparison : no esoteric optimized systems , just a setup like i could be using myself, doing what i could be doing. Nice one.
My intention was to show filesystem performance in one singular and specified case, being FTP. I didn’t mean to judge overall performance. I just wanted to show people the differences between the filesystems when running a FTP server.
And indeed the results were expected, FTP heavily relies on CPU processing power and harddrive speed, not the filesystem. But still there was a noticeable difference between the filesystems.
And wasn’t that what my article was about, running a ftp server on old hardware.
Also about other comment about the ranges, well there actually were almost no ranges with the download speed. The maximum variance I got was about 30Kb/s higher or lower.
Also this article wasn’t written for Mr. Kernel-Hacker “I’ll write my own filesystem from scratch”, but for an average Joe at home. Many starting linuxers don’t want to bother recompiling a kernel.
Goede Avond Pascal,
first off, thanks for the article. It is really very simple.
You don’t need a PHD to read it like you do you many other
benchmarks.
The only three things that I’ve been missing are:
1 — (Already been mentioned) Ranges for the download speeds
2 — A measure for many small files. Not everyone is coping
with big files only!
3 — A measure for *copying* files (large ones and small
ones), since for people who’re using Linux as their
main workstation (not everyone is just using it as a
file server), this is much more interesting
Keep up the good work!
Groetje van Duitsland,
– Raphael
I agree I like it too, a good one, plain and simple.
Well, the fluctuation with the uploads can easily be explained by Linux flushing it’s write buffer to make way for new data. Writing data takes times. So when the speed was at it’s lowest. Linux was probably writing the files to the drive.
Next the lesser upload performance of ext3 can be explaned because of ext3’s Data Journaling, it’s a feature that ReiserFS/XFS don’t have, which does impact write performance. Data Journaling delivers some increased robustness.
(ReiserFS and XFS only do Meta-Data Journaling
If there are problems with XFS it is most likely because of a couple bugs integrating it into the Linux kernel. It wasn’t integrated until like 2.5.54 or something. If you’re running XFS now you better have the latest development kernel or one supported by one of the distros, none of which boot off of XFS, AFAIK. (Any distro that ships with it now has probably patched a 2.4.x kernel prematurely)
XFS has been in use by SGI for years, and I can assure you that the filesystem itself is extremely stable. While working there I never once saw a crash that destroyed data, even when I accidently pulled the motherboard out of a running O2 and only blew the power supply. And I RMAd harddrives like they were going out of style.
This sounds like a perfectly relevant benchmark for your system if the planned use is to have a single user download large files.
If that’s not the planned use, then it’s essentially irrelevant. The benchmark says nothing on its to handle several simultaneous requests, on the FS ability to organize files and keep them grouped together and avoid fragmentation, or how well it works with several little files.
Finally, the difference in download speeds from the fastest to the slowest is only ~4%, meaning a one minute download will take only ~2.5 seconds more on the slowest file system. On a 1 gig download, this equals roughly 10 seconds (out of ~4 minutes).
So, it may be a worthy measurement, for trivia but for those not using the winning FS, I wouldn’t be losing any sleep or get in a rush to swap to it solely on this benchmark.
From what I’ve tasted of XFS, it is indeed rock solid and hyper fast. I mean that thing was flying compared to the other file systems. The only downside to it is if you unproperly shutdown your machine.
XFS seems to keep multiple revisions of the same file, when the machine improperly shuts down it uses a _VERY OLD_ revision of the file, screwing up your latest distro upgrades, configuration changes etc, almost bringing the system to an unusable state.
I am not whether or not methods/tools exists to flush the latest revisions. Other than this issue, XFS is a killer
And indeed the results were expected, FTP heavily relies on CPU processing power and harddrive speed, not the filesystem. But still there was a noticeable difference between the filesystems.
Wait… Noticeable or statistically significant? These are two very different things. A “noticeable difference” can be (and frequently is in experiments) just the luck of the draw, and not reflect any underlying real difference among the parameters.
Also, do you have numbers on server load during the transfers? Iostat output? CPU time used by operating system vs. FTP server process?
Yours truly,
Jeffrey Boulier
My current main Linux partition (Gentoo 1.4 rc1) uses XFS. And I have lost several files on it during power loss. Am sticking to Ext3 for now. Should look at ReiserFS soon.
Well, I have no exact reading, I had “top” running while transfering, and in all cases the CPU was maxed out. Each time the proftpd thread was running at about 95% of the CPU.
Greets,
Pascal de Bruijn
You know, just for kicks I searched google for “xfs problems linux -gentoo” and got nearly nothing relevant. I’m wondering if there isn’t some conflict with Gentoo’s other patches and XFS.
For my data point, I’ve never had trouble with XFS on Gentoo, nor have I have trouble with XFS on IRIX.
Gentoo doesn’t recommend against XFS because of corruption issues. It is because of conflicts with the other kernel patches (preemption in particular) that Gentoo includes in its stock kernel. Most of these issues are resolved in 2.5.x anyway.
Read the first sentence of my last post. And might I suggest that you try using a full release, non-beta, distribution next time (none of them should support XFS at this time since the 2.5 kernel is still beta).
Don’t forget that ReiserFS has had some problems with NFS exporting in the past. Supposedly this is fixed in v 4.0. I like its performance but haven’t had the time to verify if this critical bug has been fixed.
I have not has as good performance with ext3 but the fact that you can use it while in Windows is a plus.
The benefit of using ext3 is that while it can still compete on the same level (not always trumping the other FSes, but not too far off) it’s the most stable and mature filesystem FOR LINUX atm. XFS _is_ mature and stable, and performs quite well on Irix, but until the Linux kernel has a proper implementation it might be a little premature.
Similarly with ReiserFS, except that the whole filesystem is quite new, and I would personally wait until it’s a little further into mature development until I put it on my box.
Okay, and where’s JFS?
Well, I’m not sure, here, whether this test was about how well the hardware performed given the role of the server or how well each filesystem performed. I know that the article was *intended* to compare filesystems, but the idea of throwing FTP into the mix broadens the scope of this case such that the point of this article becomes ambiguous. Why even use FTP at all? Use `dd’. Why was JFS not tested? ReiserFS is a good filesystem for large numbers of small files, but poor for large files due to its layout strategy. If, as you say, you’re using this server for transferring large files, I would recommend not using ReiserFS.
One question: you wrote, “But ReiserFS would be best suited for my own ends, because my server only has anonymous access without an upload area.” What does access control have to do with the ReiserFS? ReiserFS does not store ACL information in its inodes.
ReiserFS doesn’t store any data regarding INodes. The default install of any distro under ext3 puts many tasks in cron which are not present in case of ReiserFS. Searches under ReiserFS are much faster compared to ext3. My vote goes to ReiserFS.
Just wondering why you chose to exclude BeOS from your comparison.
BeOS can run an ftp server just as fine as linux, as long as you have BONE installed.
The OS doesn’t necessarily matter if you are strictly testing filesystems, and using standard ftp protocals.
-Chris Simmons,
Avid BeOS User.
The BeOSJournal.
FtA:
“Some time ago the industry criticized Linux because it lacked one, the community responded with an overkill of 4 filesystems:…”
Notice his use of linux in that statement? BeOS is not linux, hence its BeFS not being listed..
When the new implimentation of BeFS reaches maturity, I’d like to see some benchmarks on it…but not in it’s alpha/beta state that it is now. Be’s original BeFS doesn’t matter anymore, it’s dead…when the one that people will use from now is mature, it will be an important benchmark.
I still wouldn’t run a FTP server on BeOS or any other networking daemon due to the fact that it’s still single user. It’s like running a FTP or webserver on Win98…ew.
Using XFS on Gentoo for 5 months now, (they say it might be dangerous if used along with the kernel preemption patch, which I also happen to use). I did not have one single problem so far, and been updating the system regularly so I don’t have that many “stable” components. I would still recommend ext3 to any newbie asking me for advice, but XFS is doing an awesome job for me.
Introducing a networking element is pointless, and it makes the results less reliable. If you wanted to test the relative strengths and weaknesses of these filesystems, an array of tests would be needed. Firstly, I know that certain FS excel at creation of sparse files. One would need to create say 100,000 files with each filesystem and examine the relative difference in creation time. You would also want to test the lookup of file attributes(i.e., mtime, atime) by doing an ls -l on the directory full of sparse files. Unlinking is also worth examining because they all use different methods to keep reference counts, so I would do a rm * test, while I was at it.
Then you would want to test medium sized files. You could do this using a while loop and the dd command to create say 1000 3 megabyte files. I would then move the directory to another partition in order to see how long it takes the tested filesystem to copy and unlink that data. Another unlink test would also be nice, but it wouldn’t really say much because there aren’t nearly as many inodes as with the 100k files above.
Finally, do roughly the same treatment with say a 2GB file in order to see how the FS handles large files.
These tests aren’t anywhere near perfect, but they’re a hell of a lot more indicative of FS throughput than how long it takes to FTP a file…
I’m disappointed in OSNews for posting such an unresearched and unscientific benchmark. It’s far below the quality I’ve come to expect.
To whoever said ReiserFS isnt journaled…..sorry bro, it is. ReiserFS is journaled, but not in the same way as ext3. Ext3 has the best journaling and data integrity, but pays for it with performance. ReiserFS, to me is a great compromise (slightly less strict journaling with much better performance) Ive run ReiserFS in SuSE for over 2 years without and data integrity issues. The test in this article is obviously not a good indicator of FS performance. XFS is geared more toward large file performance. Which is why SGI developed it.
At home, I run ReiserFS on my desktops, either ReiserFS or Ext3 on the servers, and I have seperate XFS partitions for storing large movies and games etc on my FTP server. All of them are much better than NTFS though ;-). One thing I still cant understand though, is why the hell RH still only supports Ext3. I cant speak for the other distros, but SuSE defaults to ReiserFS and readily offers the choice of Ext3 and XFS as well.
Strange article.
If I want to make an FTP linux server and I have to chose filesystem – all I’ve got the tests on large file transfer.
Which only shows how good is file cache on the system. How about small files ? This is more related to filesystem structure and ReiserFS is the king here. How about directory listings ? And where data for ext2 ? That’s what you have to compare with at the beginning and I’m afraid it would beat all those journaled systems in this particular test.
What’s the point to use journaled filesystem on FTP server anyway ? Does ProFTPD make uses of extended atributes or ACLs?
Recompiling the kernel will result in much better performance gain.
Does anyone here got a “Clear Perspective on Filesystems” as author claimed ?
I hope we get more high school projects on OS News.
I agree with you, its not a typo thing i would usually read at OSNews and im also dissapointed with this horrible mmmmmm lets say “benchmark”.. Sorry
waiting for reiserfs 4.
the delivery date has been postponed again and again, from 2002-04-30 -> 2002-09-30 -> 2002-12-31 and now 2003-06-30. it looks ugly.
OTOH, it is good that reiserfs team make sure reiserfs 4 is solid, stabil, secure…. until it become the best jfs.
I don’t see how this could possibly rate as a news
story…except for maybe the well-informed comments
afterwards.
I/O performance is really very hard to do in any
meaningful way since there are potentially so many
layers of buffering (application, stdio, buffer
cache, raid, disk), different types of I/O (buffered,
direct, syncronous, asyncronous) ways of
configuring the filesystem (stripes, mirrors,
block sizes), and the size of the I/O (put 100,000
small files in one directory and see what your
filesystem does).