OSNews

RedHat Linux typically uses the ext2 or ext3 file systems for storage, while Solaris 9 x86 uses the Unix stalwart UFS. Many people complain about the slowness of UFS, similar to the complaints about BSD's FFS until the advent of soft-updates. But, since Solaris 7, Sun has provided a remedy of this UFS slowness by the way of UFS logging which many sysadmins are surprised to learn of.

UFS with logging provides a remarkable speed boost to file systems for some types of operations, so much so that I'm surprised that Sun hasn't just turned it on by default. You can run UFS logging safely on all file systems, including /. To enable UFS logging, just add logging to the option column (it's the last column) in /etc/vfstab and reboot. The file systems will automatically come up with logging enabled. Of course be careful when editing /etc/fstab, as fudging it up can really ruin your day.

/dev/dsk/c0t0d0s0 /dev/rdsk/c0t0d0s0 / ufs 1 no logging

/dev/dsk/c0t0d0s6 /dev/rdsk/c0t0d0s6 /usr ufs 1 no logging

I ran a quick check to see how UFS logging helped speed, so I did an untar of the gcc-3.3.1.tar source code from ftp://www.gnu.org/gnu/gcc/ and then deleted the directory. The results are quite dramatic:

Task | Without UFS Logging | With UFS Logging

tar -xvf gcc-3.3.1.tar 4m 2s 0m 41s

rm -rf gcc-3.3.1 2m 53s 0m 4s

Going from 4 minutes to 40 seconds for the tar, and almost 3 minutes to just under 4 seconds is quite a speedup. Of course tar and rm operations are by no means a comprehensive of I/O operations in a production server environment, but you get the picture.

For Linux, the difference between ext2 and ext3 was minimal, and both were faster than Solaris (even with UFS with logging):

Task | ext2 | ext3

tar -xvf gcc-3.3.1.tar 0m 24s 0m 22s

rm -rf gcc-3.3.1 0m 2s 0m 2s

For a bit more of a CPU and I/O intensive operation, I ran the benchmarking utility included with mysql-4.0.15a, the sql-bench test suite.

Figure 1: Basic file system performance

What surprised me about this test is that when I ran the tests for Linux in ext2 and ext3, the results were essentially identical. The same was true with Solaris with and without logging; the results were the same.

With that in mind, here are the results for Linux ext3 versus Solaris with logging.

Figure 2: MySQL benchmarks

Figure 3: MySQL benchmarks continuted

The results are mixed, with Solaris coming out on top for some operations (most notably the insert operations), and Linux coming out on top for others.

Another test of CPU speed was a test of compilation times for a number of popular open-source applications. Again, compilation time can depend on a number of metrics, including many that aren't tied to system speed (such as the number of system include files), so these benchmarks serve to satiate curiosity more than anything.

Figure 4: Compilation time

My first step was to use a common compiler, so I chose GNU GCC's latest, 3.3.1. For Solaris, I used the 2.95 GCC build I obtained from Sun's own freeware site (http://wwws.sun.com/software/solaris/freeware/). It compiled without any errors, and took 22 minutes and 26 seconds to compile.

For Linux, I used the RedHat's pre-installed GCC 3.2.2 to compile GCC 3.3.1. This took quite a bit longer, clocking in at 94 minutes, 10 seconds. Of course since it was 3.2.2 versus 2.95, it's not a fair comparison and thus I didn't include them on the graph.

I then compiled MySQL 4.0.15a (which was then used for the MySQL benchmarks), Perl 5.8.1, OpenSSH 3.7.1p2, Apache 1.3.28, and PHP 4.3.3.

So, for the most part, Solaris seems to be the faster operating system when compiling software, but again, this isn't really a critical metric.

Encryption operations are one way to put a system through its paces, so I compiled OpenSSL 0.9.7c and used its speed test: opeenssl speed.

When I initially ran it on Solaris and Linux, the results showed a huge disparity. Upon investigation, I found that the default building process for Solaris compiled it with -m486, optimized for the i486 processor, and the Linux default build compiled with -mcpu=pentium, or 586 flag. This system is a Pentium III system, so really the best compilation would be for -march=i686 (Red Hat distributes an i686-optimized version of OpenSSL).

So I went back and re-compiled with the i686 optimization flag for both operations, and re-ran the tests.

Figure 5: DSA Operation rate

Figure 6: RSA operation rate

Linux held a very slight lead in all of the results, which is interesting since the same exact hardware was used for both.

As a side note, running each system with i486, i586, and i686 optimizations showed that they make a very big difference on both platforms, around 20% between the i486 and i686 results.

Web performance was probably the biggest surprise of this evaluation, as the results strongly favored one OS over the other, and the OS I suspected would win didn't.

One of the most critical performance metrics that web servers must contend with is connections per second, also referred to as operations per second. The opening of a new TCP connection, handing it off to the web server, serving up the request, closing it off, and all the little things that go in in between the steps I just mentioned can be very CPU intensive.

Testing web performance normally entails racks and racks of systems, using sophisticated software and analysis tools and tests a variety of scenarios. I don't have racks and racks of systems, nor do I have the software necessary to test a dynamic range of metrics.

However, I do have more computers than is good for any one person to own, and I have freeware tool called http_load, written by Jef Poskanzer. You can find this great little load generator at http://www.acme.com.

I used a small 590 byte text file (small files are best for testing ops/second, whereas large files are best for testing throughput) and directed http_load on two different load systems acting as load generators to retrieve it as fast as possible from the test machine. One system is a Pentium III 1 GHz, and the other is a AMD 2200+ system, both running Red Hat Linux 9.

I compiled a stock Apache 1.3.28 build with modules enabled (mod.so) and loaded the PHP 4.3.3 module, although the test file was not PHP-based. Compilation options and environment were identical for both systems. The only change to the stock Apache configuration file was kicking MaxClients up to 256 (the hard-coded maximum without re-compiling).

I ran these operations for a period of 10 minutes, and periodically checked the file with a web browser to ensure the system was still serving up OK. I never had a problem retrieving the file during the test of either system. The http_load utility showed no errors for either run. Here are the results:

In this test, Linux was the clear winner showing double the performance, with 2057 total operations per second to Solaris' 946. This result was the most surprising, as I had hypothesized that Solaris would come out on top by a slight margin, mostly because of Solaris' renowned networking stack.

It would be interesting to see what effect specific operating system tweaks would have on these tests, but that would be a whole other article.

1. "Background and setup"
2. "Installation and usage"
3. "Everyone loves benchmarks"
4. "Conclusions"