posted by Tony Bourke on Wed 28th Jan 2004 19:51 UTC
IconWhen doing research for my evaluation of Solaris 9 on my Ultra 5, I kept running into one comment over and over again: Sun's C compiler produces much faster code than GCC does. However, I couldn't find one set of benchmarks to back this up. (If you know of any, drop me an email.) Could this be yet another case of rumor-taken-as-fact?

If indeed Sun's compilers are faster, how much faster? 5 percent? 20 percent? twice as fast? I've heard them all from various people, so I decided to do some tests on my own to see if I could see this supposed performance difference between GCC and Sun's compiler.

Another potential question would be which version of GCC to use: That latest (as of this writing) 3.3.2, or GCC 2.95 which is still in wide use (especially on Linux, FreeBSD, NetBSD, and OpenBSD running on SPARC). So I included them both, to also answer the questions of which GCC produces the better code (on Solaris/SPARC). 64-bit binary creation was also tested, for binaries created by GCC 3.3.2 and Sun's compiler.

Bling Bling

But Tony, Sun's compiler is so expensive! Yes, it is expensive, especially when compared to the free (as in beer and freedom) GCC. However, Sun does offer a free 60-day evaluation license for their compiler suite, which is what I used.

Compilers

For Sun's compiler, I used the 60-day trial for Sun ONE Studio 7, which can be found here. It includes C, C++, and Fortran compilers, as well as Java other development tools. The full version lists for $2,995.

For GCC 3.3.2, I used a binary version from SunFreeware, built for making 32-bit and 64-bit binaries. For GCC 2.95.3, I downloaded the source from GNU, and built it using GCC 3.3.2.

Applications

I'm using the same applications as I did for my previous article on 32-bit versus 64-bit binaries. In fact, the tests for this article and the 32-bit versus 64-bit article were all done at the same time, and these articles done separate in the interest of brevity. Those applications are OpenSSL 0.9.7c, GNU gzip 1.2.4a, and MySQL 4.0.17.

For the GCCs, the compiler options were “-O3 -mcpu=ultrasparc” for optimized 32-bit binaries, and “-O3 -mcpu=ultrasparc -m64” for optimized 64-bit binaries. For Sun's compiler, I used “-fast -xarch=v8plusa” for optimized 32-bit binaries, and “-fast -xarch=v9a” for optimized 64-bit binaries.

GNU gzip 1.2.4a

I used the latest version from GNU's ftp site, and compiled in the 32-bit and 64-bit variations with the various compilers. The gzip and gunzip operations were run 3 times each, and the results averaged.

benchmark graph

In this test, Sun's c compiler created a GNU gzip binary that zipped up the 624 MB file about 11% faster than GCC did. Surprisingly, GCC 2.95 produced a gzip binary slightly (2%) faster than GCC 3.3.2. The unzip operation, GCC 2.95 won at 89 seconds, with GCC 3.3.2 coming in at 90, followed by Sun's compiler at 91 seconds.

In the 64-bit run, Sun's compiler produced the faster code.

benchmark graph

As with the 32-bit tests, the gunzip operation produced very little delta between the three binaries.

After seeing my 64-bit versus 32-bit article, Shane Pearson wrote me posing the question whether or not disk I/O was a bottleneck in the gunzip operations, thus causing the results to come out essentially the same. After I ran some tests, it turns out he my very well be right. The 624 MB file is written in 91 seconds, which is about 6.9 MB/s, and then you include reading the 126 MB gzip file in, for a total of around 8.4 MB/s, read and write. This is pretty close to the effective maximum of the system, which for a 100 MB dd if=/dev/zero test and 100 MB mkfile test, comes to about 10 MB/s, with no other CPU operations going on.

And here is a graph showing how they all stacked up:

benchmark graph

Sun's compiler was the clear winner. Surprisingly, the older version of GNU's GCC beat 3.3.2 by a very slim margin.

Table of contents
  1. "Intro and GNUZip"
  2. "OpenSSL 0.9.7c"
  3. "MySQL 4.0.17 and Conclusion"
e p (0)    40 Comment(s)

Technology White Papers

See More