To read all comments associated with this story, please click here.
Your concerns are noted at the end of the article:
"Some of the Sun Studio slowdowns may be explained by a majority of the open-source projects targeting GCC as their compiler and catering their compiler flags to the GNU Compiler Collection with little or no optimizations for Sun Studio. At the same time, however, it is also worth noting the current stable series for GCC is 4.3. The BlastWave packages for GCC4 were at version 4.0.2, which is outdated and since then GCC has picked up support for SSE4 and various other features."
Also consider that they are doing these tests on the same hardware and same OS. My very light googling suggests that getting a hold of 4.3 on OpenSolaris is a bit of task and is not at all a "stable" package. Not the best if you're trying to operate psuedo-scientific tests.




Member since:
2006-01-02
I stopped reading here.
What's the point of benchmarking GCC 4.0.2 in 2009?
GCC 4.1 was a big improvement over GCC 4.0.
GCC 4.2 was a big improvement over GCC 4.1.
GCC 4.3 was a big improvement over GCC 4.2.
GCC 4.3 was released one year ago. Even debian stable has it. GCC 4.4 is nearing release and is an especially important upgrade.
I can't see any reason why one would want to benchmark GCC 4.0 in 2009 except trying to make GCC4 look bad, or at least, less impressive than it really is.
If any OS is still stuck at GCC 4.0, all that means is that you can't use this OS to benchmark GCC4 in 2009, end of the story. That's by no means an excuse to publish a "GCC4" benchmark without at least a very prominent notice that "This benchmark is of course irrelevant because we tested a GCC version that's completely outdated".
EDIT: I read a bit the article, and there's a small mention of that fact on page 6, but that's not enough for something that makes the whole benchmark totally irrelevant (since the other gcc benchmarked here, 3.4, is even more outdated)
Edited 2009-02-20 17:10 UTC