Linked by Thom Holwerda on Mon 9th Feb 2009 13:24 UTC, submitted by Michael
Benchmarks "In January we published a review of the AMD Shanghai Opteron CPUs on Linux- when we looked at four of the Opteron 2384 models. The performance of these 45nm quad-core workstation/server processors were great when compared to the earlier AMD Barcelona processors on Ubuntu Linux, but how is their performance when running Sun's OpenSolaris operating system? Up for viewing today are dual AMD Shanghai benchmarks when running OpenSolaris 2008.11, Ubuntu 8.10, and a daily build of the forthcoming Ubuntu 9.04 release."
Order by: Score:
Comment by averycfay
by averycfay on Mon 9th Feb 2009 13:49 UTC
averycfay
Member since:
2005-08-29

I'm genuinely surprised at how poorly OpenSolaris did in their tests. It won 1/19 tests and that was java-based. Also, these seem to be somewhat realistic tests unlike the last set of benchmarks posted (how many clicks does it take to install?).

Reply Score: 5

RE: Comment by averycfay
by dean_fry on Mon 9th Feb 2009 14:15 UTC in reply to "Comment by averycfay"
dean_fry Member since:
2005-07-06

yeah, definitely surprised. I just tested the liveCD and it "felt" a bit sluggish. But I thought it was just the liveCD. Any idea why?! maybe old GCC?

Slowlaris indeed ;) (*no bashing intended!*)

Reply Score: 1

RE[2]: Comment by averycfay
by pantheraleo on Mon 9th Feb 2009 15:59 UTC in reply to "RE: Comment by averycfay"
pantheraleo Member since:
2007-03-07

> Any idea why?! maybe old GCC?

Some of this might very well be a GCC issue.

It would be interesting to see these tests redone using Sun's compiler instead of GCC (which can be installed using ss-dev on OpenSolaris).

Reply Score: 1

RE[3]: Comment by averycfay
by Dubhthach on Mon 9th Feb 2009 16:55 UTC in reply to "RE[2]: Comment by averycfay"
Dubhthach Member since:
2006-01-12

GCC 3.4 is fairly ancient that's for sure. I know there is a plan to integrate 4.3 into OpenSolaris, I know I came across a package on opensolaris.org that was pre-built by someone that gave you this.

Reply Score: 2

RE[3]: Comment by averycfay
by fithisux on Tue 10th Feb 2009 19:02 UTC in reply to "RE[2]: Comment by averycfay"
fithisux Member since:
2006-01-22

GCC is defintely an issue.

Reply Score: 2

RE[2]: Comment by averycfay
by dvzt on Mon 9th Feb 2009 17:25 UTC in reply to "RE: Comment by averycfay"
dvzt Member since:
2008-10-23

Definitely GCC. Not only it's old, it's also not 'native' to Solaris. GCC is developed and optimized for Linux. (Open)Solaris is built with Sun Studio. These benchmarks are always interresting, too bad this one is kind of spoiled by choosing incorrect compiler.

Reply Score: 1

RE[3]: Comment by averycfay
by Ford Prefect on Mon 9th Feb 2009 19:34 UTC in reply to "RE[2]: Comment by averycfay"
Ford Prefect Member since:
2006-01-16

How is the compiler optimized to the OS? Can you give more details on that?

I would expect the standard library to be highly optimized, but not the compiler (optimizing only on a lower, hardware level).

Or does GCC on Solaris still use the GNU C Library in contrary to Sun's?

Reply Score: 2

RE: Comment by averycfay
by vermaden on Mon 9th Feb 2009 19:09 UTC in reply to "Comment by averycfay"
vermaden Member since:
2006-11-18
RE[2]: Comment by averycfay
by kaiwai on Tue 10th Feb 2009 07:48 UTC in reply to "RE: Comment by averycfay"
kaiwai Member since:
2005-07-06



Thank you for the link but it doesn't change the fact that the GCC included with OpenSolaris needs to be upgraded immediately - and it should have been done years ago. This has been a known problem in Solaris and yet no one can be bothered sorting it out.

Why hasn't Sun got its act together and updated its GCC bundled with Solaris? its been a known issue for year - it isn't as though it something that has cropped up recently. They wonder why people keep going back to Linux from Solaris - the issues are right in their face, too bad they ignore then in favour of blaming the boogyman.

Reply Score: 1

RE[3]: Comment by averycfay
by pantheraleo on Tue 10th Feb 2009 15:15 UTC in reply to "RE[2]: Comment by averycfay"
pantheraleo Member since:
2007-03-07

> Thank you for the link but it doesn't change
> the fact that the GCC included with OpenSolaris
> needs to be upgraded immediately - and it should
> have been done years ago. This has been a known
> problem in Solaris and yet no one can be bothered
> sorting it out.

Well, arguably you shouldn't be using GCC on OpenSolaris anyway. You should be using the Sun Studio compilers, which are also free and can be installed from the package repository.

> Why hasn't Sun got its act together and updated
> its GCC bundled with Solaris?

Again... Because you should be using the Sun Studio compilers with OpenSolaris. Not GCC.

It's not a question of Sun not having their act together here. It's a question of users who switch from Linux not grokking that there are better compilers out there than GCC that they should be using instead when they switch to a different OS.

Edited 2009-02-10 15:16 UTC

Reply Score: 1

RE[4]: Comment by averycfay
by kaiwai on Tue 10th Feb 2009 15:41 UTC in reply to "RE[3]: Comment by averycfay"
kaiwai Member since:
2005-07-06

Well, arguably you shouldn't be using GCC on OpenSolaris anyway. You should be using the Sun Studio compilers, which are also free and can be installed from the package repository.


Because the Sun compiler isn't compatible with everything out there; XChat will not compile with the Sun compile unless you muck around with a setting inside one of the source code files - IIRC relating to the NTLM proxy authentication. That is one example - I'm sure there are many more out there too.

Again... Because you should be using the Sun Studio compilers with OpenSolaris. Not GCC.

It's not a question of Sun not having their act together here. It's a question of users who switch from Linux not grokking that there are better compilers out there than GCC that they should be using instead when they switch to a different OS.


People would use something other than GCC once their compiler is actually compatible with the source code out there. Yes, the code should be compliant out of the tar ball but at the same time to expect that every project should change their code so that works out of the box with the Sun Studio compiler is just ludicrous. The code is there, for better or worse, its up to Sun to get their compiler to compile code that would compile with gcc. If the code isn't compiling, all the elongated intellectual wankery regarding the superiority of Studio Compiler will be thrown out the window in frustration when someone can't get their opensource application to compile.

The aim should be that people can grab a random application with source code (that compiles on Linux and *BSD) - once it can compile out of the box and with their compilers, then they can say there has been progress. Want proof of the cl*sterf*ck that is Studio compiler, check out the patches upon patches for JDS, pkgbuild's opensource repository, and some things that just can't be built using Studio Compiler - then maybe you'll change your tune.

Edited 2009-02-10 15:46 UTC

Reply Score: 3

Might be some bias there, but ...
by johjeff on Mon 9th Feb 2009 19:45 UTC
johjeff
Member since:
2007-11-06

My guess is they are much more familiar with Linux and did the comparison based on default installations. It's the old argument of whether each OS should be optimized by experts for benchmarking, or whether each OS should ship optimized by default.

There is a good argument for each method depending on who your intended customer is. For Desktop use, the OS should definitely be shipped pre-optimized as much as possible. For experienced UNIX server admins, they often like to optimize and customize things themselves so they know exactly how their system is set up.

Reply Score: 2

Quite dubious
by Kebabbert on Mon 9th Feb 2009 20:53 UTC
Kebabbert
Member since:
2007-07-27

I wonder what the result would be if Linux used GCC v3.x vs Linux GCC v4.x? I bet that GCC v3.x would loose big. Therefore that bench is not really fair.

I remember the old benchmark there, OpenSolaris vs FreeBSD vs Linux, and OpenSolaris used 32 bit binaries vs Linux 64 bit binaries! Someone compiled the same binaries on OpenSolaris, 32bit and 64bit - and his benchmarks doubled going from 32bit to 64bit! That means that the OpenSolaris benches should also have been doubled, easily winning.

I think the phoronix tester is quite dumb. Seriously.

Someone posted evidence that Linux is way faster than Solaris; an article migrated 800MHz SPARC to 2.8 dual CPU intel Xeon Linux and concluded that Linux is faster. Quite dumb. I wonder, why are there so many dumb people in this world? Seriously. I mean it? How in earth can you compare 800MHz CPU vs 2.8GHz CPU? If you do that, you are really really stupid.

Reply Score: 1

Not to start a flame war....
by pantheraleo on Tue 10th Feb 2009 00:54 UTC
pantheraleo
Member since:
2007-03-07

Not to start a flame war, but it seems clear that the people who ran this benchmark didn't know what the hell they were doing:

(A) They ran it on OpenSolaris using an outdated GCC.
(B) They never considered that the native Sun compiler might be faster.
(C) They never bothered to check whether Ubuntu GCC was producing 64 bit binaries by default, and that was not happening on OpenSolaris.

Ultimately, this comes down to a test that was biased in favor of Linux to begin with. And the test is totally meaningless because of that.

Edited 2009-02-10 00:56 UTC

Reply Score: 1

performance scalability
by project_2501 on Tue 10th Feb 2009 02:01 UTC
project_2501
Member since:
2006-03-20

The phoronix test suite is still under development.

One crucial way to make it more useful is to include scalability tests.

OS-A is faster than OS-B on 1 cpu core for a test. But will OS-B maintain performance as number of cpu cores increases while OS-A degrades?

Solaris is known to be slow for many microbenchmarks. That's not why peopel buy it. Its scalable to large hardware and workloads. Better tests would reveal the nature of this scalability.

http://queue.acm.org/detail.cfm?id=1095419

Reply Score: 4

RE: performance scalability
by fithisux on Tue 10th Feb 2009 19:04 UTC in reply to "performance scalability"
fithisux Member since:
2006-01-22

They should start contributing to OpenSolaris. How about Bluetooth!!! ;-)

Reply Score: 2

OpenSolaris slow...
by pantheraleo on Tue 10th Feb 2009 03:43 UTC
pantheraleo
Member since:
2007-03-07

OpenSolaris may be slow on some benchmarks. But as a long time Linux user who recently decided to give OpenSolaris a try, I can honestly say that it "feels" at least as fast as Linux to me when it comes to every day tasks on my desktop where I have been using it for the past month. In some cases it even "feels" faster. Whether it actually is or not is not something I can attest to at this point.

I will say that when it comes to multitasking, it "feels" more capable than Linux though. I can do multiple fairly CPU intensive tasks in the background (like compiling software), and still do things on the desktop foreground that "feel" more responsive than Linux (video seems to run smoother, etc). This is likely where Solaris really shines, given its extensive background on massively-multiuser systems. It is extremely capable at juggling multiple tasks. Because of this, I think these tests ultimately fail to capture the strengths of Solaris where it feels, at least in my mind, like it beats out Linux.

It would be interesting if there were some possible to way to benchmark "actual user experience speed", especially when performing multiple complex tasks at the same time", but unfortunately, as far as I know, there is not.

For full disclaimer purposes, part of the reason I decided to try OpenSolaris is cause I got bored with Linux. Linux has become too mature, and too mainstream, and there's almost no room for the average developer to contribute to it anymore. OpenSolaris is much more interesting when it comes to that. There's encumbered code that needs to be rewritten so it can be open source, tons of packages that need to be ported, etc.

So even if it is a little slower than Linux... Well, I think it is just more fun these days ;)

Edited 2009-02-10 03:47 UTC

Reply Score: 2

RE: OpenSolaris slow...
by dvzt on Tue 10th Feb 2009 07:55 UTC in reply to "OpenSolaris slow..."
dvzt Member since:
2008-10-23

How on earth can you call an OS without a stable kernel API, decent backwards binary compatibility, scattered into multiple incompatible distributions mature?

Reply Score: 2

PKGSRC..
by fithisux on Tue 10th Feb 2009 19:34 UTC
fithisux
Member since:
2006-01-22

has a gcc4.

Reply Score: 2

running in userspace
by 404error on Thu 12th Feb 2009 10:22 UTC
404error
Member since:
2009-02-12

The benchmarks that were selected for this comparison are completely inappropriate for the proposed question. Almost all of the benchmarks are highly compute-intensive. For such programs, the operating system effectively hands off control to the program, interupts it only very very briefly, and then resumes control when it is finished. In other words, the operating system does virtually nothing. It is mentioned above that the compiler might affect things, but this is indeed a dramatic understatement. These benchmarks are effectively only testing the compiler. Running these programs compiled using the same compiler across different platforms would produce differences of only a couple percent (from scheduling overhead, context switch times, memory management routines, cpu affinities, etc). The only potentially valid benchmarks are the SQLite and File Encryption benchmarks, and even these require greater scrutiny.

I am not a big opensolaris fan myself, but using such programs to benchmark an operating system is exceedingly foolish.

Reply Score: 1