Linked by Kroc Camen on Fri 19th Dec 2008 09:04 UTC, submitted by Michael
Benchmarks From Phoronix: "Have you ever wondered on what operating system Java works the best? While by no means is it a conclusive multi-platform comparison, for this article we ran a number of Java benchmarks on both Windows Vista Premium and Ubuntu Linux to see how the Java Virtual Machine performance differs. In addition, when running Ubuntu we had tested Sun's official Java package as well as the OpenJDK alternative."
Order by: Score:
The Tests
by OSGuy on Fri 19th Dec 2008 09:54 UTC
OSGuy
Member since:
2006-01-01

They should have included XP in the tests.

Reply Score: 2

RE: The Tests
by spiderman on Fri 19th Dec 2008 10:03 UTC in reply to "The Tests"
spiderman Member since:
2008-10-23

And solaris

Reply Score: 8

RE[2]: The Tests
by ichi on Fri 19th Dec 2008 10:06 UTC in reply to "RE: The Tests"
ichi Member since:
2007-03-06

And a pony.

Reply Score: 54

RE[3]: The Tests
by spiderman on Fri 19th Dec 2008 10:28 UTC in reply to "RE[2]: The Tests"
spiderman Member since:
2008-10-23

Sun Solaris is specifically designed to run Sun java with good performance. Its threading model is tuned for java.
It would be interesting to compare its performances with other OS.

Reply Score: 8

RE[4]: The Tests
by rexstuff on Fri 19th Dec 2008 16:13 UTC in reply to "RE[3]: The Tests"
rexstuff Member since:
2007-04-06

I should hope not. Solaris (and Sun OS) predate Java by a wide margin. As with operating system, Solaris will have been designed to work under a large variety of workloads. To tune your whole OS just so another tool released by your company gets a performance edge seems silly to me.

Edit: typo

Edited 2008-12-19 16:14 UTC

Reply Score: 2

RE[5]: The Tests
by Bill Shooter of Bul on Fri 19th Dec 2008 17:33 UTC in reply to "RE[4]: The Tests"
Bill Shooter of Bul Member since:
2006-07-14

The Java market is huge. If it gives them an advantage in a huge market, it might make business sense for them to do it. Increasing profit, is not a silly reason.

Reply Score: 2

RE[5]: The Tests
by spiderman on Fri 19th Dec 2008 21:23 UTC in reply to "RE[4]: The Tests"
spiderman Member since:
2008-10-23

Actually, I believe it is the other way around: Sun optimizes java for Solaris. Anyway, Solaris has several threading models...

Reply Score: 4

RE[3]: The Tests
by google_ninja on Fri 19th Dec 2008 13:13 UTC in reply to "RE[2]: The Tests"
google_ninja Member since:
2006-02-05

read all the comments, then came back and voted up cause I was still laughing

Reply Score: 2

RE[3]: The Tests
by bryanv on Fri 19th Dec 2008 14:42 UTC in reply to "RE[2]: The Tests"
bryanv Member since:
2005-08-26

I want a pony!

Reply Score: 2

RE[4]: The Tests
by OMRebel on Fri 19th Dec 2008 15:12 UTC in reply to "RE[3]: The Tests"
OMRebel Member since:
2005-11-14

I want a pony!


I hear they taste like chicken.

Reply Score: 7

RE[5]: The Tests
by Moredhas on Fri 19th Dec 2008 20:22 UTC in reply to "RE[4]: The Tests"
Moredhas Member since:
2008-04-10

What's chicken taste like? I've been eating pony all my life! Everything tastes like pony!

Reply Score: 2

RE[3]: The Tests
by BSDfan on Fri 19th Dec 2008 18:30 UTC in reply to "RE[2]: The Tests"
BSDfan Member since:
2007-03-14

You can't have my pony.

Reply Score: 1

RE[3]: The Tests
by toblerone on Fri 19th Dec 2008 23:14 UTC in reply to "RE[2]: The Tests"
toblerone Member since:
2008-12-11

:D brilliant

Reply Score: 1

RE[3]: The Tests
by Michael on Sat 20th Dec 2008 00:16 UTC in reply to "RE[2]: The Tests"
Michael Member since:
2005-07-01

Dugg

Reply Score: 2

RE[3]: The Tests
by Wowbagger on Mon 22nd Dec 2008 01:16 UTC in reply to "RE[2]: The Tests"
Wowbagger Member since:
2005-07-06

They even forgot Poland.

Reply Score: 1

RE[3]: The Tests
by pixel8r on Mon 22nd Dec 2008 04:04 UTC in reply to "RE[2]: The Tests"
pixel8r Member since:
2007-08-11

And a pony.


do ponies even like java?

I prefer Riva coffee myself, although I was previously a Nescafe Gold drinker...

Reply Score: 2

RE: The Tests
by CodeMonkey on Fri 19th Dec 2008 15:08 UTC in reply to "The Tests"
CodeMonkey Member since:
2005-09-22

Let us not forget FreeBSD. Especially now with the ULE scheduler in 7 and the native ports build of Sun's JDK.

Reply Score: 6

Client vs Server VM
by evangs on Fri 19th Dec 2008 10:16 UTC
evangs
Member since:
2005-07-07

Looking at the Scimark benchmark, it's obvious that they're comparing the Server VM on Linux to the Client VM on Windows. It _might_ also explains why the rendering tests do so much better on Windows.

Server VM is optimized for long running tasks and number crunching in general. Client VM is optimised for fast start up and smooth Swing performance. They should retest, with the JVM being the same on both Windows and Linux. Use JDK _11 on both instead of _07 on Windows and _10 on Linux. Yes, they do make performance improvements between minor versions.

Reply Score: 11

RE: Client vs Server VM
by spiderman on Fri 19th Dec 2008 10:35 UTC in reply to "Client vs Server VM"
spiderman Member since:
2008-10-23

Actually, if you look closer, they didn't benchmark Sun JVM at all on Ubuntu for the rendering (the last benchmark). It looks like they forgot because the results only show openJDK results on Ubuntu vs Sun JVM on Windows.
did I miss something?

Edited 2008-12-19 10:36 UTC

Reply Score: 1

RE[2]: Client vs Server VM
by evangs on Fri 19th Dec 2008 11:41 UTC in reply to "RE: Client vs Server VM"
evangs Member since:
2005-07-07

You didn't miss anything. They benchmarked the OpenJDK which according to the 1st page utilized the Server VM.

"with OpenJDK we were using Java 1.6.0_0 with the OpenJDK Server VM build 1.6.0_0-b12"

Reply Score: 4

RE[3]: Client vs Server VM
by Panajev on Fri 19th Dec 2008 14:00 UTC in reply to "RE[2]: Client vs Server VM"
Panajev Member since:
2008-01-09

Then it is a very unfair benchmark, there were notable performance improvements between u07 and u10 versions of the 1.6 JDK on Windows and it is also unfair testing the Server VM against the Client VM (sure it uses more RAM, but AFAICR it is faster).

Reply Score: 3

RE: Client vs Server VM
by Savior on Fri 19th Dec 2008 11:00 UTC in reply to "Client vs Server VM"
Savior Member since:
2006-09-02

They should retest, with the JVM being the same on both Windows and Linux. Use JDK _11 on both instead of _07 on Windows and _10 on Linux. Yes, they do make performance improvements between minor versions.


And I am not even sure that u10 could be called a "minor version". Though perhaps there is not such a big jump in performance, compared to u7.

Also, there is a mistake in the Sunflow rendering test. According to the figure, the results are in seconds, and Ubuntu is 19.xx, while Vista is over 20; yet in the text they write "Windows Vista was faster, but not even by a few points."

Reply Score: 3

RE: Client vs Server VM
by mdoverkil on Fri 19th Dec 2008 16:02 UTC in reply to "Client vs Server VM"
mdoverkil Member since:
2005-09-30

That's typical Phoronix benchmarking unfortunately. They always try to run comparisons between operating systems and yet they run completely different versions of the same software and expect to get accurate benchmarks.

Reply Score: 3

Comment by progster
by progster on Fri 19th Dec 2008 15:03 UTC
progster
Member since:
2005-07-27

at my uni with did some java execution speed benchmarks about 3 years back, out of xp, debian (testing) and mac osx... osx suprisingly won

Edited 2008-12-19 15:04 UTC

Reply Score: 1

RE: Comment by progster
by Bending Unit on Fri 19th Dec 2008 15:36 UTC in reply to "Comment by progster"
Bending Unit Member since:
2005-07-06

Wheee! 3 years back it would have been glorious. Nowadays, I'm afraid, it's worth nothing ;)

Reply Score: 3

Silly benchmark
by suryad on Fri 19th Dec 2008 16:13 UTC
suryad
Member since:
2005-07-09

In my opinion considering the jvms are not uniform

Reply Score: 2

RE: Silly benchmark
by sbergman27 on Fri 19th Dec 2008 17:28 UTC in reply to "Silly benchmark"
sbergman27 Member since:
2005-07-24

From TFA:

These Java versions are the stock versions for what was available with each operating system.

That's the most uniform in the way that actually counts for most users. These are the versions that are actually used, in real life, on the tested platforms.

What I'm hearing is the old classic: "They should have tweaked my contestant to perform optimally" claim, which doesn't make a whole lot of sense in this case. If we were talking "important server application" being deployed by a seasoned administrator, then sure. But in that case it wouldn't be running on Vista.

Edited 2008-12-19 17:29 UTC

Reply Score: 8

RE[2]: Silly benchmark
by evangs on Fri 19th Dec 2008 19:13 UTC in reply to "RE: Silly benchmark"
evangs Member since:
2005-07-07


What I'm hearing is the old classic: "They should have tweaked my contestant to perform optimally" claim, which doesn't make a whole lot of sense in this case.


They run a Server VM and find that numerically heavy code (i.e. Scimark) and file access operations run faster than on the Client VM. Then they find that the Client VM runs graphically intensive applications faster than the Server VM.

To use the standard car analogy (sorry guys), it's like complaining that a Ferarri Modena sucks because you can't haul much cargo with it, and then proceed to complain that the Ford F-150 sucks because it doesn't do 0-60 in under 5 seconds.

Reply Score: 3

RE[2]: Silly benchmark
by suryad on Fri 19th Dec 2008 23:08 UTC in reply to "RE: Silly benchmark"
suryad Member since:
2005-07-09

You are totally assuming that that is what I had wanted. I hate Vista with a passion and therefore I could care not care less about it's Java benchmark results.

What I do care about is already pointed out by other people and that is that client vs server JVM switches will result in different numbers...drastically different numbers so it results in bias.

Reply Score: 3

trembovetski
Member since:
2006-09-30

> That's the most uniform in the way that actually counts for most users. These are the versions that are actually used, in real life, on the tested platforms.

That's odd because the latest available update from java.com is 6u11 (and it's even auto-update-enabled).

Dmitri

Reply Score: 2

Graphics performance
by trembovetski on Fri 19th Dec 2008 17:41 UTC
trembovetski
Member since:
2006-09-30

From TFA:
> The j2dbench performance was significantly better on Windows Vista Premium, but that's likely attributed to the Intel graphics stack on Linux having little in the way of performance optimizations.

That really depends on what tests were selected to be ran in J2DBench. 6u7 on Windows has only few hw-accelerated rendering primitives.

6u10/11 has new Direct3D rendering pipeline which would have given MUCH better results on Windows. Except that the test was ran on a Intel integrated hw, on which the new Direct3D pipeline is disabled because of drivers quality and performance issues.

Dmitri

Reply Score: 2

RE: Graphics performance
by sbergman27 on Fri 19th Dec 2008 18:24 UTC in reply to "Graphics performance"
sbergman27 Member since:
2005-07-24

Except that the test was ran on a Intel integrated hw, on which the new Direct3D pipeline is disabled because of drivers quality and performance issues.

It's worth noting that Intel graphics come on more new machines than any other graphics chipset. The Intel driver situation on both OSes is a matter of some concern.

Reply Score: 3

Interesting article
by factotum218 on Fri 19th Dec 2008 19:14 UTC
factotum218
Member since:
2007-03-20

Another great benchmark article. Love 'em.
The only real experience I've had with java applications are netbeans and aptana.
Aptana was installed through the Eclipse plugin on both systems, using the default packages available for each system.
Aptana flew on Vista, on Ubuntu I had to stop typing every few seconds to wait for the keystrokes to appear on the monitor.
Netbeans on the other hand was pretty slick and quick on both.

Sorry, nothing informative here, just killing a few minutes while I take a break from shoveling.

Reply Score: 2

Results don't make sense
by PlatformAgnostic on Fri 19th Dec 2008 21:13 UTC
PlatformAgnostic
Member since:
2006-01-02

If you're testing number crunching (like SciMark seems to), the OS shouldn't matter much at all (definitely not 3X). Those tests should be stressing the CPU more than anything else.

Drawing and certain types of File IO may determine the relative merits of the OSes, but doing unbuffered or large IOs should once again be consistent on the same hardware because the OS can't make the disk spin faster.

One more thing: if they do these tests on a dual-boot machine with one harddrive, which OS is closer to the beginning of the disk? That one will experience faster sequential IOs in general.

Reply Score: 2

Mac
by wanker90210 on Sat 20th Dec 2008 13:16 UTC
wanker90210
Member since:
2007-10-26

It would have been interesting to see this test if they ran it on an Apple machine and included MacOSX in the test. From what I understand Apple maintains its own branch with some interesting optimizations.

Reply Score: 1

intel gfx
by ideasman42 on Sun 21st Dec 2008 14:56 UTC
ideasman42
Member since:
2007-07-20

comparing OpenGL performance with win/linux for intel gfx is dumb. Use an NVidia (even a cheap one) for a fair comparison.
They did this in one of their other benchmarks too.

aside from that, its nice to see a banchmarking mistake favor linux for a change ;)

Would be nice if all the different java versions wernt so damn confusing - see http://www.mattryall.net/blog/2008/06/java-technobabble-explained

Edited 2008-12-21 14:58 UTC

Reply Score: 1

RE: intel gfx
by sbergman27 on Sun 21st Dec 2008 20:00 UTC in reply to "intel gfx"
sbergman27 Member since:
2005-07-24

comparing OpenGL performance with win/linux for intel gfx is dumb. Use an NVidia (even a cheap one) for a fair comparison.

Why? Why not test the JVM that most people are using on the respective platforms, along with the graphics chipset family that they are most likely using. Because that's what this benchmark is about. It's not about Linux vs Windows as some seem to want to frame it. It's not about client vs server JVM as others seem to want to frame it. And it's not about Intel vs Nvidia. It's about how the most commonly used (default) JVMs on the respective OSes perform on the most common hardware.

Edited 2008-12-21 20:01 UTC

Reply Score: 2

RE[2]: intel gfx
by ideasman42 on Mon 22nd Dec 2008 00:45 UTC in reply to "RE: intel gfx"
ideasman42 Member since:
2007-07-20

There just seems to be a reoccuring case of...

"Some blah blah benchmark between windows and linux shows linux is bad at 3D"

- If you have intel like most people do, then sure, 3D will suffer, this is also not news, any benchmark in linux using Intel+3D will do baddly for linux.

If you assume 3D performance is anywhere important to the user, they wont be using an intel card.

Reply Score: 1

RE[3]: intel gfx
by sbergman27 on Mon 22nd Dec 2008 01:14 UTC in reply to "RE[2]: intel gfx"
sbergman27 Member since:
2005-07-24

If you have intel like most people do, then sure, 3D will suffer, this is also not news, any benchmark in linux using Intel+3D will do baddly for linux.

What's wrong with this picture? Intel has done *everything* that the FOSS community has claimed for years that we needed for a company to do in order to facilitate good Linux drivers. In fact, they've gone well beyond. Meanwhile, NVidia has taken the traditional closed and proprietary approach, and is lauded for their superior Linux drivers.

It was the same story with the original Radeon back when ATI was sharing their specs back in the 8500 days. FOSS driver writers took the ball and dropped it. I'm not sure that today's radeon driver is even feature complete WRT that ancient 8500 card. let alone newer ones. Later, I started hearing claims that ATI just wasn't "open enough". Why can't the FOSS world come up with good drivers even with all the help and man power Intel is providing?

Edited 2008-12-22 01:18 UTC

Reply Score: 2

RE[4]: intel gfx
by ideasman42 on Mon 22nd Dec 2008 01:32 UTC in reply to "RE[3]: intel gfx"
ideasman42 Member since:
2007-07-20

Its good intel has made efforts to improve their linux driver situation.

Its still beside the point, intel drivers are not that great. - NVidia has just had a lot more experience in this area.

Reply Score: 1

RE[5]: intel gfx
by sbergman27 on Mon 22nd Dec 2008 01:41 UTC in reply to "RE[4]: intel gfx"
sbergman27 Member since:
2005-07-24

Its still beside the point,

Near as I can tell, all of your comments have been beside the point. Neither 3D nor OpenGL graphics performance is even mentioned in TFA.

Edited 2008-12-22 01:46 UTC

Reply Score: 2

RE[6]: intel gfx
by ideasman42 on Mon 22nd Dec 2008 02:38 UTC in reply to "RE[5]: intel gfx"
ideasman42 Member since:
2007-07-20

From the article

Intel graphics stack on Linux having little in the way of performance optimizations. Mesa isn't yet performance oriented and there is no kernel memory manager or other features found in Ubuntu 8.10 for Intel graphics. This though should improve vastly with time considering the invasive work currently taking place within the X.Org / Mesa community and the eventual adoption of the Gallium3D architecture.


References to Mesa (an OpenGL implimentation) and Gallium3D would suggest OpenGL performance is a problem.

Edited 2008-12-22 02:39 UTC

Reply Score: 1

It's predictable
by portamenteff on Mon 22nd Dec 2008 17:09 UTC
portamenteff
Member since:
2008-05-25

The tests in Ubuntu that were slower than Vista were do to graphics driver issues and Xorg issues. The tests that came out slower for Vista were most likely non-graphical in nature. There are problems that have been known for a while now.

Reply Score: 0

I have never wondered or cared!
by Bit_Rapist on Mon 22nd Dec 2008 21:51 UTC
Bit_Rapist
Member since:
2005-11-13

I could care less what platform java performs best on as I try to avoid using Java at all costs!

Reply Score: 2

sbergman27 Member since:
2005-07-24

I could care less what platform java performs best on as I try to avoid using Java at all costs!

Why?

Reply Score: 2