In this article .NET 2.0 won 2 out of the 3 major tests – clearly besting Java 1.5 in both execution speed and real-world memory efficiency. Java did, however, manage to hold its own in the native types memory comparison by a pretty wide margin. This indicates that on the whole .NET is a more efficient platform, with perhaps at least one area for improvement – native type memory efficiency.
.NET 2.0 vs. Java 1.5 Shootout
About The Author
Eugenia Loli
Ex-programmer, ex-editor in chief at OSNews.com, now a visual artist/filmmaker.
Follow me on Twitter @EugeniaLoli
I’m a fan of .NET, and I have to say that this test was moronically executed. Not only does it conclude completely false things, such as claiming that .NET treats arrays of native types as arrays of objects, but it’s also very lacking in number and quality of tests.
The bigger “memory usage” of .NET was most likely because of a different garbage collector strategy. Seeing a big allocation it reserved a bunch of extra virtual memory space anticipating more. This memory will not be actually taken until used.
I’m a fan of .NET, and I have to say that this test was moronically executed. Not only does it conclude completely false things, such as claiming that .NET treats arrays of native types as arrays of objects, but it’s also very lacking in number and quality of tests.
I agree totally. This was no meaningful comparison of Java and .Net at all. Move along.
Thus, as we have seen, .NET 2.0 won 2 out of the 3 major tests – clearly besting Java 1.5 in both execution speed and real-world memory efficiency.
Since it looks like these benchmarks ended up .NET’s favor, how we can expect at least 80 comments explaining exactly how and why thest tests were biased in favor of .NET.
“Since it looks like these benchmarks ended up .NET’s favor, how we can expect at least 80 comments explaining exactly how and why thest tests were biased in favor of .NET.”
All benchmarks are biased one way or the other. Just like statistics, benchmarks is a great vehicle for “proving” your point.
Java will die of its own myopia and bloat.
The entire Java system needs redesign. The object models in .NET are much cleaner than Java.
It is no surprise .NET 2.0 is superior to Java 1.5 (5.0). Despite all the trolls that write code for Java, Sun doesn’t have a clue on how to make usuable software. Java’s performance has always sucked. Java is so bad, IBM had to make SWT. Which blows in its own ways. Java has no way to make nice apps. J2EE is a lesson in bad and bloated design. Perhaps the most bloated and least efficient framework ever made.
And the simple things… 10 years later and still people kill themselves with CLASSPATH.
The basic truth here:
Never trust your software to “big iron” golf course types.
.NET 2.0 release after Java 1.5 maybe for a half year.
If .net 2.0 won’t better I think MS very suck.
Despite all the trolls that write code for Java, Sun doesn’t have a clue on how to make usuable software.
Despite all the trolls that write code for Windows-Only .NET, Microsoft doesn’t have a clue on how to create multiplatform software.
Despite all the trolls that write code for Windows-Only .NET, Microsoft doesn’t have a clue on how to create multiplatform software.
I’m not sure. The .NET is basically not tied too strictly to windows, let see the mono project. There is a way to create platform-independent .NET compatible sytem. It is only the Microsoft’s decision. IMHO in the future the main plaform of microsoft will the .NET and not the Windows.
Despite all the trolls that write code for Windows-Only .NET, Microsoft doesn’t have a clue on how to create multiplatform software.
Why support multiplatform? Since all the other platforms, minus probably OSX, really suck when it comes to GUI. Microsoft created with .NET the most beautiful thing that happened to the computer industry in the past 10 years. They should at least get proper credits for that.
“Why support multiplatform? Since all the other platforms, minus probably OSX, really suck when it comes to GUI. Microsoft created with .NET the most beautiful thing that happened to the computer industry in the past 10 years. They should at least get proper credits for that.”
Because windows is the most suck OS that happened to the computer industry
It was true for win3.x and 9.x. But the NT-line is a relative good thing, the NT microkernel is really good (better then linux kernel), the win32 is not the best idea (but better then the 10000 little inconsistent library and application under linux) and IMHO the .NET will replace it in the near future.
NT is not a microkernel. This is straight from the kernel team. It only looked like one at the start, but it’s not anymore.
Firstly, Linux is not a microkernel.
Secondly, your priorities for what you want in a kernel will most likely be different than everyone elses. Both Linux and NT kernels have good points, and both have bad points.
Thirdly, the NT kernel is totally uncustomizable (totally closed proprietory). Whereas Linux is totally free.
Your comparation is nearly as baseless as this article. It also seems like you dont know what you are talking about. Oh, and good luck with your driver programming in C#…
Firstly, Linux is not a microkernel.
It is true. I never said any similar thing. But IMHO the NT microkernel is better then the monolithic linux kernel.
Both Linux and NT kernels have good points, and both have bad points.
I used linux as desktop from 1995. I don’t see too many good points in the linux kernel. There are many good thing in the linux, but it is not the kernel.
Thirdly, the NT kernel is totally uncustomizable (totally closed proprietory). Whereas Linux is totally free.
It is very good thing if you want to create PDA, or any hardware-centric development. But if you want to develop Java or .NET IMHO this is not a too important aspect for you.
Oh, and good luck with your driver programming in C#…
If I want to create driver I will use C/C++. But if I want to create a business application (and in most of cases I write business apps) I will use C#/.NET (now I use Kylix/Delphi, before Kylix I used C++/wxWidgets).
“But IMHO the NT microkernel is better then the monolithic linux kernel.”
The NT kernel is not a microkernel and the Linux kernel is not monolithic. They’re both hybrid kernels with varying degrees of modularity.
I used linux as desktop from 1995. I don’t see too many good points in the linux kernel. There are many good thing in the linux, but it is not the kernel.
LOL! And precisely what technical insights did you get into the kernel from being a desktop user?
You obviously are an idiot.
Linux is a Kernel – NOTHING MORE.
GNU/Linux is the total system.
IMHO if a zealot say “linux is better then windows” he is think to the full GNU/Linux and not only to the kernel.
You obviously are an idiot.
Linux is a Kernel – NOTHING MORE.
GNU/Linux is the total system.
Ahh, you are mistaken.
A quick look around my desktop reveals that the total system is:
Xorg/BSD/Apache/Adobe/PHP/Real/Sun/Novell/EasyTag/GNU/Linux.
I may have missed some. Be assured that before writing the name of the operating system in future I shall investigate the 30 or 40 different organisations, projects and companies that have provided the software which comprise my desktop PC’s operating system.
Yeah yeah, when I think about windows GUI I recall Xview/Motif/Openstep/Gnome/KDE all these are MS achievements, the other OSes are only command line.
First of all, I would say the windows GUI sucks badly too. It is just that people are so used to it that they don’t notice.
Second, most java programs runs ever written runs on the serverside without much of a GUI. In some way Java act as a modern replacement for COBOL, and just like COBOL it will take a very long time before it becomes extinkt regardless how many better techniques that becomes available.
Third, .net 2.0 is newer than Java 5 (jdk1.5), and being so similar it is only natural that .net is somewhat better, just like java 1.6 will be better than java 1.5 and .net 2.0, just like java 5 was better than .net 1.0 However, most organizations can’t afford to change their software platform as often as the version numbers of the development platforms change.
First of all, I would say the windows GUI sucks badly too
What is the problem with the windows GUI ?
and just like COBOL it will take a very long time before it becomes extinkt regardless how many better techniques that becomes available
It is propably true.
Third, .net 2.0 is newer than Java 5 (jdk1.5), and being so similar it is only natural that .net is somewhat better, just like java 1.6 will be better than java 1.5 and .net 2.0, just like java 5 was better than .net 1.0
But it is not. There are many design bugs in java, and nobody can resolve this problems without breaking the compatibility. Java x.x never will better then .NET 1.1.
Despite all the trolls that write code for Windows-Only .NET, Microsoft doesn’t have a clue on how to create multiplatform software.
With previous version(s) of Windows NT being released for multiple platforms and the Rotor project bringing a large part of the .net base framework to *nix I’d say MS is more than capable of supporting multiple platforms.
The fact that they do not support them more at this point is purely strategic for them.
javascript, php and ruby are the languages of the next ten years.
do javascript and php suck? yes, but they have huge installed bases
ruby has a killer app and only python will challenge it in the next decade
The dynamic languages always will suck. Without static type system the applications always will be slow and unstable. You can write a little bit less code, but with the dynamic typing the IDE’s code completition always will be less sophisticated. In long term (and with bigger projects) you will create your application with using more working hours then the tradidional languages and systems. And with .NET or Java you can use a very big class library. This class libraries are very well designed – oppisitely the most of FOSS class libs. And this class libraries are coherent, not 1000 little redundant, incompatible, buggy little library. And this libraries are precompiled, especially on .NET.
> This class libraries are very well designed
I remember using Swing a long time ago, its design is quite clean but it was also full of bugs and very slow at this time, which Sun took years to correct..
As for the dynamic vs static type debate, I remember seing that C# v3 will have locally inferenced types which should increase the readability.
I think that local type inference is an interesting middle ground: you get nearly the same type safety as static types with a much improved readability.
Hmm, Objective-C is nice on MacOS X, IMHO it does not “suck”.
This is so moronic I hardly know where to start. Do you know the difference between dynamic typing and strong typing? I can’t imagine why you think dynamic typing makes for slow and unstable apps. And what does code completion have to do with anything? Finally, I make a living writing .Net code and if you think the class libraries are especially well designed then it just reveals your naivety. They’re acceptable, but the collection classes in particular are unwieldy and limited. It’s no surprise that the roadmap for C# contains a lot of features inspired by Ruby, one of the most dynamic languages in common use.
I can’t imagine why you think dynamic typing makes for slow and unstable apps.
Speed: if you use dynamic typing, your app will check the type of your variable before every instruction.
if you execute the $a=$a+1 statament your app must check the type of $a before you can add to 1 to the $a.
Stability (PHP):
$a=2;
$b=”2 apple”;
$c=$a+$b;
print $c; /* 4 */
Python is better (send type error if you try a+b, but the a*b is valid, the result is 2 apple2 apple).
You can create unit tests, but IMHO better if this type of bugs filtered out in compile time.
Python is better (send type error if you try a+b, but the a*b is valid, the result is 2 apple2 apple).
You should probably read up on dynamic/static vs strong/weak typing. Python, for instance, is dynamic and strongly-typed. You might not declare types in the code, but an int is an int is an int. It is an int when it is first assigned, and it is an int throughout the program. Since the type of a variable is constant, it does not need to be checked “before every instruction”. Your a*b example is doing exactly what you are telling it to do, and has nothing to do with the type system. a+b is a type error, and flagged as such. So, given that python has the same runtime type safety as java and .Net, and does not have the performance penalties you ascribe to it, I ask again – how is it that dynamic languages necessarily result in slow and unstable apps? Before answering, you might want to consider that the type safety in C# and java can be trashed by upcasting to an object in one statement, then attempting to downcast to an incompatible object in the next statement. This will not be spotted at compile time, and so by your definition must make C# and java apps unstable. C++ and (especially) C are even easier to abuse.
Java != Javascript && Javascript != Java
Javascript was created by Netscape, Java on the other hand was created by Sun microsystems.
“JavaScript was first introduced and deployed in the Netscape browser version 2.0B3 in December of 1995. The choice of name proved to be a source of much confusion. There is no real relation between Java and JavaScript; their similarities are mostly in syntax (that is, both derived from C). Their semantics are quite different: notably, their object models are unrelated and largely incompatible.”
http://en.wikipedia.org/wiki/Javascript#Java.2C_JavaScript.2C_and_J…
Where did the original poster confuse Java and JavaScript? How would that even make sense, when he was clearly referring to it as a dynamic language?
Edited 2005-11-05 08:36
Java != Javascript and Javascript != Java have the same boolean value.
ie.
1 != 2 = T
2 != 1 = T
I stopped reading your post there
Kudos for being redundant
In an object oriented language if an object inherits from another than the one that inherits can be used in place of it’s parent class; however, depending on the language the opposite isn’t always true.
ex:
obj1 base class
obj2 extends obj1 // inheritence
void foo(obj1) can take obj2 instead, but
void bar(obj2) cannot always take obj1.
Therefore one can be the other but the opposite cannot always be true.
Oh, I agree with you, I am talking about in the since of boolean algebra though. With the operands being T/F values.
I suppose you’ll say that .NET is also better because of it’s higher version?
Last I checked .NET isn’t designed for a compile once, run anywhere. Please correct me if I’m wrong.
This I’d consider Java to be more suitable/desirable for me.
Because of that factor it might also be a reason why it’s a little slower. It has to figure out how that OS does things. This however can be solved by coding it into the JVM. Of course I have no backing to this just me thinking out loud at 1am.
Last I checked .NET isn’t designed for a compile once, run anywhere. Please correct me if I’m wrong.
This I’d consider Java to be more suitable/desirable for me.
Are you kidding? .NET was designed by MS from the beginning to be cross-platform from the ground up! MS has released versions of .NET (Rotor) for Mac OS X and at least one other OS. Then you have Mono, which takes .NET’s design onto virtually every other platform. I wrote cross-platform apps in .NET every day. It rocks!
.NET was designed by MS from the beginning to be cross-platform from the ground up!
Last time I heard Mono is not going to support System.EnterpriseServices…
Last time I heard Mono is not going to support System.EnterpriseServices…
This has nothing to do with the platform itself being designed to be OS/system neutral. There are many C++ libraries not available on other platforms, but it doesn’t mean you can’t create code that runs on all of them.
The fact is the CLI was developed to be platform neutral. .NET is one implementation of this standard. The MSR runtimes, Mono, and .GNU are other implementations of the same standards. .NET and Mono also include and/or have seperately distributed platform-specific libraries not unlike most other language distributions. These libraries in no way impede one’s ability to develop platform-neutral applications as the various CLI implementations have proven.
Only problem is Mono uses microsoft API’s that are not part of any standard. i.e. ASP.NET and ADO.NET are not part of the ECMA standard.
Heck, even on the Mono website, they are already figuring out ways to ‘get around’ the patent problems.
Java seems safer from a legal perspective.
Only problem is Mono uses microsoft API’s that are not part of any standard. i.e. ASP.NET and ADO.NET are not part of the ECMA standard.
And again, the availability of class libraries licensed only for specific OSes has nothing to do with the platform neutral design of the runtime architecture.
As you said, they aren’t part of the standard. This in no way impedes the development of cross-platform code, nor means the standard wasn’t designed with neutrality in mind. Practically all language platforms have libraries that won’t work across OSes, whether for technical reasons or only due to licensing. Whether a library is limited to a particular OS/architecture is up to the developer of that library.
Java seems safer from a legal perspective.
It is true, but just because the MS and the SUN agreed. But it is not concern to any other implementations (classpath+gcj, etc) and if SUN is out of business the legal perspective of java can be very interesting. But probably the IBM also can protect their java implementation.
This combined with SuSE getting the axe really makes it look like Uncle Bill is going to win again.
Microsoft vs. IBM… I kind of wish that IBM would just focus on hardware. They kick ass in that area. Software? Not so much.
SUSE, although a good distribution, is not the only Linux distribution out there.
Here we see .NET reversing the memory outlook and besting Java by a solid margin. This indicates that since .NET treats native types – such as double – as objects, that this incurs additional object overhead. Java, on the other hand, does not treat native types as objects and therefore can save on this overhead. However, when it comes to real-world object allocation, it appears that .NET is more efficient
What bull. .NET treats “native types” (more accurately put as value types) the same as Java. They are an int takes 4 bytes a double 8 bytes etc.
There’s no need for value types to have more overhead in .NET than Java an, indeeed, they don’t.
When a value type needs to act as an object (like when it’s used via an object or interface referfence), then and only then does it get boxed and take up more space. Normal regular calls directly on on the value type (such as Int32.ToString() etc don’t require boxing).
How can you trust an article where the author clearly doesn’t know what he’s talking about?
When a value type needs to act as an object (like when it’s used via an object or interface referfence), then and only then does it get boxed and take up more space.
Exactly. And of course Java does exactly the same, only you’ve got to do the boxing and unboxing manually, which is a pain in the ass.
And C# is actually more efficient when value types are combined with generics. In Java e.g. you have to manually box integers to put them into a List<Integer>. In C# you can put them into a List<int> directly, and the runtime system automatically creates a version of the List<T> code that is specialised for int’s and thus does not require any boxing at all.
>Exactly. And of course Java does exactly the same, only you’ve got to do the boxing and unboxing manually, which is a pain in the ass.
Not with J2SE 5.0, which does autoboxing.
Otherwise known as Java 1.5. Douchebag.
Before you open your yipper, why don’t you pull your head out of your ass to see if your mouth is just full off bullshit or not.
There’s another thing to note in there.
If the Java double type is 64-bit, that means each double takes 8 bytes of memory. With 5million doubles, that adds up to 40000000 bytes of memory, but the Java test shows 19.9MB which is ~half that.
What happens here? Does java compress the floats when they are not used, or is it that the JVM detects that they are not used, and so doesn’t keep all of it in memory?
The guy who made this test is a moron who don’t understand what he’s talking about, plain and simple.
I noticed that too… I’m thinking Java recognizes that he hasn’t done anything with his memory, he needs to be smarter than his vm and do something like:
char lst = new char[50000000];
for (int c; c < 50000000; ++c)
lst[c] = char(c%32 + 65);
That way, maybe he can trick the VM into thinking he’s using that RAM.
This benchmark is pretty flawed (missing details, missing options, missing code, etc.), and by the looks of it done by someone who doesn’t really have a good idea of whats going on.
I know these kinds of articles usually start a flamewar, and OSNews wants impressions – but there clearly is a lack of quality control in this case.
I know there are going to be a lot of people picking at this, in all fairness I do not care too much for either dotnet or java. I think they are great technologies they just don’t excite me.
However the neutrality of this test can be called into question as we are testing the microsofts dotnet implementation on their own platform meaning they have access to private API’s and the like that Sun does not have access to. Maybe this is not the case but we can’t really be sure of this now can we?
How about they run these tests again on a neutral platform such as FreeBSD, it would be rather interesting to see what sort of results we got then
However the neutrality of this test can be called into question as we are testing the microsofts dotnet implementation on their own platform meaning they have access to private API’s and the like that Sun does not have access to. Maybe this is not the case but we can’t really be sure of this now can we?
Which private API helps .NET during selection sort? Which private API helps .NET to take up less memory for an array of objects?
Do you really believe there’s a private API that makes the CPU run faster?
“Which private API helps .NET during selection sort? Which private API helps .NET to take up less memory for an array of objects?”
Couldn’t tell you really mate. I think the whole “private” part of it kind of gives me a reason not to know now doesn’t it…
“Do you really believe there’s a private API that makes the CPU run faster?”
I doubt it, if you find one be sure to let me know…I would it really appreciate it.
Whether or not they have a private API, Microsoft does have some advantage working off their native O/S.
{and yes, MS has in the past used undocumented apis to increase performance compared to rival products on their platform}
If .Net performed worse than Java on Windows, wouldn’t you be shocked?
How about we compare Mono performance vs. Kaffe?
“Whether or not they have a private API, Microsoft does have some advantage working off their native O/S.
{and yes, MS has in the past used undocumented apis to increase performance compared to rival products on their platform} ”
I love it everytime someone tries to use this argument. Sun themselves did a test with this in StarOffice, and what did they find? Performance gain with undocumented API = zero.
Certain APIs are undocumented because they WILL change in the future. Once MS publically documents an API, it’s very unlikely to change.
“How about we compare Mono performance vs. Kaffe?”
This has already been done, Mono won. IKVM on an old build of Mono (VM on VM, whee!) ran Java just as fast as Kaffe.
I’m guessing that’s because Sun still doesn’t make it all that easy for FreeBSD to even run Java 1.5, which isn’t a great selling point for the people using it.
That was quite possibly the lamest article I’ve ever read. Honestly, is this fluff worthy of being posted?
When I was 12 I benchmarked my commodore 64 by running a for loop. This ‘test’ seems to have the same level of sophistication. It’s naive, flawed and tells you the tester probably just finished the C# for dummies book and now thinks he’s ready to take on the world.
to read this article. As if either Java or .NET existed for performance reasons. .NET was not built to compete with Java in that respect, a platform comparison would be more adequate.
On the other hand I don´t think anyone could compare these two breeds to full extent.
It´s mostly a matter of personal preferences? If I were programming for Windows only, and if I hadn´t programmed in Java for so long, maybe I would consider .NET for my projects. IDEs are a real problem for .NET, you can get several powerful Java IDEs for free, and there are no counterparts in the .NET world. VS is a laugh for code-based work
Soon we will be comparing a knife to a fork.
As much as I like .NET, this ‘benchmark’ was absolutely stupid. We don’t even know HOW he arrived at those numbers. For all we know, the doubles didn’t do anything and didn’t get allocated. Or got GCed early, in which then we’d also need to see the running time. Etc.
We don’t know what sorting algorithm was used. How was it implemented? Without anything of value to actually study and draw conclusions from, this ‘benchmark’ has absolutely no value and smack in the face of regular .NET benchmarks that do provide full disclosure.
But the whole doubles as objects thing should have tipped off everyone that the clue is clueless.
“Last I checked .NET isn’t designed for a compile once, run anywhere. Please correct me if I’m wrong.
This I’d consider Java to be more suitable/desirable for me. ”
Tell that to the people who only have to maintain have to maintain 3 different codebases for 100% ‘Pure Java’ applications on Win, Linux and OSX. Or different builds for different appservers.
“Because of that factor it might also be a reason why it’s a little slower. It has to figure out how that OS does things.” No, it doesn’t. It did all the figuring out when you picked which version of the JRE to install.
The only thing this ‘benchmark’ proved was the Hotspot VM does not do any optimizations for long running processes, which is expected to anyone with any idea about runtime optimizations, contrary to the Hotspot’s proponents.
In short, this benchmark = stupid.
Actually you do know what algorithm was used: selection sort. Though knowing that is not strictly necessary so long as the same sorting algorithm is used in both cases and it is implemented in terms of the same functionality.
Your last meaningful paragraph doesn’t make any sense to me, and I’ve only glossed over the middle of the comment. The benchmark is stupid, and the absence of source code makes it even more useless than it clearly is from the exposition. The source code though sounds like it would take about six minutes to write from scratch if anyone really cares.
Knowing it was selection sort is about as useful as knowing another benchamrk implemented a shopping cart. For all we know, the implementation could be assbackwards.
The real value of benchmarks is being able to study the results and the implementation that lead to the results so you can appropriately plan the best way to do something. In-depth and detailed benchmarks are hard to come across because it takes a lot of time, effort and resources to conduct one, which is why they’re usually funded. Being a funded benchmark doesn’t lessen its validity as long as you get to inspect the details from full disclosure and understand what’s going on.
I vote for knife vs fork shootout. Fork is clearly superior to knife.
Is that your really weak manner of saying “I don’t know what I’m talking about, stop correcting me? I didn’t read the first paragraph.”
As long as it’s implemented in terms of precisely the same functionality on both platforms (and it clearly is since they just ‘ported’ the same code from one to the other) they are testing essentially the same aspects of the virtual machines. What would be much more interesting is the exact parameters used for conducting the benchmarking.
Providing the source code in this case, or even saying what algorithm is used is unimportant. It’s unimportant because a single test case is completely worthless for benchmarking. It affords absolutely no advantage here because the nature of the algorithm places most of the overhead in the effectiveness of the JIT compiler, and there aren’t enough tests being made to conclude anything about the VMs.
Oh, starting with the personal attacks already? Is this your really lame hypocrytical attempt to act holier than thou?
“As long as it’s implemented in terms of precisely the same functionality on both platforms (and it clearly is since they just ‘ported’ the same code from one to the other) they are testing essentially the same aspects of the virtual machines. What would be much more interesting is the exact parameters used for conducting the benchmarking.”
No. Firstly, .NET != Java. Secondly, no proof has been offered that it is just a simple port. .NET and Java are implemented differently, their VM behaviours will be different, and the class libraries will perform differently. .NET and Java are not equal. The algorithm used could have been specifically chosen to exploit weaknesses in Java. We don’t even know if the tests did anything with the doubles. The only thing we got is a digested number spitted out claiming “.NET is faster than Java”.
“Providing the source code in this case, or even saying what algorithm is used is unimportant. It’s unimportant because a single test case is completely worthless for benchmarking. It affords absolutely no advantage here because the nature of the algorithm places most of the overhead in the effectiveness of the JIT compiler, and there aren’t enough tests being made to conclude anything about the VMs.”
So now you’re saying the benchmark is worthless because it one uses a single test case, but it wouldn’t be if the VM paramaters were given out? So now the switches are be be all and end all? Well, Hello contradictory. I’m saying it’s worthless because it’s too simplistic and does not disclose the full details. You can make your benchmark as complex as you want, but it will still have little real value because your implementation could be retarded, or overly biased. You can learn something about performance even from biased benchmarks by studying the details.
If you can’t understand that, then there’s no point continuing this discussion with you. Your arguments strike as exactly the same kind of arguments Sun uses to avoid disclosing the details of their benchmarks, and the poor arguments made for ‘apples to apples’ comparisons which deliberately perform unnecessary ‘tweaks’ to the competing system causing poorer performance.
No question benchmark is plain dumb.
As far as this statement:
“3 different codebases for 100% ‘Pure Java’ applications on Win, Linux and OSX” … could you elaborate. People always throw around various complaints, without ever quantifying or at least qualifying their points.
In spirit of free discovery,
Someone who thinks benchmark is stupid.
P.S: Even though Java used to run billions of e-commerce transactions per year, with vulnerabilities counting in single digits over a period of 10 years, nothing is ever 100% perfect and no one ever writes the same code. Java provides a _specificiation_, which is free to be implemented so long as it complies with “the test suite”. No test suite is ever perfect.
I like the article. It is simple, well written and balanced. I would like to see a comparison between dynamic languages like Python and Ruby presented in a similar fashion. Usually, benchmark articles bore me, but this is an exception.
Edited 2005-11-05 09:09
Now a competive benchmark between mono and java 5, that would have been informative.
95% of all the Java i’ve written is not intended for the desktop, but middleware running on BSD of linux. No .NET there.
And most people using .NET are using it on Windows, so how does that make anything better?
This article has to be the absolute dumbest comparison of two platforms. The comparison has very little if any application to real life and no one would be stupid enough to base their platform choice and that bunch of garbage. On a more applicable note Java has it all over .NET and is pretty much a clear winner across the board as far as the real world is concerned (vendor support, applications, maturity, performance, etc.). The marketing hype from MS and their whores pretty much completely deflated by now and the promisses of stealing the market share from Java and making it irrelevant never materialized. Actually the gap between Java and .NET is widenning now — Java is still becoming more popular and .NET pretty much stopped growing. There are still many more Java jobs than .NET and they pay better, there are still many more vendors basing their products on Java, there is infinitely more open source stuff written in Java than in .NET crap — do the math and decide which side is superior. My take on it is .NET will never catch up with Java and will always be relegated to the realm of MS whores and ass kissers.
You never can compare a real word applications, because IMHO if you write a real word application on .NET and Java the code will be very different. The best thing what you can do the synthetic tests…
Yes, java at this moment very popular. But at this moment is nobody behind the Java, only the SUN. And the SUN is not too stable and ritch company. Yes, IBM, Oracle, etc supports Java, but this companies also supports .NET.
I mean come on, if .net is the absolute ruler except for native type memory efficiency is narrow minded to say the least. The evalutation and comparison was – to say the least – lacking and lame, not good for anything, let alone taking it as a base of comparison of .net/java. I’m not taking part of either, don’t get me wrong, I’m just saying this article proves nothing beyond the lack of any working quality control in article review.
And, as if all those books/articles on managed code taught us nothing: Who cares if you sacrifice half of your clock cycles if you can save 10% of your development costs.
Not to mention, managed languages have a tendency (especially Java) to try and push you into the more efficient ways of using the VM to get the job done.. Where as, in say c, you can do whatever you want and there’s no real push (although you likely won’t be declaring structs with functors in them).
I really don’t see the point in doing a benchmark of them. Maybe doing a benchmark of their graphics libraries, that might help out some aspiring 2d game programmers. Or, benchmark their 3d graphics libraries, maybe you’ll help an aspiring cross platform 3d game programmer.
> Yes, java at this moment very popular. But at this moment is nobody behind the Java, only the SUN. And the SUN is not too stable and ritch company. Yes, IBM, Oracle, etc supports Java, but this companies also supports .NET.
Ha ha, are you too stoned out of your freaking mind or are just stupid? Java is IBM’s biggest cash cow as far software is concerned. The entire WebSphere suite of products is built around Java (all middleware from IBM built on Java with Java in mind). Same thing applies to all middleware from Oracle. IBM and Oracle provide only *compatibility* with .NET, but do not provide a single product based on anything related to .NET. IBM and Oracle absolutely hate anything coming out of MS and hell will freeze over before either of them will touch .NET with a barge pole. As I said .NET will never catch up with Java simply because it is too damn proprietary and comes out of Microsoft, which nearly everyone hates.
> But it is not. There are many design bugs in java, and nobody can resolve this problems without breaking the compatibility. Java x.x never will better then .NET 1.1.
What the f*ck are you talking about? You wouldn’t know your own assh*le from a design bug in Java. Judging by your previous posts you have absolutely no passing clue about what the f*ck you’re talking, so you would better STFU or pull out something more concrete about the design flaws or whatever you’re trying to pull here. Less bullsh*t and more concrete evidence brother!
I don’t see any design flaws in Java compared with .NET and I thing you’re completely full of sh*t. Feel free to proove me wrong.
> But it is not. There are many design bugs in java, and nobody can resolve this problems without breaking the compatibility. Java x.x never will better then .NET 1.1.
What the f*ck are you talking about? You wouldn’t know your own assh*le from a design bug in Java. Judging by your previous posts you have absolutely no passing clue about what the f*ck you’re talking, so you would better STFU or pull out something more concrete about the design flaws or whatever you’re trying to pull here. Less bullsh*t and more concrete evidence brother!
I don’t see any design flaws in Java compared with .NET and I thing you’re completely full of sh*t. Feel free to proove me wrong.
Kiss your mother with that mouth??
No, but he does kiss yours. Apparently she likes it that way.
Let see: operator overloading, properties, databinding…
And this features are wasn’t a new things when java was born (let see Delphi, C++, etc). You can implement this features in java, but if you want to use them you must rewrite all existing class libraries.
Ps: Java is a really good thing. But IMHO .NET is simply better. Not because the coders of MS are genious, only because the .NET is younger and the MS developers was draw the conclusions from java.
So, another completly useless test.. Big deal!
In order to compare the two frameworks you’d need a very detailed case scenario and then compare building time, code performance etc. etc.
But even then you run into heaps of issues, like flexibility and extensability.
But a performance test could be useful, but this one was just utter BS. Like many of you already commented on; how can doubles require different amounts of memory?!
A Windows funded site doing a .Net propoganda piece and handily omitting to publish the sources is about as credible as Bush not invding Iraq for oil.
Just look any 95% of Enterprise deployments (Java)
When last did ANYONE “switch” to .Nyet? It’s dying, just like Microsoft and their Office revenue stream.
It’s theoretically not possible to do better than a dynamic JIT compilation for certain operations.
When I saw .NET for the first time, I predicted that this platform will beat JAVA and this is becoming true. .NET have many cool sort of features that JAVA don’t have. I was once a fan of JAVA but now I love .NET.
“Positive articles about Microsoft are just not true and are FUD. Nothing positive can come out of MS”
Signed your average Linux user aka Linux Zealot(notice I didn’t say open source user as Open source is way beyond linux and it has hijacked the term).
Open source the idea is great, the linux political movement–the world could do without.
With regard’s to java’s multiplatform capabilities, as long as your not using jni then chances are that your app will work with multiple platforms, I develop in-house mathematical applications in swing and I compile the system in my xp laptop and distribute it to different test machines, solaris, linux, windows. always the same result on all the machines. So as far as i’m concerned java’s multiplatform ability is there
it’s guess guess why .net beats java in the selection sort test. Autoboxing in Java isn’t done in an especially efficient way – primitive types are wrapped into the heavyweight Numeric classes dynamically. I’d be willing to be the person conducting this benchmark used a Collection to store the doubles being sorted. This would result in extremely excessive overhead which could be easily dismissed by using a primitive array. So, whilst one could say that this is a flaw in Java 5, it’s one that is easy to avoid.
As someone who has to write code in both languages, my experiences are that Java is faster, has better GC performance and has a better designed class library. One need only look at the .net collection classes to realise this. (This is an anecdote, so treat it as such).
So .NET runs on native platform, and java runs on all platforms… where are you compairing this point? The portability is sure to compromise some performance… You should actually compair .NET with similiar product which is native!
For me this was the stupid comparision. Looking at the flaws .NET has one will mostly never dare to implement .NET product on server side.. where java can reach .NET can only dream of!
In past MS tried to port hotmail on NT and failed miserabiliy and had to go back to Sun Solaris os for hotmail. The basic purpose of buying hotmail failed! They wanted to prove that NT is as stable as Solaris. For me this comparision is similar to that. MS should write products that run on desktop… not on servers. Servers are special… not their type of food!
one of the worst benchmarks i have ever seen. No code, tuned for the strength of one platform (primitive collections and storing Strings as ASCII if possible..) i laugh people who takes this test serious.
I thought I’d missed a few more pages. Three, very poor, tests? I’m surprised the author didn’t chuck in some code to print and time “Hello World” a few thousand times on the screen as well.
I don’t know which is more embarrassing: the author thinking this was some kind of useful comparison test, or OSNews for posting such a sorry excuse for an article.
A real test would be Mono vs. Java (or if MS ever ported something to non-Windows).
.NET can be optimized for a single operating system, while Java has to be able to have an average performance on many, many operating systems.
.NET is Windows native code that just so happens to be running in a VM for the purpose of garbage collection and security. Big difference from the purpose of Java.
Let’s see the performance of Java on a UNIX/Linux box, eh?
Oh! You mean Java runs slower than .NET on Windows?! How suprising?
Of course, if I want to write code only for Windows, I’ll use .NET or Win32. For cross-platform, I’ll use Java!
Look at the industry, whenever you talk about simple web applications, you may use .NET for faster development by using some drag & drop skills(in fact you can do that with many Java IDEs too!), but when it comes down to enterprise business, it is all about multi-tiers and you are not telling me that you are using .NET for middleware?! A non-multi-platform, slow and messy piece of junk as middleware and you have to expect bluescreen for your middle-tier?! wake up guys! Uncle bill’s bluescreen in your middleware production server?!
Java beat the snot out of .Net, 100 to 0.
This is pointless. I shouldn’t even be writing this post, but i couldn’t let it go this time.
Compare all you want, but in the end the practical value of Java vs .NET, will differ in diffrent situations. And i’m pretty sure the “fittest” language will survive in the end. Or the “best” if you will.
“My dad is better than yours”. “Gnome is better than KDE”.
Please, go ahead and always look at something from only ONE perspective. Be narrow minded.
These tests are clearly biased towards .NET. Let’s see how:
The machine where they did the testing has 2Gb of RAM, which puts the JVM in “server mode” automatically. In server mode, the JVM optimizes for long run execution, meaning that the startup penalty will be bigger in exchange for a potential increase in performance as the execution progresses. This works great for server applications, as they tend to have more predictable execution patterns and end up being faster than native code sometimes, due to run-time optimizations.
.NET, on the other hand, seems to be targeted (and used) mainly on the client side, where startup time matters. For this reason, and because of the JVM’s server mode thing, .NET wins hands down in these tests.
For an unbiased benchmark the tests should have been longer (minutes or hours) to reduce the influence of startup overhead.
Just to back my claims up, in Server-Class Machine Detection [ http://java.sun.com/j2se/1.5.0/docs/guide/vm/server-class.html ] one can read:
“For J2SE 5.0, the definition of a server-class machine is one with at least 2 CPUs and at least 2GB of physical memory.” — The test machine is a Pentium D (dual-core) with 2Gb of RAM.
“The aim is to improve performance even if no one configures the VM to reflect the application it’s running. In general, the server VM starts up more slowly than the client VM, but over time runs more quickly.” — This alone debunks these benchmarks.
It’s been proven that the Hotspot does not actually do any additional optimizations for long running processes. It has different allocation and GC strategies, but JIT optimizations are zilch.
Furthermore, if you paid any attention to the ‘benchmark’, as stupid as it is, you’d see that the server VM must definitely start in less 375ms if the times took it into account, which certainly doesn’t account for the 10000+ ms different in the second result, shooting your argument full of holes. Additionally, the graphs are proportionate, which suggests that the time taken was recorded via APIs, which in case you didn’t notice, requires the VM to start first. Now, anyone can tell you that even on a 20 processor E20K, the server VM does not start in less than 375ms. Let’s not even get started about the other dozen switches needed by Java to not make it look like a total laggard in properly conducted benchmarks. Switches that only help in those specific tasks and configurations.
This benchmark may be stupid and pointless, but the Java camp trying to defend Java with broken arguments is even worse.
“It’s been proven that the Hotspot does not actually do any additional optimizations for long running processes. It has different allocation and GC strategies, but JIT optimizations are zilch.”
Is it? Then point us all to some “proof”, otherwise we will assume you are just taking that out of your ass.
The startup overhead isn’t just about the startup time of the VM itself, it is about how aggressively the VM JIT compiles the code after it starts running.
If you are trying to benefit long running applications you don’t start doing JIT compiling to the code right away, you collect statistics first to see what code runs frequently enough to be compiled without the JIT overhead becoming larger than the interpretation overhead of the bytecodes themeselves.
.NET, on the other hand, seems to be targeted (and used) mainly on the client side, where startup time matters. For this reason, and because of the JVM’s server mode thing, .NET wins hands down in these tests.
.NET also has a server mode. One way of setting it is to make an app.config file (text file named appname.exe.config in the same directory as the app executable) with the following declaration:
<Configuration>
<runtime>
<gcServer enabled=“true“ />
</runtime>
</Configuration>
The truth is, no one really cares about benchmarks like this (i.e. my language can do a gazillion loops half a second faster than your language). Does the author really think that a business stakeholder in a project cares about stuff like that? What matters is time to market, return on investment, etc…that kind of stuff is nearly impossible to “benchmark.” A real world test would be to build a large, distributed application and see:
A) Who could build it the quickest
B) Number of bugs per (n) lines of code
C) Maximum sustained number of concurrent users
D) Maximum sustained transactions/second
E) Cost of hardware to support the system at maximum throughput
The problem with this is that MS won’t allow benchmarks like this with their languages/platforms unless they approve it, which of course means the results will always be in MS’s favor…so we’ll probably never really know the truth. In the end, it’s whatever tool is best for the project at hand…and it’s the stakeholders in businesses that drive projects, NOT the IT dept.
The problem with this is that MS won’t allow benchmarks like this with their languages/platforms unless they approve it, which of course means the results will always be in MS’s favor…so we’ll probably never really know the truth.
MS does allow such benchmarks (excerpt from .NET 2.0 RTM EULA located in %systemroot%Microsoft.NET[backslash]Framework[backslash]v2.0.5 0727[backslash]Microsoft .NET Framework 2.0)
2. MICROSOFT .NET FRAMEWORK BENCHMARK TESTING. This supplement includes the .NET Framework component of the Windows operating systems (“.NET Component”). You may conduct internal benchmark testing of the .NET Component. You may disclose the results of any benchmark test of the .NET Component, provided that you comply with the following terms: (1) you must disclose all the information necessary for replication of the tests, including complete and accurate details of your benchmark testing methodology, the test scripts/cases, tuning parameters applied, hardware and software platforms tested, the name and version number of any third party testing tool used to conduct the testing, and complete source code for the benchmark suite/harness that is developed by or for you and used to test both the .NET Component and the competing implementation(s); (2) you must disclose the date (s) that you conducted the benchmark tests, along with specific version information for all Microsoft software products tested, including the .NET Component; (3) your benchmark testing was performed using all performance tuning and best practice guidance set forth in the product documentation and/or on Microsoft’s support web sites, and uses the latest updates, patches and fixes available for the .NET Component and the relevant Microsoft operating system; (4) it shall be sufficient if you make the disclosures provided for above at a publicly available location such as a website, so long as every public disclosure of the results of your benchmark test expressly identifies the public site containing all required disclosures; and (5) nothing in this provision shall be deemed to waive any other right that you may have to conduct benchmark testing. The foregoing obligations shall not apply to your disclosure of the results of any customized benchmark test of the .NET Component, whereby such disclosure is made under confidentiality in conjunction with a bid request by a prospective customer, such customer’s application(s) are specifically tested and the results are only disclosed to such specific customer. Notwithstanding any other agreement you may have with Microsoft, if you disclose such benchmark test results, Microsoft shall have the right to disclose the results of benchmark tests it conducts of your products that compete with the .NET Component, provided it complies with the same conditions above.
Edited 2005-11-06 08:42
These tests were very narrowly defined, covering only specific data types. What happens in each test if you swap out data types (double for long, char[] for String, etc)?
I’m not disputing the finding that .Net is more memory efficient than Java — that may very well be true. I’m disputing whether this test is sufficient to show that.
Well apart all above, where is the test?
I see no any. I see someone clames .NET is better than Java. Well I think .NOT so what? Do you wish I make some graph where Java is 10^10 times faster? I mean in any benchmark we have to ask WHAT are we testing? In what conditions? I even haven’t seen the java -XX parameters… I think for the raw data they have to be at least comparable (with java a little better ;-)). I still like to see some tests of .net on some solaris with 16 micros and 128Gb 🙂 …or linux at least :-).
.Net has failed. It’s almost official.
Cheers
Let me quote from one of them:
“Thou Shalt Render Software and Religion Asunder”
Just have all the programmers vote on the language to use and use it. If some company guideline forces you to force the programmers to use a language wich didn’t win or was a close second in the vote, quit your job.
I wrote a scripting engine in .NET and the EXACT EXE and DLL files were copied to my linux box and executed under mono perfectly.
Mono also has a beta Windows.Forms implementation so even GUI stuff should work now.
Far superior to java in my opinion.
Neither .Net nor Java is a panacea. Anyone who claims either is far superior to the other is an idiot.
I prefer Java, but I, unlike the idiot .Zealots and JZealots, recognize that it is a tool. Use whatever tool works for you, but don’t try to be all high and mighty claiming .Net is far superior to Java, or vice versa.
.Net supports far more languages. Java has a number of languages that target the JVM, but .Net has far more. Java isn’t good enough to hold .Net’s jock strap in this regard.
Java supports far more platforms. Mono is a good start, but it doesn’t have the currency or platform coverage to even think about holding Java’s jock strap in this area.
Why are you wasting your time reading this article? Did you think that .net wouldn’t run faster on windows? Better yet, why is anyone wasting there time with .net? If you want speed under windows write c or c++ with their libs. Why are you wasting your time with .net? .Net only runs under windows so does c and c++ compiled with MS tools. Why did you think that Java would be faster in windows when it’s designed to run on many platforms. How about a .net comparison with Java in linux? Oops, Java won that one since .net runs at 0 speed in linux. Everyone writing .NEt must be an idiot. They should be coding in c or c++. If you use Java, you are writing for write once run anywhere not for blazing speed.
Does no one get this? If I want to write something once and it doesn’t have to run blazing fast. I write in Java. If I want to run something super fast, I code in c and assembler with the native platform tools. Could someone tell me why I would want to use a language that only runs on 1 platform that is slower then other languages available for the same platform?
Java has a purpose, what the hell is .net for? Why should a business care about .net?
Did they use the -server Java VM or the default client?
I know this makes a HUGE difference in performance for some benchmarks.
Does anyone know?
I would like to try these tests using the following VM’s:
1) IBM JDK 1.4.2 (or 1.5)
2) JRocket 1.5
3) Sun using 1.5.0_05 under both -client and -server modes
4) Sun 1.6 ea (for kicks…I know this is not the current VM)
I bet if the correct VM tuning params were used, the numbers would be different. Also, I have found some performance differences when running (same box) on SuSE Linux 10.
Oh, by the way, who’s VM did they use in the test? Suns? IBM’s? JRockets?
Also, did they set the initial and maximum heap size? This can have a huge impact on performance. I have found that for short benchmarks, setting the initial heap size made a significant difference.
Where can you get the source code and VM params used?
If the author can not provide these, then I consider these tests purely bogus.
why do so many messages get modded down here? it’s not like someone critiscizing java in a non-flame like manner deserves it.
there wont be any complications, because Java is dominated by Sun, but not owned by Sun anymore, Java is controlled by the JCP a consortium, if Sun goes down, someone else (very likely IBM) will take over as the dominant part in the JCP.
Judging by the lack of information about command line switches it is safe to assume he
didn’t know to use the -server switch
This is a classic mistake made by newbies trying to benchmark Java.
The default Java (client) compiler lacks the advanced optimization provided by the server compiler. Using SciMark the Java server compiler is much faster than .NET 2.0. Using the client compiler Java is slower than .NET. Classic newbie stuff.
Tiered compilation is being worked on (either for JDK 6 or JDK 7), which will result in the client and server compilers being combined, with the result that client applications will have access to all the optimisations provided by server compiler.
The C# and Java versions of SciMark are freely available on the net if anyone wants to compare Java (just use the -server switch!) and .NET
I’m afraid this benchmark lacks sufficient substance to have been posted on the front page.