Linked by Christopher W. Cowell-Shah on Thu 8th Jan 2004 19:33 UTC
General Development This article discusses a small-scale benchmark test run on nine modern computer languages or variants: Java 1.3.1, Java 1.4.2, C compiled with gcc 3.3.1, Python 2.3.2, Python compiled with Psyco 1.1.1, and the four languages supported by Microsoft's Visual Studio .NET 2003 development environment: Visual Basic, Visual C#, Visual C++, and Visual J#. The benchmark tests arithmetic and trigonometric functions using a variety of data types, and also tests simple file I/O. All tests took place on a Pentium 4-based computer running Windows XP. Update: Delphi version of the benchmark here.
Permalink for comment
To read all comments associated with this story, please click here.
Re: Rayiner Hashem
by Bascule on Thu 8th Jan 2004 22:59 UTC

Numerics are really the best case for an optimizer, because they are so low level. All the JIT compilers should have compiled the loop once to native code, and gotten out of the way. This is fine if all you are doing is inner-loop numeric code (some scientific stuff, graphics) but not really a good indicator of general performance.

Yes, this benchmark could really give people the wrong idea about Java. Obviously HotSpot is doing its job, and it performs comparable to native code.

Even for scientific code, this benchmark probably isn't representative, because you often need proper mathematical semantics for your calculations, which C/C++/Java/C# don't provide.

Fortran is a wonderful language for the scientific community, not only for its language semantics but also its optimization potential. While this potential is not fully realized on most platforms (the Alpha is the only ISA where the Fortran compiler has been optimized to the point that for scientific computing Fortran is the clear performance winner over C) Fortran does have a distinct advantage in that many mathematical operations which work as library functions in languages like C (i.e. exponents, imaginary numbers) are part of the language syntax in Fortran, and thus complex mathematical expressions involving things like exponents can be highly optimized, as opposed to C where a function invocation is required to perform exponentation. Algorithms for doing things like modular exponents can be applied to cases where they are found in the language syntax and be applied to the code at compile time, whereas C requires programmers to implement these sort of optimizations themselves. With more and more processors getting vector units, a language which allows such units to be used effectively really is in order.

Java really dropped the ball on mathematical code. C at least has a rationale behind why things like exp() and log() are functions rather than part of the language syntax. C is designed to be a language with a relatively simple mapping between language syntax and processor features. Java/.NET could have made exponentation a language feature rather than a library function... after all, they certainly aren't bound by the limitations of processors. Instead we find these sorts of things in java.lang.Math and System.Math because they are clinging to C/C++'s legacy rather than thinking about the rationale behind the C language syntax and how the syntax could be better designed when a simple mapping between processor features and language syntax isn't required.

Lack of operator overloading is the biggest drawback to mathetmatical code in Java. Complex mathematical expressions, which are hard enough to read with conventional syntax, become completely indecipherable when method calls are used in their stead. I had the unfortunate expreience of working on a Java application to process the output of our atmospheric model, which is an experience I would never like to repeat. Working with a number of former Fortran programmers, everyone grew quickly disgusted with the difficulty of analyzing matrix math as method calls, and were quite amazed when I told them that with C++ operator overloading such code could be written with conventional mathematical syntax (although there are some issues differentiating dot products from cross products, but it's still much less ugly than method invocations)

A more telling test would be to get higher-level language features involved. Test virtual C++ function calls vs Java method calls (which are automatically virtual).

Java will almost certainly win on the speed of virtual methods because it can inline them at run-time. Again, the solution in C++ is not to use virtual methods within performace critical portions of the code, especially within large loops, and the simple solution is to replace such uses with templates where applicable.