Linked by Christopher W. Cowell-Shah on Thu 8th Jan 2004 19:33 UTC
General Development This article discusses a small-scale benchmark test run on nine modern computer languages or variants: Java 1.3.1, Java 1.4.2, C compiled with gcc 3.3.1, Python 2.3.2, Python compiled with Psyco 1.1.1, and the four languages supported by Microsoft's Visual Studio .NET 2003 development environment: Visual Basic, Visual C#, Visual C++, and Visual J#. The benchmark tests arithmetic and trigonometric functions using a variety of data types, and also tests simple file I/O. All tests took place on a Pentium 4-based computer running Windows XP. Update: Delphi version of the benchmark here.
Permalink for comment
To read all comments associated with this story, please click here.
Please post your sources
by Ben Maurer on Thu 8th Jan 2004 20:05 UTC


Given the large differences between and C#, it is very likely you are doing something wrong. You may be mistakenly using a different construct. The `native' VB IO functions may be much slower than the standard CLR classes (System.IO). If you are using the Visual Basic library, you are not fairly testing the language. Again, you must post your source code to allow independant review.

As well, I would love to run the C# tests on Mono.

Another thing I should point out, most applications *DO NOT* involve intensive IO or Math alone. This is not a measure of true application performance. You are mearly measuring how well the JIT or compiler emits code for a specific case. I am sure any of the JIT developer could optimize for this specific test case. I think prehaps the more interesting view you could take is `what language provides highspeed building blocks -- such as collections classes, callback functionality, and object creation.' The answer to this question is *MUCH* less of a micro-benchmark.

Also, I would add, for JIT'd languages, you should call the function you are requesting once before you do the call that you time. Depending on how you structure your run, you may end up counting JIT time. Although JIT time can matter in a 60 second benchmark, when running a web server for days, weeks or even months at a time it really does not matter. In fact, many applications use a native code precompiler to reduce startup time (under Mono, Miguel de Icaza often reports performance improvements of over 30% by using AOT compilation of our C# compiler mcs.exe [times are for the compilation of our mscorlib library, consisting of 1000 C# source files]). However, AOT does loose out in a large bench mark like this because it is forced to generate sub-optimal code (like a C++ compiler). So, it is much more fair to allow for a warm up runt to allow the runtime to JIT the code.

-- Ben