With the release of Ubuntu 8.10 only a few days away, Phoronix decided to take a look at the performance figures over the past releases – from Ubuntu 7.04 to Ubuntu 8.10. Phoronix used its own extensive test suite on fresh installations, with the same parameters, on the identical hardware. The results are rather surprising. Update: I’ve added some more information about this, gathered from the Ubuntu mailing list. You can find it in the ‘read more’.
The Phoronix test suite is rather extensive, as the website describes.
The tests we used had included the BYTE Unix Benchmark, SciMark 2.0, SQLite, Tandem XML, eSpeak Speech Engine, timed Apache compilation, timed PHP compilation, timed ImageMagick compilation, Bonnie++, Flexible IO Tester, GnuPG, OpenSSL, LAME MP3 encoding, Ogg encoding, FLAC encoding, WavPack encoding, FFmpeg encoding, OpenArena, World of Padman, Unreal Tournament 2004 Demo, GtkPerf, Bork File Encrypter, Java SciMark 2.0, and RAMspeed.
Almost all the tests show a worrying pattern: over time, Ubuntu is getting slower. The Phoronix guys were surprised by this, so they re-ran their tests to rule out the possibility of a fluke – even though the tests run autonomously and multiple times each – but the results were consistent. “Major slowdowns after Ubuntu 7.04 ‘Feisty Fawn’ in so many different tests certainly weren’t what we had expected.”
While the benchmarks offer an intriguing insight, it’s important to note that each new release also offered new features which might affect performance. In addition, it’s a more or less grudgingly accepted rule that as time goes by, software gets fatter, slower, and more complicated, and that you need ever faster hardware in order to keep up. It appears that Ubuntu isn’t exempt from this – almost – universal constant.
Update: The news of this story hit the ubuntu-devel mailing list. It appears that the people on the mailing list are glad that Phoronix did not point fingers towards Ubuntu itself, but rather tried to look at the components that make up Ubuntu. The developers are already trying to see which major changes could account for some of the performance drops. Phoronix itself has stated that they are planning on running the same test suite on other distributions and hardware as well – Fedora’s results might be up by the end of the week.
This thread is worth a read:
Phoronix didn’t do a very good job, which is why I personally continue not to trust them as a source