Benchmarks Archive

‘The Ultimate Lab Test’?

PopularMechanics has performed three types of benchmarks on Apple and 'ordinary' PC machines, with the former running Mac OS X Leopard, and the latter Windows Vista. The first type of benchmark consisted of users giving ratings to things like design, ergonomics, web browsing experience, and so on. The second benchmark focused on real-world performance (launching applications, boot/shutdown times, and so on), while the third and final benchmark consisted of Geekbench and Cinebench runs. PM concludes: "The results gave us a clear winner in the performance categories, but the big surprise was how little difference we found in user preferences. Turns out, both platforms are capable and easy to use, but only one was the victor." Even before I got to the results, I noticed a whole set of problems with the benchmarks performed in this article, that would seriously skew the results.

Performance: Vista 32-bit vs. Vista 64-bit

"Testing 64-bit performance is still a somewhat dicey proposition. Major benchmarks are either lacking, or don't work properly. For example, SYSmark 2007 simply doesn't run on a 64-bit OS (Vista or XP). And while there's now a 3ds Max 9 SPECapc benchmark, the benchmark crashes consistently with a scripting error before it completes when running on 3ds Max 9 64-bit under Vista 64-bit. On the other hand, there are more 64-bit applications and benchmarks now. That system-sapping game, Crysis, ships with a 64-bit client. 3ds Max 9, Lightwave 9, POV-Ray, and the Cinebench rendering benchmark all have 64-bit versions. Futuremark's PCMark Vantage offers a 64-bit version of that Vista-centric, synthetic test. On top of that, anyone using 64-bit Vista will still be running a lot of 32-bit applications. So we benchmarked some of those as well. Let's take a look at the benchmarks and test system."

Benchmark: FreeBSD 7.0, Dragonfly BSD 1.12, More

"In May 2007 I ran some benchmarks of Dragonfly 1.8 to evaluate progress of its SMP implementation, which was the original focus of the project when it launched in 2003 and is still widely believed to be an area in which they had made concrete progress. This was part of a larger cross-OS multiprocessor performance evaluation comparing improvements in FreeBSD to Linux, NetBSD and other operating systems. The 2007 results showed essentially no performance increase from multiple processors on dragonfly 1.8, in contrast to the performance of FreeBSD 7.0 which scaled to 8 CPUs on the benchmark. Recently Dragonfly 1.12 was released, and the question was raised on the dragonfly-users mailing list of how well the OS performs after a further year of development. I performed several benchmarks to study this question."

Becnhmark: New Mac Pro

"Earlier this week Apple released updated Mac Pros that use Intel's new Penryn processors. Also new is the fact that the standard Mac Pro configuration now comes with eight (instead of four) cores. Of course, what I've been wondering (as I sit here and think about getting a new Mac Pro) is how does the new standard eight-core Mac Pro perform compared to the old high-end Mac Pro? I've gathered Geekbench 2 results for both Mac Pros to find out."

Theora vs. h.264

A lot was said lately about the Vorbis/Theora vs h.264/AAC situation on the draft of the HTML5. As some of you know, video is my main hobby these days (I care not about operating systems anymore), so I have gain some experience on the field lately, and at the same time this has made me more demanding about video quality. Read on for a head to head test: OGG Theora/Vorbis vs MP4 h.264/AAC. Yup, with videos. And pictures.

‘What Intel Giveth, Microsoft Taketh Away’

"'What Intel giveth, Microsoft taketh away'. Such has been the conventional wisdom surrounding the Windows/Intel duopoly since the early days of Windows 95. In practical terms, it means that performance advancements on the hardware side are quickly consumed by the ever-increasing complexity of the Windows/Office code base. Case in point: Microsoft Office 2007 which, when deployed on Windows Vista, consumes over 12x as much memory and nearly 3x as much processing power as the version that graced PCs just 7 short years ago (Office 2000). But despite years of real-world experience with both sides of the duopoly, few organizations have taken the time to directly quantify what my colleagues and I at Intel used to call 'The Great Moore's Law Compensator'. In fact, the hard numbers below represent what is perhaps the first ever attempt to accurately measure the evolution of the Windows/Office platform in terms of real-world hardware system requirements and resource consumption."

Ubuntu 7.10 with WINE vs. Windows XP

"This week's release of Ubuntu 7.10 is a significant win for the free software community. Not only does this release incorporate an updated package set - most notably with the Linux 2.6.22 kernel and GNOME 2.20, but it also delivers on new desktop innovations from BulletProofX and displayconfig-gtk to Compiz Fusion being enabled by default on supported systems. However, for those business professionals and gamers that remain dependent on some Windows-only binary applications, the WINE project has been making some excellent headway into supporting Windows applications on the Linux desktop. With Ubuntu 7.10 and WINE 0.9.46 in hand, we had set out to compare the performance between Windows XP and Gutsy Gibbon with WINE on two popular DirectX benchmarks."

Serverwide Performance Benchmarking

"Before installing a new shared Linux database- and webserver I wanted to get an idea of the real world performance of various filesystems combined with the noatime and atime flags. This article contains a brief analysis of the results of my testing. To get a good idea of the relative impact of the chosen filesystem I also tuned the amount of shared buffers used in the PostgreSQL database software, which powers a number of sites on the server. On top of that, I also took a look at the benefit of using APC, the Advanced PHP Cache."

Comprehensive Linux Terminal Speed Comparison

"Linux has an abundance of excellent terminal applications. Interestingly, I could not find any decent comparison of their text display performance. Since I use the command line a lot, I want text output that is as fast as possible. When you compile a large project, you don’t want the console output to be the limiting factor. I took the burden on me to do a comprehensive comparison of the text through of all possible terminals."

OPIUM: Optimal Package Install/Uninstall Manager

"We have developed a new package-management tool, called Opium, that improves on current tools in two ways: Opium is complete, in that if there is a solution, Opium is guaranteed to find it, and Opium can optimize a user-provided objective function, which could for example state that smaller packages should be preferred over larger ones. We performed a comparative study of our tool against Debian's apt-get on 600 traces of real-world package installations. We show that Opium runs fast enough to be usable, and that its completeness and optimality guarantees provide concrete benefits to end users."

Fedora vs. Ubuntu: a Performance Look

"We have compared the 32-bit and 64-bit performance of Ubuntu and started a performance comparison of Ubuntu/Kubuntu/Xubuntu, but how does the performance of the upcoming Feisty Fawn release compare to that of Fedora 7? In this article we have enclosed benchmarks from Fedora Core 6, Ubuntu 6.10 Edgy Eft, Fedora 7 Test 2, and Ubuntu Feisty Fawn Herd 5. In gaming and desktop benchmarks, which of these Linux distributions is faster?"

Comparison of Desktop Indexers

"A number of search engines are available for the Gnome and KDE desktop environments, many based around the open source Lucene search engine. It would be tremendous if we could adopt one of these search engines for the Gnome platform, so we can provide the type of integrated search experience for our users that they really need, irrespective of which distort they are using. So to help in this assessment we have carried out a comparison of four different Unix based indexers ."

Performance Comparison of Most Current CPUs

Most current Intel and AMD CPUs, as well as almost every Macintosh model from the last few years, geekbenched and compared. Interesting results if you just want to have an overall generic idea of how fast is what. Interesting to see that my 12" 867 Mhz Powerbook G4 is at least 6 times slower than a new MacBook Pro or that new Core2Duo CPUs at 2 GHz are way faster than my pretty recent 3 GHz P4 (which came with hyperthreading and 64bit support nonetheless). Technology flies fast past you!

Benchmarks: the Unofficial Eight-Core Mac Pro

"We might be ahead of Apple's product release cycle, and we've probably violated our Mac Pro's warranty, but we just had to see what the Apple Mac Pro could do when populated with a pair of Intel's brand-new quad-core Xeon 5355 processors," Daniel A. Begun reports for CNET. They conclude: "Unless you do work normally relegated to high-end workstations, perform massively multitasking workloads, or just want the bragging rights, eight cores is definitely overkill - at least for now."

MacBook Pro (Core 2 Duo) Performance Comparison

Geek Patrol has benchmarked the new (Core 2 Duo) MacBook Pro against the old (Core Duo) model. "Moving from the Core Duo to the Core 2 Duo means 32-bit MacBook Pro performance is up 10% without an increase in processor clock speed. That’s impressive! Even more impressive is the Core 2 Duo is 20% faster (when executing 64-bit code) than the Core Duo (when executing 32-bit code)."