Linked by Thom Holwerda on Thu 3rd Oct 2013 16:07 UTC
Benchmarks

With the exception of Apple and Motorola, literally every single OEM we've worked with ships (or has shipped) at least one device that runs this silly CPU optimization. It's possible that older Motorola devices might've done the same thing, but none of the newer devices we have on hand exhibited the behavior. It's a systemic problem that seems to have surfaced over the last two years, and one that extends far beyond Samsung.

Pathetic, but this has been going on in the wider industry for as long as I can remember - graphics chip makers come to mind, for instance. Still, this is clearly scumbag behaviour designed to mislead consumers.

On the other hand, if you buy a phone based on silly artificial benchmark scores, you deserve to be cheated.

Thread beginning with comment 573798
To read all comments associated with this story, please click here.
Deserve to be cheated?
by RshPL on Thu 3rd Oct 2013 16:20 UTC
RshPL
Member since:
2009-03-13

Why is that? If one wants to have the smoothest web experience it would be perfectly reasonable to look at these particular benchmarks. I am rarely looking at benchmark results myself but isn't it a bit unfair to judge people that do?

Edited 2013-10-03 16:24 UTC

Reply Score: 6

Bill Shooter of Bul Member since:
2006-07-14

Well, first of all, no one "deserves to be cheated". That's just a lie con men tell themselves to help them sleep at night.

But seriously, benchmarks aren't very accurate to real life scenarios on phones. I don't understand why anyone would take the results very seriously. I think this is what Thom is trying to say. Anyone who took them seriously has already been conned into thinking they were useful.

Reply Parent Score: 10

RE: Deserve to be cheated?
by Alfman on Thu 3rd Oct 2013 17:31 in reply to "Deserve to be cheated?"
Alfman Member since:
2011-01-28

RshPL,

Agreed. I don't really do much with phones, however I do consider benchmarks when it comes to buying personal computer components and I have to wonder the extent of cheating that goes on there.

"On the other hand, if you buy a phone based on silly artificial benchmark scores, you deserve to be cheated."

-1 for this Thom.

Benchmarks can certainly can be valuable for people who understand what they mean and how it relates to their needs. However if the results are being manipulated into producing unfair results then it completely defeats the point in having them for comparison. The shame should be squarely on those artificially manipulating the results, not those who were mislead into believing those results were genuine.


As an aside: if it's easy (and safe) to control the performance characteristics of the chips per application, then perhaps this functionality could be exposed as a bona fide feature for boosting performance in general (presumably at the expense of battery life). This way it would not be misleading, and the users could turn on and off the performance boost as they deem fit.

Edited 2013-10-03 17:46 UTC

Reply Parent Score: 3

RE: Deserve to be cheated?
by lucas_maximus on Thu 3rd Oct 2013 21:15 in reply to "Deserve to be cheated?"
lucas_maximus Member since:
2009-08-18

I think the issue is that they disabled the thermal limits to bench better, which isn't going to reflecting real world usage.

Reply Parent Score: 4