How Many Android OEMs Cheat Benchmark Scores? Pretty Much All of Them 189
An anonymous reader writes "After Samsung got caught out cheating on benchmarks (Note 3, Galaxy S4) AnandTech has done a detailed analysis of the state of benchmark cheating amongst Android OEMs. With the exception of Motorola, literally every single OEM they've looked at ships (or has shipped) at least one device that does benchmark-specific CPU optimizations. AnandTech also thinks it will get worse before it gets better. 'The hilarious part of all of this is we’re still talking about small gains in performance. The impact on our CPU tests is 0 - 5%, and somewhere south of 10% on our GPU benchmarks as far as we can tell. I can't stress enough that it would be far less painful for the OEMs to just stop this nonsense and instead demand better performance/power efficiency from their silicon vendors.' The article notes that Apple doesn't do any of the frequency gaming stuff."
Or, alternatively (Score:5, Insightful)
Re:And Apple (Score:3, Insightful)
I fear Motorola is dead.
The Moto X was supposed to be a device with high amounts of post Google input. It is a high priced mediocre mid range device. They still cater to carriers unlike Apple. They need to compete with the 5S, One X and S4. Instead they are competing with last years product.
Moto needs to copy Apple in some things. Make 1 main device, sell the old one or a cheaper version as well. Max 2-3 devices. Refresh them once a year. This lets third parties make all kinds of doodads for the device. Put them on all carriers and update them at the same time. If need be get the FCC to lean on Verizon's C block requirements. Other than that, unlock the bootloaders and make donations of devices to the popular mod groups. Most users will never care, but the PR among the tech crowd would be huge and they influence other buyers.
Re:Probably wont get better (Score:4, Insightful)
They started that game long before you could get 1 GB hard drives.
Hilarious... (Score:4, Insightful)
The hilarious part of all of this is we’re still talking about small gains in performance.
The hilarious part of all this is that most people really don't give a rat's as about performance when selecting a phone or even a tablet. The criteria are things like: how does it handle? How intuitive is the UI? Can I watch my favorite online video feeds on this thing? Are any buttons in annoying palaces? What's the price? Does this thing have software to view and edit MS Office files I get sent by mail? The only performance tests these smartphone and tablet things usually get is playing around with a display example in the shop and seeing if the UI is nice and snappy. Nobody excepts tech nerds gives a rats ass that a Samsun Galaxy 4 get a few more FPS in Modern Combat than an iPhone 5.
Comment removed (Score:5, Insightful)
If you measure it, it gets better (Score:5, Insightful)
You can't measure everything, so you're best bet is to try to keep the measurement methods secret and change them frequently. Unless, of course, your measurements are intended to improve a particular area, then by all means, measure on.
Re:And Apple (Score:4, Insightful)
And a little deeper analysis will show....
a) Apple accounted for numbers the same way they always did.
b) Apple does not count phones as sold if they are still in Apple Stores inventories.
c) If Apple were channel stuffing, phones would readily be available at carriers.
The fact is that the "analyst" were wrong and are trying to cover their tails.
Do you really believe that there is not enough of a demand in the launch countries to sale 9 million phones in one weekend?
Do you think that Apple will be forced to take a write down (like MS and BlackBerry) for unsold inventory in the channel?
Re:Or, alternatively (Score:5, Insightful)
On the contrary, the testers should just be bigger dicks. "We detected benchmark-specific optimizations in products #1, #2 and #3, so they all got zero points."
That seems quite arbitrary. What about "To test the battery, we tested how many minutes the battery lasts while running benchmark X". The cheaters will get shorter battery life.
Re:Hilarious... (Score:4, Insightful)
Apple decided the only way they can differentiate their iPhone 5s amidst all the comments that they are no longer innovative is to create a competitive market based on useless CPU performance numbers, just like what Apple did with Retina displays. Before Retina, nobody cared about pixel density. Before A7, nobody cared about CPU performance or its bittyness. Before the iPhone 5s camera, nobody cared about the size of the CCD pixel on their phone camera.
Dude, It's not that long ago that articles like this slashvertisment [slashdot.org] were plastered all over the web accompanied by comments filled with enthusiastic boasts by hoards of Android fans detailing how iPhone performance sucks ass. Even if blisteringly fast benchmark performance is pretty low down on the list of most people out to buy a smartphone I still can't fault Apple for trying to put a sock in the collective mouth of the Android community. There is a certain personal satisfaction to be had from making the choir of hard core Android fans shut up about benchmarks until Samsung comes up with a still faster device (hopefully free of benchmark cheating this time) even if the customers will probably hardly notice this stupid pissing contest.
Re:Or, alternatively (Score:4, Insightful)
Just buy an iPhone. Apple doesn't cheat apparently.
Re:"Pretty Much All of Them" (Score:5, Insightful)
No, not really. Comparing iOS and Android directly on performance is silly. They're two totally different ecosystems and hard performance numbers don't change much. That's like a typical user picking a Mac or Windows PC because one performs 5% better at random tasks, ignoring the fact that the offerings between each machine is radically different and pure performance numbers are only a tiny part of the whole picture.
Apple has no reason to cheat because they have no competition that merits the risk of cheating on. It might have been a different story had iOS hardware been available from multiple vendors.