"We've already started work on making sure that all future versions of benchmarks we get will come with unique package names." - Anandtech
After that Samsung will just figure out some other identifying feature to recognize the benchmarks. Naturally we might go overboard and do random obfuscation on everything, that might solve the issue for the journalists who use them one off.
However for benchmarks that are published in the Google play store that is impossible, so there is no real way for benchmark developers to 'immunize' their products for this sort of approach.
Disclaimer: I am employed in developing Mobile benchmarks, and our product is something they cheat on. Avoiding devices that cheat is very much in our interest.
> After that Samsung will just figure out some other identifying feature to recognize the benchmarks.
Yep, reminds me of 3DMark2003 when NVidia and ATi "optimized" their drivers by disabling, altering or swapping the benchmark's shaders for their own or less complex versions
> The phone giant has proved that benchmarking apps are vulnerable
The only thing they have proved is their willingness to game benchmarks. ATI and NVidia were gaming 3DMark 10 years ago.
Also interesting to note, though Samsung is definitely more than willing to game benchmark despite their protests (and looks like the worst offender by a pretty long shot), Anandtech found out they're far from the only company doing so: http://anandtech.com/show/7384/state-of-cheating-in-android-...
Edit: turns out this was mentioned in the article, but the way it's presented, what the fuck Guardian?
> Anandtech, a site where you could get the (wrong) impression that benchmarks are the only reason to own a gadget
Perfectly correct. The only way to produce real benchmarks is to run useful software that are identical on two different hardware setups. Sadly because of to DRM, this is impossible to do on lock down devices.
You could run two similar applications (say angry avialaes), but the only thing you would test is that device implementation of said game.
The funny thing is that despite being caught out doing this a while ago with the S4, they have no choice but to continue doing it regardless how embarrassing it becomes. If they don't, their next generation devices may score only slightly more (or less than) than the current gen devices.
They painted themselves into a corner from which there is no escape. And they only have themselves to blame.
People still buys. In business sense, it is funny. They cheated and impressed as much as they can and made really large volume of sale. Media reporting like this does not seem to make much difference. Other than few hackers/engineers/programmers, majority of consumers don't give a thing on benchmarks. Too bad.
The CPU/GPUs keep getting faster but this idiotic PPI race is forcing manufactures to cheat so they can appear faster than the previous generation. On a 5" display anything over 720p is overkill.
A number of benchmarks have an "off-screen" mode with fixed-resolution rendering specifically to avoid native resolution artefacts. That provides both the absolute hardware power in comparison to other devices and the device's "feeling", how its hardware meshes together (aka if it has enough power to drive its own screen). The second one is arguably the more important one wrt user experience and the ability to actually use the device.
GLBenchmark has both onscreen and offscreen modes, and Anandtech posts both every time.
If Apple moves to a 5" screen or something close next year, I'm sure they'll double up the resolution again to 2272x1280. That could force Android OEM's to think they need to go to 2560x1440 now, to be higher than Apple, but I agree that would be silly.
It could help devices like Oculus Rift, though, and it could also lead to 4k displays on 10-12" tablets and notebooks faster than otherwise ("only" 2x the pixels from the current resolutions for 10" tablets).
So the chairman was found guilty of financial wrongdoing and tax evasion.
Tax evasion is indeed serious business, but murder? I guess I would like to see several large business CEO's charged with facilitating murder by tax evasion. Would make for some interesting times, and likely tax cuts for the general public.
Concerning that Samsung operates like this even though they are the dominant market leader - with a plurality of the market share for mobile devices, 30.4% of the global market Q2, 2013 [1].
1 http://bgr.com/2013/07/26/smartphone-market-share-q2-2013-sa...
[+] [-] sharpneli|12 years ago|reply
After that Samsung will just figure out some other identifying feature to recognize the benchmarks. Naturally we might go overboard and do random obfuscation on everything, that might solve the issue for the journalists who use them one off.
However for benchmarks that are published in the Google play store that is impossible, so there is no real way for benchmark developers to 'immunize' their products for this sort of approach.
Disclaimer: I am employed in developing Mobile benchmarks, and our product is something they cheat on. Avoiding devices that cheat is very much in our interest.
[+] [-] masklinn|12 years ago|reply
Yep, reminds me of 3DMark2003 when NVidia and ATi "optimized" their drivers by disabling, altering or swapping the benchmark's shaders for their own or less complex versions
[+] [-] masklinn|12 years ago|reply
The only thing they have proved is their willingness to game benchmarks. ATI and NVidia were gaming 3DMark 10 years ago.
Also interesting to note, though Samsung is definitely more than willing to game benchmark despite their protests (and looks like the worst offender by a pretty long shot), Anandtech found out they're far from the only company doing so: http://anandtech.com/show/7384/state-of-cheating-in-android-...
Edit: turns out this was mentioned in the article, but the way it's presented, what the fuck Guardian?
> Anandtech, a site where you could get the (wrong) impression that benchmarks are the only reason to own a gadget
[+] [-] NatW|12 years ago|reply
Cheaters: Samsung, LG, Asus, HTC
non-cheaters: Motorola, Google, Nvidia, Apple
See any patterns?
[+] [-] belorn|12 years ago|reply
You could run two similar applications (say angry avialaes), but the only thing you would test is that device implementation of said game.
[+] [-] steve19|12 years ago|reply
They painted themselves into a corner from which there is no escape. And they only have themselves to blame.
[+] [-] bsullivan01|12 years ago|reply
Apple is the only that held them accountable in court, at least a little it. These b*stards are shameless http://cdn.mactrast.com/wp-content/uploads/2011/09/Samsung-A...
[+] [-] hclee|12 years ago|reply
[+] [-] slacka|12 years ago|reply
[+] [-] masklinn|12 years ago|reply
GLBenchmark has both onscreen and offscreen modes, and Anandtech posts both every time.
[+] [-] devx|12 years ago|reply
It could help devices like Oculus Rift, though, and it could also lead to 4k displays on 10-12" tablets and notebooks faster than otherwise ("only" 2x the pixels from the current resolutions for 10" tablets).
[+] [-] badman_ting|12 years ago|reply
[+] [-] bsullivan01|12 years ago|reply
http://en.wikipedia.org/wiki/Lee_Kun-hee#Samsung_scandal
They do make decent electronics, but I wouldn't trust anything they say.
[+] [-] belorn|12 years ago|reply
Tax evasion is indeed serious business, but murder? I guess I would like to see several large business CEO's charged with facilitating murder by tax evasion. Would make for some interesting times, and likely tax cuts for the general public.
[+] [-] NatW|12 years ago|reply
[+] [-] epo|12 years ago|reply