Thanks for discovering and pointing this out! And Huawei clearly choose a politics-style answer: they say something true, roughly on topic, yet ignore the real question completely.
In my opinion optimization with Ascend P7 didn't do anything as it remains poorly rated based on consumer satisfaction. /Craig from http://www.topreport.org/phones/
If Huawei performs this same tweak for popular games from the PlayStore then I would consider this misguided rather than an attempt to cheat benchmarks. Joshua, did you evaluate if any games receive this optimization? Keep up the good work.
Well considering a renamed benchmark caused the performance to drop significally, I wouldn't expect this to kick in during gaming, unless the game is called "GFX Bench 3.0".
You missed the angle of his question; he is suggesting the (to my mind, unlikely) possibility that some actual games might be whitelisted for the high-performance governor in addition to gfxbench. I think it's a reasonable suggestion and is worth a quick check, but I am not holding my breath.
Yeah, shocking to see Huawei copying Samsung's mistakes when Sammy was caught some time ago. DO they think they can get away with it this time as testers become very good in ensuring they get the cheating cases. It does show however, the merits of a triple core system which no one has built to date!.
Maybe because Motorola did not even engage in the premium tier performance fight of the others at that time... Cheating with a MSM8960 to shine among others who run on higher tier silicon already is maybe not a promising endeavor to begin with...
It took some public shaming for Samsung to stop this type of behaviour. Hopefully Huawei chooses to respond in the same way and not spew out the same tired excuses we've already heard from others.
You assume that a benchmark can only be detected by the file name of the executable. Obviously, this is not true. A shrewd smart phone manufacture would use other methods to detect benchmarks running (byte code signatures, etc.). Just saying.
The test demonstrates that Huawei is using the name of the executable to detect the benchmark. There would be little sense in doing complicated programming-gymnastics to surreptitiously detect benchmarks since the altered behavior would still be noticeable in that the CPU/GPU behavior would be substantially different from real workloads.
Not that there's any sense in this benchmark cheating in the first place, mind you.
No it wouldn't, because the reviewer has no way of knowing what the normal behaviour is. The reviewer's only method of detecting cheating is changing the file name and presumably the people who make the cheats know this, so it would really a no-brainer to invent a more sophisticated cheat system
I know they got some special benchmarks that did more than "rename" the program.
I feel like an old man now remembering the quack 3 debacle and to see it repeated is disheartening.
Although phones for the Chinese market are already half built by marketing considering there are 8 core A7 chips out there where core-count is all that they market for in China...
While the Ascend P7 is in the high-end range in terms of materials, aesthetics and features, it's not Huawei's flagship. That place belongs to the Honor 6, which carries a Kirin 920 with 4*A15+4*A7 in big.LITTLE together with a Mali T628 MP4 and dual-channel memory.
Not that this makes any difference in the article itself, but for the record this model hasn't been Huawei's flagship for a few months.
Typical political answer from any business, regardless of nationality, that has been discovered doing something unscrupulous. Or it's possible they misunderstood the accusation due to something getting lost in translation.
I have tested my oppo find 5 and found similar result to this, but they have done this only with software. Once I installed cyanogenmod it's CPU frequency graph looks a lot like the one with your anti cheat and scored significantly lower.
If a mobile phone cpu can be safely overclocked to reach better performance so be it. But I doubt it can be overclocked safely for a longer time. Just like overclocked CPU/GPU will crash sooner or later if you don't have the cooling under control. So maybe one way to overcome/combat this issue is, besides evaluating phones by benchmark scores, also evaluate endurance. Say "overclocked" phones get better score, but if you run it over and over would it crash the phone, make it unstable, or excessively hot? These can also be included along with benchmarks to gauge the usefulness of a particular phone.
We’ve updated our terms. By continuing to use the site and/or by logging into your account, you agree to the Site’s updated Terms of Use and Privacy Policy.
Facebook
Twitter
RSS
23 Comments
Back to Article
MrSpadge - Thursday, August 21, 2014 - link
Thanks for discovering and pointing this out! And Huawei clearly choose a politics-style answer: they say something true, roughly on topic, yet ignore the real question completely.craighamilton - Saturday, December 6, 2014 - link
In my opinion optimization with Ascend P7 didn't do anything as it remains poorly rated based on consumer satisfaction. /Craig from http://www.topreport.org/phones/joe0185 - Thursday, August 21, 2014 - link
If Huawei performs this same tweak for popular games from the PlayStore then I would consider this misguided rather than an attempt to cheat benchmarks. Joshua, did you evaluate if any games receive this optimization? Keep up the good work.GC2:CS - Thursday, August 21, 2014 - link
Well considering a renamed benchmark caused the performance to drop significally, I wouldn't expect this to kick in during gaming, unless the game is called "GFX Bench 3.0".teiglin - Thursday, August 21, 2014 - link
You missed the angle of his question; he is suggesting the (to my mind, unlikely) possibility that some actual games might be whitelisted for the high-performance governor in addition to gfxbench. I think it's a reasonable suggestion and is worth a quick check, but I am not holding my breath.DanNeely - Friday, August 22, 2014 - link
IIRC when they were first caught doing this there weren't any games on the cheater lists people pulled out of the ROMs; just benchmarks.fteoath64 - Sunday, August 24, 2014 - link
Yeah, shocking to see Huawei copying Samsung's mistakes when Sammy was caught some time ago. DO they think they can get away with it this time as testers become very good in ensuring they get the cheating cases.It does show however, the merits of a triple core system which no one has built to date!.
blackmagnum - Thursday, August 21, 2014 - link
Do you really expect a show of scruple from a thriving Chinese manufacturer?kyuu - Thursday, August 21, 2014 - link
Because we all know that American/insert-your-favored-nationality-here businesses are shining beacons of scrupulous behavior.Klimax - Friday, August 22, 2014 - link
In this case, yes. When the case was originally reported by Anandtech it was found all smartphones by Asian vendors cheated, while Motorola's did not.rDeck - Friday, August 29, 2014 - link
Maybe because Motorola did not even engage in the premium tier performance fight of the others at that time...Cheating with a MSM8960 to shine among others who run on higher tier silicon already is maybe not a promising endeavor to begin with...
dawheat - Thursday, August 21, 2014 - link
It took some public shaming for Samsung to stop this type of behaviour. Hopefully Huawei chooses to respond in the same way and not spew out the same tired excuses we've already heard from others.lefty2 - Thursday, August 21, 2014 - link
You assume that a benchmark can only be detected by the file name of the executable. Obviously, this is not true. A shrewd smart phone manufacture would use other methods to detect benchmarks running (byte code signatures, etc.). Just saying.kyuu - Thursday, August 21, 2014 - link
The test demonstrates that Huawei is using the name of the executable to detect the benchmark. There would be little sense in doing complicated programming-gymnastics to surreptitiously detect benchmarks since the altered behavior would still be noticeable in that the CPU/GPU behavior would be substantially different from real workloads.Not that there's any sense in this benchmark cheating in the first place, mind you.
lefty2 - Friday, August 22, 2014 - link
No it wouldn't, because the reviewer has no way of knowing what the normal behaviour is. The reviewer's only method of detecting cheating is changing the file name and presumably the people who make the cheats know this, so it would really a no-brainer to invent a more sophisticated cheat systemerrorr - Friday, August 22, 2014 - link
I know they got some special benchmarks that did more than "rename" the program.I feel like an old man now remembering the quack 3 debacle and to see it repeated is disheartening.
Although phones for the Chinese market are already half built by marketing considering there are 8 core A7 chips out there where core-count is all that they market for in China...
ToTTenTranz - Thursday, August 21, 2014 - link
While the Ascend P7 is in the high-end range in terms of materials, aesthetics and features, it's not Huawei's flagship.That place belongs to the Honor 6, which carries a Kirin 920 with 4*A15+4*A7 in big.LITTLE together with a Mali T628 MP4 and dual-channel memory.
Not that this makes any difference in the article itself, but for the record this model hasn't been Huawei's flagship for a few months.
Andrei Frumusanu - Thursday, August 21, 2014 - link
I'm reviewing the Honor 6 at the moment, the points in this article still apply for it too.Sushisamurai - Thursday, August 21, 2014 - link
Typical china BS answerkyuu - Thursday, August 21, 2014 - link
Typical political answer from any business, regardless of nationality, that has been discovered doing something unscrupulous. Or it's possible they misunderstood the accusation due to something getting lost in translation.DIYEyal - Thursday, August 21, 2014 - link
I have tested my oppo find 5 and found similar result to this, but they have done this only with software. Once I installed cyanogenmod it's CPU frequency graph looks a lot like the one with your anti cheat and scored significantly lower.hlovatt - Thursday, August 21, 2014 - link
Glad AnandTech are still testing for people gaming benchmarks; one of the few sites that take testing seriously. Keep up the good fight :)flyingfiddle - Friday, August 22, 2014 - link
If a mobile phone cpu can be safely overclocked to reach better performance so be it. But I doubt it can be overclocked safely for a longer time. Just like overclocked CPU/GPU will crash sooner or later if you don't have the cooling under control. So maybe one way to overcome/combat this issue is, besides evaluating phones by benchmark scores, also evaluate endurance. Say "overclocked" phones get better score, but if you run it over and over would it crash the phone, make it unstable, or excessively hot? These can also be included along with benchmarks to gauge the usefulness of a particular phone.