So the mainland phone makers are cheating in benchmarks as well? I know this isn't a China only thing, but seems like they're trying to grab more than what they can chew.
I am saying the tech world in general is far to hung up on it. Companies, tech sites and their visitors - so hung up on it and its perceived importance that companies pull crappy moves to appear to benchmark better.
Maybe 5-10 years ago, such benchmarks were important, as the performance gain was quite noticeable. However, now I think we are well beyond the point of tangible gains on a smartphone, at least until the time that we expect more from the devices than the current usage model.
I'm not sure how you don't notice 10-40% improvements in peak performance and efficiency between generations, gains are very tangible for everyone, in multiple ways, regardless of the usage model. Even if you just use your phone for making actual phone calls, you can notice the standby time increase, better radio reception, ability to answer calls while on LTE or wireless only. Maybe YOU don't notice these things, but please speak for yourself. Thank you!
As for Huawei, the company is shady beyond belief. I consider the Nexus 6p the only Huawei phone I've ever wanted to get. I don't trust them, not one bit. Then again I don't trust Google either but Google seems to be an unavoidable evil I have to live with, and I do trust them quite a bit more than Huawei.
Benchmarks are always important. If a customer is shopping for a device based on performance, the metric they have to depend on is that measured by...benchmarks.
Those brands aren't being sold here, so it's more of a deflection than a real answer. The only other recent example of this problem is the OnePlus 5, which is another Chinese phone. All Huawei is doing is making Chinese brands look bad.
No, it's is normal behavior for a heavy GPU test to peak initially and then throttle back down as thermal limitations are reached. What Huawei is doing is ignoring those thermal limitations and actually overheating devices for specifically named benchmarks.
This right here. A lot of us would agree that benchmark results are not the end all/be all of a device. But in no way is that an appropriate response to an article about benchmark cheating.
I think it speaks directly to the subject matter. Far too many companies, tech sites, and users hang far too much on bench-marking. Does 1% better "geekmarks" actually make a difference in usage?
But it's not 1%, or even close to that. The OnePlus 6 is 3x faster than the P20 Pro in most of the sustained performance benchmarks.
I think it's important for users to know that the Kirin 970 has a significantly weaker GPU than the S845, especially when Huawei is marketing some of these phones as 'gaming phones'.
Now you are comparing 2 different class phones (flagship OP6 vs mid-range P20) and CPU from 2 different years (2017's 970 vs 2018's S845). In most areas like vs. like is very close, and there is alot more complexity to it than just that. It's all way over-hyped and over-emphasized
No he's right, current QC flagship SoC vs reigning Hisilicon Flasghip. In case you din't know SD845 is available in phones priced as low as $300 lower than P20 in fact.
Fair enough, but the Kirin 980 is out in a month. It is last years product and when it came out it compared to the S835. Both on yearly cadence appx 6 months apart.
The P20 Pro came out this year and actually cost more at launch than the OP6 (~700-800 euro vs ~500-600 euro). The P20 Pro also came out after the GS9, which has the S845. Even the P20 launched at 649 euro. If you'd prefer to compare phones that came out at the same time, then the GS9 is still over 2x faster. If you want to compare the Kirin 970 to the S835, since both came out in 2017, then the 835 is still 2x as fast.
From a GPU perspective you aren't incorrect... If you are buying a phone as a gaming device you might want to look at GPU benchmarks more, but you do however seem to be overly obsessed with bench-marking... You are also not comparing like for like. The P20 is not Huawei's flagship, the Mate is. Oneplus is a crappy budget brand with massive quality issues and the price shows it. IF you want to buy one go for it.
The P20 Pro is priced like a flagship and has their top-end SOC. Huawei just has two high-end models for some reason.
Again, even if you insist on comparing only flagships from big companies, Huawei comes out looking bad. The Pixel 2 and Galaxy S8 are both significantly faster than the Mate 10.
I'm not obsessed with benchmarks, but I think they have value. We aren't talking about modern SSD benchmarks where one brand reads at 3100mbps and the other at 3200mbps. That's a small difference. This is very different. We're talking about at least double the performance.
> I think it's important for users to know that the Kirin 970 has a significantly weaker GPU than the S845
How so? If some popular game needs 10,000 shader OPS to run at 800x600 at 30 frames/sec what difference does it make if one SoC can pump out 8000 (admittedly synthetic - are you really going to tell me you're going to notice 24FPS vs 30? pahlease), or 15,000 or another 40,000? Ok, so does OPS/Watt actually matter in anybody's evaluation metric? No. Does anyone choose a phone based on this one lets me run X game for 30 minutes before running out of batt but I can get 40 minutes with this other one because in "game mode" the manufacturer took liberties with wattage?
What modern phone runs at 800x600? Also, faster GPUs can get closer to 60fps, which is definitely a noticeable improvement over 30fps.
If all you're playing is Candy Crush, then it doesn't matter what GPU you have, but if you're playing Fortnite or the upcoming Elder Scrolls game, then GPU performance is important. If two phones are roughly the same price, but one of them has 3x the GPU power with no downsides, I'm going to go with the faster one every time.
The human eye in games like Fortnite etc can only process a very limited frame rate. So anything over 30 is basically pointless. Plus factor in using a 27+ monitor(s) vs a piddly-ass phone screen with lousy (by comparison to "gaming" monitors) refresh characteristics the benchmark is even less useful.
That article make it very clear that people can tell the difference between 60fps and 30fps. Its claim is that it's only an improvement in smoothness, not an improvement in our ability to track changes. A higher frame rate won't improve my ability to pick out movement.
60fps looks better than 30fps. If I can choose between the two, at the same resolution, I'm always going to pick 60fps. Will it make me better at the game? No. Does it make the game look at feel better? Yes.
@shogun18 - I always find it amusing when people present "evidence" to support their position only to find out the evidence they are producing very clearly refutes their position. The article very clearly states: "Certainly 60 Hz is better than 30 Hz, demonstrably better." - Professor Thomas Busey
From my own perspective, I would suggest to you that games need to have a 30 fps at minimum to be playable and to appear to be somewhat fluid. 60 fps is clearly better, but not "twice as good". You can see the difference though. On my iPad, I can do 120 fps on games like World of Tanks Blitz and can even notice that difference. For some games, reaction time is critical and network performance also plays a role in this. However, higher frame rates can indeed provide a competitive advantage.
did you BOTHER to read to the end let alone comprehend what was being put forth? The human brain is SLOW! It's massively parallel but it's SLOW. Just like our ears are crap compared to other creatures who actually have good hearing. If you're playing FPS on a phone you're an idiot to begin with. Fluidity or more properly the perception of same doesn't make your performance better. Your reaction time is also completely shit compared to the theoretical frame rate you think you are perceiving. Anyone who cares about game play on a phone is a moron.
Buyers should be able to value whatever they wish when making their purchasing decisions. Lying to them denies them the right to make decisions based on the criteria that matter most to them, whether it be nice cameras, great screens, excellent call quality, or yes, 'geekmarks' or whatever.
It's not for you to determine what is most important to a customer, nor is it ethical to lie about one of those or other items in order to trick people who value them into buying your product.
You need to put a performance metric on things somehow. Cars have horsepower and torque, batteries have volts and milliamps, and food has protein and carbs.
Unfortunately these metrics do not come from the SoC manufacturer, but the phone vendor. That therein lies the problem. "Overclocking" or boosting a SoC beyond reasonable thermal design limitations is blatant cheating if it can't be sustained throughout, say, a game, that the benchmark is momentarily mimicking.
At the end of the day, this is really an Android problem too, because the freedom the OS gives phone vendors to manipulate the kernel, scheduler, and frequency curve of the CPU/GPU. This kind of flexibility didn't exist (and still doesn't exist) in other mobile operating systems.
So imagine if this were happening in the PC space. Where vendors were selling overclocked systems WITHOUT SAYING they were overclocked. Where vendors were manipulating the real-world benefits of a GPU with software that faked benchmark results.
I would liken it to what happened with the game console clones of the 80's, when there were third-party Atari's, Intellivisions, etc, that had custom CPU's running at higher frequencies. In that case, it actually hurt developers more than consumers (but still hurt consumers) because developers couldn't even depend on a performance metric for the platform they were developing for. This is partially why there were virtually no third party developers (Activision and Hudsonsoft - who later developed their own console simply to have some control over the hardware environment! - were effectively the first cross-platform developers.)
Benchmarking is incredibly important when people are planning and building systems, and by systems I mean those that actually support business. I'd argue you're right for most consumers though, as things are just "more than fast enough."
That said, I'd still argue they're an important metric. Just might not be important to you.
This is not helping the perception that Chinese manufacturers will cheat and lie to their customers. That's already a big problem for Huawei in western markets. I think they need to fire everyone responsible for marketing their phones to western markets and hire native teams who actually understand the customer base.
You're confusing "the western markets" with the USA. Elsewhere people don't have such an obsession, actually they know American devices are probably more likely to provide backdoors.
The Chinese culture is synonymous with cheating. It's an *aspiration* to cheat and "put one over" their competitor and has been for for thousands of years. You're not going to convince anyone to play by "modern" Western notions of "fair and honest" benchmarks. Even so-called Westerners have put their values aside for the almighty buck in droves. Not to suggest criminal enterprises like the East India Company (of Britain) had any intention of acting in any civil/honorable manner from the get go. Human beings are for the most part filthy, rotten, scoundrels and far too many of them will gladly commit atrocities in the name of profit or something else. Cheating is basic human behavior.
Therefore, you should ALWAYS expect humans to be dishonest. See Intel for a brazen example.
Thanks for doing the article. But I also have to wonder, are there really more than a 100 basement-dwelling morons who give a flying F about how fast a GPU is on a stupid phone? And furthermore play games on such pointless platforms where frame rates or triangles/sec matters? Seriously? PC Graphics cards from 10 years ago are still stupid fast. Phone games are throwaways. Sure, some publishers some how manage to make a tidy sum but it could run in CGA and people would still buy them to fritter away their boredom. If every Youtube ran in 360p nobody would give a damn. Oh some would bitch and moan but they'd still watch. And that's the point.
Graphics performance of the SoC is just dick measuring and the percentage of people who consider the number in their purchasing valuation metric is basically zero.
No You're 1 in 10+ million consumers (out of just English-reading population) and therefore irrelevant except that you read Anand and/or Toms and such. :)
Actually, social science style studies reveal that (ballpark) 10% of people always cheat, 10% never cheat no matter what, and the 80% in the middle are a continum who are mostly swayed by perceived risks and incentives. If you are always mistrusting everyone it prevents one from building profitable relations.
Not sure about Chinese culture but you'll find more than a few people care about phone GPU performance, I can only wonder why you are on a site basically dedicated to people who are interested in such things Phones are the new computer, with a decent docking set up many people probably don't need a desktop or laptop. I've been gaming since the days of the first civilisation game and currently do a lot of gaming on my phone.
@shogun18 - You said: "But I also have to wonder, are there really more than a 100 basement-dwelling morons who give a flying F about how fast a GPU is on a stupid phone? And furthermore play games on such pointless platforms where frame rates or triangles/sec matters?"
It seems your comments are very much out of touch with what is happening today, even in the world of gaming.
I don't get the point in cheating on phone benchmarks or even the point of phone benchmarks in general. I also don't get the point of sticking an auto playing video about buying the right CPU on EVERY DAMNED PAGE of this article. The same video. Every time! If seeing it on the first page pissed me off and I didn't watch it there (while also being enraged at the auto starting nature of it) what on Earth would make you think it's a good idea to do it to me three more times in the same article? Did the Purch buyout somehow involve a lobotomy that removed logic and reason or was that SOP already in place beforehand?
I would use stronger words than that to really convey the egregious nature of repetitive onslaughts of their bloody (not a swear word) videos against us. Their blatant "up yours, screw you" attitude turns rage to a frothing mouth frenzy ready to spill blood (especially on Toms Hardware). What the flying F is wrong with allowing users to decide whether to hit the play button or not? Or am I really only talking to a lobotomized miscreant that should be restrained from interacting with the public. Cease and desist auto play videos! A "up yours, screw you" attitude is not a smart way to deal with the public. Toms are you listening, or are sending us all another finger image?
er it's 2018. How the hell have you been using the internet for the last decade without all browser plugins disabled or at the very least noScript (or similar) with or without AdBlock (or similar)?
No, I totally get your point and I agree (with) you have every right to be incensed with auto-play videos. However a random comment here is unlikely to get the author or site-admin's attention. And said person is doubtless unable to oppose corporate policy. When Anand's site readership takes a major nose dive (further?) and the morons in marketing (but I repeat myself) can be beaten over the head that one of the major reasons they are losing revenue and clicks is because of auto-play, THEN and ONLY then will the practice stop.
There haven't been more than a minuscule handful (again, basically zero given the side of the ecosystem) that could be described as "responsible" or "good" website operators for well over 10 years. Therefore if you haven't been black-listing every feature in your browser and toggling the choices conveniently given to you, then your ire is misplaced IMO. I am an equal opportunity banner of site content. Join the club and be amazed at far more enjoyable the experience is.
Well, Purch has been pushing those annoying ads on Tom's Hardware for quite some time now, so I guess it's only to be expected they'd be forcing them on this site as well.
TBH I hadn't even noticed the ads because I used my hosts file to block assets.purch.com, because my adblocker wasn't taking care of it. You could try that too.
At some point, Huawei (and other Chinese OEMs) need to decide whether they want to build their brands globally or just in their home market.
"Other Chinese OEMs lie so we've got to as well" ends up doing nothing but providing ammunition for those that say that Chinese phones are "cheap," under-performing knock offs.
The Nexus 6p showed many years ago that Huawei can make good hardware. HiSilicon's chips obviously aren't doing them many favors in the GPU department, but that just means they need to target appropriate segments where they are competitive (I'm convinced that there's a large niche of people who want stylish devices that feel premium but don't really care much about performance), rather than lying and perpetuating a stereotype that will hurt their brand long after they've abandoned those practices.
Why not simply do a combined performance & power test where you run the benchmark continuously until the phone shuts down? If a phone maker tries to cheat for the performance side, they'll look bad on the run-time side. If a phone throws up a "I'm running too hot" screen, consider that the end of the test. Such a test not only shows how fast your game may perform, but also for how long you can game.
Screen resolution, V-Sync and other device differences makes this kinda hard. In my view there's no added value over just peak & sustained performance as well as just measuring power.
The worst one should be the "Intel's repeatedly not following the specs" that causes the problems of "Meltdown", requiring OS memory relocation, the industry's 1st and only, and "Foreshadow" that the mitigation can only "reduce" the risk but "not eliminated" it!
Because I don't consider phones to be gaming devices, for that I have a PC and a console, so benchmarks are worthless to me, the most important things in a phone are the OS (Unfortunately I'm stuck with android, iOS isn't well suited for my use), screen (high resolution and no notch) and finally battery life (around one day, an OLED screen).
Android increasingly uses the GPU to render the OS, and apps like Google Maps use it extensively as well. Sustained GPU performance isn't just relevant to gamers.
First and foremost: Thanks Andrei and Ian! This kind of article is why I come to Anandtech again and again (and more frequently than other computer tech websites). Yes, those benchmarks are not only misleading, they also steer the manufacturers towards optimizing for an artificial use (benchmarking), often at the expense of actually optimizing their smartphones for real world use. Who knows just how much better Huawei's phones could have been for everyday use if the time and energy invested in cheating for benchmarks would have instead gone into optimizing their phones for productivity, real world applications, and battery live (Huawei gets a bonus for historically having large capacity batteries, though!). As they are, those benchmarking suites would probably come in handy if one needs to use the phone as a hand warmer in winter.
One minor edit: the "High" and "Normal"labels on the the Aztec Ruins graphs are probably switched around. The fps numbers for "high" are really high for all devices. However, that is a very minor, cosmetic, point in an otherwise very good article!
@Andrei and Ian: I also wonder how the apparent superiority of highly customized GPUs in Apple and QC SoCs reflects a conundrum that ARM faces with their Mali designs: Basically, Mali GPUs have to work in configurations ranging from as few as 1-2 to as many as 20 (or 24) units and with widely varying CPU cores (dual-core A53 or 55 at the low end to now A76 or mongoose octacores). In contrast, Apple GPUs and QC's Adrenos appear to be a lot more "tailored" to the SoCs they end up in, which, together with optimized drivers, probably gives them a leg up. Andrei, as you are truly an seasoned expert in this field, I wonder if you could comment on this, maybe even in some detail in a dedicated article in the future?
Qualcomm also need to ship smaller version of Adreno to their lower-end SoC like Snapdragon 625 or so. While Anandtech haven't cover the efficiency of lower-end Snapdragon SoC, my experience confirmed that Snapdragon 625 had better much better GPU efficiency than Kirin 960 (lower temp at the same frame rate)
> So Honor is trying to promote the Honor Play as a gaming-centric phone, making bold marketing claims about its performance and experience. This is a quite courageous marketing strategy given the fact that the SoC powering the phone is currently the worst of its generation when it comes to gaming.
I know Oneplus used to do something similar but they did it for benchmarks and games, witch is'nt cheating I my book because end users do get the performance shown in benchmarks. Is oneplus still doing this? The oneplus 6 seems to benchmark faster than other sd845 based phones, so this would kind of explain how.
Is this click bait or something? One cannot simply add more power/speed to a device without throttling or shutting down. If one would run gaming benchmark for more than 10 minutes then it levels out. It is a different story with browser benchmarks where burst speeds for a few seconds is valuable.
It would have been cheating if a phone starts dimming or turning off the display when there is a benchmark run.
I wouldn't put much more value on battery efficiency when running games or benchmarks because it will be difficult to regulate. Everyone will have differing opinions about it. When you turbo/boost (any chip), it is the least efficient anyway.
I always need extra cooling in order to make PUBG playable on My poor Mate 9. Sometimes I use wet tissue to paste on the back of the chassis, sometimes I just put Ice and let it melt to cool the phone down. (which usually take less than 3 minutes to melt all). Yes I know this phone is not water resistance certified but come on! its just a hot potatoes that need to be cooled and I don't expect flagship phone to have this behavior. The ironic is I had buy Xiaomi Redmi Note 4X with Snapdragon 625 for 150$ and this devices play PUBG much better than Huawei's Flagship.
After read this article, I choose to not believe Huawei's marketing anymore, I don't believe that Kirin 980 can close the gap in GPU performance with Snapdragon 845, let alone acceptable performance.
The most interesting aspect is that it shows that ARM also struggles with power once they get into x86 performance area. No free lunch. And I wonder how the other devices cheat. Probably most due somehow. Huawei just wan't that clever.
Great work on this piece. I really appreciate good journalism giving light to industry issues while having the technical expertise to dive deep and explain everything in a concise manner. And I wouldn't worry about catching this earlier, what's important is we know now. And hopefully at least some consumers now won't fall for the marketing/benchmarking hype.
The GFXBench T-Rex Offscreen Power Efficiency benchmark in the Kirin 970 piece still shows the cheating result for the Mate 10.
It's astonishing to see the difference in sustained performance cooling alone can attribute for - P20 Pro and Honor Play have the same maker, same SoC, similar dimentions, and yet, the performance is quite different.
I thought that ever since Samsung was caught doing the same thing in 2013 you put in active countermeasures (randomly named benchmark software, etc.) or at least a test for cheating as a standard part of your setup?
These tests show similar behavior with iPhone. It's not any faster than the other leading brands. The difference between peak and sustained is huge. Same goes for Samsung and Xiaomi.
I understand why the UI seems so fast and responsive, and why many people complained about the performance. It just can't stay at peak forever.
To clarify up front: I don't own or like iOS devices. However, I have to give Apple its due here: the idea of really high, short burst performance coupled with okay longer-term speed is pretty much what I (and probably many other mobile users) want in smartphones. This is useful for multitasking while opening multiple browser windows etc., i.e. scenarios that really benefit from well above-normal CPU/GPU speeds for the few seconds, resulting in a fluid user experience. This is different from running the SoC to heat exhaustion and shutdown whenever a benchmarking app is recognized. Some current Android flagships are sort-of able to do that short burst ("turbo" in PCs) also, but none has yet the (momentary) peak performance of Apple's wide and deep cores. The Mongoose M3 was an attempt, the Kirin 980 was an apparent step towards this, sort of, but is now marred by this benchmark cheating BS. Let's see what QC can cook up, they tend to get closest to Apple's top SoC.
Thermal throttling happens on ALL phones. That's not what's in question. The issue is with companies that artificially white list specific benchmarks in order to achieve results that would not be seen in real applications.
To that end, Anandtech's battery tests have always demonstrated the difference between peak and sustained performance in mobile devices. Up through the iPhone 6s, there was very little throttling going on with iPhones on peak loads. To your point, the level of throttling in iPhones has been approaching practices of common Android equivalents.
Naughty. Makes running a benchmark in a 'loop mode' until the battery runs out very important IMO. If the device dies in an hour in benchmarks, but 3 hours elsewhere, then you know something's awry.
However there is a potential positive - it shows that the Kirin 970 can perform well at higher power consumption - there's no performance wall between 3.5W and 9W, and the perf/W scales fairly well too.
So - why not look into a 'docked' mode option in the future? One option could be a Switch-like dock, using external power (to protect the battery), optional cooling assistance, HDMI out to a TV, provide a controller in this pack as well, and allow the SoC to run as fast as this setup can keep the device from damaging itself. That's flippin' marketable. The dock would cost a few dollars, and it sounds like the software is already there in the main.
Hopefully the Mali G76 in the Kirin 980 actually fixes a lot of the performance issues with Mali, which surely were a factor in this sad situation (also clearly saving money by using a smaller GPU, wide and slow beats narrow and fast for GPUs where power consumption matters.
Obviously you haven't read the article, the so-called whitelisting's performance can't be sustained, it's not as simple as merely activating some sort of game mode automatically.
I think the real questions Andrei and Ian should be asking is whether the devices are detecting not only benchmarking software but also popular software (games included) that people use for extended periods. If that is the case, the device's behavior will shorten its lifespan over a shorter amount of time.
I know this thread is old but I wanted to share my story as well, so yesterday EMUI 9 updated arrived for my Honor Play in my country. bringing the "performance mode" toggle from the battery settings. anyone watching "Gary Explains" on youtube knows that Kirin 980 can have increased performance turning on "performance mode" in the battery settings. before updating i ran a few antutu benchmarks, my scores were 206-210,000. my highest was 212,000. using antutu sensors my Hottest CPU & battery were 41-43C right after updating I was getting 165-170,000. and my lowest score was 153,000. which is basically SD660 & Kirin 710 Performance, but my phone was not getting warm at all, like max CPU & Battery was 32C after 3 tests. at this moment I was like "HUAWEI THOSE LYIN CHEATIN BASTARDS" then I remembered Performance Mode, Turned it on, and BOOM, ran 3 Tests, was getting around 206-210,000. with my LOWEST Score being 198,000 after 4 looping tests and my SoC & battery were peaked at 41C. and after letting the phone cooldown to around 19C I ran a final test and got 216,000 with SoC & battery at like 36C. my point is as follows. SoC sustained performance vs peak can vary by as much as 20,000 on antutu, it all depends on the phone's cooling ability. the Mate 10 Pro has thermal paste on the Kirin 970. the honor Play does not. (check Jerryrigeverything for teardown of M10Pro). "performance mode" is not "cheat mode". the kirin 970 may not be as efficient as SD835 but it does manage to go neck and neck. it's like desktop GPUs. the RX 580 vs 1060. yes in phones batteries are the main culprit. but that peak power draw only happens when gaming, and let's face it, no one games for 1-2 hours on their phones, it's more like 2-3 games of PUBG mobile or a few fifa soccer matches. 30mins or so. so Huawei are shady af yes, Qualcomm's SD is better yes. but kirin is respectable as well.
We’ve updated our terms. By continuing to use the site and/or by logging into your account, you agree to the Site’s updated Terms of Use and Privacy Policy.
Facebook
Twitter
RSS
84 Comments
Back to Article
goatfajitas - Tuesday, September 4, 2018 - link
The tech world is far to hung up on benchmarking these days. Benchmarking is like the Kardashians of tech sites. The lowest form of entertainment. :PR0H1T - Tuesday, September 4, 2018 - link
So the mainland phone makers are cheating in benchmarks as well? I know this isn't a China only thing, but seems like they're trying to grab more than what they can chew.goatfajitas - Tuesday, September 4, 2018 - link
I am saying the tech world in general is far to hung up on it. Companies, tech sites and their visitors - so hung up on it and its perceived importance that companies pull crappy moves to appear to benchmark better.MonkeyPaw - Tuesday, September 4, 2018 - link
Maybe 5-10 years ago, such benchmarks were important, as the performance gain was quite noticeable. However, now I think we are well beyond the point of tangible gains on a smartphone, at least until the time that we expect more from the devices than the current usage model.niva - Tuesday, September 4, 2018 - link
I'm not sure how you don't notice 10-40% improvements in peak performance and efficiency between generations, gains are very tangible for everyone, in multiple ways, regardless of the usage model. Even if you just use your phone for making actual phone calls, you can notice the standby time increase, better radio reception, ability to answer calls while on LTE or wireless only. Maybe YOU don't notice these things, but please speak for yourself. Thank you!As for Huawei, the company is shady beyond belief. I consider the Nexus 6p the only Huawei phone I've ever wanted to get. I don't trust them, not one bit. Then again I don't trust Google either but Google seems to be an unavoidable evil I have to live with, and I do trust them quite a bit more than Huawei.
Samus - Wednesday, September 5, 2018 - link
Benchmarks are always important. If a customer is shopping for a device based on performance, the metric they have to depend on is that measured by...benchmarks.Flunk - Tuesday, September 4, 2018 - link
Those brands aren't being sold here, so it's more of a deflection than a real answer. The only other recent example of this problem is the OnePlus 5, which is another Chinese phone. All Huawei is doing is making Chinese brands look bad.techconc - Friday, September 21, 2018 - link
No, it's is normal behavior for a heavy GPU test to peak initially and then throttle back down as thermal limitations are reached. What Huawei is doing is ignoring those thermal limitations and actually overheating devices for specifically named benchmarks.kirsch - Tuesday, September 4, 2018 - link
They very well may be. But that is a completely orthogonal discussion to companies cheating to show better results than they should.Reflex - Tuesday, September 4, 2018 - link
This right here. A lot of us would agree that benchmark results are not the end all/be all of a device. But in no way is that an appropriate response to an article about benchmark cheating.goatfajitas - Tuesday, September 4, 2018 - link
I think it speaks directly to the subject matter. Far too many companies, tech sites, and users hang far too much on bench-marking. Does 1% better "geekmarks" actually make a difference in usage?cfenton - Tuesday, September 4, 2018 - link
But it's not 1%, or even close to that. The OnePlus 6 is 3x faster than the P20 Pro in most of the sustained performance benchmarks.I think it's important for users to know that the Kirin 970 has a significantly weaker GPU than the S845, especially when Huawei is marketing some of these phones as 'gaming phones'.
goatfajitas - Tuesday, September 4, 2018 - link
Now you are comparing 2 different class phones (flagship OP6 vs mid-range P20) and CPU from 2 different years (2017's 970 vs 2018's S845). In most areas like vs. like is very close, and there is alot more complexity to it than just that. It's all way over-hyped and over-emphasizedR0H1T - Tuesday, September 4, 2018 - link
No he's right, current QC flagship SoC vs reigning Hisilicon Flasghip. In case you din't know SD845 is available in phones priced as low as $300 lower than P20 in fact.R0H1T - Tuesday, September 4, 2018 - link
>reply meant for goatfajitasgoatfajitas - Tuesday, September 4, 2018 - link
Fair enough, but the Kirin 980 is out in a month. It is last years product and when it came out it compared to the S835. Both on yearly cadence appx 6 months apart.Cicerone - Friday, September 7, 2018 - link
A phone price is not standing from it's SOC. Display, cameras quality, type of storage and software is accounted.cfenton - Tuesday, September 4, 2018 - link
The P20 Pro came out this year and actually cost more at launch than the OP6 (~700-800 euro vs ~500-600 euro). The P20 Pro also came out after the GS9, which has the S845. Even the P20 launched at 649 euro. If you'd prefer to compare phones that came out at the same time, then the GS9 is still over 2x faster. If you want to compare the Kirin 970 to the S835, since both came out in 2017, then the 835 is still 2x as fast.goatfajitas - Tuesday, September 4, 2018 - link
From a GPU perspective you aren't incorrect... If you are buying a phone as a gaming device you might want to look at GPU benchmarks more, but you do however seem to be overly obsessed with bench-marking... You are also not comparing like for like. The P20 is not Huawei's flagship, the Mate is. Oneplus is a crappy budget brand with massive quality issues and the price shows it. IF you want to buy one go for it.cfenton - Tuesday, September 4, 2018 - link
The P20 Pro is priced like a flagship and has their top-end SOC. Huawei just has two high-end models for some reason.Again, even if you insist on comparing only flagships from big companies, Huawei comes out looking bad. The Pixel 2 and Galaxy S8 are both significantly faster than the Mate 10.
I'm not obsessed with benchmarks, but I think they have value. We aren't talking about modern SSD benchmarks where one brand reads at 3100mbps and the other at 3200mbps. That's a small difference. This is very different. We're talking about at least double the performance.
Cicerone - Friday, September 7, 2018 - link
But sometimes Kirin 970 is on the same level with 2016 Exynos 8890 found on Samsung S7.shogun18 - Tuesday, September 4, 2018 - link
> I think it's important for users to know that the Kirin 970 has a significantly weaker GPU than the S845How so? If some popular game needs 10,000 shader OPS to run at 800x600 at 30 frames/sec what difference does it make if one SoC can pump out 8000 (admittedly synthetic - are you really going to tell me you're going to notice 24FPS vs 30? pahlease), or 15,000 or another 40,000? Ok, so does OPS/Watt actually matter in anybody's evaluation metric? No. Does anyone choose a phone based on this one lets me run X game for 30 minutes before running out of batt but I can get 40 minutes with this other one because in "game mode" the manufacturer took liberties with wattage?
cfenton - Tuesday, September 4, 2018 - link
What modern phone runs at 800x600? Also, faster GPUs can get closer to 60fps, which is definitely a noticeable improvement over 30fps.If all you're playing is Candy Crush, then it doesn't matter what GPU you have, but if you're playing Fortnite or the upcoming Elder Scrolls game, then GPU performance is important. If two phones are roughly the same price, but one of them has 3x the GPU power with no downsides, I'm going to go with the faster one every time.
shogun18 - Tuesday, September 4, 2018 - link
The human eye in games like Fortnite etc can only process a very limited frame rate. So anything over 30 is basically pointless. Plus factor in using a 27+ monitor(s) vs a piddly-ass phone screen with lousy (by comparison to "gaming" monitors) refresh characteristics the benchmark is even less useful.eg. https://www.pcgamer.com/how-many-frames-per-second...
cfenton - Tuesday, September 4, 2018 - link
That article make it very clear that people can tell the difference between 60fps and 30fps. Its claim is that it's only an improvement in smoothness, not an improvement in our ability to track changes. A higher frame rate won't improve my ability to pick out movement.60fps looks better than 30fps. If I can choose between the two, at the same resolution, I'm always going to pick 60fps. Will it make me better at the game? No. Does it make the game look at feel better? Yes.
techconc - Monday, September 10, 2018 - link
@shogun18 - I always find it amusing when people present "evidence" to support their position only to find out the evidence they are producing very clearly refutes their position. The article very clearly states:"Certainly 60 Hz is better than 30 Hz, demonstrably better." - Professor Thomas Busey
From my own perspective, I would suggest to you that games need to have a 30 fps at minimum to be playable and to appear to be somewhat fluid. 60 fps is clearly better, but not "twice as good". You can see the difference though. On my iPad, I can do 120 fps on games like World of Tanks Blitz and can even notice that difference. For some games, reaction time is critical and network performance also plays a role in this. However, higher frame rates can indeed provide a competitive advantage.
shogun18 - Tuesday, September 11, 2018 - link
did you BOTHER to read to the end let alone comprehend what was being put forth? The human brain is SLOW! It's massively parallel but it's SLOW. Just like our ears are crap compared to other creatures who actually have good hearing. If you're playing FPS on a phone you're an idiot to begin with. Fluidity or more properly the perception of same doesn't make your performance better. Your reaction time is also completely shit compared to the theoretical frame rate you think you are perceiving. Anyone who cares about game play on a phone is a moron.Reflex - Tuesday, September 4, 2018 - link
Buyers should be able to value whatever they wish when making their purchasing decisions. Lying to them denies them the right to make decisions based on the criteria that matter most to them, whether it be nice cameras, great screens, excellent call quality, or yes, 'geekmarks' or whatever.It's not for you to determine what is most important to a customer, nor is it ethical to lie about one of those or other items in order to trick people who value them into buying your product.
boozed - Tuesday, September 4, 2018 - link
Funny you should say that, considering the reason for the existence of this website.Samus - Wednesday, September 5, 2018 - link
You need to put a performance metric on things somehow. Cars have horsepower and torque, batteries have volts and milliamps, and food has protein and carbs.Unfortunately these metrics do not come from the SoC manufacturer, but the phone vendor. That therein lies the problem. "Overclocking" or boosting a SoC beyond reasonable thermal design limitations is blatant cheating if it can't be sustained throughout, say, a game, that the benchmark is momentarily mimicking.
At the end of the day, this is really an Android problem too, because the freedom the OS gives phone vendors to manipulate the kernel, scheduler, and frequency curve of the CPU/GPU. This kind of flexibility didn't exist (and still doesn't exist) in other mobile operating systems.
So imagine if this were happening in the PC space. Where vendors were selling overclocked systems WITHOUT SAYING they were overclocked. Where vendors were manipulating the real-world benefits of a GPU with software that faked benchmark results.
I would liken it to what happened with the game console clones of the 80's, when there were third-party Atari's, Intellivisions, etc, that had custom CPU's running at higher frequencies. In that case, it actually hurt developers more than consumers (but still hurt consumers) because developers couldn't even depend on a performance metric for the platform they were developing for. This is partially why there were virtually no third party developers (Activision and Hudsonsoft - who later developed their own console simply to have some control over the hardware environment! - were effectively the first cross-platform developers.)
aebiv - Wednesday, September 5, 2018 - link
Benchmarking is incredibly important when people are planning and building systems, and by systems I mean those that actually support business. I'd argue you're right for most consumers though, as things are just "more than fast enough."That said, I'd still argue they're an important metric. Just might not be important to you.
websitetrafficchecker - Wednesday, February 5, 2020 - link
best website traffic estimator If you are looking for free website traffic checker, try https://seowebsitetraffic.net/check-website-traffi...Flunk - Tuesday, September 4, 2018 - link
This is not helping the perception that Chinese manufacturers will cheat and lie to their customers. That's already a big problem for Huawei in western markets. I think they need to fire everyone responsible for marketing their phones to western markets and hire native teams who actually understand the customer base.SetiroN - Tuesday, September 4, 2018 - link
You're confusing "the western markets" with the USA.Elsewhere people don't have such an obsession, actually they know American devices are probably more likely to provide backdoors.
shogun18 - Tuesday, September 4, 2018 - link
The Chinese culture is synonymous with cheating. It's an *aspiration* to cheat and "put one over" their competitor and has been for for thousands of years. You're not going to convince anyone to play by "modern" Western notions of "fair and honest" benchmarks. Even so-called Westerners have put their values aside for the almighty buck in droves. Not to suggest criminal enterprises like the East India Company (of Britain) had any intention of acting in any civil/honorable manner from the get go. Human beings are for the most part filthy, rotten, scoundrels and far too many of them will gladly commit atrocities in the name of profit or something else. Cheating is basic human behavior.Therefore, you should ALWAYS expect humans to be dishonest. See Intel for a brazen example.
Thanks for doing the article. But I also have to wonder, are there really more than a 100 basement-dwelling morons who give a flying F about how fast a GPU is on a stupid phone? And furthermore play games on such pointless platforms where frame rates or triangles/sec matters? Seriously? PC Graphics cards from 10 years ago are still stupid fast. Phone games are throwaways. Sure, some publishers some how manage to make a tidy sum but it could run in CGA and people would still buy them to fritter away their boredom. If every Youtube ran in 360p nobody would give a damn. Oh some would bitch and moan but they'd still watch. And that's the point.
Graphics performance of the SoC is just dick measuring and the percentage of people who consider the number in their purchasing valuation metric is basically zero.
goatfajitas - Tuesday, September 4, 2018 - link
American companies do it too.R0H1T - Tuesday, September 4, 2018 - link
I agree with most of what you've said about the Chinese culture but let's be 100% clear - "fair and honest" is not a modern "western" notion!Entropyq3 - Tuesday, September 4, 2018 - link
I care about SoC graphics performance, and would prefer valid data.I don’t live in a basement though, do I still count?
shogun18 - Tuesday, September 4, 2018 - link
No You're 1 in 10+ million consumers (out of just English-reading population) and therefore irrelevant except that you read Anand and/or Toms and such. :)skoondi - Sunday, September 9, 2018 - link
Utter rubbishAmandtec - Wednesday, September 5, 2018 - link
Actually, social science style studies reveal that (ballpark) 10% of people always cheat, 10% never cheat no matter what, and the 80% in the middle are a continum who are mostly swayed by perceived risks and incentives. If you are always mistrusting everyone it prevents one from building profitable relations.skoondi - Sunday, September 9, 2018 - link
Not sure about Chinese culture but you'll find more than a few people care about phone GPU performance, I can only wonder why you are on a site basically dedicated to people who are interested in such things Phones are the new computer, with a decent docking set up many people probably don't need a desktop or laptop. I've been gaming since the days of the first civilisation game and currently do a lot of gaming on my phone.techconc - Monday, September 10, 2018 - link
@shogun18 - You said:"But I also have to wonder, are there really more than a 100 basement-dwelling morons who give a flying F about how fast a GPU is on a stupid phone? And furthermore play games on such pointless platforms where frame rates or triangles/sec matters?"
It seems your comments are very much out of touch with what is happening today, even in the world of gaming.
PeachNCream - Tuesday, September 4, 2018 - link
I don't get the point in cheating on phone benchmarks or even the point of phone benchmarks in general. I also don't get the point of sticking an auto playing video about buying the right CPU on EVERY DAMNED PAGE of this article. The same video. Every time! If seeing it on the first page pissed me off and I didn't watch it there (while also being enraged at the auto starting nature of it) what on Earth would make you think it's a good idea to do it to me three more times in the same article? Did the Purch buyout somehow involve a lobotomy that removed logic and reason or was that SOP already in place beforehand?[email protected] - Tuesday, September 4, 2018 - link
I would use stronger words than that to really convey the egregious nature of repetitive onslaughts of their bloody (not a swear word) videos against us. Their blatant "up yours, screw you" attitude turns rage to a frothing mouth frenzy ready to spill blood (especially on Toms Hardware). What the flying F is wrong with allowing users to decide whether to hit the play button or not? Or am I really only talking to a lobotomized miscreant that should be restrained from interacting with the public. Cease and desist auto play videos! A "up yours, screw you" attitude is not a smart way to deal with the public. Toms are you listening, or are sending us all another finger image?shogun18 - Tuesday, September 4, 2018 - link
er it's 2018. How the hell have you been using the internet for the last decade without all browser plugins disabled or at the very least noScript (or similar) with or without AdBlock (or similar)?[email protected] - Tuesday, September 4, 2018 - link
You missed the point. Most of us are clueless.shogun18 - Tuesday, September 4, 2018 - link
No, I totally get your point and I agree (with) you have every right to be incensed with auto-play videos. However a random comment here is unlikely to get the author or site-admin's attention. And said person is doubtless unable to oppose corporate policy. When Anand's site readership takes a major nose dive (further?) and the morons in marketing (but I repeat myself) can be beaten over the head that one of the major reasons they are losing revenue and clicks is because of auto-play, THEN and ONLY then will the practice stop.There haven't been more than a minuscule handful (again, basically zero given the side of the ecosystem) that could be described as "responsible" or "good" website operators for well over 10 years. Therefore if you haven't been black-listing every feature in your browser and toggling the choices conveniently given to you, then your ire is misplaced IMO. I am an equal opportunity banner of site content. Join the club and be amazed at far more enjoyable the experience is.
[email protected] - Tuesday, September 4, 2018 - link
Put that way you may be right.1_rick - Tuesday, September 4, 2018 - link
Well, Purch has been pushing those annoying ads on Tom's Hardware for quite some time now, so I guess it's only to be expected they'd be forcing them on this site as well.TBH I hadn't even noticed the ads because I used my hosts file to block assets.purch.com, because my adblocker wasn't taking care of it. You could try that too.
sing_electric - Tuesday, September 4, 2018 - link
At some point, Huawei (and other Chinese OEMs) need to decide whether they want to build their brands globally or just in their home market."Other Chinese OEMs lie so we've got to as well" ends up doing nothing but providing ammunition for those that say that Chinese phones are "cheap," under-performing knock offs.
The Nexus 6p showed many years ago that Huawei can make good hardware. HiSilicon's chips obviously aren't doing them many favors in the GPU department, but that just means they need to target appropriate segments where they are competitive (I'm convinced that there's a large niche of people who want stylish devices that feel premium but don't really care much about performance), rather than lying and perpetuating a stereotype that will hurt their brand long after they've abandoned those practices.
A5 - Tuesday, September 4, 2018 - link
Calling the 6P "good" hardware is a bit generous. The battery subsystem has a devastating defect rate, especially since the phone is sealed.At one point Google ran out of refurbs and had to give out Pixel XLs to people as replacements.
ventrolis - Tuesday, September 4, 2018 - link
Are the charts for Aztec Normal/High flipped? Somehow I imagine the 'High' test would be more difficult and have lower frame rates than 'Normal'.Andrei Frumusanu - Tuesday, September 4, 2018 - link
Thank you for pointing it out, indeed the labels were flipped.CityZ - Tuesday, September 4, 2018 - link
Why not simply do a combined performance & power test where you run the benchmark continuously until the phone shuts down? If a phone maker tries to cheat for the performance side, they'll look bad on the run-time side. If a phone throws up a "I'm running too hot" screen, consider that the end of the test. Such a test not only shows how fast your game may perform, but also for how long you can game.Andrei Frumusanu - Tuesday, September 4, 2018 - link
Screen resolution, V-Sync and other device differences makes this kinda hard. In my view there's no added value over just peak & sustained performance as well as just measuring power.wow&wow - Tuesday, September 4, 2018 - link
"A Cheating Headache"The worst one should be the "Intel's repeatedly not following the specs" that causes the problems of "Meltdown", requiring OS memory relocation, the industry's 1st and only, and "Foreshadow" that the mitigation can only "reduce" the risk but "not eliminated" it!
Xex360 - Tuesday, September 4, 2018 - link
Because I don't consider phones to be gaming devices, for that I have a PC and a console, so benchmarks are worthless to me, the most important things in a phone are the OS (Unfortunately I'm stuck with android, iOS isn't well suited for my use), screen (high resolution and no notch) and finally battery life (around one day, an OLED screen).A5 - Tuesday, September 4, 2018 - link
Android increasingly uses the GPU to render the OS, and apps like Google Maps use it extensively as well. Sustained GPU performance isn't just relevant to gamers.eastcoast_pete - Tuesday, September 4, 2018 - link
First and foremost: Thanks Andrei and Ian! This kind of article is why I come to Anandtech again and again (and more frequently than other computer tech websites). Yes, those benchmarks are not only misleading, they also steer the manufacturers towards optimizing for an artificial use (benchmarking), often at the expense of actually optimizing their smartphones for real world use. Who knows just how much better Huawei's phones could have been for everyday use if the time and energy invested in cheating for benchmarks would have instead gone into optimizing their phones for productivity, real world applications, and battery live (Huawei gets a bonus for historically having large capacity batteries, though!). As they are, those benchmarking suites would probably come in handy if one needs to use the phone as a hand warmer in winter.One minor edit: the "High" and "Normal"labels on the the Aztec Ruins graphs are probably switched around. The fps numbers for "high" are really high for all devices. However, that is a very minor, cosmetic, point in an otherwise very good article!
Andrei Frumusanu - Tuesday, September 4, 2018 - link
As noted, the Aztec labels have been fixed.eastcoast_pete - Tuesday, September 4, 2018 - link
@Andrei and Ian: I also wonder how the apparent superiority of highly customized GPUs in Apple and QC SoCs reflects a conundrum that ARM faces with their Mali designs: Basically, Mali GPUs have to work in configurations ranging from as few as 1-2 to as many as 20 (or 24) units and with widely varying CPU cores (dual-core A53 or 55 at the low end to now A76 or mongoose octacores). In contrast, Apple GPUs and QC's Adrenos appear to be a lot more "tailored" to the SoCs they end up in, which, together with optimized drivers, probably gives them a leg up. Andrei, as you are truly an seasoned expert in this field, I wonder if you could comment on this, maybe even in some detail in a dedicated article in the future?mekpro - Wednesday, September 5, 2018 - link
Qualcomm also need to ship smaller version of Adreno to their lower-end SoC like Snapdragon 625 or so. While Anandtech haven't cover the efficiency of lower-end Snapdragon SoC, my experience confirmed that Snapdragon 625 had better much better GPU efficiency than Kirin 960 (lower temp at the same frame rate)vladx - Tuesday, September 4, 2018 - link
What about Honor 9, is it affected after upgrading to EMUI 8?nfriedly - Tuesday, September 4, 2018 - link
> So Honor is trying to promote the Honor Play as a gaming-centric phone, making bold marketing claims about its performance and experience. This is a quite courageous marketing strategy given the fact that the SoC powering the phone is currently the worst of its generation when it comes to gaming.Do you mean "courageous" in the Apple sense?
V900 - Tuesday, September 4, 2018 - link
I’m shocked and stunned, nay SHOCKED AND STUNNED that Chinese Smartphone/SOC vendors have been caught cheating in benchmarks.Such a surprising development from a country that’s known for being the high watermark in ethical and honest business practices.
(Industrial espionage and stealing stealth fighters through hacking not withstanding.)
Allan_Hundeboll - Tuesday, September 4, 2018 - link
I know Oneplus used to do something similar but they did it for benchmarks and games, witch is'nt cheating I my book because end users do get the performance shown in benchmarks.Is oneplus still doing this? The oneplus 6 seems to benchmark faster than other sd845 based phones, so this would kind of explain how.
zodiacfml - Tuesday, September 4, 2018 - link
Is this click bait or something?One cannot simply add more power/speed to a device without throttling or shutting down. If one would run gaming benchmark for more than 10 minutes then it levels out. It is a different story with browser benchmarks where burst speeds for a few seconds is valuable.
It would have been cheating if a phone starts dimming or turning off the display when there is a benchmark run.
I wouldn't put much more value on battery efficiency when running games or benchmarks because it will be difficult to regulate. Everyone will have differing opinions about it.
When you turbo/boost (any chip), it is the least efficient anyway.
unixfg - Wednesday, September 5, 2018 - link
No, not Huawei.https://www.engadget.com/2018/08/20/huawei-caught-...
mekpro - Wednesday, September 5, 2018 - link
I always need extra cooling in order to make PUBG playable on My poor Mate 9.Sometimes I use wet tissue to paste on the back of the chassis, sometimes I just put Ice and let it melt to cool the phone down. (which usually take less than 3 minutes to melt all). Yes I know this phone is not water resistance certified but come on! its just a hot potatoes that need to be cooled and I don't expect flagship phone to have this behavior. The ironic is I had buy Xiaomi Redmi Note 4X with Snapdragon 625 for 150$ and this devices play PUBG much better than Huawei's Flagship.
After read this article, I choose to not believe Huawei's marketing anymore, I don't believe that Kirin 980 can close the gap in GPU performance with Snapdragon 845, let alone acceptable performance.
beginner99 - Wednesday, September 5, 2018 - link
The most interesting aspect is that it shows that ARM also struggles with power once they get into x86 performance area. No free lunch. And I wonder how the other devices cheat. Probably most due somehow. Huawei just wan't that clever.ncsaephanh - Wednesday, September 5, 2018 - link
Great work on this piece. I really appreciate good journalism giving light to industry issues while having the technical expertise to dive deep and explain everything in a concise manner. And I wouldn't worry about catching this earlier, what's important is we know now. And hopefully at least some consumers now won't fall for the marketing/benchmarking hype.yhselp - Wednesday, September 5, 2018 - link
The GFXBench T-Rex Offscreen Power Efficiency benchmark in the Kirin 970 piece still shows the cheating result for the Mate 10.It's astonishing to see the difference in sustained performance cooling alone can attribute for - P20 Pro and Honor Play have the same maker, same SoC, similar dimentions, and yet, the performance is quite different.
Hyper72 - Wednesday, September 5, 2018 - link
I thought that ever since Samsung was caught doing the same thing in 2013 you put in active countermeasures (randomly named benchmark software, etc.) or at least a test for cheating as a standard part of your setup?tommo1982 - Wednesday, September 5, 2018 - link
These tests show similar behavior with iPhone. It's not any faster than the other leading brands. The difference between peak and sustained is huge. Same goes for Samsung and Xiaomi.I understand why the UI seems so fast and responsive, and why many people complained about the performance. It just can't stay at peak forever.
eastcoast_pete - Thursday, September 6, 2018 - link
To clarify up front: I don't own or like iOS devices. However, I have to give Apple its due here: the idea of really high, short burst performance coupled with okay longer-term speed is pretty much what I (and probably many other mobile users) want in smartphones. This is useful for multitasking while opening multiple browser windows etc., i.e. scenarios that really benefit from well above-normal CPU/GPU speeds for the few seconds, resulting in a fluid user experience. This is different from running the SoC to heat exhaustion and shutdown whenever a benchmarking app is recognized. Some current Android flagships are sort-of able to do that short burst ("turbo" in PCs) also, but none has yet the (momentary) peak performance of Apple's wide and deep cores. The Mongoose M3 was an attempt, the Kirin 980 was an apparent step towards this, sort of, but is now marred by this benchmark cheating BS. Let's see what QC can cook up, they tend to get closest to Apple's top SoC.techconc - Monday, September 10, 2018 - link
Thermal throttling happens on ALL phones. That's not what's in question. The issue is with companies that artificially white list specific benchmarks in order to achieve results that would not be seen in real applications.To that end, Anandtech's battery tests have always demonstrated the difference between peak and sustained performance in mobile devices. Up through the iPhone 6s, there was very little throttling going on with iPhones on peak loads. To your point, the level of throttling in iPhones has been approaching practices of common Android equivalents.
psychobriggsy - Thursday, September 6, 2018 - link
Naughty. Makes running a benchmark in a 'loop mode' until the battery runs out very important IMO. If the device dies in an hour in benchmarks, but 3 hours elsewhere, then you know something's awry.However there is a potential positive - it shows that the Kirin 970 can perform well at higher power consumption - there's no performance wall between 3.5W and 9W, and the perf/W scales fairly well too.
So - why not look into a 'docked' mode option in the future? One option could be a Switch-like dock, using external power (to protect the battery), optional cooling assistance, HDMI out to a TV, provide a controller in this pack as well, and allow the SoC to run as fast as this setup can keep the device from damaging itself. That's flippin' marketable. The dock would cost a few dollars, and it sounds like the software is already there in the main.
Hopefully the Mali G76 in the Kirin 980 actually fixes a lot of the performance issues with Mali, which surely were a factor in this sad situation (also clearly saving money by using a smaller GPU, wide and slow beats narrow and fast for GPUs where power consumption matters.
hanselltc - Friday, September 7, 2018 - link
wut if: the white list includes popular games as well? is that still cheating?s.yu - Monday, September 10, 2018 - link
Obviously you haven't read the article, the so-called whitelisting's performance can't be sustained, it's not as simple as merely activating some sort of game mode automatically.arpit - Saturday, September 8, 2018 - link
Benchmarking these phone after few times will degrade its SOC due to overheating. Idk why Huawei/Honor has resorted to using this cheat tricks.LemmingOverlord - Tuesday, September 11, 2018 - link
I think the real questions Andrei and Ian should be asking is whether the devices are detecting not only benchmarking software but also popular software (games included) that people use for extended periods. If that is the case, the device's behavior will shorten its lifespan over a shorter amount of time.ashraf mahdy - Friday, February 8, 2019 - link
I know this thread is old but I wanted to share my story as well, so yesterday EMUI 9 updated arrived for my Honor Play in my country. bringing the "performance mode" toggle from the battery settings. anyone watching "Gary Explains" on youtube knows that Kirin 980 can have increased performance turning on "performance mode" in the battery settings.before updating i ran a few antutu benchmarks, my scores were 206-210,000. my highest was 212,000. using antutu sensors my Hottest CPU & battery were 41-43C
right after updating I was getting 165-170,000. and my lowest score was 153,000. which is basically SD660 & Kirin 710 Performance, but my phone was not getting warm at all, like max CPU & Battery was 32C after 3 tests. at this moment I was like "HUAWEI THOSE LYIN CHEATIN BASTARDS" then I remembered Performance Mode, Turned it on, and BOOM, ran 3 Tests, was getting around 206-210,000. with my LOWEST Score being 198,000 after 4 looping tests and my SoC & battery were peaked at 41C. and after letting the phone cooldown to around 19C I ran a final test and got 216,000 with SoC & battery at like 36C.
my point is as follows.
SoC sustained performance vs peak can vary by as much as 20,000 on antutu, it all depends on the phone's cooling ability. the Mate 10 Pro has thermal paste on the Kirin 970. the honor Play does not. (check Jerryrigeverything for teardown of M10Pro).
"performance mode" is not "cheat mode". the kirin 970 may not be as efficient as SD835 but it does manage to go neck and neck. it's like desktop GPUs. the RX 580 vs 1060.
yes in phones batteries are the main culprit. but that peak power draw only happens when gaming, and let's face it, no one games for 1-2 hours on their phones, it's more like 2-3 games of PUBG mobile or a few fifa soccer matches. 30mins or so.
so Huawei are shady af yes, Qualcomm's SD is better yes. but kirin is respectable as well.
jessicacoles - Saturday, December 12, 2020 - link
https://www.shampooadvice.com/