Huawei phones removed from a best smartphone list after caught gaming performance test
Huawei joins the benchmark manipulation club
Huawei admitted that some of its phones run faster in benchmarking tests than in normal performance... then said everyone else does it too, and performance tests aren't real anyway.
UL, the makers of a computing performance testing app called 3DMark, removed a number of Huawei smartphones from its listings after the phones were discovered to run faster during its benchmark tests.
3DMark's benchmarks are widely respected, and offer a standard way to compare performance across different devices, whether they run iOS, Android, or even Windows. They're often used in ads by manufacturers to show why their phones are better than the competition, giving handset makers an incentive to try to top the rankings.
AnandTech found that four Huawei devices (and one from sub-brand Honor) run faster in benchmarking tests than normal performance: The Nova 3, Honor Play, and the flagship P20 and P20 Pro. The phones literally get a boost in power by maxing out power consumption, delivering much better performance at the expense of battery life -- something impractical in normal use, as anyone who's seen their phone battery die before the day is out can attest to.
Funnily enough, Huawei -- who have in the past made a big deal about the AI capabilities of its handsets -- tried to spin it as a positive, telling us in a statement that the phone will "intelligently" switch to a high-performance mode when it detects a benchmarking app.
Except... that's not entirely true. According to 3DMark, it was able to prove the misleading test results because Huawei's phones only recognized the public version of its benchmarking app. A private version of the app -- the same tests, but not available outside of 3DMark maker UL -- showed a much lower level of performance.
Huawei's defense? Everyone does it. “In the Android ecosystem, other manufacturers also mislead with their numbers”, said Huawei's Wang Chenglu to AnandTech.
To be fair, they're not wrong. Last year, OnePlus was found to have boosted the performance of the 3T for benchmark tests. And in 2013, Samsung and a slew of Android device makers including HTC, LG and Sony, was also caught doing the same thing.
In their statement, Huawei also argued that current benchmarking tests don't accurately evaluate the user experience anyway... which is an odd position to take after admitting to gaming those same tests, but hey.
Anyway, if you feel like running your own benchmarking tests, you're in luck! Huawei said it will soon allow users to toggle performance mode themselves.