Thing about benchmark is: it’s how we evaluate a device on how it performs in those tests to give a rough idea on how its kind-of translates to real world testing. In the past, other manufacturers have been caught for this mischief but this time, MediaTek did something that made UL Benchmarks hit the sledgehammer on 8 MediaTek Processors.
MediaTek Processor’s Misleading Benchmarks
Earlier, AnandTech – a site where they are pretty darn great at what they do, reported in their article that quite the number of MediaTek processor powered devices produced artificially high and misleading benchmark scores – this considering coming from one of the biggest semiconductor manufacturer is quite the news. That said, UL Benchmark – a tool we here at The Adventures of Vesper use to benchmark our review units to show you the benchmark scores, has delisted over 50 smartphone models powered by MediaTek Processors. If you happen to go to PCMark for the scores on these devices, it will now appear as unranked and UL recommends that these devices are better not used for comparison.
List of Delisted Models
UL Benchmarks have delisted the following processors temporarily – and these processors happens to be powered by devices like Xiaomi Redmi Note 8 Pro, Oppo Reno3 Pro, Vivo Y19 and Realme 6. In total, over 50 devices from more than 25 different vendors have been delisted. The processors are as follows:
- MediaTek Helio G90
- MediaTek Helio G70
- MediaTek Helio P95
- MediaTek Helio P90
- MediaTek Helio P65
- MediaTek Helio P60
- MediaTek Helio P20
- MediaTek Helio A22
Why UL Delisted these Devices?
UL Benchmark said the following:
Anandtech tested the Oppo Reno3 Pro with MediaTek Helio P95 chipset with the public version of PCMark for Android, available from Google Play, and a private version of the app that is not available to the public or manufacturers. Anandtech found a big difference in the scores from the public and private apps, even though the tests in each are identical. In Anandtech’s words,
“…the two versions of the benchmark did differ in their scores – but I was still aghast at the magnitude of the score delta: a 30% difference in the overall score, with up to a 75% difference in important subtests such as the writing workload.”
With a bit of digging, Anandtech found a benchmark whitelist in the device’s firmware files. Further investigation found similar whitelist files on a host of devices powered by MediaTek chipsets,
“Inspect the file, there we find amongst what seems to be a list of popular applications with various power management tweaks applied to them, with lo and behold, also a list of various benchmarks. We find the APK ID for PCMark, and we see that there’s some power management hints being configured for it, one common one being called a ‘Sports Mode.'”
Among our Malaysian tech journalist, Nasi Lemak Tech mentioned about Realme 6 having the script, which can be found in the .xml screenshot in their article. In fact, head over to AnandTech to read theirs for a much better in-depth on what’s going on. Honestly, the smartphones manufacturers aren’t at fault here – in fact, they didn’t see it coming, but this does affect the brands reputation and truth be told, MediaTek doesn’t have a good reputation to begin with and with this now, its going to be quite the challenge to redeem themselves. For what it’s worth, it’s good UL Benchmarks took the initiative to delist these devices.
We are reaching out to UL Benchmarks in regard to the whole situation to get a better clarification on how it works. So, keep your eyes peeled for that.