While there might be people who believe that a prospective buyer does not base their product purchases just over benchmark score results, there is no doubt about the fact they are at some point important to an end user. For the same reason, Futuremark, the company that makes the popular 3D Mark benchmarking tool has just de-listed multiple devices from Samsung and HTC for allegedly tweaking certain handsets to perform well when subjected to benchmarking processes. This tweaking resulted in inflated benchmark scores – usually higher than what a similar device from another manufacturer with the same specs might score.
The delisted devices include popular handsets like the HTC One, HTC One Mini, Samsung Galaxy S4 and the recently launched Galaxy Note 3. These devices have been stripped off their ranks and scores and a prominent “Delisted” tag appears right next to their images. Their rankings too have been demoted to the bottom of the list.
It was earlier this year that news first came out about the fact that Samsung (and later HTC) were optimizing their handsets to perform “better” whilst running certain benchmarking applications. This resulted in these devices scoring better than their competition and thereby giving them an unfair advantage. Apparently, this was against Futuremark’s benchmarking policies and they have finally decided to delist the offending devices.
Futuremark’s benchmarking policies clearly state that hardware makers aren’t allowed to make optimizations specific to 3DMark. In fact, devices and platforms are not even allowed to detect the fact that 3D mark is being run at any given point of time.
In a statement released by Futuremark, they assert:
“People rely on Futuremark benchmarks to produce accurate and unbiased results. That’s why we have clear rules for hardware manufacturers and software developers that specify how a platform can interact with our benchmark software. In simple terms, a device must run our benchmarks without modification as if they were any other application.”
It is unclear at this stage if these devices would return to the Futuremark website any time soon. Samsung and HTC are yet to comment on this.
