Huawei Caught Cheating Performance Test For New Phones (techcrunch.com) 74
An anonymous reader quotes a report from TechCrunch: UL, the company behind the tablet and phone performance benchmark app 3DMark, has delisted new Huawei phones from its "Best Smartphone" leaderboard after AnandTech discovered the phone maker was boosting its performance to ace the app's test. The phones delisted were the P20, P20 Pro, Nova 3 and the Honor Play. "After testing the devices in our own lab and confirming that they breach our rules, we have decided to delist the affected models and remove them from our performance rankings," the company said in a statement.
For the Huawei case, the rules are actually a little fuzzy. Phones are permitted to adjust performance based on workload, which results in peaks or dips in performance for different apps, but they are not permitted to hard-code peaks in performance specifically for the benchmark app. Huawei reportedly claimed that the peak in performance seen during the run of the benchmark app was an intuitive jump determined by AI; however, when an unlabeled version of the benchmark test was run, the phones were unable to recognize it and, as a result, displayed lower performances. In other words, the phones aren't so smart after all.
For the Huawei case, the rules are actually a little fuzzy. Phones are permitted to adjust performance based on workload, which results in peaks or dips in performance for different apps, but they are not permitted to hard-code peaks in performance specifically for the benchmark app. Huawei reportedly claimed that the peak in performance seen during the run of the benchmark app was an intuitive jump determined by AI; however, when an unlabeled version of the benchmark test was run, the phones were unable to recognize it and, as a result, displayed lower performances. In other words, the phones aren't so smart after all.
From the country that puts lead in children's toys (Score:1)
Me Chinese. Me play joke. Me go peepee in your Coke.
- Huawei CEO
Re: (Score:2)
Alzheimer on devices - here we come.
Re:Stop using foreign products (Score:5, Informative)
Re: (Score:2)
You DO know Apple phones are made in China, right?
Name 1 that is manufactured in the USA.
Re: (Score:2)
Apple has an American design team, but outsources all production and is incredibly expensive....
It be nice to see more American production, and maybe some mid grade stuff at a lower price point....
Samsung's flagship phones are every bit as expensive as Apple's flagship phone.
Now where's your Hate against Samsung?
Re: Stop using foreign products (Score:2)
At least many Samsung phones are made in South Korea
Re: (Score:2)
At least many Samsung phones are made in South Korea
Since the OP was talking about AMERICAN production, that is a Strawman argument.
Re: (Score:2)
Well he was talking about foreign production. The USA is as foreign to me as South Korea as I live in either.
But unlike China, South Korea actually has decent wages, which I thought was the point.
Re: (Score:2, Insightful)
There is no such thing as an American-made phone. In fact, there is almost no American-made anything when it comes to electronics. Prior administrations have allowed the manufacturing sector to move almost everything to enemy states.
Make no mistake. If there were ever a war, the US would lose, and quickly, because we cannot maintain, repair, or build anything without China's help. We have nearly zero electronics manufacturing capability compared to what would be needed to prosecute a War.
Re: (Score:3)
Also, U.S. manufacturing output has been increasing year over year even though we were moving large chunks of it overseas. I think now would be a good time to start reinvesting in local manufacturing, but that would be done with machines. A lot of the jobs aren't coming back, but that's okay because it means that labor is free to do something that's more product
Re: (Score:2)
Oh, no, wait,I forgot.... we're all supposed to re-tool for the new economy. Because businesses want to hire 40 year olds for entry level tech jobs....
Re: (Score:2)
The U.S. still does a lot of chip design and fabrication. It's just that the chips get shipped overseas for packaging and final product assembly.
Also, U.S. manufacturing output has been increasing year over year even though we were moving large chunks of it overseas. I think now would be a good time to start reinvesting in local manufacturing, but that would be done with machines. A lot of the jobs aren't coming back, but that's okay because it means that labor is free to do something that's more productive instead.
Ya know, for all the Apple-bashing around here, they have actually been doing manufacturing/final-assembly of at least one of their products here since 2013: The (often-maligned) Mac Pro.
Granted, it's not the highest-volume product Apple sells; but even if it represents just 1% of Apple's nearly 20 million Mac units sold in 2017, that still represents a quantity of nearly 2 million units of $2k-5k Mac Pros per year (for an average yearly income of $5 BEELION in gross sales), which is a production rate and i
Re: Stop using foreign products (Score:1)
Just build a wall against them. Call it the Chinese wall to make a statement.
Re: (Score:2)
Stick with Apple. Buy American, support American companies and the Americans they employ, and put your money where your mouth is.
Don't you mean "Buy Chinese?" Apple phones are made in China, not America.
Re: (Score:2)
Re: Stop using foreign products (Score:1)
Could you remind me, when did the US stop being a foreign country?
American made is a myth (Score:2)
Stick with Apple. Buy American, support American companies and the Americans they employ, and put your money where your mouth is.
Where do you think Apple products are made?
There are very few products that are "American made" and you can say the same thing about every other country too. Furthermore brands are useless in determining where something was made. I have a truck that is Honda branded but it was the most American made [tfltruck.com] vehicle in 2017 (yes more than the Ford F150. But the content in it is from all over the globe. The concept of "American products" or "Chinese products" is largely a myth for any non trivial product. Manufa
Re: Stop using foreign products (Score:1)
Sucking gold songs?
Also I'm Swede. I'm only allowed to buy political correctness and progressivism?
Guess that's in line with the parliament. They think people talking bad about the situation of the country are traitors. Preferably forbidden.
Strike two! (Score:2)
Cheating and spying are no way to go through life, son.
Re: (Score:1)
Damn right. Let that be a lesson to Americans, because there is no other country that has cheated, rigged, lied, spied, and manipulated as much as America to get to where it is.
AI (Score:5, Insightful)
Whenever someone makes an AI claim you know they are lying.
Re: (Score:2)
No, not all car brands. VW/Audi/Porsche all had variations on the diesel emissions cheat. They're in a class by themselves.
https://en.wikipedia.org/wiki/... [wikipedia.org]
Hardly seems worth it (Score:4, Insightful)
If you know what chipset is in a phone, then you know as much as you usefully need to know about the real world performance of that phone relative to other phones with the same chipset, i.e. pretty much the same. If you see a benchmark saying one such phone is outperforming its peers, then you assume that, if it's not down to actual cheating, then it's a quirk of the benchmark, because you know the same cpu/gpu at the same speed gives the same performance. Does anybody actually look at benchmarks when buying a phone?
Re: (Score:1)
What's more: using the same SoC, performance may still vary due to cpu / gpu / memory clocks. But higher clocks translate to higher power consumption. For a portable device, that means more battery drain. So even if you can run apps a few % faster, you'll only be able to do (roughly) the same amount of work before battery runs empty.
In battery operated devices, you'll often see that components aren't clocked up to their highest possible spec. The above is one important reason for that. Local heating / st
Re: (Score:1)
I assume they either have connected the phone to AC with the charger, or not, and have not changed this aspect when testing with the original 3DMark app and then with a replacement named differently.
This is cheating, and thus I will avoid Huawei/Honor in the future when making purchase decisions. It is the same pattern as with Volkswagen (VW).
Re: (Score:3)
"For a portable device, that means more battery drain. So even if you can run apps a few % faster, you'll only be able to do (roughly) the same amount of work before battery runs empty"
Actually no. In the olden days the supply voltage was fixed, leakage current was low resulting in a constant energy per cycle for a given operation. In modern times the supply voltage is carefully adjusted with clock speed. Run the part down at 1V and it might only run at 1 GHz, crank it up to 1.5V and it goes at 2 GHz, bu
Re: (Score:2)
Not true at all. The same SoC can have a wide difference in performance due to differing thermal headroom depending on the device.
Re: (Score:2)
Volkswagon (Score:1)
So now we know where ex-Volkswagen engineers end up.
Quack Quack Quack (Score:1)
Quake/Quack3 was 17 years ago and people still haven't learnt.
Heavens (Score:3)
Re: (Score:2)
Yeah because American companies would never cheat benchmarks or add backdoors. Right? RIGHT?
Re: (Score:1)
Re: (Score:2)
Then they would have not caught this current bullshit...
Re: Why aren't all benchmarks run "unlabelled"? (Score:2)
Yeah, this is my thought too. Except they should run it in both modes, so they can easily detect a discrepancy and hence a cheating attempt.
They should be running the benchmark at least 2 or 3 times and averaging the results anyway, and so 1 or 2 extra runs with the unlabeled version, and see if the performance is out of line with the standard average, would not be increasing the overall test duration by much.
Re: (Score:2)
But is this what GPU manufacturers have been doing (Score:1)
But is this what GPU manufacturers have been doing for years?
Re: (Score:2)
So far as I know, the above sites are still looking for cheating and GPU manufacturers have stopped. They've at least stopped obviously cheating by looking for when a know