Contents
The world of technology is developing rapidly, of course performance standards are needed, because the pure technical data are no longer sufficient to evaluate a device. For this reason, the so-called benchmarks were developed. But why are they used? And do they provide reliable information about the performance of a device?
- WikiPIT: All your questions are answered here
- Do you know the new technology that will make your WiFi more secure?
What are these benchmarks anyway?
Benchmarks are programs that are designed to load CPUs, GPUs, memory or practically every aspect of a smartphone or computer and measure their performance. In the end there is a result in the form of a score, a time, a speed or the like. There are also very special tests, such as from DxOMark for evaluating camera focus times or test programs to determine the speed of the Internet network. These also fall under the category of benchmarks. Of course, we mainly use smartphone benchmarks, which we use to evaluate smartphones for our performance tests.
We can divide benchmarks into two broad categories:
- Synthetic benchmarks
- Everyday benchmarks
The first and most frequently used benchmarks simulate an extreme workload and demand the tested components with their full capacity. This enables us to determine the maximum performance of a device.
Everyday real-world benchmarks simulate realistic (albeit still very demanding) workloads and are more useful in assessing a device’s actual capabilities for daily purposes and applications.
What is the point of benchmarks?
Benchmark applications are specifically designed for one purpose: to measure the performance of the device they are running on, to compare them with other models and their measurements. The word benchmark can be translated as a comparative analysis.
As the complexity of electronic devices increases, it becomes increasingly difficult to compare their performance against the data on paper. That is why new measurement programs have been and are being developed all the time.
Can we trust benchmarks?
Theoretically yes. The values calculated by benchmarks are always determined by performing the same operations on each machine, so that there is no difference in the treatment of the tested devices. However, you need to pay attention to the condition of the device before running the benchmark.
A practical example: If apps run in the background on a smartphone while the benchmark is running, the result is of course different than on a second smartphone, on which only the smartphone benchmark runs without other apps open. This is because some of the resources of the device under test are used by other programs or applications, which affects the bottom line.
But manufacturers have also been caught doing lazy tricks. For example, OnePlus has been found to increase the performance of its smartphones while running the most popular and widely used Android benchmarks. By recognizing the running application, certain manufacturers can implement special procedures to improve hardware performance and achieve higher values. If you know exactly how these tests work, you can optimize your product so that it scores high in benchmarks, but this does not always translate into better performance in real life.
- Xiaomi Mi 8 in the performance test: strong eighth misses benchmark gold
Looks at benchmark values as a pointer rather than a fact
Benchmarks are only really useful if all devices used for comparison are treated equally. If all tests are carried out under the same conditions, you can at least judge on paper which product is objectively better. Why only “on paper”? Benchmarks are not only easy to influence, but generally do not reflect the use of a device in practice.
Sometimes smartphones can achieve astonishing benchmark results, but cannot offer acceptable everyday performance. Just think of Samsung’s Galaxy S series, which has the highest level of hardware, but sometimes because of its software is not as smooth and responsive as smartphones with less power on paper than Google’s pixel devices .
- Samsung Galaxy S9 and S9 + in direct comparison: bigger wins
Opinion by Luca ZaninelloBenchmarks give me an idea of the performance level of a smartphone, but have no influence on the choice of my new device.Do you agree?505070 participants
Do you rely on benchmarks when choosing a new device?