Difference between revisions of "Benchmarking"

From ArchWiki
Jump to: navigation, search
(Adding iperf)
m (OS X is now called macOS)
 
(63 intermediate revisions by 19 users not shown)
Line 1: Line 1:
 
[[Category:Hardware]]
 
[[Category:Hardware]]
 +
[[ja:ベンチマーク]]
 +
[[zh-cn:Benchmarking]]
 +
{{Related articles start}}
 +
{{Related|Benchmarking/Data storage devices}}
 +
{{Related|Improving performance}}
 +
{{Related|Sysstat}}
 +
{{Related articles end}}
 +
 
Benchmarking is the act of measuring performance and comparing the results to another system's results or a widely accepted standard through a unified procedure. This unified method of evaluating system performance can help answer questions such as:
 
Benchmarking is the act of measuring performance and comparing the results to another system's results or a widely accepted standard through a unified procedure. This unified method of evaluating system performance can help answer questions such as:
  
Line 7: Line 15:
  
 
Many tools can be used to determine system performance, the following provides a list of tools available.
 
Many tools can be used to determine system performance, the following provides a list of tools available.
== Stand alone tools ==
+
== Standalone tools ==
===glxgears (and why not to use this as a benchmark) ===
+
===glxgears===
 +
glxgears is a popular OpenGL test that renders a very simple OpenGL performance and outputs the frame rate. Though glxgears can be useful as a test of direct rendering capabilities of the graphics driver, it is an outdated tool that is not representative of the current state of GNU/Linux graphics and overall OpenGL possibilities. glxgears only tests a small segment of the OpenGL capabilities that might be used in a game. Performance increases noted in glxgears will not necessarily be realized in any given game. See [http://wiki.cchtml.com/index.php/Glxgears_is_not_a_Benchmark here] for more information.
  
# pacman -S mesa-demos
+
glxgears can be installed via the {{pkg|mesa-demos}} and {{pkg|lib32-mesa-demos}} (for [[Multilib]]) packages.
  
Glxgears is a popular OpenGL test that renders a very simple OpenGL performance and outputs the frame rate. Though glxgears can be useful as a test of direct rendering capabilities of the graphics driver, glxgears is an outdated tool that is not representative of the current state of linux graphics and overall OpenGL possibilities.
+
=== UnixBench ===
 +
A unixbench package is available in the [[AUR]]: {{AUR|unixbench}}.
 +
To run benchmark run ''ubench'' in terminal.
  
So to summarize, glxgears only tests a small part of what you typically see in a 3D game. You could have glxgears FPS performance increase, but your 3D game performance decrease. Likewise, you could have glxgears performance decrease and your 3D game performance increase. [http://wiki.cchtml.com/index.php/Glxgears_is_not_a_Benchmark source]
+
See also:
 
+
* https://github.com/kdlucas/byte-unixbench
=== superpi ===
+
* https://github.com/kdlucas/byte-unixbench/blob/master/UnixBench/USAGE
http://myownlittleworld.com/miscellaneous/computers/pilargetable.html
+
  
 
=== interbench ===
 
=== interbench ===
This benchmark application is designed to benchmark interactivity in Linux.  
+
interbench is an application designed to benchmark interactivity in Linux. It is designed to measure the effect of changes in Linux kernel design or system configuration changes such as CPU, I/O scheduler and filesystem changes and options. {{Tip|With careful benchmarking, different hardware can be compared.}}
  
It is designed to measure the effect of changes in Linux kernel design or system configuration changes such as CPU, I/O  scheduler and filesystem changes and options. With careful benchmarking, different hardware can be compared.
+
interbench is available in the [[AUR]]: {{AUR|interbench}}.
  
interbench is available in the AUR: https://aur.archlinux.org/packages.php?ID=2093
+
See also:
 +
* [[Realtime process management]]
 +
* [[Advanced traffic control]]
 +
* [[Linux-ck]]
 +
* [[Linux-pf]]
  
=== (n)ttcp ===
+
=== ttcp ===
(n)ttcp measures point-to-point bandwidth over any network connection, the binary must be provided on both nodes between which you want to determine bandwidth. nttcp is a a much more convenient rewrite of ttcp.
+
(n)(nu)ttcp measures point-to-point bandwidth over any network connection. The program must be provided on both nodes between which bandwidth is to be determined.
 +
 
 +
Various flavors of ttcp can be found in the [[AUR]] (see links below).
 +
 
 +
See also:
 +
* {{AUR|ttcp}}
 +
* {{AUR|nttcp}}{{Broken package link|{{aur-mirror|nttcp}}}}
 +
* {{AUR|nuttcp}}
  
 
=== iperf ===
 
=== iperf ===
iperf is a very easy to use point-to-point bandwidth testing tool that can use either TCP or UDP. It has nicely formatted output and a parallel test mode.
+
iperf is an easy to use point-to-point bandwidth testing tool that can use either TCP or UDP. It has nicely formatted output and a parallel test mode.
 +
 
 +
{{pkg|iperf}} can be installed, or a different version of iperf is available with {{Pkg|iperf3}}.
  
nttcp is available on the AUR: https://aur.archlinux.org/packages.php?ID=11469
 
 
=== time ===
 
=== time ===
 
The time command provides timing statistics about the command run by displaying the time that passed between invocation and termination. Time is available on most basic linux systems.
 
The time command provides timing statistics about the command run by displaying the time that passed between invocation and termination. Time is available on most basic linux systems.
  time tar -zxvf archive.tar.gz
+
 
=== times ===
+
  $ time tar -zxvf archive.tar.gz
{{Expansion|Obvious! Write sth and delete this. Or delete the heading.}}
+
  
 
=== hdparm ===
 
=== hdparm ===
You can Benchmark storage media with [[Hdparm]].
+
Storage media can be benchmarked with [[Hdparm]] ({{pkg|hdparm}}).
  
This can also be done with dd and Co. There is an Article about [[Benchmarking disk wipes]].
+
See also [[Benchmarking/Data storage devices#Using hdparm]].
  
=== unigine (Heavens, Tropics and Sanctuary OpenGL Benchmark) ===
+
=== Unigine Engine ===
Unigine[http://www.unigine.com/] has produced several modern OpenGL benchmarks based on their graphics engine with features such as:
+
[http://www.unigine.com/ Unigine corp.] has produced several modern OpenGL benchmarks based on their graphics engine with features such as:
* Per-pixel dynamic lighting
+
* Per-pixel dynamic lighting
* Normal & parallax occlusion mapping
+
* Normal & parallax occlusion mapping
* 64-bit HDR rendering
+
* 64-bit HDR rendering
* Volumetric fog and light
+
* Volumetric fog and light
* Powerful particle systems: fire, smoke, explosions
+
* Powerful particle systems: fire, smoke, explosions
* Extensible set of shaders (GLSL / HLSL)
+
* Extensible set of shaders (GLSL / HLSL)
* Post-processing: depth of field, refraction, glow, blurring, color correction and much more.
+
* Post-processing: depth of field, refraction, glow, blurring, color correction and much more.
  
unigine-heavens is available in the AUR: https://aur.archlinux.org/packages.php?ID=35901
+
Unigine benchmarks have found recent usage by those looking to overclock their systems. Heaven especially has been used for initial stability testing of overclocks.
  
unigine-tropics is available in the AUR: https://aur.archlinux.org/packages.php?ID=25420
+
These benchmarks can be found in the [[AUR]] (see links below).
  
unigine-sanctuary is available in the AUR: https://aur.archlinux.org/packages.php?ID=45959
+
See also:
 +
* {{AUR|unigine-heaven}}
 +
* {{AUR|unigine-tropics}}
 +
* {{AUR|unigine-sanctuary}}
 +
* {{AUR|unigine-valley}}
  
 
== Software suites ==
 
== Software suites ==
 
=== Bonnie++ ===
 
=== Bonnie++ ===
This C++ rewrite of the original Bonnie benchmarking suite is aimed at performing several tests of hard drive and filesystem performance.
 
  
Bonnie++ is available in package {{Pkg|bonnie++}}.
+
{{pkg|bonnie++}} is a C++ rewrite of the [http://www.textuality.com/bonnie/ original Bonnie] benchmarking suite is aimed at performing several tests of hard drive and filesystem performance.
  
=== iozone ===
+
{{Note|The original Bonnie suite does not appear to have been released under the GPL or other compatible license.}}
Iozone is useful for performing a broad filesystem analysis of a vendor’s computer platform. See this thread: https://bbs.archlinux.org/viewtopic.php?pid=969463
+
  
iozone is available in {{AUR|iozone}}.
+
See also:
  
=== Hardinfo ===
+
* [http://www.coker.com.au/bonnie++/ Author's site]
HardInfo can gather information about your system's hardware and operating system, perform benchmarks, and generate printable reports either in HTML or in plain text formats. Hardinfo performs CPU and FPU benchmarks and has a very clean GTK UI.
+
* [[Wikipedia:Bonnie++]]
  
hardinfo is available in {{Pkg|hardinfo}}.
+
=== IOzone ===
 +
IOzone is useful for performing a broad filesystem analysis of a vendor’s computer platform.
 +
 
 +
This program is available in the [[AUR]]: {{AUR|iozone}}.
 +
 
 +
See also BBS Article: [https://bbs.archlinux.org/viewtopic.php?pid=969463 iozone to evaluate I/O schedulers... results NOT what you'd expect!].
 +
 
 +
=== HardInfo ===
 +
{{pkg|hardinfo}} can gather information about your system's hardware and operating system, perform benchmarks, and generate printable reports either in HTML or in plain text formats. HardInfo performs CPU and FPU benchmarks and has a very clean GTK-based interface.
 +
 
 +
See also [http://wiki.hardinfo.org/HomePage Author's site].
  
 
=== Phoronix Test Suite ===
 
=== Phoronix Test Suite ===
The [http://www.phoronix-test-suite.com/ Phoronix Test Suite] is the most comprehensive testing and benchmarking platform available for the Linux operating system. This software is designed to effectively carry out both qualitative and quantitative benchmarks in a clean, reproducible, and easy-to-use manner. This software is based upon the extensive Linux benchmarking work and internal tools developed by Phoronix.com since 2004 along with input from leading tier-one computer hardware vendors. This software is open-source and licensed under the GNU GPLv3. The Phoronix Test Suite consists of a lightweight processing core (pts-core) with each benchmark consisting of an XML-based profile with related resource scripts. The process from the benchmark installation, to the actual benchmarking, to the parsing of important hardware and software components is heavily automated and completely repeatable, asking users only for confirmation of actions.
+
''The [http://www.phoronix-test-suite.com/ Phoronix Test Suite] is the most comprehensive testing and benchmarking platform available that provides an extensible framework for which new tests can be easily added. The software is designed to effectively carry out both qualitative and quantitative benchmarks in a clean, reproducible, and easy-to-use manner.
  
This suite can be [[pacman|Installed]] with the package {{pkg|phoronix-test-suite}} , available in the [[Official Repositories]].
+
''The Phoronix Test Suite is based upon the extensive testing and internal tools developed by Phoronix.com since 2004 along with support from leading tier-one computer hardware and software vendors. This software is open-source and licensed under the GNU GPLv3.
  
=== PTS Desktop Live 2010.1 ===
+
''Originally developed for automated Linux testing, support to the Phoronix Test Suite has since been added for OpenSolaris, Apple macOS, Microsoft Windows, and BSD operating systems. The Phoronix Test Suite consists of a lightweight processing core (pts-core) with each benchmark consisting of an XML-based profile and related resource scripts. The process from the benchmark installation, to the actual benchmarking, to the parsing of important hardware and software components is heavily automated and completely repeatable, asking users only for confirmation of actions.
 +
 
 +
''The Phoronix Test Suite interfaces with OpenBenchmarking.org as a collaborative web platform for the centralized storage of test results, sharing of test profiles and results, advanced analytical features, and other functionality. Phoromatic is an enterprise component to orchestrate test execution across multiple systems with remote management capabilities.''
 +
 
 +
This suite can be [[pacman|installed]] with the package {{pkg|phoronix-test-suite}}. There is also a developmental version available with {{AUR|phoronix-test-suite-git}}.
 +
 
 +
=== PTS Desktop Live ===
 +
{{Warning|The live image does not look like it has been maintained since 2010.}}
 
As an alternative to the installation of the Phoronix Test Suite to the system, Phoronix also provides a Live-CD. This Live-CD offers all the features of the Phoronix Test Suite and includes the latest [[ATI]] and [[NVIDIA]] binary drivers. It will allow you to run 40+ benchmarks from a live environment without the need to store anything on your hard drive and includes a working GUI interface.
 
As an alternative to the installation of the Phoronix Test Suite to the system, Phoronix also provides a Live-CD. This Live-CD offers all the features of the Phoronix Test Suite and includes the latest [[ATI]] and [[NVIDIA]] binary drivers. It will allow you to run 40+ benchmarks from a live environment without the need to store anything on your hard drive and includes a working GUI interface.
  
Website: http://www.phoronix-test-suite.com/?k=pts_desktop_live
+
See also:
 +
* [http://www.phoronix-test-suite.com/?k=pts_desktop_live Official link]
 +
* [http://www.phoronix-test-suite.com/documentation/2.4/pts_desktop_live.html Documentation]
 +
 
 +
== Flash media ==
 +
 
 +
Performance characteristics can be measured quantitatively using {{AUR|iozone}}. ''Sustained'' read and write values can, but often do not, correlate to real-world use cases of I/O heavy operations, such as unpacking and writing a number of files on a system update. A relevant metric to consider in these cases is the '''random write''' speed for small files.
 +
 
 +
The example invocation tests a 10M file using a 4k record size:
 +
 
 +
$ iozone -e -I -a -s 10M -r 4k -i 0 -i 1 -i 2
 +
...
 +
 +
                                                                random  random
 +
              kB  reclen    write  rewrite    read    reread    read    write
 +
            10240      4      661      649    5802    5822    3892      624
  
Documentation: http://www.phoronix-test-suite.com/documentation/2.4/pts_desktop_live.html
+
{{Note|
 +
* Test values are reported in KB/s.
 +
* For performance charts on SD cards and other flash media, see for example [http://www.tomshardware.com/charts/memory-cards,39.html Tom's Hardware].
 +
}}
  
== Related Links ==
+
See also:
 
* [http://lbs.sourceforge.net/ Linux Benchmarking Homepage]
 
* [http://lbs.sourceforge.net/ Linux Benchmarking Homepage]
* [http://www.phoronix.com/scan.php?page=home Phoronix.com]
+
* [http://www.phoronix.com/ Phoronix.com]
 
* [http://users.on.net/~ckolivas/interbench/ Interbench Homepage]
 
* [http://users.on.net/~ckolivas/interbench/ Interbench Homepage]
 
* [http://unigine.com/download/ Unigine.com]
 
* [http://unigine.com/download/ Unigine.com]

Latest revision as of 10:51, 14 September 2016

Benchmarking is the act of measuring performance and comparing the results to another system's results or a widely accepted standard through a unified procedure. This unified method of evaluating system performance can help answer questions such as:

  • Is the system performing as it should?
  • What driver version should be used to get optimal performance?
  • Is the system capable of doing task x?

Many tools can be used to determine system performance, the following provides a list of tools available.

Standalone tools

glxgears

glxgears is a popular OpenGL test that renders a very simple OpenGL performance and outputs the frame rate. Though glxgears can be useful as a test of direct rendering capabilities of the graphics driver, it is an outdated tool that is not representative of the current state of GNU/Linux graphics and overall OpenGL possibilities. glxgears only tests a small segment of the OpenGL capabilities that might be used in a game. Performance increases noted in glxgears will not necessarily be realized in any given game. See here for more information.

glxgears can be installed via the mesa-demos and lib32-mesa-demos (for Multilib) packages.

UnixBench

A unixbench package is available in the AUR: unixbenchAUR. To run benchmark run ubench in terminal.

See also:

interbench

interbench is an application designed to benchmark interactivity in Linux. It is designed to measure the effect of changes in Linux kernel design or system configuration changes such as CPU, I/O scheduler and filesystem changes and options.
Tip: With careful benchmarking, different hardware can be compared.

interbench is available in the AUR: interbenchAUR.

See also:

ttcp

(n)(nu)ttcp measures point-to-point bandwidth over any network connection. The program must be provided on both nodes between which bandwidth is to be determined.

Various flavors of ttcp can be found in the AUR (see links below).

See also:

iperf

iperf is an easy to use point-to-point bandwidth testing tool that can use either TCP or UDP. It has nicely formatted output and a parallel test mode.

iperf can be installed, or a different version of iperf is available with iperf3.

time

The time command provides timing statistics about the command run by displaying the time that passed between invocation and termination. Time is available on most basic linux systems.

$ time tar -zxvf archive.tar.gz

hdparm

Storage media can be benchmarked with Hdparm (hdparm).

See also Benchmarking/Data storage devices#Using hdparm.

Unigine Engine

Unigine corp. has produced several modern OpenGL benchmarks based on their graphics engine with features such as:

  • Per-pixel dynamic lighting
  • Normal & parallax occlusion mapping
  • 64-bit HDR rendering
  • Volumetric fog and light
  • Powerful particle systems: fire, smoke, explosions
  • Extensible set of shaders (GLSL / HLSL)
  • Post-processing: depth of field, refraction, glow, blurring, color correction and much more.

Unigine benchmarks have found recent usage by those looking to overclock their systems. Heaven especially has been used for initial stability testing of overclocks.

These benchmarks can be found in the AUR (see links below).

See also:

Software suites

Bonnie++

bonnie++ is a C++ rewrite of the original Bonnie benchmarking suite is aimed at performing several tests of hard drive and filesystem performance.

Note: The original Bonnie suite does not appear to have been released under the GPL or other compatible license.

See also:

IOzone

IOzone is useful for performing a broad filesystem analysis of a vendor’s computer platform.

This program is available in the AUR: iozoneAUR.

See also BBS Article: iozone to evaluate I/O schedulers... results NOT what you'd expect!.

HardInfo

hardinfo can gather information about your system's hardware and operating system, perform benchmarks, and generate printable reports either in HTML or in plain text formats. HardInfo performs CPU and FPU benchmarks and has a very clean GTK-based interface.

See also Author's site.

Phoronix Test Suite

The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available that provides an extensible framework for which new tests can be easily added. The software is designed to effectively carry out both qualitative and quantitative benchmarks in a clean, reproducible, and easy-to-use manner.

The Phoronix Test Suite is based upon the extensive testing and internal tools developed by Phoronix.com since 2004 along with support from leading tier-one computer hardware and software vendors. This software is open-source and licensed under the GNU GPLv3.

Originally developed for automated Linux testing, support to the Phoronix Test Suite has since been added for OpenSolaris, Apple macOS, Microsoft Windows, and BSD operating systems. The Phoronix Test Suite consists of a lightweight processing core (pts-core) with each benchmark consisting of an XML-based profile and related resource scripts. The process from the benchmark installation, to the actual benchmarking, to the parsing of important hardware and software components is heavily automated and completely repeatable, asking users only for confirmation of actions.

The Phoronix Test Suite interfaces with OpenBenchmarking.org as a collaborative web platform for the centralized storage of test results, sharing of test profiles and results, advanced analytical features, and other functionality. Phoromatic is an enterprise component to orchestrate test execution across multiple systems with remote management capabilities.

This suite can be installed with the package phoronix-test-suite. There is also a developmental version available with phoronix-test-suite-gitAUR.

PTS Desktop Live

Warning: The live image does not look like it has been maintained since 2010.

As an alternative to the installation of the Phoronix Test Suite to the system, Phoronix also provides a Live-CD. This Live-CD offers all the features of the Phoronix Test Suite and includes the latest ATI and NVIDIA binary drivers. It will allow you to run 40+ benchmarks from a live environment without the need to store anything on your hard drive and includes a working GUI interface.

See also:

Flash media

Performance characteristics can be measured quantitatively using iozoneAUR. Sustained read and write values can, but often do not, correlate to real-world use cases of I/O heavy operations, such as unpacking and writing a number of files on a system update. A relevant metric to consider in these cases is the random write speed for small files.

The example invocation tests a 10M file using a 4k record size:

$ iozone -e -I -a -s 10M -r 4k -i 0 -i 1 -i 2
...

                                                                random   random
              kB  reclen    write  rewrite    read    reread    read     write
           10240       4      661      649     5802     5822     3892      624
Note:
  • Test values are reported in KB/s.
  • For performance charts on SD cards and other flash media, see for example Tom's Hardware.

See also: