Difference between revisions of "Benchmarking"
(Adding iperf) |
(→Software suites: added S) |
||
| (73 intermediate revisions by 25 users not shown) | |||
| Line 1: | Line 1: | ||
[[Category:Hardware]] | [[Category:Hardware]] | ||
| + | [[ja:ベンチマーク]] | ||
| + | [[zh-hans:Benchmarking]] | ||
| + | {{Related articles start}} | ||
| + | {{Related|Improving performance}} | ||
| + | {{Related|Sysstat}} | ||
| + | {{Related articles end}} | ||
| + | |||
Benchmarking is the act of measuring performance and comparing the results to another system's results or a widely accepted standard through a unified procedure. This unified method of evaluating system performance can help answer questions such as: | Benchmarking is the act of measuring performance and comparing the results to another system's results or a widely accepted standard through a unified procedure. This unified method of evaluating system performance can help answer questions such as: | ||
| Line 7: | Line 14: | ||
Many tools can be used to determine system performance, the following provides a list of tools available. | Many tools can be used to determine system performance, the following provides a list of tools available. | ||
| − | == | + | == Standalone tools == |
| − | ===glxgears | + | ===glxgears=== |
| + | glxgears is a popular OpenGL test that renders a very simple OpenGL performance and outputs the frame rate. Though glxgears can be useful as a test of direct rendering capabilities of the graphics driver, it is an outdated tool that is not representative of the current state of GNU/Linux graphics and overall OpenGL possibilities. glxgears only tests a small segment of the OpenGL capabilities that might be used in a game. Performance increases noted in glxgears will not necessarily be realized in any given game. See [http://wiki.cchtml.com/index.php/Glxgears_is_not_a_Benchmark here] for more information. | ||
| − | + | glxgears can be installed via the {{pkg|mesa-demos}} and {{pkg|lib32-mesa-demos}} (for [[Multilib]]) packages. | |
| − | + | === UnixBench === | |
| + | A unixbench package is available in the [[AUR]]: {{AUR|unixbench}}. | ||
| + | To run benchmark run ''ubench'' in terminal. | ||
| − | + | See also: | |
| + | * https://github.com/kdlucas/byte-unixbench | ||
| + | * https://github.com/kdlucas/byte-unixbench/blob/master/UnixBench/USAGE | ||
| − | === | + | === interbench === |
| − | + | interbench is an application designed to benchmark interactivity in Linux. It is designed to measure the effect of changes in Linux kernel design or system configuration changes such as CPU, I/O scheduler and filesystem changes and options. {{Tip|With careful benchmarking, different hardware can be compared.}} | |
| − | + | interbench is available in the [[AUR]]: {{AUR|interbench}}. | |
| − | |||
| − | + | See also: | |
| + | * [[Realtime process management]] | ||
| + | * [[Advanced traffic control]] | ||
| + | * [[Linux-ck]] | ||
| + | * [[Linux-pf]] | ||
| − | + | === ttcp === | |
| + | ttcp (Test TCP) measures point-to-point bandwidth over any network connection. The program must be provided on both nodes between which bandwidth is to be determined. | ||
| − | + | Various flavors of ttcp can be found in the [[AUR]]: | |
| − | + | * {{AUR|ttcp}} | |
| + | * {{AUR|nuttcp}} | ||
=== iperf === | === iperf === | ||
| − | iperf is | + | iperf is an easy to use point-to-point bandwidth testing tool that can use either TCP or UDP. It has nicely formatted output and a parallel test mode. |
| + | |||
| + | {{pkg|iperf}} can be installed, or a different version of iperf is available with {{Pkg|iperf3}}. | ||
| − | |||
=== time === | === time === | ||
The time command provides timing statistics about the command run by displaying the time that passed between invocation and termination. Time is available on most basic linux systems. | The time command provides timing statistics about the command run by displaying the time that passed between invocation and termination. Time is available on most basic linux systems. | ||
| − | time tar -zxvf archive.tar.gz | + | |
| − | + | $ time tar -zxvf archive.tar.gz | |
| − | |||
=== hdparm === | === hdparm === | ||
| − | + | Storage media can be benchmarked with [[hdparm]] ({{pkg|hdparm}}). Using hdparm with the -Tt switch, one can time sequential reads. This method is independent of partition alignment! | |
| + | |||
| + | # hdparm -Tt /dev/sdX | ||
| + | /dev/sdX: | ||
| + | Timing cached reads: x MB in y seconds = z MB/sec | ||
| + | Timing buffered disk reads: x MB in y seconds = z MB/sec | ||
| + | |||
| + | {{Note|One should run the above command 4-5 times and manually average the results for an accurate evaluation of read speed per the hdparm man page.}} | ||
| + | |||
| + | === Unigine Engine === | ||
| + | [http://www.unigine.com/ Unigine corp.] has produced several modern OpenGL benchmarks based on their graphics engine with features such as: | ||
| + | * Per-pixel dynamic lighting | ||
| + | * Normal & parallax occlusion mapping | ||
| + | * 64-bit HDR rendering | ||
| + | * Volumetric fog and light | ||
| + | * Powerful particle systems: fire, smoke, explosions | ||
| + | * Extensible set of shaders (GLSL / HLSL) | ||
| + | * Post-processing: depth of field, refraction, glow, blurring, color correction and much more. | ||
| + | |||
| + | Unigine benchmarks have found recent usage by those looking to overclock their systems. Heaven especially has been used for initial stability testing of overclocks. | ||
| + | |||
| + | These benchmarks can be found in the [[AUR]] (see links below). | ||
| + | |||
| + | See also: | ||
| + | * {{AUR|unigine-heaven}} | ||
| + | * {{AUR|unigine-tropics}} | ||
| + | * {{AUR|unigine-sanctuary}} | ||
| + | * {{AUR|unigine-valley}} | ||
| + | * {{AUR|unigine-superposition}} | ||
| + | |||
| + | === gnome-disks === | ||
| + | |||
| + | There is a graphical benchmark called gnome-disks contained in the gnome-disk-utility package that will give min/max/ave reads along with ave access time and a nice graphical display. This method is independent of partition alignment! | ||
| + | |||
| + | # gnome-disks | ||
| + | |||
| + | Users will need to navigate through the GUI to the benchmark button ("More actions..." => "Benchmark Volume..."). [http://imgur.com/Ayv1B Example] | ||
| + | |||
| + | === systemd-analyze === | ||
| + | |||
| + | systemd-analyze plot > boot.svg | ||
| + | |||
| + | Will plot a detailed graphic with the boot sequence: kernel time, userspace time, time taken by each service. [http://imgur.com/4ywt1 Example] | ||
| + | |||
| + | === dd === | ||
| + | |||
| + | The dd utility can be used to measure both reads and writes. This method is dependent on partition alignment! In other words, if you failed to properly align your partitions, this fact will be seen here since you are writing and reading to a mounted filesystem. | ||
| + | |||
| + | {{Note|This method requires the command to be executed from a mounted partition on the device of interest!}} | ||
| + | |||
| + | First, enter a directory on the SSD with at least 1.1 GB of free space (and one that obviously gives your user wrx permissions) and write a test file to measure write speeds and to give the device something to read: | ||
| + | |||
| + | $ cd /path/to/SSD | ||
| + | $ dd if=/dev/zero of=tempfile bs=1M count=1024 conv=fdatasync,notrunc status=progress | ||
| + | 1024+0 records in | ||
| + | 1024+0 records out | ||
| + | w bytes (x GB) copied, y s, z MB/s | ||
| + | |||
| + | {{Tip|See [https://romanrm.net/dd-benchmark dd-benchmark] for an explanation on the requirement to {{ic|sync}} and further related {{ic|dd}} options.}} | ||
| + | |||
| + | Next, clear the buffer-cache to accurately measure read speeds directly from the device: | ||
| + | |||
| + | # echo 3 > /proc/sys/vm/drop_caches | ||
| + | $ dd if=tempfile of=/dev/null bs=1M count=1024 status=progress | ||
| + | 1024+0 records in | ||
| + | 1024+0 records out | ||
| + | w bytes (x GB) copied, y s, z MB/s | ||
| + | |||
| + | Now that the last file is in the buffer, repeat the command to see the speed of the buffer-cache: | ||
| + | |||
| + | $ dd if=tempfile of=/dev/null bs=1M count=1024 status=progress | ||
| + | 1024+0 records in | ||
| + | 1024+0 records out | ||
| + | w bytes (x GB) copied, y s, z GB/s | ||
| + | |||
| + | {{Note|One should run the above command 4-5 times and manually average the results for an accurate evaluation of the buffer read speed.}} | ||
| + | |||
| + | Finally, delete the temp file | ||
| + | $ rm tempfile | ||
| − | + | {{Note|Some SSD controllers have compression hardware, which may skew benchmark results. See http://www.pugetsystems.com/labs/articles/SSDs-Advertised-vs-Actual-Performance-179/}} | |
| − | + | See also [[Core utilities#dd]]. | |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | + | === dcfldd === | |
| − | + | Dcfldd doesn't print the average speed in MB/s like good old dd does but with [[#time|time]] you can work around that. | |
| − | + | Time the run clearing the disk: | |
| + | # time dcfldd if=/dev/zero of=/dev/sdX bs=4M | ||
| + | 18944 blocks (75776Mb) written.dcfldd:: No space left of device | ||
| + | real 16m17.033s | ||
| + | user 0m0.377s | ||
| + | sys 0m51.160s | ||
| + | |||
| + | Calculate MB/s by dividing the output of the dcfldd command by the time in seconds. For this example: 75776Mb / (16.4 min * 60) = 77.0 MB/s. | ||
== Software suites == | == Software suites == | ||
=== Bonnie++ === | === Bonnie++ === | ||
| − | |||
| − | + | {{pkg|bonnie++}} is a C++ rewrite of the [http://www.textuality.com/bonnie/ original Bonnie] benchmarking suite is aimed at performing several tests of hard drive and filesystem performance. | |
| + | |||
| + | {{Warning|By default, bonnie++ write at least twice the RAM size on disk. If you want to preserve your SSD, use non default option.}} | ||
| + | |||
| + | {{Note|The original Bonnie suite does not appear to have been released under the GPL or other compatible license.}} | ||
| − | + | See also: | |
| − | |||
| − | + | * [http://www.coker.com.au/bonnie++/ Author's site] | |
| + | * [[Wikipedia:Bonnie++]] | ||
| − | === | + | === IOzone === |
| − | + | IOzone is useful for performing a broad filesystem analysis of a vendor’s computer platform. | |
| − | + | This program is available in the [[AUR]]: {{AUR|iozone}}. | |
| + | |||
| + | See also BBS Article: [https://bbs.archlinux.org/viewtopic.php?pid=969463 iozone to evaluate I/O schedulers... results NOT what you'd expect!]. | ||
| + | |||
| + | === HardInfo === | ||
| + | {{pkg|hardinfo}} can gather information about your system's hardware and operating system, perform benchmarks, and generate printable reports either in HTML or in plain text formats. HardInfo performs CPU and FPU benchmarks and has a very clean GTK-based interface. | ||
| + | |||
| + | See also [http://wiki.hardinfo.org/HomePage Author's site]. | ||
=== Phoronix Test Suite === | === Phoronix Test Suite === | ||
| − | |||
| − | This suite can be [[pacman| | + | ''The [http://www.phoronix-test-suite.com/ Phoronix Test Suite] is the most comprehensive testing and benchmarking platform available that provides an extensible framework for which new tests can be easily added. The software is designed to effectively carry out both qualitative and quantitative benchmarks in a clean, reproducible, and easy-to-use manner. |
| + | |||
| + | ''The Phoronix Test Suite is based upon the extensive testing and internal tools developed by Phoronix.com since 2004 along with support from leading tier-one computer hardware and software vendors. This software is open-source and licensed under the GNU GPLv3. | ||
| + | |||
| + | ''Originally developed for automated Linux testing, support to the Phoronix Test Suite has since been added for OpenSolaris, Apple macOS, Microsoft Windows, and BSD operating systems. The Phoronix Test Suite consists of a lightweight processing core (pts-core) with each benchmark consisting of an XML-based profile and related resource scripts. The process from the benchmark installation, to the actual benchmarking, to the parsing of important hardware and software components is heavily automated and completely repeatable, asking users only for confirmation of actions. | ||
| + | |||
| + | ''The Phoronix Test Suite interfaces with OpenBenchmarking.org as a collaborative web platform for the centralized storage of test results, sharing of test profiles and results, advanced analytical features, and other functionality. Phoromatic is an enterprise component to orchestrate test execution across multiple systems with remote management capabilities.'' | ||
| + | |||
| + | This suite can be [[pacman|installed]] with the package {{pkg|phoronix-test-suite}}. There is also a developmental version available with {{AUR|phoronix-test-suite-git}}. | ||
| + | |||
| + | === S === | ||
| + | |||
| + | S, an ''I/O Benchmark Suite'', is a small collection of programs to measure storage I/O performance.[https://github.com/Algodev-github/S] | ||
| + | |||
| + | It has been developed by the same group of the BFQ scheduler, lead by Paolo Valente.[http://algogroup.unimore.it/algodev/] | ||
| + | |||
| + | Download or clone the project, install its dependencies and run it as root (superuser privileges needed to change disk scheduler). | ||
| + | |||
| + | == Flash media == | ||
| + | |||
| + | Performance characteristics can be measured quantitatively using {{AUR|iozone}}. ''Sustained'' read and write values can, but often do not, correlate to real-world use cases of I/O heavy operations, such as unpacking and writing a number of files on a system update. A relevant metric to consider in these cases is the '''random write''' speed for small files. | ||
| − | + | The example invocation tests a 10M file using a 4k record size: | |
| − | |||
| − | + | $ iozone -e -I -a -s 10M -r 4k -i 0 -i 1 -i 2 | |
| + | ... | ||
| + | |||
| + | random random | ||
| + | kB reclen write rewrite read reread read write | ||
| + | 10240 4 661 649 5802 5822 3892 624 | ||
| − | + | {{Note| | |
| + | * Test values are reported in KB/s. | ||
| + | * For performance charts on SD cards and other flash media, see for example [http://www.tomshardware.com/charts/memory-cards,39.html Tom's Hardware]. | ||
| + | }} | ||
| − | == | + | == See also == |
* [http://lbs.sourceforge.net/ Linux Benchmarking Homepage] | * [http://lbs.sourceforge.net/ Linux Benchmarking Homepage] | ||
| − | * [http://www.phoronix.com/ | + | * [http://www.phoronix.com/ Phoronix.com] |
* [http://users.on.net/~ckolivas/interbench/ Interbench Homepage] | * [http://users.on.net/~ckolivas/interbench/ Interbench Homepage] | ||
* [http://unigine.com/download/ Unigine.com] | * [http://unigine.com/download/ Unigine.com] | ||
Latest revision as of 09:57, 20 August 2017
Related articles
Benchmarking is the act of measuring performance and comparing the results to another system's results or a widely accepted standard through a unified procedure. This unified method of evaluating system performance can help answer questions such as:
- Is the system performing as it should?
- What driver version should be used to get optimal performance?
- Is the system capable of doing task x?
Many tools can be used to determine system performance, the following provides a list of tools available.
Contents
Standalone tools
glxgears
glxgears is a popular OpenGL test that renders a very simple OpenGL performance and outputs the frame rate. Though glxgears can be useful as a test of direct rendering capabilities of the graphics driver, it is an outdated tool that is not representative of the current state of GNU/Linux graphics and overall OpenGL possibilities. glxgears only tests a small segment of the OpenGL capabilities that might be used in a game. Performance increases noted in glxgears will not necessarily be realized in any given game. See here for more information.
glxgears can be installed via the mesa-demos and lib32-mesa-demos (for Multilib) packages.
UnixBench
A unixbench package is available in the AUR: unixbenchAUR. To run benchmark run ubench in terminal.
See also:
- https://github.com/kdlucas/byte-unixbench
- https://github.com/kdlucas/byte-unixbench/blob/master/UnixBench/USAGE
interbench
interbench is an application designed to benchmark interactivity in Linux. It is designed to measure the effect of changes in Linux kernel design or system configuration changes such as CPU, I/O scheduler and filesystem changes and options.interbench is available in the AUR: interbenchAUR.
See also:
ttcp
ttcp (Test TCP) measures point-to-point bandwidth over any network connection. The program must be provided on both nodes between which bandwidth is to be determined.
Various flavors of ttcp can be found in the AUR:
iperf
iperf is an easy to use point-to-point bandwidth testing tool that can use either TCP or UDP. It has nicely formatted output and a parallel test mode.
iperf can be installed, or a different version of iperf is available with iperf3.
time
The time command provides timing statistics about the command run by displaying the time that passed between invocation and termination. Time is available on most basic linux systems.
$ time tar -zxvf archive.tar.gz
hdparm
Storage media can be benchmarked with hdparm (hdparm). Using hdparm with the -Tt switch, one can time sequential reads. This method is independent of partition alignment!
# hdparm -Tt /dev/sdX /dev/sdX: Timing cached reads: x MB in y seconds = z MB/sec Timing buffered disk reads: x MB in y seconds = z MB/sec
Unigine Engine
Unigine corp. has produced several modern OpenGL benchmarks based on their graphics engine with features such as:
- Per-pixel dynamic lighting
- Normal & parallax occlusion mapping
- 64-bit HDR rendering
- Volumetric fog and light
- Powerful particle systems: fire, smoke, explosions
- Extensible set of shaders (GLSL / HLSL)
- Post-processing: depth of field, refraction, glow, blurring, color correction and much more.
Unigine benchmarks have found recent usage by those looking to overclock their systems. Heaven especially has been used for initial stability testing of overclocks.
These benchmarks can be found in the AUR (see links below).
See also:
- unigine-heavenAUR
- unigine-tropicsAUR
- unigine-sanctuaryAUR
- unigine-valleyAUR
- unigine-superpositionAUR
gnome-disks
There is a graphical benchmark called gnome-disks contained in the gnome-disk-utility package that will give min/max/ave reads along with ave access time and a nice graphical display. This method is independent of partition alignment!
# gnome-disks
Users will need to navigate through the GUI to the benchmark button ("More actions..." => "Benchmark Volume..."). Example
systemd-analyze
systemd-analyze plot > boot.svg
Will plot a detailed graphic with the boot sequence: kernel time, userspace time, time taken by each service. Example
dd
The dd utility can be used to measure both reads and writes. This method is dependent on partition alignment! In other words, if you failed to properly align your partitions, this fact will be seen here since you are writing and reading to a mounted filesystem.
First, enter a directory on the SSD with at least 1.1 GB of free space (and one that obviously gives your user wrx permissions) and write a test file to measure write speeds and to give the device something to read:
$ cd /path/to/SSD $ dd if=/dev/zero of=tempfile bs=1M count=1024 conv=fdatasync,notrunc status=progress 1024+0 records in 1024+0 records out w bytes (x GB) copied, y s, z MB/s
Next, clear the buffer-cache to accurately measure read speeds directly from the device:
# echo 3 > /proc/sys/vm/drop_caches $ dd if=tempfile of=/dev/null bs=1M count=1024 status=progress 1024+0 records in 1024+0 records out w bytes (x GB) copied, y s, z MB/s
Now that the last file is in the buffer, repeat the command to see the speed of the buffer-cache:
$ dd if=tempfile of=/dev/null bs=1M count=1024 status=progress 1024+0 records in 1024+0 records out w bytes (x GB) copied, y s, z GB/s
Finally, delete the temp file
$ rm tempfile
See also Core utilities#dd.
dcfldd
Dcfldd doesn't print the average speed in MB/s like good old dd does but with time you can work around that.
Time the run clearing the disk:
# time dcfldd if=/dev/zero of=/dev/sdX bs=4M 18944 blocks (75776Mb) written.dcfldd:: No space left of device real 16m17.033s user 0m0.377s sys 0m51.160s
Calculate MB/s by dividing the output of the dcfldd command by the time in seconds. For this example: 75776Mb / (16.4 min * 60) = 77.0 MB/s.
Software suites
Bonnie++
bonnie++ is a C++ rewrite of the original Bonnie benchmarking suite is aimed at performing several tests of hard drive and filesystem performance.
See also:
IOzone
IOzone is useful for performing a broad filesystem analysis of a vendor’s computer platform.
This program is available in the AUR: iozoneAUR.
See also BBS Article: iozone to evaluate I/O schedulers... results NOT what you'd expect!.
HardInfo
hardinfo can gather information about your system's hardware and operating system, perform benchmarks, and generate printable reports either in HTML or in plain text formats. HardInfo performs CPU and FPU benchmarks and has a very clean GTK-based interface.
See also Author's site.
Phoronix Test Suite
The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available that provides an extensible framework for which new tests can be easily added. The software is designed to effectively carry out both qualitative and quantitative benchmarks in a clean, reproducible, and easy-to-use manner.
The Phoronix Test Suite is based upon the extensive testing and internal tools developed by Phoronix.com since 2004 along with support from leading tier-one computer hardware and software vendors. This software is open-source and licensed under the GNU GPLv3.
Originally developed for automated Linux testing, support to the Phoronix Test Suite has since been added for OpenSolaris, Apple macOS, Microsoft Windows, and BSD operating systems. The Phoronix Test Suite consists of a lightweight processing core (pts-core) with each benchmark consisting of an XML-based profile and related resource scripts. The process from the benchmark installation, to the actual benchmarking, to the parsing of important hardware and software components is heavily automated and completely repeatable, asking users only for confirmation of actions.
The Phoronix Test Suite interfaces with OpenBenchmarking.org as a collaborative web platform for the centralized storage of test results, sharing of test profiles and results, advanced analytical features, and other functionality. Phoromatic is an enterprise component to orchestrate test execution across multiple systems with remote management capabilities.
This suite can be installed with the package phoronix-test-suite. There is also a developmental version available with phoronix-test-suite-gitAUR.
S
S, an I/O Benchmark Suite, is a small collection of programs to measure storage I/O performance.[1]
It has been developed by the same group of the BFQ scheduler, lead by Paolo Valente.[2]
Download or clone the project, install its dependencies and run it as root (superuser privileges needed to change disk scheduler).
Flash media
Performance characteristics can be measured quantitatively using iozoneAUR. Sustained read and write values can, but often do not, correlate to real-world use cases of I/O heavy operations, such as unpacking and writing a number of files on a system update. A relevant metric to consider in these cases is the random write speed for small files.
The example invocation tests a 10M file using a 4k record size:
$ iozone -e -I -a -s 10M -r 4k -i 0 -i 1 -i 2
...
random random
kB reclen write rewrite read reread read write
10240 4 661 649 5802 5822 3892 624