Difference between revisions of "Benchmarking"

From ArchWiki
Jump to: navigation, search
m (unigine (Heavens, Tropics and Sanctuary OpenGL Benchmark))
m (hdparm: quotation marks are nonsense)
(20 intermediate revisions by 11 users not shown)
Line 1: Line 1:
{{expansion}}
+
[[Category:Hardware]]
[[Category:Hardware (English)]][[Category:HOWTOs (English)]]
+
 
Benchmarking is the act of measuring performance and comparing the results to another system's results or a widely accepted standard through a unified procedure. This unified method of evaluating system performance can help answer questions such as:
 
Benchmarking is the act of measuring performance and comparing the results to another system's results or a widely accepted standard through a unified procedure. This unified method of evaluating system performance can help answer questions such as:
  
Line 7: Line 6:
 
* Is the system capable of doing task x?
 
* Is the system capable of doing task x?
  
== Software ==
 
 
Many tools can be used to determine system performance, the following provides a list of tools available.
 
Many tools can be used to determine system performance, the following provides a list of tools available.
=== Stand alone tools ===
+
== Stand alone tools ==
==== glxgears (and why not to use this as a benchmark) ====
+
===glxgears (and why not to use this as a benchmark) ===
Glxgears is a popular OpenGL test that renders a very simple OpenGL performance and outputs the framerate. Though glxgears can be usefull as a test of direct rendering capabillities of the graphics driver, glxgears is an outdated tool that is not representative of the current state of linux graphics and overall OpenGL possibilities.
+
 
 +
# pacman -S mesa-demos
 +
 
 +
Glxgears is a popular OpenGL test that renders a very simple OpenGL performance and outputs the frame rate. Though glxgears can be useful as a test of direct rendering capabilities of the graphics driver, glxgears is an outdated tool that is not representative of the current state of linux graphics and overall OpenGL possibilities.
  
 
So to summarize, glxgears only tests a small part of what you typically see in a 3D game. You could have glxgears FPS performance increase, but your 3D game performance decrease. Likewise, you could have glxgears performance decrease and your 3D game performance increase. [http://wiki.cchtml.com/index.php/Glxgears_is_not_a_Benchmark source]
 
So to summarize, glxgears only tests a small part of what you typically see in a 3D game. You could have glxgears FPS performance increase, but your 3D game performance decrease. Likewise, you could have glxgears performance decrease and your 3D game performance increase. [http://wiki.cchtml.com/index.php/Glxgears_is_not_a_Benchmark source]
  
==== superpi ====
+
=== superpi ===
 
http://myownlittleworld.com/miscellaneous/computers/pilargetable.html
 
http://myownlittleworld.com/miscellaneous/computers/pilargetable.html
  
==== interbench ====
+
=== interbench ===
 
This benchmark application is designed to benchmark interactivity in Linux.  
 
This benchmark application is designed to benchmark interactivity in Linux.  
  
It is designed to measure the effect of changes in Linux kernel design or systemnconfiguration changes such as cpu, I/O  scheduler and filesystem changes and options. With careful benchmarking, different hardware can be compared.
+
It is designed to measure the effect of changes in Linux kernel design or system configuration changes such as CPU, I/O  scheduler and filesystem changes and options. With careful benchmarking, different hardware can be compared.
  
interbench is available in the AUR: http://aur.archlinux.org/packages.php?ID=2093
+
interbench is available in the AUR: https://aur.archlinux.org/packages.php?ID=2093
  
==== (n)ttcp ====
+
=== (n)ttcp ===
(n)ttcp measures point-to-point bandwidth over any network connection, the binary must be provided on both nodes between wich you want to determine bandwidth. nttcp is a a much more convenient rewrite of ttcp.
+
(n)ttcp measures point-to-point bandwidth over any network connection, the binary must be provided on both nodes between which you want to determine bandwidth. nttcp is a a much more convenient rewrite of ttcp.
  
nttcp is available on the AUR: http://aur.archlinux.org/packages.php?ID=11469
+
nttcp is available on the AUR: https://aur.archlinux.org/packages.php?ID=11469
==== time ====
+
=== time ===
 
The time command provides timing statistics about the command run by displaying the time that passed between invocation and termination. Time is available on most basic linux systems.
 
The time command provides timing statistics about the command run by displaying the time that passed between invocation and termination. Time is available on most basic linux systems.
 
  time tar -zxvf archive.tar.gz
 
  time tar -zxvf archive.tar.gz
==== times ====
+
=== times ===
 +
{{Expansion|Obvious! Write sth and delete this. Or delete the heading.}}
  
==== unigine (Heavens, Tropics and Sanctuary OpenGL Benchmark) ====
+
=== hdparm ===
Unigine[http://www.unigine.com/] has produced several modern OpenGL benchmarks based on their graphics engine with features as:
+
You can Benchmark storage media with [[Hdparm]].
 +
 
 +
This can also be done with dd and Co. There is an Article about [[Benchmarking disk wipes]].
 +
 
 +
=== unigine (Heavens, Tropics and Sanctuary OpenGL Benchmark) ===
 +
Unigine[http://www.unigine.com/] has produced several modern OpenGL benchmarks based on their graphics engine with features such as:
 
  * Per-pixel dynamic lighting
 
  * Per-pixel dynamic lighting
 
  * Normal & parallax occlusion mapping
 
  * Normal & parallax occlusion mapping
Line 42: Line 49:
 
  * Powerful particle systems: fire, smoke, explosions
 
  * Powerful particle systems: fire, smoke, explosions
 
  * Extensible set of shaders (GLSL / HLSL)
 
  * Extensible set of shaders (GLSL / HLSL)
  * Postprocessing: depth of field, refraction, glow, blurring, color correction and much more.
+
  * Post-processing: depth of field, refraction, glow, blurring, color correction and much more.
  
unigine-heavens is available in the AUR: http://aur.archlinux.org/packages.php?ID=35901
+
unigine-heavens is available in the AUR: https://aur.archlinux.org/packages.php?ID=35901
  
unigine-tropics is available in the AUR: http://aur.archlinux.org/packages.php?ID=25420
+
unigine-tropics is available in the AUR: https://aur.archlinux.org/packages.php?ID=25420
  
'''TODO''' AUR package for unigine-sanctuary
+
unigine-sanctuary is available in the AUR: https://aur.archlinux.org/packages.php?ID=45959
  
=== Software suites ===
+
== Software suites ==
==== Bonnie++ ====
+
=== Bonnie++ ===
 
This C++ rewrite of the original Bonnie benchmarking suite is aimed at performing several tests of hard drive and filesystem performance.
 
This C++ rewrite of the original Bonnie benchmarking suite is aimed at performing several tests of hard drive and filesystem performance.
  
Bonnie++ is available in the AUR: http://aur.archlinux.org/packages.php?ID=24114
+
Bonnie++ is available in [http://www.archlinux.org/packages/community/i686/bonnie++/ community].
==== Hardinfo ====
+
HardInfo can gather information about your system's hardware and operating system, perform benchmarks, and generate printable reports either in HTML or in plain text formats. Hardinfo performs CPU and FPU benchmarks and has a very clean GTK UI.
+
  
hardinfo is available in [http://www.archlinux.org/packages/community/i686/hardinfo/ community]
+
=== iozone ===
 +
Iozone is useful for performing a broad filesystem analysis of a vendor’s computer platform. See this thread: https://bbs.archlinux.org/viewtopic.php?pid=969463
  
==== Phoronix Test Suite ====
+
iozone is available in [https://aur.archlinux.org/packages.php?ID=22212 the AUR].
The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available for the Linux operating system. This software is designed to effectively carry out both qualitative and quantitative benchmarks in a clean, reproducible, and easy-to-use manner. This software is based upon the extensive Linux benchmarking work and internal tools developed by Phoronix.com since 2004 along with input from leading tier-one computer hardware vendors. This software is open-source and licensed under the GNU GPLv3. The Phoronix Test Suite consists of a lightweight processing core (pts-core) with each benchmark consisting of an XML-based profile with related resource scripts. The process from the benchmark installation, to the actual benchmarking, to the parsing of important hardware and software components is heavily automated and completely repeatable, asking users only for confirmation of actions.
+
  
source: http://www.phoronix-test-suite.com/?k=home
+
=== Hardinfo ===
 +
HardInfo can gather information about your system's hardware and operating system, perform benchmarks, and generate printable reports either in HTML or in plain text formats. Hardinfo performs CPU and FPU benchmarks and has a very clean GTK UI.
  
This suite is available in [community] and can be installed through your favorite package manager
+
hardinfo is available in [http://www.archlinux.org/packages/community/i686/hardinfo/ community]
pacman -S phoronix-test-suite
+
 
+
To enable the GUI
+
phoronix-test-suite gui
+
 
+
You need "php-gtk" from AUR for the GUI: [http://aur.archlinux.org/packages.php?ID=20658 php-gtk]
+
  
Add the following line to ''/etc/php/php.ini'':
+
=== Phoronix Test Suite ===
  extension=php_gtk2.so
+
The [http://www.phoronix-test-suite.com/ Phoronix Test Suite] is the most comprehensive testing and benchmarking platform available for the Linux operating system. This software is designed to effectively carry out both qualitative and quantitative benchmarks in a clean, reproducible, and easy-to-use manner. This software is based upon the extensive Linux benchmarking work and internal tools developed by Phoronix.com since 2004 along with input from leading tier-one computer hardware vendors. This software is open-source and licensed under the GNU GPLv3. The Phoronix Test Suite consists of a lightweight processing core (pts-core) with each benchmark consisting of an XML-based profile with related resource scripts. The process from the benchmark installation, to the actual benchmarking, to the parsing of important hardware and software components is heavily automated and completely repeatable, asking users only for confirmation of actions.
  
== Arch Linux Standardized Testing ==
+
This suite can be [[pacman|Installed]] with the package {{pkg|phoronix-test-suite}} , available in the [[Official Repositories]].
To be able to compare results within the Arch Linux community in a standardized manner, the following standards have been set.
+
=== General System Information Script ===
+
When comparing results, it is essential to include a complete overview of your hardware configuration and such things as driver and versions thereof. To simplify this, a system information script will be written wich output all the basic information to a humenly readable format.
+
  
In the future, a script will be provided that invokes the installed benchmarking applications of your choise and compiles a list of results along with system information.
+
=== PTS Desktop Live 2010.1 ===
=== CPU ===
+
As an alternative to the installation of the Phoronix Test Suite to the system, Phoronix also provides a Live-CD. This Live-CD offers all the features of the Phoronix Test Suite and includes the latest [[ATI]] and [[NVIDIA]] binary drivers. It will allow you to run 40+ benchmarks from a live environment without the need to store anything on your hard drive and includes a working GUI interface.
'''proposal:''' superpi with 1M digits (20 bit number) --[[User:Stefanwilkens|stefanwilkens]] 22:27, 25 October 2009 (EDT)
+
=== Graphics ===
+
'''proposal:''' phoronix-test-suite with a selection of tests aimed at graphics, preferably without having to provide a large number of dependencies.--[[User:Stefanwilkens|stefanwilkens]] 06:40, 26 October 2009 (EDT)
+
  
list of tests:
+
Website: http://www.phoronix-test-suite.com/?k=pts_desktop_live
* smokin-guns (req: openal)--[[User:Stefanwilkens|stefanwilkens]] 06:40, 26 October 2009 (EDT)
+
* ..
+
* ..
+
  
=== Storage ===
+
Documentation: http://www.phoronix-test-suite.com/documentation/2.4/pts_desktop_live.html
==== Hard Drive ====
+
'''proposal:'''
+
==== CD / DVD ====
+
'''proposal:'''
+
==== RAM ====
+
'''proposal:'''
+
=== Network ===
+
'''proposal:''' a combination of nttcp and ping should do the job here--[[User:Stefanwilkens|stefanwilkens]] 06:43, 26 October 2009 (EDT)
+
==== Internet ====
+
'''proposal:'''
+
  
 
== Related Links ==
 
== Related Links ==
* [http://bbs.archlinux.org/viewtopic.php?pid=643939 Forum Topic: Benchmarking Let's organise and standardize]
 
 
* [http://lbs.sourceforge.net/ Linux Benchmarking Homepage]
 
* [http://lbs.sourceforge.net/ Linux Benchmarking Homepage]
 
* [http://www.phoronix.com/scan.php?page=home Phoronix.com]
 
* [http://www.phoronix.com/scan.php?page=home Phoronix.com]
 
* [http://users.on.net/~ckolivas/interbench/ Interbench Homepage]
 
* [http://users.on.net/~ckolivas/interbench/ Interbench Homepage]
 
* [http://unigine.com/download/ Unigine.com]
 
* [http://unigine.com/download/ Unigine.com]

Revision as of 11:15, 27 September 2012

Benchmarking is the act of measuring performance and comparing the results to another system's results or a widely accepted standard through a unified procedure. This unified method of evaluating system performance can help answer questions such as:

  • Is the system performing as it should?
  • What driver version should be used to get optimal performance?
  • Is the system capable of doing task x?

Many tools can be used to determine system performance, the following provides a list of tools available.

Stand alone tools

glxgears (and why not to use this as a benchmark)

# pacman -S mesa-demos

Glxgears is a popular OpenGL test that renders a very simple OpenGL performance and outputs the frame rate. Though glxgears can be useful as a test of direct rendering capabilities of the graphics driver, glxgears is an outdated tool that is not representative of the current state of linux graphics and overall OpenGL possibilities.

So to summarize, glxgears only tests a small part of what you typically see in a 3D game. You could have glxgears FPS performance increase, but your 3D game performance decrease. Likewise, you could have glxgears performance decrease and your 3D game performance increase. source

superpi

http://myownlittleworld.com/miscellaneous/computers/pilargetable.html

interbench

This benchmark application is designed to benchmark interactivity in Linux.

It is designed to measure the effect of changes in Linux kernel design or system configuration changes such as CPU, I/O scheduler and filesystem changes and options. With careful benchmarking, different hardware can be compared.

interbench is available in the AUR: https://aur.archlinux.org/packages.php?ID=2093

(n)ttcp

(n)ttcp measures point-to-point bandwidth over any network connection, the binary must be provided on both nodes between which you want to determine bandwidth. nttcp is a a much more convenient rewrite of ttcp.

nttcp is available on the AUR: https://aur.archlinux.org/packages.php?ID=11469

time

The time command provides timing statistics about the command run by displaying the time that passed between invocation and termination. Time is available on most basic linux systems.

time tar -zxvf archive.tar.gz

times

Tango-view-fullscreen.pngThis article or section needs expansion.Tango-view-fullscreen.png

Reason: Obvious! Write sth and delete this. Or delete the heading. (Discuss in Talk:Benchmarking#)

hdparm

You can Benchmark storage media with Hdparm.

This can also be done with dd and Co. There is an Article about Benchmarking disk wipes.

unigine (Heavens, Tropics and Sanctuary OpenGL Benchmark)

Unigine[1] has produced several modern OpenGL benchmarks based on their graphics engine with features such as:

* Per-pixel dynamic lighting
* Normal & parallax occlusion mapping
* 64-bit HDR rendering
* Volumetric fog and light
* Powerful particle systems: fire, smoke, explosions
* Extensible set of shaders (GLSL / HLSL)
* Post-processing: depth of field, refraction, glow, blurring, color correction and much more.

unigine-heavens is available in the AUR: https://aur.archlinux.org/packages.php?ID=35901

unigine-tropics is available in the AUR: https://aur.archlinux.org/packages.php?ID=25420

unigine-sanctuary is available in the AUR: https://aur.archlinux.org/packages.php?ID=45959

Software suites

Bonnie++

This C++ rewrite of the original Bonnie benchmarking suite is aimed at performing several tests of hard drive and filesystem performance.

Bonnie++ is available in community.

iozone

Iozone is useful for performing a broad filesystem analysis of a vendor’s computer platform. See this thread: https://bbs.archlinux.org/viewtopic.php?pid=969463

iozone is available in the AUR.

Hardinfo

HardInfo can gather information about your system's hardware and operating system, perform benchmarks, and generate printable reports either in HTML or in plain text formats. Hardinfo performs CPU and FPU benchmarks and has a very clean GTK UI.

hardinfo is available in community

Phoronix Test Suite

The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available for the Linux operating system. This software is designed to effectively carry out both qualitative and quantitative benchmarks in a clean, reproducible, and easy-to-use manner. This software is based upon the extensive Linux benchmarking work and internal tools developed by Phoronix.com since 2004 along with input from leading tier-one computer hardware vendors. This software is open-source and licensed under the GNU GPLv3. The Phoronix Test Suite consists of a lightweight processing core (pts-core) with each benchmark consisting of an XML-based profile with related resource scripts. The process from the benchmark installation, to the actual benchmarking, to the parsing of important hardware and software components is heavily automated and completely repeatable, asking users only for confirmation of actions.

This suite can be Installed with the package phoronix-test-suite , available in the Official Repositories.

PTS Desktop Live 2010.1

As an alternative to the installation of the Phoronix Test Suite to the system, Phoronix also provides a Live-CD. This Live-CD offers all the features of the Phoronix Test Suite and includes the latest ATI and NVIDIA binary drivers. It will allow you to run 40+ benchmarks from a live environment without the need to store anything on your hard drive and includes a working GUI interface.

Website: http://www.phoronix-test-suite.com/?k=pts_desktop_live

Documentation: http://www.phoronix-test-suite.com/documentation/2.4/pts_desktop_live.html

Related Links