Difference between revisions of "Makepkg-optimize"

From ArchWiki
Jump to: navigation, search
(simplification and beautification of wikilinks, fixed section fragments (interactive))
(Tag: wiki-scripts)
m (Building with Profile-guided optimization: looking for the safest/simplest way to do rebuild and avoid "a package is already built" etc.)
 
(7 intermediate revisions by the same user not shown)
Line 22: Line 22:
 
== Build an optimized package ==
 
== Build an optimized package ==
  
First, edit [[#Configuration|{{ic|/etc/makepkg-optimize.conf}}]] and select your preferred optimizations for [[Makepkg#Building optimized binaries|{{ic|ARCHITECTURE, COMPILE FLAGS}}]], {{ic|BUILD ENVIRONMENT}}, {{ic|GLOBAL PACKAGE OPTIONS}} and {{ic|COMPRESSION DEFAULTS}}.
+
First, edit [[#Configuration|{{ic|/etc/makepkg-optimize.conf}}]] and select your preferred optimizations for [[Makepkg#Building optimized binaries|{{ic|ARCHITECTURE, COMPILE FLAGS}}]], [https://aur.archlinux.org/cgit/aur.git/tree/buildenv_ext.conf?h=makepkg-optimize {{ic|BUILD ENVIRONMENT}}], [https://aur.archlinux.org/cgit/aur.git/tree/pkgopts_ext.conf?h=makepkg-optimize {{ic|GLOBAL PACKAGE OPTIONS}}] and [https://aur.archlinux.org/cgit/aur.git/tree/compress-param_max.conf?h=makepkg-optimize {{ic|COMPRESSION DEFAULTS}}].
  
 
{{Warning|While many packages build with all optimizations enabled, some will not. It may take a few attempts to find which--if any--optimizations a package is compatible with. Some software may also experience runtime problems caused by over-optimization.}}
 
{{Warning|While many packages build with all optimizations enabled, some will not. It may take a few attempts to find which--if any--optimizations a package is compatible with. Some software may also experience runtime problems caused by over-optimization.}}
Line 44: Line 44:
 
==== Install required packages ====
 
==== Install required packages ====
  
First, install [[Git]] and some of the backends for the optimization macros:
+
First, install some of the backends for the optimization macros to the base chroot:
  
  $ arch-nspawn -M /etc/makepkg.conf "$CHROOT"/root pacman -S git graphite openmp upx optipng
+
  $ arch-nspawn -M /etc/makepkg.conf "$CHROOT"/root pacman -S graphite openmp upx optipng
  
Then [[Arch User Repository#Acquire build files|download]], [[DeveloperWiki:Building in a Clean Chroot#Building in the Chroot|build]] and [[DeveloperWiki:Building in a Clean Chroot#Manual package installation|install]] {{AUR|pacman-buildenv_ext-git}}, {{AUR|makepkg-optimize}}, and {{AUR|nodejs-svgo}}.
+
Then [[Arch User Repository#Acquire build files|download]] and [[DeveloperWiki:Building in a Clean Chroot#Building in the Chroot|build]] {{AUR|pacman-buildenv_ext-git}}, {{AUR|makepkg-optimize}}, and {{AUR|nodejs-svgo}}.
 +
 
 +
To install them in the base chroot, copy their package files into it, and install them with pacman, e.g.:
 +
 
 +
# cp nodejs-svgo-1.1.1-1-any.pkg.tar.xz.pkg.tar.xz "$CHROOT"/root/root/
 +
$ arch-nspawn -M /etc/makepkg.conf "$CHROOT"/root pacman -U /root/nodejs-svgo-1.1.1-1-any.pkg.tar.xz.pkg.tar.xz
  
 
==== Create a [[Wikipedia:Profile-guided optimization|Profile-guided Optimization]] cache ====
 
==== Create a [[Wikipedia:Profile-guided optimization|Profile-guided Optimization]] cache ====
Line 64: Line 69:
 
==== Build a package ====
 
==== Build a package ====
  
First, edit [[#Configuration|{{ic|$CHROOT/root/etc/makepkg-optimize.conf}}]] and select your preferred optimizations for [[Makepkg#Building optimized binaries|{{ic|ARCHITECTURE, COMPILE FLAGS}}]], {{ic|BUILD ENVIRONMENT}}, {{ic|GLOBAL PACKAGE OPTIONS}} and {{ic|COMPRESSION DEFAULTS}}.
+
First, edit [[#Configuration|{{ic|$CHROOT/root/etc/makepkg-optimize.conf}}]] and select your preferred optimizations.
  
 
When [[DeveloperWiki:Building in a Clean Chroot#Building in the Chroot|building]], pass the {{AUR|makepkg-optimize}} configuration file to makepkg:
 
When [[DeveloperWiki:Building in a Clean Chroot#Building in the Chroot|building]], pass the {{AUR|makepkg-optimize}} configuration file to makepkg:
Line 78: Line 83:
 
Once the package is [[Pacman#Additional commands|installed]], test-run its executables.
 
Once the package is [[Pacman#Additional commands|installed]], test-run its executables.
  
Then, for the second building phase, remove package cruft but ''do not'' clean the chroot (do not pass {{ic|-c}}):
+
For the second building phase, overwrite the original package file and do not pass {{ic|-c}} to {{ic|makechrootpkg}}:
  
{{Note|If you have rebooted, be sure to rebind the PGO folder. Alternatively, use [[fstab]] to bind these folders persistently.}}
+
{{Note|If you have rebooted, be sure to rebind the PGO folder before rebuilding. Alternatively, use [[fstab]] to bind these folders persistently.}}
  
  $ rm -rf *.pkg.tar.xz pkg src; makechrootpkg -r "$CHROOT" -- --config /etc/makepkg-optimize.conf
+
  $ makechrootpkg -r "$CHROOT" -- -f --config /etc/makepkg-optimize.conf

Latest revision as of 08:32, 14 October 2018

Tango-go-next.pngThis article or section is a candidate for moving to [[qq's github]].Tango-go-next.png

Notes: AUR is not a code hosting platform and the wiki is not well suited for upstream documentation, in this case for a (temporary) pacman fork. (Discuss in Talk:Makepkg-optimize#This article should not be (re)moved.)

makepkg-optimizeAUR is a collection of supplemental build_env and tidy scripts for pacman-buildenv_ext-gitAUR, a fork of pacman-gitAUR. Their purpose is to provide macros to makepkg for several kinds of optimization in build and packaging stages, both in response to the simplification of makepkg and as a proposal for a future version of pacman.

Installation

Install pacman-buildenv_ext-gitAUR and makepkg-optimizeAUR from the AUR.

To make optimizations available, install their backends: graphite, openmp, upx, optipng, and nodejs-svgoAUR

Configuration

makepkg-optimizeAUR generates an additional configuration file, from your current makepkg configuration, with supplementary options for optimization that are disabled by default.

Build an optimized package

First, edit /etc/makepkg-optimize.conf and select your preferred optimizations for ARCHITECTURE, COMPILE FLAGS, BUILD ENVIRONMENT, GLOBAL PACKAGE OPTIONS and COMPRESSION DEFAULTS.

Warning: While many packages build with all optimizations enabled, some will not. It may take a few attempts to find which--if any--optimizations a package is compatible with. Some software may also experience runtime problems caused by over-optimization.

When building, pass the makepkg-optimizeAUR configuration file to makepkg:

$ makepkg --config /etc/makepkg-optimize.conf
Note: Building with Profile-guided optimization requires that a package be built and installed twice. The first phase initiates profile generation in $PROFDEST/$pkgbase.gen while the second moves the profiles to $PROFDEST/$pkgbase.used and applies them to the software being packaged.

Build an optimized package in a clean chroot

Note: Read Building in a Clean Chroot and familiarize yourself with the "Classic Way".

It is not necessary to install pacman-buildenv_ext-gitAUR or makepkg-optimizeAUR in your Archlinux installation. Alternatively, they can be used to build optimized packages within a chroot.

Chroot Setup

After setting up a chroot, a few additional steps are needed.

Install required packages

First, install some of the backends for the optimization macros to the base chroot:

$ arch-nspawn -M /etc/makepkg.conf "$CHROOT"/root pacman -S graphite openmp upx optipng

Then download and build pacman-buildenv_ext-gitAUR, makepkg-optimizeAUR, and nodejs-svgoAUR.

To install them in the base chroot, copy their package files into it, and install them with pacman, e.g.:

# cp nodejs-svgo-1.1.1-1-any.pkg.tar.xz.pkg.tar.xz "$CHROOT"/root/root/
$ arch-nspawn -M /etc/makepkg.conf "$CHROOT"/root pacman -U /root/nodejs-svgo-1.1.1-1-any.pkg.tar.xz.pkg.tar.xz

Create a Profile-guided Optimization cache

First, create a folder in the same place--both inside and outside of the chroot--to store profiles:

# mkdir -m 777 {"$CHROOT"/root,}/mnt/pgo

Then edit $CHROOT/root/etc/makepkg-optimize.conf and set PROFDEST=/mnt/pgo under PACKAGE OUTPUT.

Using the chroot

Note: When updating the chroot, AUR packages will have to be updated manually

Build a package

First, edit $CHROOT/root/etc/makepkg-optimize.conf and select your preferred optimizations.

When building, pass the makepkg-optimizeAUR configuration file to makepkg:

$ makechrootpkg -c -r "$CHROOT" -- --config /etc/makepkg-optimize.conf
Building with Profile-guided optimization

After the first building phase, bind the PGO folder:

# mount -o bind {,"$CHROOT"/"$USER"}/mnt/pgo

Once the package is installed, test-run its executables.

For the second building phase, overwrite the original package file and do not pass -c to makechrootpkg:

Note: If you have rebooted, be sure to rebind the PGO folder before rebuilding. Alternatively, use fstab to bind these folders persistently.
$ makechrootpkg -r "$CHROOT" -- -f --config /etc/makepkg-optimize.conf