Difference between revisions of "Arch Build System"

From ArchWiki
Jump to: navigation, search
(revert latest changes; fakeroot is part of "base-devel" and is enabled by default in makepkg.conf; redundant)
m (Added pacman-src-git from AUR)
(217 intermediate revisions by 60 users not shown)
Line 1: Line 1:
[[Category:About Arch (English)]]
+
[[Category:About Arch]]
[[Category:Package development (English)]][[Category:Package management (English)]]
+
[[Category:Package development]]
[[Category:HOWTOs (English)]]
+
[[Category:Package management]]
{{i18n|Arch Build System}}
+
[[cs:Arch Build System]]
{{Article summary start}}
+
[[da:Arch Build System]]
{{Article summary text|Description of the Arch Build System, a 'ports-like' system for building and packaging software from source code.}}
+
[[de:Arch Build System]]
{{Article summary heading|Related articles}}
+
[[el:Arch Build System]]
{{Article summary wiki|Arch Packaging Standards}}
+
[[es:Arch Build System]]
{{Article summary wiki|Arch User Repository}}
+
[[fr:ABS]]
{{Article summary wiki|Building Packages}}
+
[[it:Arch Build System]]
{{Article summary wiki|Kernel Compilation with ABS}}
+
[[ja:Arch Build System]]
{{Article summary wiki|makepkg}}
+
[[ko:Arch Build System]]
{{Article summary wiki|makepkg.conf}}
+
[[pl:Arch Build System]]
{{Article summary wiki|pacman}}
+
[[pt:Arch Build System]]
{{Article summary wiki|pacman Tips}}
+
[[ro:ABS]]
{{Article summary wiki|PKGBUILD}}
+
[[ru:Arch Build System]]
{{Article summary wiki|PKGBUILD Tricks}}
+
[[zh-hans:Arch Build System]]
{{Article summary end}}
+
[[zh-hant:Arch Build System]]
 +
{{Related articles start}}
 +
{{Related|Arch packaging standards}}
 +
{{Related|Arch User Repository}}
 +
{{Related|Creating packages}}
 +
{{Related|Kernel Compilation with ABS}}
 +
{{Related|makepkg}}
 +
{{Related|Official repositories}}
 +
{{Related|pacman}}
 +
{{Related|PKGBUILD}}
 +
{{Related|Patching in ABS}}
 +
{{Related articles end}}
  
This article provides an overview of the Arch Build System along with a quick walkthrough for beginners. It is not a complete reference guide! If you need more information, please reference the man pages.
+
This article provides an overview of the Arch Build System (ABS) along with a walkthrough for beginners. It is not intended to be a complete reference guide.
  
 
== What is the Arch Build System? ==
 
== What is the Arch Build System? ==
  
The Arch Build System (ABS for short) is a 'ports-like' system for building and packaging software from source code. While [[pacman]] is the specialized Arch tool for binary package management (including packages built with the ABS) ABS is the specialized Arch tool for compiling source into an installable {{Filename|.pkg.tar.gz}} package.
+
The Arch Build System is a ''ports-like'' system for building and packaging software from source code. While [[pacman]] is the specialized Arch tool for binary package management (including packages built with the ABS), ABS is a collection of tools for compiling source into installable {{ic|.pkg.tar.xz}} packages.
  
 
=== What is a ports-like system? ===
 
=== What is a ports-like system? ===
  
'Ports' is a system used by *BSD, which allows source packages to be downloaded, unpacked, patched, compiled and installed. A 'port' is merely a small directory on the user's computer, named after the corresponding software to be installed, which contains a few files with instructions for downloading and installing an application from source (typically by navigating to the directory, or port, and doing 'make' and 'make install'). The system will then download, compile and install the desired software.
+
''Ports'' is a system used by *BSD to automate the process of building software from source code. The system uses a ''port'' to download, unpack, patch, compile, and install the given software. A ''port'' is merely a small directory on the user's computer, named after the corresponding software to be installed, that contains a few files with the instructions for building and installing the software from source. This makes installing software as simple as typing {{ic|make}} or {{ic|make install clean}} within the port's directory.
  
=== '''ABS''' is a similar concept ===
+
=== ABS is a similar concept ===
  
ABS is made up of a directory tree (the ABS tree) residing under {{Filename|/var/abs}}. This tree contains many subdirectories, each within a category and each named by their respective package. This tree represents (but does not contain) all ''official Arch software'', retrievable through the SVN system. You may refer to each package-named subdirectory as an 'ABS', much the way one would refer to a 'port'. These ABS (or subdirectories) do not contain the software package nor the source but rather a [[PKGBUILD]] file (and sometimes other files). A PKGBUILD is a simple BASH build script -- a text file containing the compilation and packaging instructions as well as the URL of the appropriate '''source''' tarball to be downloaded. (The most important component of ABS are PKGBUILDs.) By issuing the ABS [[makepkg]] command, the software is first compiled and then ''packaged'' within the build directory before being installed. Now you may use [[pacman]], the Arch Linux package manager, to install, upgrade, and remove your new package.
+
ABS is made up of a directory tree that can be checked out using SVN. This tree represents, but does not contain, all official Arch software. Subdirectories do not contain the software package nor the source but rather a [[PKGBUILD]] file and sometimes other files. By issuing [[makepkg]] inside a directory containing a PKGBUILD, the software is first compiled and then packaged within the build directory. Then you can use [[pacman]] to install or upgrade your new package.
  
 
=== ABS overview ===
 
=== ABS overview ===
  
'ABS' may be used as an umbrella term, since it includes and relies on several other components. Therefore, though not technically accurate, 'ABS' can refer to the following structure and tools as a complete toolkit:
+
'ABS' may be used as an umbrella term since it includes and relies on several other components; therefore, though not technically accurate, 'ABS' can refer to the following tools as a complete toolkit:
  
; The ABS tree: The ABS directory structure; an SVN hierarchy under {{Filename|/var/abs/}} on your (local) machine. It contains many subdirectories, named for all available official Arch Linux software from repositories specified in {{Filename|/etc/abs.conf}}, but not the packages themselves.
+
; SVN tree: The directory structure containing files needed to build all official packages but not the packages themselves nor the source files of the software. It is available in [https://www.archlinux.org/svn/ svn] and [https://projects.archlinux.org/svntogit/packages.git/ git] repositories.
  
; ABS: A set of tools to retrieve and build '''official''' Arch Linux PKGBUILDs. Example PKGBUILDs are also included.
+
; [[PKGBUILD]]: A [[Bash]] script that contains the URL of the source code along with the compilation and packaging instructions.
  
; [[PKGBUILD]]s: Text build script files residing under the ABS directories, or that are custom made, with instructions for building packages and the URL of the sources.  
+
; [[makepkg]]: shell command tool which reads the PKGBUILDs, automatically downloads and compiles the sources and creates a {{ic|.pkg.tar*}} according to the {{ic|PKGEXT}} array in {{ic|makepkg.conf}}. You may also use makepkg to make your own custom packages from the [[AUR]] or third-party sources. See [[Creating packages]] for more information.
  
; [[makepkg]]: ABS shell command tool which reads the PKGBUILDs, automatically downloads and compiles the sources and creates a {{Filename|.pkg.tar.gz}}.
+
; [[pacman]]: pacman is completely separate, but is necessarily invoked either by makepkg or manually, to install and remove the built packages and for fetching dependencies.
  
; [[pacman]]: pacman is completely separate, but is necessarily invoked either by makepkg or manually to install and remove the built packages and for fetching dependencies.
+
; [[AUR]]: The Arch User Repository is separate from ABS but AUR (unsupported) PKGBUILDs are built using makepkg to compile and package up software. In contrast to the ABS tree on your local machine, the AUR exists as a website interface. It contains many thousands of user-contributed PKGBUILDs for software which is unavailable as an official Arch package. If you need to build a package outside the official Arch tree, chances are it is in the AUR.
  
; [[AUR]]: The Arch User Repository is separate from ABS but [[AUR]] ([unsupported]) PKGBUILDs can be built using the ABS '''makepkg''' tool to compile and package up software. The AUR contains almost 16,000 user-contributed PKGBUILDs for software which is unavailable as an official Arch package. If you need to build a package outside the official Arch tree, chances are it is in the [[AUR]].
+
{{Warning|Official PKGBUILDs assume that packages are [[DeveloperWiki:Building_in_a_Clean_Chroot|built in a clean chroot]]. Building software on a ''dirty'' build system may fail or cause unexpected behaviour at runtime, because if the build system detects dependencies dynamically, the result depends on what packages are available on the build system.}}
 +
 
 +
==== SVN tree ====
 +
 
 +
The ''core'', ''extra'', and ''testing'' [[repositories]] are in the ''packages'' SVN respository for [[#Non-recursive checkout|checkout]]. The ''community'' and ''multilib'' repositories are in the ''community'' SVN repository.
 +
 
 +
Each package has its own subdirectory. Within it there are {{ic|repos}} and {{ic|trunk}} directories. {{ic|repos}} is further broken down by repository name (e.g., ''core'') and architecture. PKGBUILD's and files found in {{ic|repos}} are used in official builds. Files found in {{ic|trunk}} are used by developers in preparation before being copied to {{ic|repos}}.
 +
 
 +
For example, the tree for {{pkg|acl}} looks like this:
 +
 
 +
acl
 +
acl/repos
 +
acl/repos/core-i686
 +
acl/repos/core-i686/PKGBUILD
 +
acl/repos/core-x86_64
 +
acl/repos/core-x86_64/PKGBUILD
 +
acl/trunk
 +
acl/trunk/PKGBUILD
 +
 
 +
The source code for the package is not present in the ABS directory. Instead, the {{ic|PKGBUILD}} contains a URL that will download the source code when the package is built.
  
 
== Why would I want to use ABS? ==
 
== Why would I want to use ABS? ==
  
 
The Arch Build System is used to:
 
The Arch Build System is used to:
* Recompile a package, for any reason
+
* Compile or recompile a package, for any reason
* Make and install new packages from source of software for which no packages are yet available (see [[Building Packages]])  
+
* Make and install new packages from source of software for which no packages are yet available (see [[Creating packages]])  
 
* Customize existing packages to fit your needs (enabling or disabling options, patching)
 
* Customize existing packages to fit your needs (enabling or disabling options, patching)
* Rebuild your entire system using your compiler flags, "a la FreeBSD" (e.g. with [[pacbuilder]])
+
* Rebuild your entire system using your compiler flags, "à la FreeBSD" (e.g. with {{AUR|pacbuilder-svn}} (not available anymore) or {{AUR|pacman-src-git}})
* Cleanly build and install your own custom kernel (see [[Custom Kernel Compilation with ABS]] as well as [[Kernel Compilation]])
+
* Cleanly build and install your own custom kernel (see [[Kernels#Compilation|Kernel compilation]])
 
* Get kernel modules working with your custom kernel
 
* Get kernel modules working with your custom kernel
 
* Easily compile and install a newer, older, beta, or development version of an Arch package by editing the version number in the PKGBUILD
 
* Easily compile and install a newer, older, beta, or development version of an Arch package by editing the version number in the PKGBUILD
Line 61: Line 91:
 
ABS is not necessary to use Arch Linux, but it is useful for automating certain tasks of source compilation.
 
ABS is not necessary to use Arch Linux, but it is useful for automating certain tasks of source compilation.
  
== Walkthrough ==
+
== How to use ABS ==
 
 
''With the '''ABS tree''' in place, an Arch user has all available Arch software at their fingertips to compile from source, automatically package as a {{Filename|.pkg.tar.gz}}, and finally, install with pacman.''
 
 
 
=== Quick overview ===
 
 
 
Install the ABS package with {{Codeline|pacman -S abs}}. Running {{Codeline|abs}} as root creates the ABS tree by synchronizing with the Arch Linux server. If you wanted to build a package from source you would copy the build files (usually residing under {{Filename|/var/abs/<repo>/<pkgname>}}) to a build directory, navigate to that directory, edit the PKGBUILD (if desired/necessary) and do '''makepkg'''. According to instructions in the PKGBUILD, makepkg will download the appropriate source tarball, unpack it, patch if desired, compile according to CFLAGS specified in {{Filename|makepkg.conf}}, and finally compress the built files into a package with the extension {{Filename|.pkg.tar.gz}}. PKGBUILDs may be customized to suit your unique configuration needs, or for applying patches. Installing is as easy as doing {{Codeline|pacman -U <.pkg.tar.gz file>}}. Package removal is also handled by pacman.
 
 
 
You may also use makepkg to make your own custom packages from the [[AUR]] or third-party sources. (See the [[Building Packages]] wiki article.)
 
 
 
=== Details ===
 
 
 
To use abs, you first need to install '''abs''' from the [core] repository. This can be done simply by:
 
 
 
# pacman -S abs
 
 
 
This will grab the abs-sync scripts, various build scipts, and [[rsync]] (as a dependency, if you don't already have it).
 
 
 
Before you can actually build anything, however, you will also need to grab basic compiling tools.  These are handily collected in the package group '''base-devel'''.  This group can be installed with:
 
 
 
# pacman -S base-devel
 
 
 
{{Warning | Remember this before complaining about missing (make)dependencies. The "base" group is assumed already installed in all Arch setups. The group "base-devel" is assumed already installed when building with makepkg.}}
 
 
 
=== {{Filename|/etc/abs.conf}} ===
 
 
 
As root, edit {{Filename|/etc/abs.conf}} to include your desired repositories:
 
 
 
# vim /etc/abs.conf
 
 
 
or:
 
  
# nano /etc/abs.conf
+
To retrieve the [[PKGBUILD]] required to build a certain package from source, you can either use [[Svn]] or a [[Git]]-based approach using the {{pkg|asp}} package which is a thin wrapper around the svntogit repositories. In the following, the svn-based method as well as the [[#Retrieve PKGBUILD source using Git|git-based method]] is described.
  
Remove the ! in front of the appropriate repos, e.g.:
+
=== Retrieve PKGBUILD source using Svn ===
  
REPOS=(core extra community !testing)
+
==== Prerequisites ====
  
=== Download the ABS tree ===
+
[[Install]] the {{Pkg|subversion}} package.
  
As root, run:
+
==== Non-recursive checkout ====
  
# abs
+
{{Warning|Do not download the whole repository; only follow the instructions below. The entire SVN repository is huge. Not only will it take an obscene amount of disk space, but it will also tax the archlinux.org server for you to download it. If you abuse this service, your address may be blocked. Never use the public SVN for any sort of scripting.}}
  
Your ABS tree is now created under {{Filename|/var/abs}}. Note the appropriate branches of the ABS tree now exist and correspond to the ones you specified in {{Filename|/etc/abs.conf}}.
+
To checkout the ''core'', ''extra'', and ''testing'' [[repositories]]:
  
If you are an active developer, the abs command should also be run periodically to be in sync with the official repositories. If you are a casual developer, individual ABS package files can be downloaded by:
+
  $ svn checkout --depth=empty <nowiki>svn://svn.archlinux.org/packages</nowiki>
  
# abs repository/package
+
To checkout the ''community'' and ''multilib'' repositories:
  
=== {{Filename|/etc/makepkg.conf}} ===
+
$ svn checkout --depth=empty <nowiki>svn://svn.archlinux.org/community</nowiki>
  
{{Filename|/etc/makepkg.conf}} specifies global environment variables and compiler flags which you may wish to edit if you are using an SMP system, or to specify other desired optimizations. The default settings are for i686 and x86_64 optimizations which will work fine for those architectures on single-CPU systems. (The defaults will work on SMP machines, but will only use one core/CPU when compiling -- see [[makepkg.conf]].)
+
In both cases, it simply creates an empty directory, but it does know that it is an svn checkout.
  
=== The ABS tree ===
+
==== Checkout a package ====
  
When you run {{Codeline|abs}} for the first time, it synchronizes the ABS tree on the Arch Linux server to your computer. The ABS tree is an SVN directory hierarchy located under {{Filename|/var/abs}} and looks like this:
+
In the directory containing the svn repository you checked out (i.e., ''packages'' or ''community''), do:
  
<pre>
+
$ svn update ''package-name''
| -- core/
 
|    || -- base/
 
|    ||    || -- acl/
 
|    ||    ||    || -- PKGBUILD
 
|    ||    || -- attr/
 
|    ||    ||    || -- PKGBUILD
 
|    ||    || -- ...
 
|    || -- devel/
 
|    ||    || -- abs/
 
|    ||    ||    || -- PKGBUILD
 
|    ||    || -- autoconf/
 
|    ||    ||    || -- PKGBUILD
 
|    ||    || -- ...
 
|    || -- ...
 
| -- extra/
 
|    || -- daemons/
 
|    ||    || -- acpid/
 
|    ||    ||    || -- PKGBUILD
 
|    ||    ||    || -- ...
 
|    ||    || -- apache/
 
|    ||    ||    || -- ...
 
|    ||    || -- ...
 
|    || -- ...
 
| -- community/
 
|    || -- ...
 
</pre>
 
  
The ABS tree has exactly the same structure as the package database:
+
This will pull the package you requested into your checkout. From now on, any time you ''svn update'' at the top level, this will be updated as well.
  
* First-level: Category directories
+
If you specify a package that does not exist, svn will not warn you.  It will just print something like "At revision 115847", without creating any files.  If that happens:
* Second-level: Package name directories
+
* check your spelling of the package name
* Third level: PKGBUILD (contains information needed to build a package) and other related files (patches, other files needed for building the package)
+
* check that the package has not been moved to another repository (i.e. from community to the main repository)
 +
* check https://www.archlinux.org/packages to see if the package is built from another base package (for example, {{Pkg|python-tensorflow}} is built from the {{Pkg|tensorflow}} PKGBUILD)
  
The source code for the package is not present in the ABS directory. Instead, the '''PKGBUILD''' file contains a URL that will download the source code when the package is built.
+
{{Tip|To checkout an older version of a package, see [[#Checkout an older version of a package]].}}
  
=== Create a build directory ===
+
You should periodically update all of your checked out packages if you wish to perform rebuilds on more recent revisions of the repositories. To do so, do:
  
You must create a build directory where the actual compiling will take place. This is where you'll do everything; you should never modify the ABS tree by building within it, as data will be lost (overwritten) on each ABS update. It is good practice to use your home directory, though some Arch users prefer to create a 'local' directory under {{Filename|/var/abs/}}, owned by a normal user. Copy the ABS from the tree ({{Filename|/var/abs/branch/category/pkgname}}) to the build directory, {{Filename|/path/to/build/dir}}.
+
$ svn update
  
Create your build directory. e.g.:
+
=== Retrieve PKGBUILD source using Git ===
  
$ mkdir -p $HOME/abs
+
As a precondition, [[install]] the {{Pkg|asp}} package.
  
{{Note | The first download of the abs tree is the biggest, then only minor updates are needed. Don't be afraid about the data to download if you've got only a 56K connection; it's only text files and is compressed during the transfer. For example, as of November 24, 2009, the abs tree that includes core, extra and community repositories is a ~16MB download (~56MB on disk).}}
+
To clone the svntogit-repository for a specific package, use:
 +
$ asp checkout ''pkgname''
  
=== The build function, traditional method ===
+
This will clone the git repository for the given package into a directory named like the package.
  
If you're not familiar with compiling from source, you should know that most packages (but not all) can be built from source in this '''traditional way''':
+
To update the cloned git repository, run {{ic|asp update}} followed by {{ic|git pull}} inside the git repository.
  
* Download source tarball from remote server, using web browser, ftp, wget or alternate method.
+
Furthermore, you can use all other git commands to checkout an older version of the package or to track custom changes. For more information on git usage, see the [[git]] page.
  
* Decompress the source file:
+
If you just want to copy a snapshot of the current [[PKGBUILD]] for a specific package, use:
  $ tar -xzf foo-0.99.tar.gz
+
  $ asp export ''pkgname''
or
 
$ tar -xjf foo-0.99.tar.bz2
 
  
* Enter the directory:
+
=== Build package ===
$ cd foo-0.99
 
  
* Configure the package. Generally, there is a script called {{Filename|configure}} in the source directory that is used to configure the package (add or remove support for things, choose the install destination, etc.) and check that your computer has all the software needed by the package. It can be run by:
+
See [[makepkg#Configuration]] on how to configure ''makepkg'' for building packages from the [[PKGBUILD]]'s you have checked out.
$ ./configure [option]
 
  
You should first try the help to better understand how it works:
+
Then, copy the directory containing the [[PKGBUILD]] you wish to modify to a new location. There, make the desired modifications and use ''makepkg'' there as described in [[makepkg#Usage]] to create and install the new package.
$ ./configure --help
 
  
If a {{Codeline|--prefix}} option is not passed to the script, ''most'' scripts will use {{Filename|/usr/local}} as the install path, but others will use {{Filename|/usr}}. For the sake of consistency, it is generally advised to pass the {{Codeline|1=--prefix=/usr/local}} option. It is good practice to install personal programs in {{Filename|/usr/local}}, and to have the ones being managed by the distro in {{Filename|/usr}}. This ensures personal program versions can coexist with those being managed by the distro's package manager -- in Arch's case, ''pacman''.
+
== Tips and tricks ==
$ ./configure --prefix=/usr/local
 
  
* Compile the sources:
+
=== Preserve modified packages ===
  
$ make
+
Updating the system with pacman will replace a modified package from ABS with the package of the same name from the official repositories. See the following instructions for how to avoid this.
  
* Install:
+
Insert a group array into the PKGBUILD, and add the package to a group called {{ic|modified}}.
  
# make install
+
{{hc|PKGBUILD|2=
 +
groups=('modified')
 +
}}
  
* Removal would be accomplished by entering the source directory and running:
+
Add this group to the section {{ic|IgnoreGroup}} in {{ic|/etc/pacman.conf}}.
  
# make uninstall
+
{{hc|/etc/pacman.conf|2=
 +
IgnoreGroup = modified
 +
}}
  
However, you should always read the {{Filename|INSTALL}} file to know how the package should be built and installed! '''Not all packages use the <code>configure; make; make install</code> system!'''
+
If new versions are available in the official repositories during a system update, pacman prints a note that it is skipping this update because it is in the IgnoreGroup section. At this point the modified package should be rebuilt from ABS to avoid partial upgrades.
  
{{Note | The above traditional method of compiling source tarballs can, of course, still be used on Arch Linux. However, if you are not careful, files may become scattered throughout the filesystem that pacman (or any other package manager) will be unaware of. You should only use this method if you are experienced at manual compilation and system software tracking, as it can lead to future problems on Arch (or any distribution) if using a package manager.}}
+
=== Checkout an older version of a package ===
  
=== The build function, the ABS way ===
+
Within the svn repository you checked out as described in [[#Non-recursive checkout]] (i.e. "packages" or "community"), first examine the log:
  
ABS is an elegant tool which allows for powerful assistance and customization for the build process and creates a ''pacman-trackable'' package file for installation. The ABS method involves copying an ABS from the tree to a build directory, and doing makepkg. In our example, we will build the ''slim'' display manager package.
+
$ svn log ''package-name''
  
Copy the slim ABS from the ABS tree to a build directory:
+
Find out the revision you want by examining the history, then specify the revision you wish to checkout. For example, to checkout revision {{ic|r1729}} you would do:
$ cp -r /var/abs/extra/slim/ ~/abs
 
  
Navigate to the build directory:
+
  $ svn update -r1729 ''package-name''
  $ cd ~/abs/slim
 
  
Modify the PKGBUILD to add or remove support for components, to patch or to change package versions, etc. (optional):
+
This will update an existing working copy of ''package-name'' to the chosen revision.
$ nano PKGBUILD
 
  
Run makepkg as normal user (with {{Codeline|-s}} switch to install with automatic dependency handling):
+
You can also specify a date. If no revision on that day exists, svn will grab the most recent package before that time. The following example checks out the revision from 2009-03-03:
$ makepkg -s
 
  
Install as root:
+
  $ svn update -r{20090303} ''package-name''
  # pacman -U slim-1.3.0-2-i686.pkg.tar.gz
 
  
That's it. You have just built slim from source and cleanly installed it to your system with pacman. Package removal is also handled by pacman -- ({{Codeline|pacman -R slim}}).
+
It is possible to checkout packages at versions before they were moved to another repository as well; check the logs thoroughly for the date they were moved or the last revision number.
  
Essentially, the same steps are being executed as described in the traditional method (generally including the <code>./configure, make, make install</code> steps) but the software is installed into a ''fake root'' environment. (A ''fake root'' is simply a subdirectory within the build directory that functions and behaves as the system's root directory. In conjunction with the '''''fakeroot''''' program, makepkg creates a fake root directory, and installs the compiled binaries and associated files into it, with '''root''' as owner.) The ''fake root'', or subdirectory tree containing the compiled software, is then compressed into an archive with the extension {{Filename|.pkg.tar.gz}}, or a ''package''. When invoked, pacman then extracts the package (installs it) into the system's real root directory ({{Filename|/}}). Simple.
+
== Other tools ==
  
''The ABS method adds a level of convenience and automation, while still maintaining complete transparency and control of the build and installation functions by including them in the PKGBUILD.''
+
* [http://xyne.archlinux.ca/projects/pbget/ pbget] - retrieve PKGBUILDs for individual packages directly from the web interface. Includes AUR support.
 +
* [https://github.com/falconindy/asp asp] - a tool to manage the build source files used to create Arch Linux packages. Uses the git interface which offers more up to date sources.

Revision as of 17:08, 21 April 2018

ro:ABS

This article provides an overview of the Arch Build System (ABS) along with a walkthrough for beginners. It is not intended to be a complete reference guide.

What is the Arch Build System?

The Arch Build System is a ports-like system for building and packaging software from source code. While pacman is the specialized Arch tool for binary package management (including packages built with the ABS), ABS is a collection of tools for compiling source into installable .pkg.tar.xz packages.

What is a ports-like system?

Ports is a system used by *BSD to automate the process of building software from source code. The system uses a port to download, unpack, patch, compile, and install the given software. A port is merely a small directory on the user's computer, named after the corresponding software to be installed, that contains a few files with the instructions for building and installing the software from source. This makes installing software as simple as typing make or make install clean within the port's directory.

ABS is a similar concept

ABS is made up of a directory tree that can be checked out using SVN. This tree represents, but does not contain, all official Arch software. Subdirectories do not contain the software package nor the source but rather a PKGBUILD file and sometimes other files. By issuing makepkg inside a directory containing a PKGBUILD, the software is first compiled and then packaged within the build directory. Then you can use pacman to install or upgrade your new package.

ABS overview

'ABS' may be used as an umbrella term since it includes and relies on several other components; therefore, though not technically accurate, 'ABS' can refer to the following tools as a complete toolkit:

SVN tree
The directory structure containing files needed to build all official packages but not the packages themselves nor the source files of the software. It is available in svn and git repositories.
PKGBUILD
A Bash script that contains the URL of the source code along with the compilation and packaging instructions.
makepkg
shell command tool which reads the PKGBUILDs, automatically downloads and compiles the sources and creates a .pkg.tar* according to the PKGEXT array in makepkg.conf. You may also use makepkg to make your own custom packages from the AUR or third-party sources. See Creating packages for more information.
pacman
pacman is completely separate, but is necessarily invoked either by makepkg or manually, to install and remove the built packages and for fetching dependencies.
AUR
The Arch User Repository is separate from ABS but AUR (unsupported) PKGBUILDs are built using makepkg to compile and package up software. In contrast to the ABS tree on your local machine, the AUR exists as a website interface. It contains many thousands of user-contributed PKGBUILDs for software which is unavailable as an official Arch package. If you need to build a package outside the official Arch tree, chances are it is in the AUR.
Warning: Official PKGBUILDs assume that packages are built in a clean chroot. Building software on a dirty build system may fail or cause unexpected behaviour at runtime, because if the build system detects dependencies dynamically, the result depends on what packages are available on the build system.

SVN tree

The core, extra, and testing repositories are in the packages SVN respository for checkout. The community and multilib repositories are in the community SVN repository.

Each package has its own subdirectory. Within it there are repos and trunk directories. repos is further broken down by repository name (e.g., core) and architecture. PKGBUILD's and files found in repos are used in official builds. Files found in trunk are used by developers in preparation before being copied to repos.

For example, the tree for acl looks like this:

acl
acl/repos
acl/repos/core-i686
acl/repos/core-i686/PKGBUILD
acl/repos/core-x86_64
acl/repos/core-x86_64/PKGBUILD
acl/trunk
acl/trunk/PKGBUILD

The source code for the package is not present in the ABS directory. Instead, the PKGBUILD contains a URL that will download the source code when the package is built.

Why would I want to use ABS?

The Arch Build System is used to:

  • Compile or recompile a package, for any reason
  • Make and install new packages from source of software for which no packages are yet available (see Creating packages)
  • Customize existing packages to fit your needs (enabling or disabling options, patching)
  • Rebuild your entire system using your compiler flags, "à la FreeBSD" (e.g. with pacbuilder-svnAUR (not available anymore) or pacman-src-gitAUR)
  • Cleanly build and install your own custom kernel (see Kernel compilation)
  • Get kernel modules working with your custom kernel
  • Easily compile and install a newer, older, beta, or development version of an Arch package by editing the version number in the PKGBUILD

ABS is not necessary to use Arch Linux, but it is useful for automating certain tasks of source compilation.

How to use ABS

To retrieve the PKGBUILD required to build a certain package from source, you can either use Svn or a Git-based approach using the asp package which is a thin wrapper around the svntogit repositories. In the following, the svn-based method as well as the git-based method is described.

Retrieve PKGBUILD source using Svn

Prerequisites

Install the subversion package.

Non-recursive checkout

Warning: Do not download the whole repository; only follow the instructions below. The entire SVN repository is huge. Not only will it take an obscene amount of disk space, but it will also tax the archlinux.org server for you to download it. If you abuse this service, your address may be blocked. Never use the public SVN for any sort of scripting.

To checkout the core, extra, and testing repositories:

$ svn checkout --depth=empty svn://svn.archlinux.org/packages

To checkout the community and multilib repositories:

$ svn checkout --depth=empty svn://svn.archlinux.org/community

In both cases, it simply creates an empty directory, but it does know that it is an svn checkout.

Checkout a package

In the directory containing the svn repository you checked out (i.e., packages or community), do:

$ svn update package-name

This will pull the package you requested into your checkout. From now on, any time you svn update at the top level, this will be updated as well.

If you specify a package that does not exist, svn will not warn you. It will just print something like "At revision 115847", without creating any files. If that happens:

  • check your spelling of the package name
  • check that the package has not been moved to another repository (i.e. from community to the main repository)
  • check https://www.archlinux.org/packages to see if the package is built from another base package (for example, python-tensorflow is built from the tensorflow PKGBUILD)
Tip: To checkout an older version of a package, see #Checkout an older version of a package.

You should periodically update all of your checked out packages if you wish to perform rebuilds on more recent revisions of the repositories. To do so, do:

$ svn update

Retrieve PKGBUILD source using Git

As a precondition, install the asp package.

To clone the svntogit-repository for a specific package, use:

$ asp checkout pkgname

This will clone the git repository for the given package into a directory named like the package.

To update the cloned git repository, run asp update followed by git pull inside the git repository.

Furthermore, you can use all other git commands to checkout an older version of the package or to track custom changes. For more information on git usage, see the git page.

If you just want to copy a snapshot of the current PKGBUILD for a specific package, use:

$ asp export pkgname

Build package

See makepkg#Configuration on how to configure makepkg for building packages from the PKGBUILD's you have checked out.

Then, copy the directory containing the PKGBUILD you wish to modify to a new location. There, make the desired modifications and use makepkg there as described in makepkg#Usage to create and install the new package.

Tips and tricks

Preserve modified packages

Updating the system with pacman will replace a modified package from ABS with the package of the same name from the official repositories. See the following instructions for how to avoid this.

Insert a group array into the PKGBUILD, and add the package to a group called modified.

PKGBUILD
groups=('modified')

Add this group to the section IgnoreGroup in /etc/pacman.conf.

/etc/pacman.conf
IgnoreGroup = modified

If new versions are available in the official repositories during a system update, pacman prints a note that it is skipping this update because it is in the IgnoreGroup section. At this point the modified package should be rebuilt from ABS to avoid partial upgrades.

Checkout an older version of a package

Within the svn repository you checked out as described in #Non-recursive checkout (i.e. "packages" or "community"), first examine the log:

$ svn log package-name

Find out the revision you want by examining the history, then specify the revision you wish to checkout. For example, to checkout revision r1729 you would do:

$ svn update -r1729 package-name

This will update an existing working copy of package-name to the chosen revision.

You can also specify a date. If no revision on that day exists, svn will grab the most recent package before that time. The following example checks out the revision from 2009-03-03:

$ svn update -r{20090303} package-name

It is possible to checkout packages at versions before they were moved to another repository as well; check the logs thoroughly for the date they were moved or the last revision number.

Other tools

  • pbget - retrieve PKGBUILDs for individual packages directly from the web interface. Includes AUR support.
  • asp - a tool to manage the build source files used to create Arch Linux packages. Uses the git interface which offers more up to date sources.