Difference between revisions of "Creating packages"

From ArchWiki
Jump to: navigation, search
(Temporary save again.)
(Finished page creation.)
Line 10: Line 10:
  
 
<ul>
 
<ul>
   <li><strong>abs</strong>: <a href="http://wiki.archlinux.org/index.php/ABS">Arch
+
   <li><strong>abs</strong>: [[ABS|Arch Build System]].</li>
  Build System</a>.</li>
+
 
   <li><strong>base-devel</strong>: Contains <em>makepkg</em> and additional
 
   <li><strong>base-devel</strong>: Contains <em>makepkg</em> and additional
 
       tools for compiling from the source.</li>
 
       tools for compiling from the source.</li>
Line 95: Line 94:
 
download the software's source code and compile it according to the
 
download the software's source code and compile it according to the
 
instructions specified in the <em>PKGBUILD</em> file. The instructions must be
 
instructions specified in the <em>PKGBUILD</em> file. The instructions must be
fully interpretable by <a href="http://en.wikipedia.org/wiki/Bash">Bash</a>.
+
fully interpretable by [[Wikipedia:Bash|Bash]].
 
After successful compilation, the resulting binaries and metadata of the
 
After successful compilation, the resulting binaries and metadata of the
 
package, i.e.  package version and dependencies, are packed in a
 
package, i.e.  package version and dependencies, are packed in a
 
<em>pkgname.pkg.tar.gz</em> package file that can be cleanly installed with
 
<em>pkgname.pkg.tar.gz</em> package file that can be cleanly installed with
<em>pacman -Up [package file]</em>.</p>
+
<em>pacman -U [package file]</em>.</p>
  
 
<p>To start out with a new package, you should first create an empty working
 
<p>To start out with a new package, you should first create an empty working
Line 339: Line 338:
 
     use a subdirectory in order to keep the <em>/etc</em> area as clean as
 
     use a subdirectory in order to keep the <em>/etc</em> area as clean as
 
     possible. Use <em>/etc/<strong>pkgname</strong></em> or a suitable
 
     possible. Use <em>/etc/<strong>pkgname</strong></em> or a suitable
     alternative, e.g., <a href="http://httpd.apache.org/">Apache</a> uses
+
     alternative, e.g., [http://httpd.apache.org/ Apache] uses
 
     <em>/etc/httpd/</em>.
 
     <em>/etc/httpd/</em>.
 
   </li>
 
   </li>
Line 479: Line 478:
 
   <li>If not, repeat the last two steps.</li>
 
   <li>If not, repeat the last two steps.</li>
 
</ol>
 
</ol>
 +
 +
<h2>External links</h2>
 +
* [http://www.bytetrap.com/blog/2008/09/16/building-packages-arch-linux/ Building Packages in Arch Linux]

Revision as of 18:21, 26 September 2008


Setting Up

First, verify that you have the necessary tools installed. You need the following packages:

  • abs: Arch Build System.
  • base-devel: Contains makepkg and additional tools for compiling from the source.
  • fakeroot: Allows a normal user the necessary root permissions to create packages in the build environment without being able to alter the wider system. If the build process attempts to alter files outside of the build environment, errors are produced and the build fails. This is very useful for checking the quality, safety and integrity of the packages for distribution.

If you do not have them, install them using pacman:

$ pacman -S abs base-devel fakeroot

One of the key tools for building packages is makepkg. It does the following:

  1. Checks if package dependencies are installed.
  2. Downloads the source file(s) from the specified server(s).
  3. Unpacks the source file(s).
  4. Compiles the software and installs it under a fakeroot environment.
  5. Strips symbols from binaries and libraries.
  6. Generates the package meta file which is included with each package.
  7. Compress the fakeroot environment into a package file.
  8. Stores the package file in the configured destination directory, which is the present working directory by default.

abs is optional to use, but it gives you the option to build official packages directly from source instead of downloading the pre-built, binary copy. It also helps you keep all of your PKGBUILD files, both official and unofficial in a centralized location. A PKGBUILD file contains metadata about a package, and is the essential component for building packages. You can specify which repositories to include in your ABS tree in the /etc/abs.conf file. To obtain the ABS tree which contains all of the PKGBUILD files, just run the abs command as root:

$ abs

Although you can build packages in any directory, the most flexible and organized place to build them is in /var/abs/local. /var/abs/local is owned by root:root, but a normal user needs write access to the directory to be able to run makepkg on a PKGBUILD file. To accomplish this and maintain security, create an abs group and make yourself a member of that group:

$ groupadd abs
$ usermod -G abs [user]

If you are already logged in, log out for the changes to take place. Next, change the group ownership of the /var/abs/local directory to abs:

$ chown root:abs /var/abs/local
$ chmod 775 /var/abs/local

Preparing the Files

Now download the source tarball of the software you want to package, extract it, and note all commands needed to compile and install it. You will be repeating the same commands in the PKGBUILD file. Most software authors stick to the 3-step build cycle:

./configure
make
make install

When you run makepkg, it will look for a PKGBUILD file in the present working directory. If a PKGBUILD file is found it will download the software's source code and compile it according to the instructions specified in the PKGBUILD file. The instructions must be fully interpretable by Bash. After successful compilation, the resulting binaries and metadata of the package, i.e. package version and dependencies, are packed in a pkgname.pkg.tar.gz package file that can be cleanly installed with pacman -U [package file].

To start out with a new package, you should first create an empty working directory, preferably named /var/abs/local/pkgname. That way it's nicely integrated into the normal ABS-tree and unaltered whenever you synchronize the tree. Once you make the directory and change into it, create a PKGBUILD file to work with either by copying the prototype from /usr/share/pacman/PKGBUILD.proto or by copying a PKGBUILD from another package. The latter is quite useful if you want to modify compilation options of a package instead of creating an entirely new one.

Defining PKGBUILD Variables

The following are variables that can be filled out in the PKGBUILD file:

  • pkgname
    • The name of the package. It should consist of alphanumeric characters only and all letters should be lowercase. For the sake of consistency, pkgname should match the name of the source tarball of the software you are packaging. For instance, if the software is in foobar-2.5.tar.gz, the pkgname value should be foobar. The present working directory the PKGBUILD file is in should also match the pkgname.
  • pkgver
    • The version of the package. The value should be the same as the version released by the author of the package. It can contain letters, numbers and periods but CANNOT contain a hyphen. If the author of the package uses a hyphen in their version numbering scheme, replace it with an underscore. For instance, if the version is 0.99-10, it should be changed to 0.99_10.
  • pkgrel
    • The release number of the package specific to Arch Linux. This value allows users to differentiate between consecutive builds of the same version of a package. When a new package version is first released, the release number starts at 1. As fixes and optimizations are made to the PKGBUILD file, the package will be re-released and the release number will increment by 1. When a new version of the package comes out, the release number resets to 1.
  • pkgdesc
    • The description of the package. The description should be about 80 character or less and should not include the package name in a self-referencing way. For instance, "Nedit is a text editor for X11" should be written as "A text editor for for X11."
  • arch
    • An array of architectures that the PKGBUILD file is known to build and work on. Currently, it should contain i686 and/or x86_64. The value any can also be used for independent packages, but the official repositories of Arch does not support this value.
  • url
    • The URL of the official site of the software being packaged.
  • licenses
    • The license under which the software is distributed. A licenses package has been created in [core] that stores common licenses in /usr/share/licenses/common, i.e. /usr/share/licenses/common/GPL. If a package is licensed under one of these licenses, the value should be set to the directory name, e.g. license=('GPL'). If the appropriate license is not included in the official licenses package, several things must be done:
      1. The license file(s) should be included in: /usr/share/licenses/pkgname/, e.g. /usr/share/licenses/foobar/LICENSE.
      2. If the source tarball does NOT contain the license details and the license is only displayed elsewhere, e.g. a website, then you need to copy the license to a file and include it.
      3. Add custom to the licenses array. Optionally, you can replace custom with custom:name of license. Once a license is used in two or more packages in an official repository (including [community]), it becomes a part of the licenses package.
    • The BSD, MIT, zlib/png and Python licenses are special cases and could not be included in the licenses package. For the sake of the license array, it is treated as a common license (license=('BSD'), license=('MIT'), license=('ZLIB') and license=('Python')) but technically each one is a custom license because each one has its own copyright line. Any packages licensed under these four should have its own unique license stored in /usr/share/licenses/pkgname. Some packages may not be covered by a single license. In these cases, multiple entries may be made in the license array, e.g. license=('GPL' 'custom:name of license').
    • Additionally, the (L)GPL has many versions and perutations of those versions. For (L)GPL software, the convention is:
      • (L)GPL - (L)GPLv2 or any later version
      • (L)GPL2 - (L)GPL2 only
      • (L)GPL3 - (L)GPL3 or any later version
  • groups
    • The group the package belongs in. For instance, when you install the kdebase package, it installs all packages that belongs in the kde group.
  • depends
    • An array of package names that must be installed before this software can be run. If a software requires a minimum version of a dependency, the >= operator should be used to point this out, e.g. depends=('foobar>=1.8.0'). You do not need to list packages that your software depends on if other packages your software depends on already have those packages listed in their dependency. For instance, gtk2 depends on glib2 and glibc. However, glibc does not need to be listed as a dependency for gtk2 because it is a dependency for glib2.
  • makedepends
    • An array of package names that must be installed to build the software but unnecessary for using the software after installation. You can specify the minimum version dependency of the packages in the same format as the depends array.
  • optdepends
    • An array of package names that are not needed for the software to function but provides additional features. A short description of what each package provides should also be noted. An optdepends may look like this:
optdepends=('cups: printing support'
            'sane: scanners support'
            'libgphoto2: digital cameras support'
            'alsa-lib: sound support'
            'giflib: GIF images support'
            'libjpeg: JPEG images support'
            'libpng: PNG images support')
  • provides
    • An array of package names that this package provies the features of (or a virtual package such as cron or sh). If you use this variable, you should add the version (pkgver and perhaps the pkgrel) that this package will provide if dependencies may be affected by it. For instance, if you are providing a modified qt package named qt-foobar version 3.3.8 which provides qt then the provides array should look like provides=('qt=3.3.8'). Putting provides=('qt') will cause dependencies that require a specific version of qt to fail.
  • conflicts
    • An array of package names that may cause problems with this package if installed. You can also specify the version properties of the conflicting packages in the same format as the depends array.
  • replaces
    • An array of obsolete package name that are replaced by this package, e.g. replaces=('ethereal') for the wireshark package. After syncing with pacman -Sy, it will immediately replace an installed package upon encountering another package with the matching replaces in the repositories. If you are providing an alternate version of an already existing package, use the conflicts variable which is only evaluated when actually installing the conflicting package.
  • backup
    • An array of files to be backed up as file.pacsave when the package is removed. This is commonly used for packages placing configuration files in /etc.
  • options
    • This array allows you to override some of the default behavior of makepkg. To set an option, include the option name in the array. To reverse the default behavior, place an ! at the front of the option. The following options may be placed in the array:
      • strip - Strips symbols from binaries and libraries.
      • docs - Save /doc directories.
      • libtool - Leave libtool (.la) files in packages.
      • emptydirs - Leave empty directories in packages.
      • zipman - Compress man and info pages with gzip.
      • ccache - Allow the use of ccache during build. More useful in its negative form !ccache with select packages that have problems building with ccache.
      • distcc - Allow the use of distcc during build. More useful in its negative form !distcc with select packages that have problems building with distcc.
      • makeflags - Allow the use of user-specific makeflags during build. More useful in its negative form !makeflags with select packages that have problems building with custom makeflags.
      • force - Force the package to be upgraded by a pacman system upgrade operation, even if the version number would normally not trigger such an upgrade. This is useful when the version numbering scheme of a package changes (or is alphanumeric).
  • install
    • The name of the .install script to be included in the package. pacman has the ability to store and execute a package-specific script when it installs, removes or upgrades a package. The script contains the following functions which run at different times:
      • pre_install - The script is run right before files are extracted. One argument is passed: new package version.
      • post_install - The script is run right after files are extracted. One argument is passed: new package version.
      • pre_upgrade - The script is run right before files are extracted. Two arguments are passed in the following order: new package version, old package version.
      • post_upgrade - The script is run after files are extracted. Two arguments are passed in the following order: new package version, old package version.
      • pre_remove - The script is run right before files are removed. One argument is passed: old package version.
      • post_remove - The script is run right after files are removed. One argument is passed: old package version.
    • A prototype .install is provided at /usr/share/pacman/proto.install.
  • source
    • An array of files which are needed to build the package. It must contain the location of the software source, which in most cases is a full HTTP or FTP URL. The previously set variables pkgname and pkgver can be used effectively here:
      • source=(http://example.com/<strong>$pkgname</strong>-<strong>$pkgver</strong>.tar.gz)
    • If you need to supply which are not downloadable on the fly, e.g. self-made patches, you simply put those into the same directory where your PKGBUILD file is in and add the filename to this array. Any paths you add here are resolved relative to the directory where the PKGBUILD lies. Before the actual build process is started, all of the files referenced in this array will be downloaded or checked for existence, and makepkg will not proceed if any are missing.
  • noextract
    • An array of files listed under the source array which should not be extracted from their archive format by makepkg. This most commonly applies to certain zip files which cannot be handled by bsdtar because libarchive processes all files as streams rather than random access as unzip does. In these situations unzip should be added in the makedepends array and the first line of the build() function should contain:
cd $srcdir/$pkgname-$pkgver
unzip [source].zip
  • md5sums
    • An array of md5 checksums of the files listed in the source array. Once all files in the source array are available, an md5 hash of each file will be automatically generated and compared with the values of this array in the same order they appear in the source array. While the order of the source files itself does not matter, it is important that it matches the order of this array since makepkg cannot guess which md5sum belongs to what source file. You can generate this array quickly and easily using the command makepkg -g in the directory that contains the PKGBUILD file.

It is common practice to preserve the order of the PKGBUILD variables as shown above. However, this is not mandatory, as the only requirement in this context is correct Bash syntax.

The build() function

Now you need to implement the build() function in the PKGBUILD file. This function uses common shell commands in the <a href="http://en.wikipedia.org/wiki/Bash">Bash</a> syntax to automatically compile software and create a pkg/ directory to install the software to. This allows makepkg to pack it up without having to pick all the necessary files from your actual filesystem.

The first step in the build() function is to change into the directory created by uncompressing the source tarball. In most common cases the first command will look like this:

cd $srcdir/$pkgname-$pkgver

Now, you need to list the same commands you used when you manually compiled the software. The build() function in essence automates everything you did by hand and compiles the software in the fakeroot build environment. If the software you are packaging uses a configure script, it is good practice to use --prefix=/usr when building packages for pacman. A lot of software install their files relative to the /usr/local directory, which should only be done if you are manually building from source. All Arch Linux pakcages should use the /usr directory. As seen in the /usr/share/pacman/PKGBUILD.proto file, the next two lines should look like this:

./configure --prefix=/usr
make || return 1

The final step in the build() funciton is to put the compiled files in a directory where makepkg can retrieve to create a package. This by default is the pkg/ directory - a simple fakeroot environment. It imitates the root of your filesystem to the software's installation procedure. If you have to manually install files under the root of your filesystem, you should instead install it in the pkg/ directory under the same directory structure, e.g. if you want to install a file to /usr/bin, it should instead be placed under $pkgdir/usr/bin. Very few software require the user to copy dozens of files manually, and you usually will not have to worry about this. Instead, for most software you will have to call make install. The final line should look like the following in order to correctly install the software in the pkg/ directory:

make DESTDIR=$pkgdir install

In some odd cases, the software expects to be run from a single directory. In such cases it is wise to simply copy these to $pkgdir/opt.

More often than not the installation process of the software will create any subdirectories below the pkg/ directory. If it does not, however, makepkg will generate a lot of errors and you will need to manually create subdirectories by adding the appropriate mkdir -p commands in the build() function before the installation procedure is run.

Also, makepkg defines three variables that you should use as part of the build and install process:

  • startdir - This contains the absolute path to the directory where the PKGBUILD file is located, which is the present working directory. This variable is almost always used in combination with /src or /pkg postfixes, but the use of srcdir and pkgdir variables is preferred.
  • srcdir - This points to the directory where makepkg extracts or copies all source files. It is currently an alias for $startdir/src.
  • pkgdir - This points to the directory where makepkg bundles the installed package, which becomes the root directory of your built package. It is currently an alias for $startdir/pkg.

Additional Guidelines

Here are some additional guidelines that you should adhere to:

  • All elements in an array variable must be separated by spaces.
  • If possible, remove empty lines from the PKGBUILD including fields that have empty values.
  • Wherever posible, use || return 1 for critical build functions:
    make || return 1
    make DESTDIR=$pkgdir install || return 1
    
  • Do not introduce new variables into your PKGBUILD file unless the package cannot be built without doing so, as these could possibly conflict with variables used in makepkg itself. If a new variable is absolutely required, prefix the variable name with an underscore:
    _customvariable=
    
  • Avoid using /usr/libexec/ for anything. Use /usr/lib/pkgname instead.
  • The packager field from the package meta file can be customized by the package builder by modifying the appropriate option in the /etc/makepkg.conf file, or alternatively by exporting the PACKAGER environment variable before buliding packages with makepkg:
    # export PACKAGER="Your Name <email@domain.com>"
    
  • Configuration files should be placed in the /etc directory. If there is more than one configuration file, it is customary to use a subdirectory in order to keep the /etc area as clean as possible. Use /etc/pkgname or a suitable alternative, e.g., Apache uses /etc/httpd/.
  • Package files should follow these general directory guidelines:
    • /etc/ - System essential configuration files
    • /usr/bin - Application binaries
    • /usr/sbin - System binaries
    • /usr/lib - Libraries
    • /usr/include - Header files
    • /usr/lib/pkgname - Modules, plugins, etc.
    • /usr/share/man - Manpages
    • /usr/share/pkgname - Application data
    • /etc/pkgname - Configuration files
    • /opt - Large self-contained packages such as Java, etc.
  • Packages should not contain the following directories:
    • /dev
    • /home
    • /media
    • /mnt
    • /proc
    • /root
    • /selinux
    • /sys
    • /tmp
    • /var/tmp

Testing the Package

As you are writing the build() function, you will want to test your changes frequently to ensure there are no bugs. You can do this using the makepkg command in the directory containing the PKGBUILD file. With a properly formatted PKGBUILD, this will create a package, but with a broken or unfinished one it will throw an error.

If makepkg finishes successfully, it will place a file named pkgname-pkgver.pkg.tar.gz in your working directory. This is package that can be installed with the pacman -U command. However, just because a package file was built does not imply that it is fully functional. It might conceivably contain only the directory and no files whatsoever if, for example, a prefix was specified improperly. You can use pacman's query functions to display a list of files contained in the package and the dependencies it requires with pacman -Qlp [package file] and pacman -Qip [package file] respectively.

If the package looks sane, then you are done! However, if you plan on release the PKGBUILD file, it is imperative that you check and double check the contents of the depends array.

ldd and namcap

Dependencies are the most common packaging error. There are two excellent tools you can use to check dependencies. The first one is ldd, which will show you the shared library dependencies of dynamic executables:

$ ldd gcc
        linux-gate.so.1 =>  (0xb7f33000)
        libc.so.6 => /lib/libc.so.6 (0xb7de0000)
        /lib/ld-linux.so.2 (0xb7f34000)

The other tool is namcap which not only checks for dependencies but the overall sanity of your package. It can be used on both the PKGBUILD file and the pkg.tar.gz package. It will report any bad permissions, missing dependencies, unnecessary dependencies and other common mistakes. You can install namcap via pacman:

$ pacman -S namcap

To use namcap simply supply a PKGBUILD file or a pkg.tar.gz package as the argument:

$ namcap [<em>PKGBUILD</em> or <em>pkgname.tar.gz</em>]

Although namcap can help detect dependencies, it is not always correct. Verify dependencies by looking at the documentation of the software.

Submitting Packages to the AUR

Follow these steps to submit a package into the AUR:

  1. Register a new account if you do not already have one.
  2. Check all of the official repositories ([core], [extra], [unstable], and [community]) and see if the package already exists. If it is inside any of those repositories, DO NOT submit the package. If the package is broken, file a bug report.
  3. Check the [unsupported] repository for the package. If it is currently maintained, changes can be submitted in a comment for the maintaner's attention. If it is unmaintained, the package can be adopted and updated.
  4. To upload the package to the AUR, the directory containing the PKGBUILD file and any other required files must be compressed as a tarball. The archive name should contain the name of the package, e.g. foo.tar.gz.. You can easily build a tarball by using makepkg --source in the directory. This makes a tarball named pkgname-pkver-pkrel.src.tar.gz, which you can then upload to the AUR. The tarball cannot contain the binary tarball created by makepkg or the source tarball of the software. Packages that contain binaries or that are very poorly written will be deleted without warning.
  5. Click the Submit link by the menu in the AUR. Choose the appropriate category for your package and upload.

Summary

  1. Download the source tarball of the software you want to package.
  2. Try compiling the package and installing it into an arbitrary directory.
  3. Copy over the prototype /usr/share/pacman/PKGBUILD.proto and rename it to PKGBUILD in a temporary working directory - preferably /var/abs/local.
  4. Edit the PKGBUILD according to the needs of your package.
  5. Run makepkg and see whether the resulting package is built correctly.
  6. If not, repeat the last two steps.

External links