Difference between revisions of "Pacman/Tips and tricks"

From ArchWiki
Jump to navigation Jump to search
m (Generating the package recovery list: consistent indent level)
(Restore database only: unset var)
Line 294: Line 294:
   fakeroot pacman "$@" \
   fakeroot pacman "$@" \
     --cache  "${cache}" \
     --cache  "${cache}" \
     --dbpath "${dbpath}" \
     --dbpath "${datapath}" \
     --log    /dev/null \
     --log    /dev/null \
     --root  "${root}" \
     --root  "${root}" \

Revision as of 19:40, 11 June 2010

Template:Article summary start Template:Article summary text Template:Article summary heading Template:I18n entry Template:Article summary heading Template:Article summary wiki Template:Article summary wiki Template:Article summary wiki Template:Article summary end


Basic pacman modifications and improvements

Color output

The most effective method of colorizing pacman is installing Template:Package AUR from the AUR.

Besides that, there are numerous scripts and hacks devised by members of the Arch Linux community:

Simple Bash script

Colorized pacman search output:

alias pacs="pacsearch"
pacsearch () {
       echo -e "$(pacman -Ss $@ | sed \
       -e 's#core/.*#\\033[1;31m&\\033[0;37m#g' \
       -e 's#extra/.*#\\033[0;32m&\\033[0;37m#g' \
       -e 's#community/.*#\\033[1;35m&\\033[0;37m#g' \
       -e 's#^.*/.* [0-9].*#\\033[0;36m&\\033[0;37m#g' )"

For a system-wide script, do:

# touch /usr/bin/pacs && chmod 755 /usr/bin/pacs

Then paste this into Template:Filename:

echo -e "$(pacman -Ss $@ | sed \
     -e 's#core/.*#\\033[1;31m&\\033[0;37m#g' \
     -e 's#extra/.*#\\033[0;32m&\\033[0;37m#g' \
     -e 's#community/.*#\\033[1;35m&\\033[0;37m#g' \
     -e 's#^.*/.* [0-9].*#\\033[0;36m&\\033[0;37m#g' )"

To use it simply type Template:Codeline instead of Template:Codeline.

Python script

This one also searches the AUR.

Using acoc

There is another, more general possibility of colorizing arbitrary command output. Download the small Ruby tool acoc, and its requirements, term-ansicolor and tpty. Some applications like Template:Codeline will not run without tpty because they need to be started from a terminal (or pseudo terminal, in this case), or else they behave differently.

Installation is relatively straightforward:

$ tar xvzf tpty-0.0.1.tar.gz
$ cd tpty-0.0.1
$ ruby extconf.rb
$ make
$ ruby ./test.rb
# make install
$ tar xvzf term-ansicolor-1.0.1.tar.gz
$ cd term-ansicolor-1.0.1
# ruby install.rb

And now acoc itself:

$ tar xvzf acoc-0.7.1.tar.gz
$ cd acoc-0.7.1
# make install

Now, just read the section Advanced Installation in acoc's Template:Filename file, and configure acoc as prefered. Create a link for pacman as well, since that is primarily what this is being done for.

Once acoc runs, optionally add these lines to Template:Filename:

[pacman -Si]
/^Name\s+:\s([\w.-]+)/                                  bold
[pacman -Qi]
/^Name\s+:\s([\w.-]+)/                                  bold
[pacman -Qi$]
/^([\w.-]+)\s([\w.-]+)/                                 bold,clear
[pacman -Ss]
/^([\w.-]+)\/([\w.-]+)\s+([\w.-]+)/                     clear,bold,clear
[pacman -Qs]
/^([\w.-]+)\/([\w.-]+)\s+([\w.-]+)/                     clear,bold,clear
[pacman -Sl]
/^([\w.-]+)\s([\w.-]+)\s([\w.-]+)/                      clear,bold,clear
[pacman -Qo]
/^([\w.-\/]+)\sis\sowned\sby\s([\w.-]+)\s([\w.-]+)/     clear,bold,clear
[pacman -Qe$]
/^([\w.-]+)\s([\w.-]+)/                                 bold,clear
[pacman -Qg$]
/^([\w.-]+)\s([\w.-]+)/                                 clear,bold

The above lines make pacman print all package names in bold, which is particularly helpful when doing, e.g. Template:Codeline. If desiring a more colorful output, modify the lines to suit. Read the acoc documentation contained in the source package for more information.


The following instructions allow users to run some of the more common pacman commands without the need to type them fully.

Configure the shell

Add the following examples, which work in both Bash and Zsh:

# Pacman alias examples
alias pacupg='sudo pacman -Syu'        # Synchronize with repositories before upgrading packages that are out of date on the local system.
alias pacin='sudo pacman -S'           # Install specific package(s) from the repositories
alias pacins='sudo pacman -Up'         # Install specific package not from the repositories but from a file 
alias pacre='sudo pacman -R'           # Remove the specified package(s), retaining its configuration(s) and required dependencies
alias pacrem='sudo pacman -Rns'        # Remove the specified package(s), its configuration(s) and unneeded dependencies
alias pacrep='pacman -Si'              # Display information about a given package in the repositories
alias pacreps='pacman -Ss'             # Search for package(s) in the repositories
alias pacloc='pacman -Qi'              # Display information about a given package in the local database
alias paclocs='pacman -Qs'             # Search for package(s) in the local database
# Additional pacman alias examples
alias pacupd='sudo pacman -Sy && sudo abs'     # Update and refresh the local package and ABS databases against repositories
alias pacinsd='sudo pacman -S --asdeps'        # Install given package(s) as dependencies of another package
alias pacmir='sudo pacman -Syy'                # Force refresh of all package lists after updating /etc/pacman.d/mirrorlist


Perform the respective commands by simply typing the alias name. For example, to synchronize with repositories before upgrading packages that are out of date on the local system:

$ pacupg

Install packages from repositories:

$ pacin <package1> <package2> <package3>

Install a custom built package:

$ pacins /path/to/<package>

Completely remove a locally installed package:

$ pacrem <package>

Search for available packages in the repositories:

$ pacreps <keywords>

Display information about a package (e.g. size, dependencies) in the repositories:

$ pacrep <keywords>


The aliases used above are merely examples. By following the syntax samples above, rename the aliases as convenient. For example:

alias pacrem='sudo pacman -Rns'
alias pacout='sudo pacman -Rns'

In the case above, the commands Template:Codeline and Template:Codeline both call Bash to execute the same command.

Operations and Bash syntax

In addition to pacman's standard set of features, there are ways to extend it's usability through rudimentary Bash commands/syntax.

  • To install a number of packages sharing similar patterns in their names -- not the entire group nor all matching packages; eg. Template:Package Official:
 pacman -S kdemod-{applets,theme,tools}
  • Of course, that is not limited and can be expanded to however many levels needed:
 pacman -S kdemod-{ui-{kde,kdemod},kdeartwork}
  • pacman has the Template:Codeline operand to hide the version column, so it is possible to query and reinstall packages with "compiz" as part of their name:
 pacman -S `pacman -Qq | grep compiz`
 pacman -S `pacman -Q | awk '/compiz/ { print $1 }'`

Installation and recovery

Alternative ways of getting and restoring packages

Installing packages from a CD/DVD/iso

  • First mount the CD (replace cdrom with dvd if needed):
# mount /mnt/cdrom
If working with an .iso file instead, first create a directory under /mnt:
# mkdir /mnt/iso
Then mount the image:
# mount -t iso9660 -o ro,loop /path/to/iso /mnt/iso
  • Configure pacman:
# nano -w /etc/pacman.conf
  • Add the following before other repositories (e.g. extra, core, etc.). This ensures the files from the CD/DVD/iso take precedence over those in the standard repositories:
# Settings for using a cd-rom as a repository.
Server = file:///mnt/cdrom/arch/pkg
Again, replace cdrom as appropiate.

Once Template:Filename has been edited, sync pacman in order to be able to use the new repository.

Custom local repository

pacman 3 introduced a new script named Template:Codeline which makes generating a database for a personal repository much easier. Use Template:Codeline for more details on its usage.

Simply store all of the built packages to be included in the repository in one directory, and execute the following command (where repo is the name of the custom repository):

repo-add /path/to/repo.db.tar.gz /path/to/*.pkg.tar.gz

The last argument will add all Template:Filename files to the repository, so be careful. If having multiple versions of a package in the directory, it is unclear which one will take precedence and end up in the repository.

Note that when using Template:Codeline, the database and the packages do not need to be in the same directory. But when using pacman with that database, they should be together.

To add a new package (and remove the old if it exists), simply run

repo-add /path/to/repo.db.tar.gz /path/to/packagetoadd-1.0-1-i686.pkg.tar.gz
Note: If there is a package that needs to be removed from the repository, read up on Template:Codeline.

Once the local repository has been made, add the repository to Template:Filename. The name of the Template:Filename file is the repository name. Reference it directly using a file:// url, or access it via FTP using ftp://localhost/path/to/directory.

If willing, add the custom repository to the list of unofficial user repositories, so that the community can benefit from it.

Network shared pacman cache

Note: As of version 3.1, Template:Codeline changed behavior. By default it does no longer remove all outdated package files but all package files which are not installed on the machine the command was issued on. Because pacman cannot predict what packages are installed on all machines that share the cache, pacman will end up deleting files that shouldn't be. To clean up the cache so that only the newest packages are kept, put Template:Codeline in the Template:Codeline section of Template:Filename.

In order to share packages between multiple computers, simply share Template:Filename using any network-based mount protocol. This section shows how to use shfs or sshfs to share a package cache plus the related library-directories between multiple computers on the same local network. Keep in mind that a network shared cache can be slow depending on the file-system choice, among other factors.

First, install any network-supporting filesystem; for example sshfs, shfs, ftpfs, smbfs or nfs

Tip: To use sshfs or shfs, consider reading Using SSH Keys.

Then, to share the actual packages, mount Template:Filename from the server to Template:Filename on every client machine.

To have shared package databases, mount Template:Filename in the same way. Proceed to place the appropriate lines in Template:Filename.

Backing up and retrieving a list of installed packages

It is good practice to keep periodic backups of all pacman-installed packages. In the event of a system crash which is unrecoverable by other means, pacman can then easily reinstall the very same packages onto a new installation.

  • First, backup the current list of packages (which are available in a repository):
 $ comm -13 <(pacman -Qmq | sort) <(pacman -Qqe | sort) > pkglist
  • Store the pkglist on a USB key or other convenient medium.
  • Copy the pkglist file to the new installation, and navigate to the directory containing it.
  • Issue the following command to install from the backup list:
 # pacman -S $(cat pkglist)

Redownloading all installed packages (minus AUR)

Listing currently installed packages will output everything, including packages that are foreign and/or might not be available for installation any longer. So simply running:

# pacman -S `pacman -Qq`

will output errors because some (or many) of the packages were not found in the repositories. There needs to be a way to list only packages that can be installed from the repositories present in Template:Codeline's database. In order to do so, combining a command to list all packages, and another to hide the list of foreign packages is required.

This can be done by using Template:Codeline to filter out foreign packages:

# pacman -S $(pacman -Qq | grep -Fvx "$(pacman -Qmq)")

A less contrieved way of achieving the above is achieved by using Template:Codeline:

# pacman -S $(comm -3 <(pacman -Qq) <(pacman -Qqm))
Tip: When powerpill installed, replace pacman with powerpill in the commands above to increase download speed and alleviate the bandwidth load per server.

Restore pacman's local database

Signs that pacman needs a local database restoration:

Most likely, pacman's database of installed software, Template:Filename, has been corrupted or deleted. While this is a serious problem, it can be restored by following the instructions below.

Firstly, make sure pacman's log file is present:

$ ls /var/log/pacman.log

If it does not exist, it is not possible to continue with this method. You may be able to use Xyne's package detection script to recreate the database. If not, then the likely solution is to re-install the entire system.

Log filter script

  • Create a script with the following content based on [1]:


  • Make the script executable:
$ chmod +x log2pkglist.awk

Generating the package recovery list

  • Run the script and pipe the output to a temporary list:
$ ./log2pkglist.sh /var/log/pacman.log > pkglist.orig
Template:Filename now contains a list of all the software installed or upgraded.
  • Optionally edit Template:Filename and remove anything that shouldn't be re-installed. This might be the situation with custom packages made with ABS, for example.
Here is a way to automatically restrict the list to packages available in a repository:
$ { cat pkglist.orig; pacman -Slq; } | sort | uniq -d > pkglist
Check if some important base package are missing, and add them to the list:
$ pacman -Sgq base | grep -Fvx -f pkglist.orig > pkglist

Proceed once the contents of Template:Filename are satisfactory, since they will be used it restore pacman's installed package database; Template:Filename.

Performing the recovery

There are two ways of approaching the situation: one is to only recover the database and avoid file conflicts by placing the newly installed files in a disposable fake root, and another is to perform a full reinstall and later deal with conflicts and backed-up configuration files.

At first glance, the second method may seem more straight forward, but using the partial recovery method makes it easier to deal with files conflicts and backed-up configuration files.

Restore database only
  • As a normal user, make temporary directories for the cache, database, and root:
tmp=$(readlink -f ./tmp)
mkdir "${tmp}" && cd "${tmp}"

cache=$(readlink -f    ./cache)
database=$(readlink -f ./database)
root=$(readlink -f     ./root)
mkdir -p "${database}" "${cache}" "${root}"

recovery-pacman() {
  fakeroot pacman "$@" \
    --cache  "${cache}" \
    --dbpath "${datapath}" \
    --log    /dev/null \
    --root   "${root}" \
  • Populate the temporary sync database:
$ recovery-pacman -Sy
Or copy the system's sync database:
$ cp -r /var/lib/pacman/sync "${database}"
  • To avoid downloading and/or processing packages that are present in the system's local database (or whatever remains of it), optionally copy it into the temporary location:
$ cp -r /var/lib/pacman/local "${database}"
  • Generate the temporary local recovery database from the previously generated Template:Filename:
$ xargs -d$'\n' -- recovery-pacman -S --nodeps --needed < pkglist
  • After revising the database, conclude by copying it to the real destination:
# cp -r "${database}"/local /var/lib/pacman
  • Before deleting the temporary root, check if some upgraded packages have missing files:
$ pacman -Qkq
Full reinstall
  • The reinstall needs to be "forced" because some of the files may already be on the system:
# xargs -d$'\n' -- pacman -S --force --nodeps --needed < pkglist
  • This operation may have appended previously installed configuration files with a Template:Filename suffix to make room for those contained in the newly installed packages.
Find the backed-up files:
$ find /etc -name \*.pacorig
After review, teplace the new files with those that have the Template:Filename suffix to restore the original configuration for each package that may have been affected:
# find /etc -name \*.pacorig | while read -r file; do mv -- "${file}" "${file%.pacorig}"; done
  • Finally, check for changed directory permission if a program refuses to start.

Removing everything but base group

If it is ever necessary to remove all packages except the base group on a severely broken system, try this one liner:

# pacman -R $(pacman -Qq | grep -Fvx "$(pacman -Qqg base)")

From the archlinux.org mailing list:[2]

Recovering a USB key from existing install

If you have Arch installed on a USB key and manage to mess it up (e.g. removing it while it's still being written to), then it's possible to re-install all the packages and hopefully get it back up and working again (assuming USB key is mounted to /newarch)

# pacman -r /newarch -b /newarch/var/lib/pacman/ -S `pacman -Qq -b /newarch/var/lib/pacman/`


House keeping, in the interest of keeping a clean system and following The Arch Way

Miscellaneous procedures

For recursively removing orphans (be careful):

# pacman -Rs $(pacman -Qtdq)

To get a sorted list of local packages and their size:

$ LANG=C pacman -Qi | sed -n '/^Name[^:]*: \(.*\)/{s//\1 /;x};/^Installed[^:]*: \(.*\)/{s//\1/;H;x;s/\n//;p}' | sort -nk2

Getting a list of files not owned by any package

Periodic checks for files outside of pacman database are recommended. These files are often some 3rd party applications installed using the usual procedure (e.g. ./configure; make; make install). Search the file-system for these files (or symlinks) using this simple script:

time /usr/bin/pacman -Ql | cut -d ' ' -f 2- | sort -u > db.txt
time find /arch /bin /boot /etc /lib /opt /sbin /usr 2>/dev/null | while read i; do
    if [ ! -d "$i" ]; then
        echo "$i"
done | sort -u > fs.txt
time comm -23 fs.txt db.txt > non-db.txt
#test it:
#cat non-db.txt | while read i; do pacman -Qo "$i" 2>&1; done | tee test.txt

Note that one should not delete all files listed in non-db.txt without confirming each entry. There could be various configuration files, logs, etc., so use this list responsibly and only proceed after extensively searching for cross-references using Template:Codeline.

Manage Explicitly installed packages that should be Dependancies

Once a system has a full package list, there often packages that were Explicitly installed (at initial install) but should have been installed as a dependancy. Packages like common libraries are often installed explicitly but is required by many other packages after you add X11, a desktop enviroment, etc.

Here's a little script that will have pacman prompt you for install before re-installing the package as a dependancy:



pacman -Qt > $T;
pacman -Qe > $E;

if [ $# -ne 1 ]; then {
    CMD="pacman -S --asdeps ";
} else {
    CMD="echo ";
}; fi; 

for pkg in `diff -y $T $E | grep ">" | gawk '{ print $2 }'`; do {
    $CMD $pkg;
}; done;

The if statement will see if you passed any arguements at all, if you did, then it simply echo back the list packages it will prompt for.

The only reason to do this is that if you want to clone your install by copying your explicit packages, then you want only those that also are in the unrequired list. That being said, certain packages should never be installed as a dependancy for another package, such as kernel26.

Selective cache purge

Here is a Python script to clean the Template:Filename directory while allowing to specify how many package versions should be retained. [3]