Pacman/Tips and tricks

From ArchWiki
< Pacman
Revision as of 20:49, 12 June 2010 by Self (talk | contribs) (Redownloading all installed packages (minus AUR): only install explicit to keep install reason intact)
Jump to navigation Jump to search

Template:Article summary start Template:Article summary text Template:Article summary heading Template:I18n entry Template:Article summary heading Template:Article summary wiki Template:Article summary wiki Template:Article summary wiki Template:Article summary end

General

Basic pacman modifications and improvements

Color output

The most effective method of colorizing pacman is installing Template:Package AUR from the AUR.

Besides that, there are numerous scripts and hacks devised by members of the Arch Linux community:

Simple Bash script

Colorized pacman search output:

alias pacs="pacsearch"
pacsearch () {
       echo -e "$(pacman -Ss "$@" | sed \
       -e 's#core/.*#\\033[1;31m&\\033[0;37m#g' \
       -e 's#extra/.*#\\033[0;32m&\\033[0;37m#g' \
       -e 's#community/.*#\\033[1;35m&\\033[0;37m#g' \
       -e 's#^.*/.* [0-9].*#\\033[0;36m&\\033[0;37m#g' )"
}

For a system-wide script, do:

# touch /usr/bin/pacs && chmod 755 /usr/bin/pacs

Then paste this into Template:Filename:

#!/bin/bash
echo -e "$(pacman -Ss "$@" | sed \
     -e 's#core/.*#\\033[1;31m&\\033[0;37m#g' \
     -e 's#extra/.*#\\033[0;32m&\\033[0;37m#g' \
     -e 's#community/.*#\\033[1;35m&\\033[0;37m#g' \
     -e 's#^.*/.* [0-9].*#\\033[0;36m&\\033[0;37m#g' )"

To use it simply type Template:Codeline instead of Template:Codeline.

Python script

This one also searches the AUR.

Using acoc

There is another, more general possibility of colorizing arbitrary command output. Download the small Ruby tool acoc, and its requirements, term-ansicolor and tpty. Some applications like Template:Codeline will not run without tpty because they need to be started from a terminal (or pseudo terminal, in this case), or else they behave differently.

Installation is relatively straightforward:

$ tar xf tpty-0.0.1.tar.gz
$ cd tpty-0.0.1
$ ruby extconf.rb
$ make
$ ruby ./test.rb
# make install
$ tar xf term-ansicolor-1.0.1.tar.gz
$ cd term-ansicolor-1.0.1
# ruby install.rb

And now acoc itself:

$ tar xf acoc-0.7.1.tar.gz
$ cd acoc-0.7.1
# make install

Now, just read the section Advanced Installation in acoc's Template:Filename file, and configure acoc as prefered. Create a link for pacman as well, since that is primarily what this is being done for.

Once acoc runs, optionally add these lines to Template:Filename:

[pacman -Si]
/^Name\s+:\s([\w.-]+)/                                  bold
[pacman -Qi]
/^Name\s+:\s([\w.-]+)/                                  bold
[pacman -Qi$]
/^([\w.-]+)\s([\w.-]+)/                                 bold,clear
[pacman -Ss]
/^([\w.-]+)\/([\w.-]+)\s+([\w.-]+)/                     clear,bold,clear
[pacman -Qs]
/^([\w.-]+)\/([\w.-]+)\s+([\w.-]+)/                     clear,bold,clear
[pacman -Sl]
/^([\w.-]+)\s([\w.-]+)\s([\w.-]+)/                      clear,bold,clear
[pacman -Qo]
/^([\w.-\/]+)\sis\sowned\sby\s([\w.-]+)\s([\w.-]+)/     clear,bold,clear
[pacman -Qe$]
/^([\w.-]+)\s([\w.-]+)/                                 bold,clear
[pacman -Qg$]
/^([\w.-]+)\s([\w.-]+)/                                 clear,bold

The above lines make pacman print all package names in bold, which is particularly helpful when doing, e.g. Template:Codeline. If desiring a more colorful output, modify the lines to suit. Read the acoc documentation contained in the source package for more information.

Aliases

The following instructions allow users to run some of the more common pacman commands without the need to type them fully.

Configure the shell

Add the following examples, which work in both Bash and Zsh:

# Pacman alias examples
alias pacupg='sudo pacman -Syu'        # Synchronize with repositories before upgrading packages that are out of date on the local system.
alias pacin='sudo pacman -S'           # Install specific package(s) from the repositories
alias pacins='sudo pacman -Up'         # Install specific package not from the repositories but from a file 
alias pacre='sudo pacman -R'           # Remove the specified package(s), retaining its configuration(s) and required dependencies
alias pacrem='sudo pacman -Rns'        # Remove the specified package(s), its configuration(s) and unneeded dependencies
alias pacrep='pacman -Si'              # Display information about a given package in the repositories
alias pacreps='pacman -Ss'             # Search for package(s) in the repositories
alias pacloc='pacman -Qi'              # Display information about a given package in the local database
alias paclocs='pacman -Qs'             # Search for package(s) in the local database
# Additional pacman alias examples
alias pacupd='sudo pacman -Sy && sudo abs'     # Update and refresh the local package and ABS databases against repositories
alias pacinsd='sudo pacman -S --asdeps'        # Install given package(s) as dependencies of another package
alias pacmir='sudo pacman -Syy'                # Force refresh of all package lists after updating /etc/pacman.d/mirrorlist

Usage

Perform the respective commands by simply typing the alias name. For example, to synchronize with repositories before upgrading packages that are out of date on the local system:

$ pacupg

Install packages from repositories:

$ pacin <package1> <package2> <package3>

Install a custom built package:

$ pacins /path/to/<package>

Completely remove a locally installed package:

$ pacrem <package>

Search for available packages in the repositories:

$ pacreps <keywords>

Display information about a package (e.g. size, dependencies) in the repositories:

$ pacrep <keywords>

Notes

The aliases used above are merely examples. By following the syntax samples above, rename the aliases as convenient. For example:

alias pacrem='sudo pacman -Rns'
alias pacout='sudo pacman -Rns'

In the case above, the commands Template:Codeline and Template:Codeline both call Bash to execute the same command.

Operations and Bash syntax

In addition to pacman's standard set of features, there are ways to extend it's usability through rudimentary Bash commands/syntax.

  • To install a number of packages sharing similar patterns in their names -- not the entire group nor all matching packages; eg. Template:Package Official:
 pacman -S kdemod-{applets,theme,tools}
  • Of course, that is not limited and can be expanded to however many levels needed:
 pacman -S kdemod-{ui-{kde,kdemod},kdeartwork}
  • Sometimes, Template:Codeline's builtin BRE can cause a lot of unwanted results, so it has to be limited to match the package name only; not the description nor any other field:
 pacmac -Ss '^vim-.*$'
  • pacman has the Template:Codeline operand to hide the version column, so it is possible to query and reinstall packages with "compiz" as part of their name:
 pacman -S $(pacman -Qq | grep compiz)

Installation and recovery

Alternative ways of getting and restoring packages

Installing packages from a CD/DVD/iso

  • First mount the CD (replace cdrom with dvd if needed):
# mount /mnt/cdrom
If working with an .iso file instead, first create a directory under /mnt:
# mkdir /mnt/iso
Then mount the image:
# mount -t iso9660 -o ro,loop /path/to/iso /mnt/iso
  • Configure pacman:
# nano -w /etc/pacman.conf
  • Add the following before other repositories (e.g. extra, core, etc.). This ensures the files from the CD/DVD/iso take precedence over those in the standard repositories:
# Settings for using a cd-rom as a repository.
[custom]
Server = file:///mnt/cdrom/arch/pkg
Again, replace cdrom as appropiate.

Once Template:Filename has been edited, sync pacman in order to be able to use the new repository.

Custom local repository

pacman 3 introduced a new script named Template:Codeline which makes generating a database for a personal repository much easier. Use Template:Codeline for more details on its usage.

Simply store all of the built packages to be included in the repository in one directory, and execute the following command (where repo is the name of the custom repository):

$ repo-add /path/to/repo.db.tar.gz /path/to/*.pkg.tar.gz

The last argument will add all Template:Filename files to the repository, so be careful. If having multiple versions of a package in the directory, it is unclear which one will take precedence and end up in the repository.

Note that when using Template:Codeline, the database and the packages do not need to be in the same directory. But when using pacman with that database, they should be together.

To add a new package (and remove the old if it exists), run:

$ repo-add /path/to/repo.db.tar.gz /path/to/packagetoadd-1.0-1-i686.pkg.tar.gz
Note: If there is a package that needs to be removed from the repository, read up on Template:Codeline.

Once the local repository has been made, add the repository to Template:Filename. The name of the Template:Filename file is the repository name. Reference it directly using a file:// url, or access it via FTP using ftp://localhost/path/to/directory.

If willing, add the custom repository to the list of unofficial user repositories, so that the community can benefit from it.

Network shared pacman cache

Note: As of version 3.1, Template:Codeline changed behavior. By default it does no longer remove all outdated package files but all package files which are not installed on the machine the command was issued on. Because pacman cannot predict what packages are installed on all machines that share the cache, pacman will end up deleting files that shouldn't be. To clean up the cache so that only the newest packages are kept, put Template:Codeline in the Template:Codeline section of Template:Filename.

In order to share packages between multiple computers, simply share Template:Filename using any network-based mount protocol. This section shows how to use shfs or sshfs to share a package cache plus the related library-directories between multiple computers on the same local network. Keep in mind that a network shared cache can be slow depending on the file-system choice, among other factors.

First, install any network-supporting filesystem; for example sshfs, shfs, ftpfs, smbfs or nfs

Tip: To use sshfs or shfs, consider reading Using SSH Keys.

Then, to share the actual packages, mount Template:Filename from the server to Template:Filename on every client machine.

To have shared package databases, mount Template:Filename in the same way. Proceed to place the appropriate lines in Template:Filename.

Backing up and retrieving a list of installed packages

It is good practice to keep periodic backups of all pacman-installed packages. In the event of a system crash which is unrecoverable by other means, pacman can then easily reinstall the very same packages onto a new installation.

  • First, backup the current list of non-local packages:
 $ comm -23 <(pacman -Qeq) <(pacman -Qmq) > pkglist
  • Store the pkglist on a USB key or other convenient medium.
  • Copy the pkglist file to the new installation, and navigate to the directory containing it.
  • Issue the following command to install from the backup list:
 # pacman -S $(< pkglist)

Redownloading all installed packages (minus AUR)

Listing currently installed packages will output everything, including packages that are foreign and/or might not be available for installation any longer. So simply running:

# pacman -S $(pacman -Qeq)

will output errors because some (or many) of the packages were not found in the repositories. There needs to be a way to list only packages that can be installed from the repositories present in Template:Codeline's database. In order to do so, combining a command to list all packages, and another to hide the list of foreign packages is required.

A way of achieving the above:

# pacman -S $(comm -23 <(pacman -Qeq) <(pacman -Qmq))
Tip: When powerpill installed, replace pacman with powerpill in the commands above to increase download speed and alleviate the bandwidth load per server.

Restore pacman's local database

Signs that pacman needs a local database restoration:

Most likely, pacman's database of installed software, Template:Filename, has been corrupted or deleted. While this is a serious problem, it can be restored by following the instructions below.

Firstly, make sure pacman's log file is present:

$ ls /var/log/pacman.log

If it does not exist, it is not possible to continue with this method. You may be able to use Xyne's package detection script to recreate the database. If not, then the likely solution is to re-install the entire system.

Log filter script

  • Create a script with the following content based on [1]:

Template:File

  • Make the script executable:
$ chmod +x log2pkglist.awk

Generating the package recovery list

  • Run the script and pipe the output to a temporary list:
$ ./log2pkglist.awk /var/log/pacman.log > pkglist.orig
Template:Filename now contains a list of all the software installed or upgraded.
  • Optionally edit Template:Filename and remove anything that shouldn't be re-installed. This might be the situation with custom packages made with ABS, for example.
Here is a way to automatically restrict the list to packages available in a repository:
$ { cat pkglist.orig; pacman -Slq; } | sort | uniq -d > pkglist
And/or Check if some important base package are missing, and add them to the list:
$ comm -23 <(pacman -Sgq base) pkglist.orig >> pkglist

Proceed once the contents of Template:Filename are satisfactory, since they will be used it restore pacman's installed package database; Template:Filename.

Performing the recovery

There are two ways of approaching the situation: one is to only recover the database and avoid file conflicts by placing the newly installed files in a disposable fake root, and another is to perform a full reinstall and later deal with conflicts and backed-up configuration files.

At first glance, the second method may seem more straight forward, but using the partial recovery method makes it easier to deal with files conflicts and backed-up configuration files.

Restore database only
  • As a normal user, make temporary directories for the cache, database, and root:
tmp=$(readlink -f ./tmp)
mkdir "${tmp}" && pushd "${tmp}"

cache=$(readlink -f    ./cache)
dbpath=$(readlink -f ./dbpath)
root=$(readlink -f     ./root)
mkdir -p "${dbpath}" "${cache}" "${root}"

recovery-pacman() {
  fakeroot pacman "$@" \
    --cache  "${cache}" \
    --dbpath "${dbpath}" \
    --log    /dev/null \
    --root   "${root}" \
    #
}

popd
  • Populate the temporary sync database:
$ recovery-pacman -Sy
Or copy the system's sync database:
$ cp -r /var/lib/pacman/sync "${dbpath}"
  • To avoid downloading and/or processing packages that are present in the system's local database (or whatever remains of it), optionally copy it into the temporary location:
$ cp -r /var/lib/pacman/local "${dbpath}"
  • Generate the temporary local recovery database from the previously generated Template:Filename:
$ xargs -d$'\n' -- recovery-pacman -S --nodeps --needed < pkglist
  • After revising the database, conclude by copying it to the real destination:
# cp -r "${dbpath}"/local /var/lib/pacman
  • Before deleting the temporary root, check if some upgraded packages have missing files:
$ pacman -Qkq
Full reinstall
  • The reinstall needs to be "forced" because some of the files may already be on the system:
# xargs -d$'\n' -- pacman -S --force --nodeps --needed < pkglist
  • This operation may have appended previously installed configuration files with a Template:Filename suffix to make room for those contained in the newly installed packages.
Find the backed-up files:
$ find /etc -name \*.pacorig
After review, replace the new files with those that have the Template:Filename suffix to restore the original configuration for each package that may have been affected:
# find /etc -name \*.pacorig | while read -r file; do mv -- "${file}" "${file%.pacorig}"; done
  • Finally, check for changed directory permission if a program refuses to start.

Removing everything but base group

If it is ever necessary to remove all packages except the base group on a severely broken system, try this one liner:

# pacman -Rs $(comm -23 <(pacman -Qeq) <(pacman -Qgq base))

From the archlinux.org mailing list:[2]

Recovering a USB key from existing install

If you have Arch installed on a USB key and manage to mess it up (e.g. removing it while it's still being written to), then it's possible to re-install all the packages and hopefully get it back up and working again (assuming USB key is mounted to /newarch)

# pacman -r /newarch -b /newarch/var/lib/pacman/ -S $(pacman -Qq -b /newarch/var/lib/pacman/)

Maintenance

House keeping, in the interest of keeping a clean system and following The Arch Way

Miscellaneous procedures

For recursively removing orphans (be careful):

# pacman -Rs $(pacman -Qtdq)

To get a sorted list of local packages and their size:

$ LC_ALL=C pacman -Qi | sed -n '/^Name[^:]*: \(.*\)/{s//\1 /;x};/^Installed[^:]*: \(.*\)/{s//\1/;H;x;s/\n//;p}' | sort -nk2

Getting a list of files not owned by any package

Periodic checks for files outside of pacman database are recommended. These files are often some 3rd party applications installed using the usual procedure (e.g. ./configure; make; make install). Search the file-system for these files (or symlinks) using this simple script:

#!/bin/bash
time /usr/bin/pacman -Qlq  > db.txt
time find /bin /boot /etc /lib /opt /sbin /usr ! -type d 2>/dev/null > fs.txt
time comm -23 fs.txt db.txt > non-db.txt
 
# Test it:
#while read i; do pacman -Qo "$i" 2>&1; done < non-db.txt | tee test.txt

Note that one should not delete all files listed in Template:Filename without confirming each entry. There could be various configuration files, logs, etc., so use this list responsibly and only proceed after extensively searching for cross-references using Template:Codeline.

Manage explicitly installed packages that should be dependencies

Sometimes, packages that were explicitly installed; e.g., at initial setup, should have been installed as dependencies. These packages are usually libraries that are required by "end" packages such as those related to X11 and desktop environments.

Here's a little script that will have pacman prompt before re-installing the package as a dependency:

#!/bin/sh
tmp=$(mktemp --tmpdir --directory pacman-dependency-fix.XXXXXX) || exit;
T=${tmp}/unrequired;
E=${tmp}/explicit;
 
pacman -Qt > $T;
pacman -Qe > $E;
 
if [ $# -ne 1 ]; then
    CMD="pacman -S --asdeps";
else
    CMD="echo";
fi
 
for pkg in $(diff -y $T $E | awk '/>/ { print $2 }'); do
    $CMD $pkg;
done

If any arguments are passed to the script, then it will simply echo back the list of packages that would've been acted upon.

A situation were this might be needed is when an Arch install is to be cloned by copying explicit packages, or when there's interest in keeping an accurate dependency database. pacman 3.4 will not require this script because its new Template:Codeline options can reasign "install reasons" without reinstalling.

That being said, certain packages should never be installed as a dependency for another package, such as kernel26 and glibc.

Selective cache purge

Here is a Python script to clean the Template:Filename directory while allowing to specify how many package versions should be retained. [3]