User:Gen2ly/System backup and reinstall

From ArchWiki
< User:Gen2ly
Revision as of 07:17, 15 March 2010 by Gen2ly (Talk | contribs) (Backup Script)

Jump to: navigation, search

This article will show you how to add configurations to a backup include file, backup those files, and restore configurations and previously installed programs when you reinstall your system.

This is an uncommon document for the reason that the need to restore from 'configurations only' is only necessary when:

  1. You would like to change your system architecture (e.g. 32bit to 64bit).
  2. If a program or programs begin to behave unexpectedly and no help in the forums (or elsewhere) is available is able to fix the problem. By chance reinstalling your programs and other configurations might fix the problem.
  3. You have limited hard disk space and are not capable of doing a full restore from backup.
Tip: If you are a regular computer user, generally it is good practice to backup your primary drive to a backup hard drive after getting your system installed. Secondary storage disks can be found relatively inexpensively and are able to store/restore an entire drive of programs, configurations, documents... in a safe and quick way. If you have a hardware setup like this, look at programs like the Clonezilla CD, or the Parted Magic CD (includes Clonzilla and other tools) which are both open-source and can image your hard drive to another drive.

Backup

The tar compression tool with a couple helper scripts can backup your configurations in a few steps.

Include/Exclude Files

Tar has the ability to read from an "include" file that has a list of files (configurations, documents, etc.) to be included when compressing. The format is stored line by line with its full base path. For example:

/etc/pacman.conf
/etc/rc.conf
/home/user
...

And is invoked like this:

tar --files-from=include.txt --exclude-from=exclude.txt -cvpzf backupname.tar.gz

The name of the include file can be anything you want, and an exclude file can be used like in the example above. The exclude file is like the include file but additionally has the ability to be able to use regexps, can be commented, and have blank lines:

# Trashes not necessary
/home/*/.local/share/Trash/files
/home/*/.Trash

Adding to (In|Ex)clude Files in One Step

To help add file names, folders... to the include and exclude files, a couple bash scripts can be used to accomplish this. bca (backup-config-add) and bce (backup-cfg-exclude) can be used from the command line like this:

bca /etc/X11/xorg.conf
cd /home/user
bce .thumbnails/

Both scripts detect the full path so writing a relative or partial path (like in the example above) are acceptable. Put them in your script directory and make them executable to add a file anytime you think of it.

#!/bin/bash
# bce (backup-config-exclude) - add files to be excluded in backup

excludeloc=/home/user/.scripts/backup/exclude.txt

# Add file/folder/link to list
if [[ -z "$@" ]]; then
  echo " bce <file/folder/list> - add exclusions to backing up configurations"
  else
  echo "`realpath -s "$@"`" >> $excludeloc
fi
#!/bin/bash
# add file/folders to the backup configurations include file

# Include file location
backlsdir="/home/user/.bin/root/backup"
incfile="$backlsdir/bckcfg-inc.txt"

# Program name from it's filename
prog=${0##*/}

# Text color variables
txtund=$(tput sgr 0 1)          # Underline
txtbld=$(tput bold)             # Bold
bldred=${txtbld}$(tput setaf 1) #  red
bldblu=${txtbld}$(tput setaf 4) #  blue
bldwht=${txtbld}$(tput setaf 7) #  white
txtrst=$(tput sgr0)             # Reset
info=${bldwht}*${txtrst}        # Feedback
pass=${bldblu}*${txtrst}
warn=${bldred}*${txtrst}

# Display usage if full argument isn't given
if [[ -z "$@" ]]; then
  echo " $prog <file/folder/link> -  add files/folders/links to backup-cfgs' include file"
  exit
fi

# Check if folder exists
if [ ! -d $backlsdir ]; then
  echo "Directory and include file do not exist: $incfile"
  echo "Exiting"
  exit
fi

# Check if file exists
if [ ! -f $incfile ]; then
  echo "Include file doesn't exist: $incfile"
  echo "Exiting"
  exit
fi

# Check link valid, then add
for file in "$@"; do
  fullpath=$(realpath -s "$file")
  if [ ! -e "$fullpath" ]; then
    echo "$warn File ${txtund}$fullpath${txtrst} doesnt exist."
    continue
  fi
  echo "$fullpath" >> "$incfile"
  echo "$pass Added ${txtund}$fullpath${txtrst} in backup include file"
done

# Sort entries
sort -u "$incfile" -o "$incfile"

Backup Script

A tar backup script can be built and then be automated to run on a regular basis.

This backup script names your backup by several identifying variables, specifies your include and exclude file locations, and removes old backups:

#!/bin/bash
# backup configurations with tar

# Backup destination
backdest="/opt/backup"

# Backup name
machine=${HOSTNAME}
distro=arch
type=configs
date=`date "+%F"`
backupfile="$backdest/$distro-$type-$date.tar.gz"

# Include and exclude file locations
prog=${0##*/} # Program name from filename
bcdir="/home/<user>/.bin/root/backup"
include_file="$bcdir"/$prog-inc.txt
exclude_file="$bcdir"/$prog-exc.txt

# Text color variables
txtbld=$(tput bold)             # Bold
bldred=${txtbld}$(tput setaf 1) #  red
bldwht=${txtbld}$(tput setaf 7) #  white
txtrst=$(tput sgr0)             # Reset
info=${bldwht}*${txtrst}        # Feedback
warn=${bldred}!${txtrst}

# Verify that the target directory exists.
if [ ! -d $backdest ]; then
  echo "$warn Backup directory does not exist, exiting."
fi

# Delete backups older than a month
if [[ -n "$(find "$backdest" -mtime +30)" ]]; then
  echo "$info Deleting backups older than one month..."
  find "$backdest" -name "$distro-$type-*" -mtime +30 -exec rm {} \;
fi

# Backup
tar --exclude-from=$exclude_file --files-from=$include_file -cvpzf \
$backupfile

What Not to Include in a Backup

There are some files that should not be included in the backup directly. The reason for this will become apparently later in the document. For now, if you want these files backed up copy them to a new name:

cp /boot/grub/menu.lst /boot/grub/menu.lst-good
cp /etc/fstab /etc/fstab-good
cp /etc/sudoers /etc/sudoers-good

Then add them to your include file.

Packages

A package list can be created that can restore your installed packages. If you have the hard disk space available, you might want to also consider saving the install packages (Template:Filename) as well.

Creating a Package List

You can create a list of official installed packages with:

pacman -Qqet | grep -v "$(pacman -Qqm)" > /opt/backup/pkg-official
bca /opt/backup/pkg-official

This will create a list of all packages in the enabled pacman repositories (i.e. core, extra, community and testing).

To create a list of all local packages (includes packages installed from the AUR):

pacman -Qqmt > /opt/backup/pkg-local
bca /opt/backup/pkg-local

Alternately, you can use the packup utility in the AUR. Packup is a backup and restore from list utility with a menu driven interface. Packup will create a list and save it in the root folder. If you use packup, consider creating a separate list for official and local packages for easier debugging later.

Saving Package Tarballs

Pacman saves all package tarballs in Template:Filename. Saving these will increase the re-install speed so consider adding this folder to your include file. You might want to think about reducing the size of the cache before backing up, though. Pacman has the ability to remove any un-installed packages from the cache with:

pacman -Sc

If you use Yaourt to install packages from the AUR, you might want to consider setting up a cache for it too (Yaourt by default does not save the built package tarballs). To setup a cache directory, edit Template:Filename to include one:

  ExportToLocalRepository /var/cache/yaourt/pkg

Then give the directory the necessary permissions so Yaourt can write to it as a regular user:

mkdir -p /var/cache/yaourt/pkg
chmod 766 /var/cache/yaourt/pkg/

And add it to your include file:

bca /var/cache/yaourt/pkg/

Storing the Backup

The examples above save the backups to the same partition (provided you do not have a separate /opt partition). If you have the hard disk space, use a separate partition or drive. The Parted Magic CD has Gparted on it to help do this. If you do not you will need to put them on a separate media before you restore. If you absolutely have to, you can break up tar files and span them over multiple disks.

Restoring

Restoring will require: reinstalling the base system, extracting the backup configurations back to the new system, checking for conflicts, and reinstalling the previously installed packages.

Extracting Configurations

Install Arch Linux as you normally would and then mount the partition with your backup in it:

mkdir /backup-part
mount /dev/<your-partition> /backup-part

Then extract your backup onto the new install:

tar xvf hostname-arch-configs-date-tar.gz -C /mnt

The installer will have already mounted your install partition on /mnt. This is where not including those files directly above comes in handy. Doing this will overwrite any files that match with your backup and the new install. Check your good mount file (fstab) and bootloader file (menu.lst) with the new install version and update where necessary. Reboot.

Adding a User

When creating a user, consider giving the user a unique user id (UID). This will help prevent conflicts in the future with other users and programs having the same UID (UIDs for users generally start at 1000):

useradd -m -u 1050 -G audio,optical,power,storage,users,video -s /bin/bash user

If you have restored a user home directory (/home/user) from your backup configurations, the -m switch will give a warning about an already existing home directory but will not alter the directory. Do not forget to change permissions in your home directory if your UIDs differ:

chown -R username:users /home/user

Installing Yaourt

If you previously had any AUR packages installed, you can install Yaourt to restore the packages:

wget http://aur.archlinux.org/packages/yaourt/yaourt.tar.gz
tar xvf yaourt.tar.gz && cd yaourt*
makepkg -s
pacman -U yaourt-*.pkg.tar.gz

Reinstalling packages

Reinstall packages from the official repositories, the AUR, and locally installed packages separately to better diagnose problems if they occur.

Official

First reinstall packages from the official repositories;

pacman -Sy
pacman -S --needed $(cat /opt/backup/pkg-official)

Pacman will not install packages with file conflicts (i.e. if you have configurations placed that pacman detects an installing package has as well). Take a note of the conflicting packages and either: move the conflicting configuration (if you know what it is), or force pacman to overwrite the conflicting files:

pacman -S -f --needed $(cat /opt/backup/pkg-official)

If you do this, you will need to find out the conflicting file and reinstall it again. Pacman can tell you what files a certain package installs:

pacman -Ql <package>

And you can extract the file or files from an archive with:

tar xvf hostname-arch-configs-date-tar.gz etc/hosts etc/resolv.conf

and place the file(s) manually where they belong.

The AUR

To reinstall packages from the AUR:

yaourt -S $(cat /opt/backup/pkg-local | grep -vx "$(pacman -Qqm)") 

The additional command is to remove from the list any packages already installed. For example, if there are any problems installing a package from the AUR you will not have to go through the whole list again and tell Yaourt what to install (i.e. Yaourt does not suppport the --needed switch). Also, if you created a Yaourt cache before, note that Yaourt does not have the ability to read from the cache before an install. If you do not want to build all your AUR packages again, these packages will have to be installed manually - see the next step.

Local

If you created a Yaourt cache or have your own packages saved, you can install them with:

pacman -U <*--asdeps> <package>

Resources