Difference between revisions of "User:Gen2ly/System backup and reinstall"

From ArchWiki
Jump to: navigation, search
(Backup Script)
(41 intermediate revisions by 6 users not shown)
Line 1: Line 1:
[[Category: System recovery (English)]]
+
[[Category:System recovery]]
[[Category: HOWTOs(English)]]
+
This article is intended to show you how to backup your configurations and your package list, then to do a full system restore, restore packages, and finally restore your configurations..
This is an uncommon document for the reason that the need to restore from 'configurations only' is only necessary when:
+
# When a program or programs begin to behave unexpectedly and no help in the forums (or elsewhere) is available is able to fix the problem.  By chance reinstalling your programs and other configurations might fix the problem.
+
# You have limited hard disk space and are not capable of doing a full restore from backup.
+
  
{{Tip|If you are a regular computer user, generally it is good practice to backup your primary drive to a backup hard drive after getting your system installed. Secondary storage disks can be found relatively inexpensively and are able to store/restore an entire drive of programs, configurations, documents... in a safe and quick way.  If you have a hardware setup like this, look at programs like the [http://clonezilla.org/ Clonezilla CD], or the [http://partedmagic.com/ Parted Magic CD] (includes Clonzilla and other tools) which are both open-source and can image your hard drive to another drive.}}
+
== Motivation ==
 +
 
 +
The need for this documentation is uncommon in the sense that the need to restore from ''configurations only'' is only really necessary for the following reasons:
 +
 
 +
# You would like to change your system architecture (e.g. 32bit to 64bit).
 +
# If a program or programs begin to behave unexpectedly and no help in the forums (or elsewhere) is available is able to fix the problem. By chance reinstalling your programs and other configurations might fix the problem.
 +
# You have limited hard disk space and are not capable of doing a full restore from backup.
  
This document is designed to show you how to: add commonly-used/necessary configurations in an include file in a single step, back them up on a regular basis, and how to restore all your configurations and packages when rebuilding your system.
+
{{Tip|If you are a regular computer user, generally it is good practice to backup your primary drive to a backup hard drive after getting your system installed.  Secondary storage disks can be found relatively inexpensively these days and are able to store/restore an entire drive of programs, configurations, documents, etc. in a safe and reliable way.  If you have a hardware setup like this, look at programs like the [http://clonezilla.org/ Clonezilla CD], or the [http://partedmagic.com/ Parted Magic CD] (includes Clonzilla and other tools) which are both open-source and can image your hard drive for later restoration.}}
  
==Backup==
+
== Backup ==
  
The tar compression tool with a couple helper scripts can backup your configurations in a few steps.
+
Using {{Ic|tar}} in a script can make archiving configurations get done in just a couple steps.
  
===Include/Exclude Files ===
+
=== Include and Exclude Files ===
  
Tar has the ability to read from an "include" file that has a list of files (configurations, documents, etc.) to be included when compressing. The format is stored line by line with its full base path. For example:
+
Tar has the ability to read from both an include and an exclude file.  This means that you can tell tar everything you would like to include in the backup and exclude just by using two files. The format used is one line per file or directory that indicates the full path. For example:
  
 
<pre>
 
<pre>
Line 26: Line 29:
 
And is invoked like this:
 
And is invoked like this:
  
  tar --files-from=include.txt --exclude-from=exclude.txt -cvpzf backupname.tar.gz
+
  tar --files-from=include.txt --exclude-from=exclude.txt -c --xz -f backupname.tar.xz
  
The name of the include file can be anything you want, and an exclude file can be used like in the example above.  The exclude file is like the include file but additionally has the ability to be able to use [http://www.regular-expressions.info/ regexps], can be commented, and have blank lines:
+
The name of the files can be anything you want.  The exclude file is like the include file but additionally has the ability to be able to use [http://www.regular-expressions.info/ regexps], as well as being able to be commented and have blank lines.
  
<pre>
+
=== Backup Script ===
# Trashes not necessary
+
/home/*/.local/share/Trash/files
+
/home/*/.Trash
+
</pre>
+
  
===Adding to (In|Ex)clude Files in One Step===
+
This backup script has the ability to add to the include and exclude files and the ability to backup the configurations.  The usage is:
  
To help add file names, folders... to the include and exclude files, a couple bash scripts can be used to accomplish this.  '''bca''' (backup-config-add) and '''bce''' (backup-cfg-exclude) can be used from the command line like this:
+
<pre> bckcfg <i|e|n|c> - backup configurations
 +
  i - add to the include list a file or folder
 +
  e - add to the exclude list a file or folder or regexp pattern
 +
  n - add note
 +
  c - create backup</pre>
  
<pre>
+
For example to add to the include file:
bca /etc/X11/xorg.conf
+
cd /home/user
+
bce .thumbnails/
+
</pre>
+
  
Both scripts detect the full path so writing a relative or partial path (like in the example above) are acceptable. Put them in your script directory and make them executable to add a file anytime you think of it.
+
  bckcfg i /etc/rc.conf
  
<pre>
+
And to add to the exclude file:
#!/bin/bash
+
# bce (backup-config-exclude) - add files to be excluded in backup
+
  
excludeloc=/home/user/.scripts/backup/exclude.txt
+
bckcfg e .thumbnails/
  
# Add file/folder/link to list
+
The script is able to detect the full path so writing a relative or partial path is acceptable.
if [[ -z "$@" ]]; then
+
  echo " bce <file/folder/list> - add exclusions to backing up configurations"
+
  else
+
  echo "`readlink -f "$@"`" >> $excludeloc
+
fi
+
</pre>
+
  
<pre>
+
To backup, this script names your backup by several definable variables, removes old backups if desired, then backs up the configurations.  The include and exclude files are placed in the same directory as the backup script.
#!/bin/bash
+
# bca (backup-cfg-add) - add file/folders to the backup-cfg include file
+
  
incfile=/home/user/.scripts/backup/include.txt
+
{{hc|bckcfg|<pre>#!/bin/bash
fullpath=$(readlink -f "$@")
+
# Backup configurations
  
# Display usage if full argument is not given
+
# Backup destination directory
if [[ -z "$@" ]]; then
+
#bckp_dest="/home/todd/Desktop"
  echo " bca <file/folder/link> -  add files/folders/links to backup-cfgs' include file"
+
bckp_dest="/run/media/todd/Backup/backup-configs"
  exit
+
fi
+
 
+
# Test if path exists
+
if [ ! -e "$fullpath" ]; then
+
  echo -e " File "$@" does not exist."
+
  exit
+
fi
+
 
+
# Add entry
+
echo "$fullpath" >> "$incfile"
+
echo -e "${bldblu}*${txtrst} Added ${undwht}$fullpath${txtrst} in backup include file"
+
 
+
# Sort entries
+
sort -u "$incfile" -o "$incfile"
+
</pre>
+
 
+
===Backup Script===
+
A tar backup script can be built and then be automated to run on a regular basis.
+
 
+
This backup script names your backup by hostname and date, specifies your include and exclude file locations, and removes old backups:
+
 
+
<pre>
+
#!/bin/bash
+
# backup-cfg - backup configurations with tar
+
  
 
# Backup name
 
# Backup name
machine=$HOSTNAME
+
comp="${HOSTNAME}"  # Computer name
distro=arch
+
dist="arch"        # Distro
type=configs
+
type="configs"
date=`date "+%F"`
+
date="$(date "+%F")"
backupname=$machine-$distro-$type-$date
+
bckp_file="$bckp_dest/$comp-$dist-$type-$date.tar.xz"
  
# Backup destination
+
# Include and exclude file locations
backdest="/opt/backup"
+
scrpt_dir="/home/todd/.scripts/backup"
 +
incl_file="$scrpt_dir"/${0##*/}-inc.txt
 +
excl_file="$scrpt_dir"/${0##*/}-exc.txt
 +
note_file="$scrpt_dir"/${0##*/}-nte.txt
  
# Files containing information to include and exclude
+
scrp_help () {
include_file="/home/user/.scripts/backup/include.txt"
+
echo " ${0##*/} <i|e|n|c> - backup configurations
exclude_file="/home/user/.scripts/backup/exclude.txt"
+
  i - add to the include list a file or folder
 +
  e - add to the exclude list a file or folder or regexp pattern
 +
  n - add note
 +
  c - create backup"
 +
}
  
# Verify that the target directory exists.
+
del_old_bckps () {
if [ ! -d $backdest ]; then
+
if [[ -n "$(find "$bckp_dest" -mtime +30)" ]]; then
   echo " Backup directory does not exist, creating"
+
   find "$bckp_dest" -name "$distro-$type-*" -mtime +30 -exec rm {} \
  mkdir $backdest
+
   \; && echo " Deleted backups older than one month."
  else
+
   echo " Backup directory exists."
+
 
fi
 
fi
 +
}
  
# Delete backups older than two months
+
case $1 in
if [[ -n "$(find "$backdest" -mtime +60)" ]]; then
+
  # Add to include list file or folder
  find "$backdest"/$machine-$distro-$type-* -mtime +60 -exec rm {} \;
+
  i ) shift
   echo -e " Backups older than two months deleted"
+
      for f in "$@"; do
fi
+
        # Check if selection(s) exist
 +
        if [ ! -e "$f" ]; then
 +
          echo " File \""$f"\" does not exist."
 +
          continue
 +
        fi
 +
        # Append file/folder to include list
 +
        full_path=$(readlink -f "$f")
 +
        echo "$full_path" >> "$incl_file" && \
 +
        echo " Added \""$f"\" to ${0##*/} include file."
 +
      done
 +
      # Sort entries
 +
      sort -u "$incl_file" -o "$incl_file" ;;
 +
  # Add to exclude list file, folder, or regexp pattern
 +
  e ) shift
 +
      echo " * \"${0##*/}\" doesn't check if patch is correct because the exclude file can contain regexps.  Be sure the path is correct (e.g. '/mnt/win/*')" | fmt -c -u -w 80
 +
      read -p " Add \""$@"\" to ${0##*/} exclude file? (y/n): " add_exclude
 +
      if [[ "$add_exclude" == [Yy] ]]; then
 +
        echo "$@" >> "$excl_file" && \
 +
        echo " Added \""$@"\" to ${0##*/} exclude file."
 +
      else
 +
        echo " Error: \""$@"\" not added."
 +
      fi ;;
 +
   # Add to note file
 +
  n ) shift
 +
      echo ""$date"-"$(date "+%r")": "$@"" >> "$note_file" && \
 +
      echo " Added string to \""$note_file"\"." ;;
 +
  # Create backup
 +
  c ) # Check if backup directory exists
 +
      if [ ! -d "$bckp_dest" ]; then
 +
        echo " Directory \""$bckp_dest"\" does not exist."
 +
        exit
 +
      fi
 +
      # Delete old backups
 +
      #del_old_bckps
 +
      # Backup configurations
 +
      sudo tar --exclude-from=$excl_file --files-from=$incl_file -c --xz -f $bckp_file ;;
 +
  * ) # Display usage if no parameters give
 +
      scrp_help ;;
 +
esac</pre>}}
  
# Unmount other drives/partitions
+
=== Packages ===
#umount -a
+
  
# Backup
+
Package lists can be created that can re-install your programs upon a restore.  If you have the hard disk space available, you might also want to consider saving the install packages ({{ic|*.pkg.tar.gz}}) as well.
tar --exclude-from=$exclude_file --files-from=$include_file -cvpzf \
+
$backdest/$backupname.tar.gz
+
</pre>
+
  
To do automated backups (for example, every week), put the script in your root crontab ({{Codeline|crontab -e}}):
+
==== Creating a Package List ====
  
<pre>
+
You can create a list of all installed ''official'' packages with:
#
+
# DO NOT EDIT THIS FILE MANUALLY!! USE crontab -e INSTEAD.
+
#
+
  
  50 00  *  *  0      /home/user/.scripts/backup/backup-cfg
+
  pacman -Qqe | grep -v "$(pacman -Qqm)" > pkglist-off.txt
# |   |  |  |  |  user
+
# |  |  |  |  weekday (0-6) (Sunday=0)
+
# |  |  |  month (1-12)
+
# |  |  day (0-31)
+
# |  hour (0-23)
+
# minute (0-59)
+
</pre>
+
  
===What Not to Include in a Backup===
+
This will create a list of all packages in the official, enabled [[pacman]] repositories (i.e. core, extra, community and testing).
  
There are some files that should not be included in the backup directly.  The reason for this will become apparently later in the document.  For now, if you want these files backed up copy them to a new name:
+
To create a list of all ''local'' packages (includes packages installed from the [[AUR]]):
  
<pre>
+
pacman -Qqm > pkglist-loc.txt
cp /boot/grub/menu.lst /boot/grub/menu.lst-good
+
cp /etc/fstab /etc/fstab-good
+
cp /etc/sudoers /etc/sudoers-good
+
</pre>
+
  
Then add them to your include file.
+
==== Saving Package Tarballs ====
  
===Packages===
+
Pacman saves all package tarballs in {{ic|/var/cache/pacman/pkg/}}.  Saving these will increase your re-install speed so consider saving these as well.  You might want to think about reducing the size of the cache before backing up too.  Pacman has the ability to remove any uninstalled packages from the cache with:
  
A package list can be created that can restore your installed packages. If you have the hard disk space available, you might want to also consider saving the install packages ({{Filename|*.pkg.tar.gz}}) as well.
+
  pacman -Sc
  
====Creating a Package List====
+
===== Yaourt =====
  
You can create a list of official installed packages with:
+
If you use [[Yaourt]] to install packages from the AUR, you might want to consider setting up a cache for it (Yaourt by default does not save the built package tarballs).  To setup a cache directory, edit {{ic|/etc/yaourtrc}} to include one:
  
pacman -Qqet | grep -v "$(pacman -Qqm)" > /opt/backup/pkg-official
+
  ExportToLocalRepository /var/cache/pacman/pkg-local
bca /opt/backup/pkg-official
+
  
This will create a list of all packages in the enabled [[pacman]] repositories (i.e. core, extra, community and testing).
+
Then give the directory the necessary permissions so Yaourt can write to it as a regular user:
  
To create a list of all local packages (includes packages installed from the [[AUR]]):
+
mkdir -p /var/cache/pacman/pkg-local
 +
chmod 766 /var/cache/pacman/pkg-local
  
pacman -Qqmt > /opt/backup/pkg-local
+
Copy these packages to your seperate medium.
bca /opt/backup/pkg-local
+
  
Alternately, you can use the [http://aur.archlinux.org/packages.php?ID=22747 packup] utility in the AUR.  Packup is a backup and restore from list utility with menu driven interface.  Packup will create a list and save it in the root folder.  If you use packup, consider creating a separate list for official and local packages for easier debugging later.
+
=== Storing the Backup ===
  
====Saving Package Tarballs====
+
After you have made up your tarred configurations, package lists, and (optionally) your install packages, you are going to need to store them on a seperate medium than your install partition/drive.  Do not put your package lists and install packages in your tarred configurations. This is because all packages must be reinstalled first before you restore your configurations to prevent file conflicts (pacman will not install packages with file conflicts).  If you have large enough USB Flash Drive these work well.  Optionally you can burn them to a CD or use a partition utility like {{Pkg|gparted}} to create an extra partition.  If using CD's you can span large archives by using the [http://en.gentoo-wiki.com/wiki/Backup_to_DVD split utility].  To create a new partition consider using the [http://partedmagic.com/ Parted Magic CD] which has gparted on it.
  
Pacman saves all package tarballs in {{Filename|/var/cache/pacman/pkg/}}.  Saving these will increase the re-install speed so consider adding this folder to your include file.  You might want to think about reducing the size of the cache before backing up, though.  Pacman has the ability to remove any un-installed packages from the cache with:
+
== Restoring ==
  
pacman -Sc
+
Restoring will involve:
  
If you use [[Yaourt]] to install packages from the AUR, you might want to consider setting up a cache for it too (Yaourt by default does not save the built package tarballs). To setup a cache directory, edit {{Filename|/etc/yaourtrc}} to include one:
+
# Installing the base system through the AIF (Arch Installation Framework).
 +
# Changing root.
 +
# Reinstalling all your packages.
 +
# Extracting your configurations.
 +
# Adding a new user.
  
  ExportToLocalRepository /var/cache/yaourt/pkg
+
=== AIF Install ===
  
Then give the directory the necessary permissions so Yaourt can write to it as a regular user:
+
Install Arch Linux as you normally would through the AIF on the LiveCD.
  
mkdir -p /var/cache/yaourt/pkg
+
=== Change Root ===
chmod 766 /var/cache/yaourt/pkg/
+
  
And add it to your include file:
+
When finished, mount your USB Flash Drive (or whatever medium you choose to save your configurations... on).
  
  bca /var/cache/yaourt/pkg/
+
  mkdir /backup-files
 +
mount /dev/<disk-drive-parition> /backup-files
  
===Storing the Backup===
+
Your Arch install will already be mounted on {{ic|/mnt}} so now copy these files to your Arch install:
  
The examples above save the backups to the same partition (provided you do not have a separate /opt partition). If you have the hard disk space, use a separate partition or drive.  The [http://partedmagic.com/ Parted Magic CD] has Gparted on it to help do this.  If you do not you will need to put them on a separate media before you restore. If you absolutely have to, you can break up tar files and span them over [http://en.gentoo-wiki.com/wiki/Backup_to_DVD multiple disks].
+
  mkdir -p /mnt/opt/restore
 +
  cd /backup-files
 +
cp -a * /mnt/opt/restore
  
==Restoring==
+
Now you will need to {{Ic|chroot}} ([[Change Root]]) to your Arch install:
  
Restoring will require: reinstalling the base system, extracting the backup configurations back to the new system, checking for conflicts, and reinstalling the previously installed packages.
+
cd /mnt
 +
cp /etc/resolv.conf /mnt/etc
 +
mount -t proc none /mnt/arch/proc
 +
mount -t sysfs none /mnt/arch/sys
 +
mount -o bind /dev /mnt/arch/dev
 +
chroot . /bin/bash
  
===Extracting Configurations===
+
=== Reinstall your Packages ===
  
Install Arch Linux as you normally would and then mount the partition with your backup in it:
+
Reinstall packages from the official repositories, the AUR, and locally installed packages separately to better diagnose problems if they occur.
  
mkdir /backup-part
+
==== Official ====
mount /dev/<your-partition> /backup-part
+
  
Then extract your backup onto the new install:
+
First reinstall packages from the official repositories;
 
+
tar xvf hostname-arch-configs-date-tar.gz -C /mnt
+
 
+
The installer will have already mounted your install partition on /mnt.  This is where not including those files directly above comes in handy.  Doing this will overwrite any files that match with your backup and the new install.  Check your good mount file (fstab) and bootloader file (menu.lst) with the new install version and update where necessary.  Reboot.
+
 
+
===Adding a User===
+
 
+
When creating a user consider giving the user a unique user id (UID).  This will help prevent conflicts in the future with other users and programs having the same UID (UIDs for users generally start at 1000):
+
  
 
<pre>
 
<pre>
useradd -m -u 1050 -G audio,optical,power,storage,users,video -s /bin/bash user
+
pacman -Sy
 +
pacman -S --needed $(cat /opt/restore/pkglist-off.txt)
 
</pre>
 
</pre>
  
If you have restored a user home directory (/home/user) from your backup configurations, the '''-m''' switch will give a warning about an already existing home directory but will not alter the directory.  Do not forget to change permissions in your home directory if your UIDs differ:
+
==== The AUR ====
  
  chown -R username:users /home/user
+
Yaourt comes in handy here. To quickly install yaourt again:
 
+
===Installing Yaourt===
+
 
+
If you previously had any AUR packages installed, you can install Yaourt to restore the packages:
+
  
 
<pre>
 
<pre>
Line 245: Line 233:
 
</pre>
 
</pre>
  
===Reinstalling packages===
+
Then to install AUR pakages from the list:
  
Reinstall packages from the official repositories, the AUR, and locally installed packages separately to better diagnose problems if they occur.
+
yaourt -S $(cat /opt/restore/pkglist-loc.txt | grep -vx "$(pacman -Qqm)")
  
====Official====
+
{{Ic|grep -vx ...}} here is used to remove packages that are already installed.  This comes in useful in case you have to restart the command because you had trouble installing one of the packages.  If you have packages already built by yaourt and in your yaourt cache, you can avoid recompiling again by going to that cache and installing the packages manually ({{Ic|pacman -U ...}}).
  
First reinstall packages from the official repositories;
+
=== Extract Configurations ===
  
<pre>
+
Once all packages have been installed you can extract the configurations:
pacman -Sy
+
pacman -S --needed $(cat /opt/backup/pkg-official)
+
</pre>
+
  
Pacman will not install packages with file conflicts (i.e. if you have configurations placed that pacman detects an installing package has as well).  Take a note of the conflicting packages and either: move the conflicting configuration (if you know what it is), or force pacman to overwrite the conflicting files:
+
tar xvf /opt/restore/hostname-arch-configs-date-tar.gz -C /mnt
  
pacman -S -f --needed $(cat /opt/backup/pkg-official)
+
A couple things to look out for:
  
If you do this, you will need to find out the conflicting file and reinstall it again. Pacman can tell you what files a certain package installs:
+
* Be aware of any changes to your partition layout.  If you changed your partition, you will need to edit both {{ic|/etc/fstab}} and {{ic|/boot/grub/menu.lst}}.
 +
* If you had special options for the kernel ram disk (initrd), then you will have to re-compile it before your reboot to get your expected behavior.
  
pacman -Ql <package>
+
=== Final Details ===
  
And you can extract the file or files from an archive with:
+
Good time to add your user now before you reboot.  When creating a user, consider giving the user a unique user id (UID).  This will help prevent conflicts in the future with other users and programs having the same UID (UIDs for users generally start at 1000):
  
tar xvf hostname-arch-configs-date-tar.gz etc/hosts etc/resolv.conf
+
<pre>
 +
useradd -m -u 1050 -G audio,optical,power,storage,users,video -s /bin/bash user
 +
</pre>
  
and place the file(s) manually where they belong.
+
If you have restored a user home directory (/home/user) from your backup configurations, the '''-m''' switch will give a warning about an already existing home directory but will not alter the directory. Do not forget to change permissions in your home directory if your UIDs differ:
  
====The AUR====
+
  chown -R username:users /home/user
 
+
To reinstall packages from the AUR:
+
 
+
  yaourt -S $(cat /opt/backup/pkg-local | grep -vx "$(pacman -Qqm)")
+
 
+
The additional command is to remove from the list any packages already installed.  For example, if there are any problems installing a package from the AUR you will not have to go through the whole list again and tell Yaourt what to install (i.e. Yaourt does not suppport the '''--needed''' switch).  Also, if you created a Yaourt cache before, note that Yaourt does not have the ability to read from the cache before an install.  If you do not want to build all your AUR packages again, these packages will have to be installed manually - see the next step.
+
 
+
====Local====
+
 
+
If you created a Yaourt cache or have your own packages saved, you can install them with:
+
 
+
pacman -U <*--asdeps> <package>
+
 
+
==External Links==
+
  
* [http://bbs.archlinux.org/viewtopic.php?id=76218 BBS - List AUR packages and only AUR packages]
+
Now, '''reboot'''. Expect a few unexpected things here. No re-install is perfect.  ALSA may pop up a warning and may have to be configure again and there may be a few other things unconsidered.  That's it.  Congratulations on your reinstall.

Revision as of 13:53, 13 June 2012

This article is intended to show you how to backup your configurations and your package list, then to do a full system restore, restore packages, and finally restore your configurations..

Motivation

The need for this documentation is uncommon in the sense that the need to restore from configurations only is only really necessary for the following reasons:

  1. You would like to change your system architecture (e.g. 32bit to 64bit).
  2. If a program or programs begin to behave unexpectedly and no help in the forums (or elsewhere) is available is able to fix the problem. By chance reinstalling your programs and other configurations might fix the problem.
  3. You have limited hard disk space and are not capable of doing a full restore from backup.
Tip: If you are a regular computer user, generally it is good practice to backup your primary drive to a backup hard drive after getting your system installed. Secondary storage disks can be found relatively inexpensively these days and are able to store/restore an entire drive of programs, configurations, documents, etc. in a safe and reliable way. If you have a hardware setup like this, look at programs like the Clonezilla CD, or the Parted Magic CD (includes Clonzilla and other tools) which are both open-source and can image your hard drive for later restoration.

Backup

Using tar in a script can make archiving configurations get done in just a couple steps.

Include and Exclude Files

Tar has the ability to read from both an include and an exclude file. This means that you can tell tar everything you would like to include in the backup and exclude just by using two files. The format used is one line per file or directory that indicates the full path. For example:

/etc/pacman.conf
/etc/rc.conf
/home/user
...

And is invoked like this:

tar --files-from=include.txt --exclude-from=exclude.txt -c --xz -f backupname.tar.xz

The name of the files can be anything you want. The exclude file is like the include file but additionally has the ability to be able to use regexps, as well as being able to be commented and have blank lines.

Backup Script

This backup script has the ability to add to the include and exclude files and the ability to backup the configurations. The usage is:

 bckcfg <i|e|n|c> - backup configurations
  i - add to the include list a file or folder
  e - add to the exclude list a file or folder or regexp pattern
  n - add note
  c - create backup

For example to add to the include file:

bckcfg i /etc/rc.conf

And to add to the exclude file:

bckcfg e .thumbnails/

The script is able to detect the full path so writing a relative or partial path is acceptable.

To backup, this script names your backup by several definable variables, removes old backups if desired, then backs up the configurations. The include and exclude files are placed in the same directory as the backup script.

bckcfg
#!/bin/bash
# Backup configurations

# Backup destination directory
#bckp_dest="/home/todd/Desktop"
bckp_dest="/run/media/todd/Backup/backup-configs"

# Backup name
comp="${HOSTNAME}"  # Computer name
dist="arch"         # Distro
type="configs"
date="$(date "+%F")"
bckp_file="$bckp_dest/$comp-$dist-$type-$date.tar.xz"

# Include and exclude file locations
scrpt_dir="/home/todd/.scripts/backup"
incl_file="$scrpt_dir"/${0##*/}-inc.txt
excl_file="$scrpt_dir"/${0##*/}-exc.txt
note_file="$scrpt_dir"/${0##*/}-nte.txt

scrp_help () {
echo " ${0##*/} <i|e|n|c> - backup configurations
  i - add to the include list a file or folder
  e - add to the exclude list a file or folder or regexp pattern
  n - add note
  c - create backup"
}

del_old_bckps () {
if [[ -n "$(find "$bckp_dest" -mtime +30)" ]]; then
  find "$bckp_dest" -name "$distro-$type-*" -mtime +30 -exec rm {} \
  \; && echo " Deleted backups older than one month."
fi
}

case $1 in
  # Add to include list file or folder
  i ) shift
      for f in "$@"; do
        # Check if selection(s) exist
        if [ ! -e "$f" ]; then
          echo " File \""$f"\" does not exist."
          continue
        fi
        # Append file/folder to include list
        full_path=$(readlink -f "$f")
        echo "$full_path" >> "$incl_file" && \
        echo " Added \""$f"\" to ${0##*/} include file."
      done
      # Sort entries
      sort -u "$incl_file" -o "$incl_file" ;;
  # Add to exclude list file, folder, or regexp pattern
  e ) shift
      echo " * \"${0##*/}\" doesn't check if patch is correct because the exclude file can contain regexps.  Be sure the path is correct (e.g. '/mnt/win/*')" | fmt -c -u -w 80
      read -p " Add \""$@"\" to ${0##*/} exclude file? (y/n): " add_exclude
      if [[ "$add_exclude" == [Yy] ]]; then
        echo "$@" >> "$excl_file" && \
        echo " Added \""$@"\" to ${0##*/} exclude file."
      else
        echo " Error: \""$@"\" not added."
      fi ;;
  # Add to note file
  n ) shift
      echo ""$date"-"$(date "+%r")": "$@"" >> "$note_file" && \
      echo " Added string to \""$note_file"\"." ;;
  # Create backup
  c ) # Check if backup directory exists
      if [ ! -d "$bckp_dest" ]; then
        echo " Directory \""$bckp_dest"\" does not exist."
        exit
      fi
      # Delete old backups
      #del_old_bckps
      # Backup configurations
      sudo tar --exclude-from=$excl_file --files-from=$incl_file -c --xz -f $bckp_file ;;
  * ) # Display usage if no parameters give
      scrp_help ;;
esac

Packages

Package lists can be created that can re-install your programs upon a restore. If you have the hard disk space available, you might also want to consider saving the install packages (*.pkg.tar.gz) as well.

Creating a Package List

You can create a list of all installed official packages with:

pacman -Qqe | grep -v "$(pacman -Qqm)" > pkglist-off.txt

This will create a list of all packages in the official, enabled pacman repositories (i.e. core, extra, community and testing).

To create a list of all local packages (includes packages installed from the AUR):

pacman -Qqm > pkglist-loc.txt

Saving Package Tarballs

Pacman saves all package tarballs in /var/cache/pacman/pkg/. Saving these will increase your re-install speed so consider saving these as well. You might want to think about reducing the size of the cache before backing up too. Pacman has the ability to remove any uninstalled packages from the cache with:

pacman -Sc
Yaourt

If you use Yaourt to install packages from the AUR, you might want to consider setting up a cache for it (Yaourt by default does not save the built package tarballs). To setup a cache directory, edit /etc/yaourtrc to include one:

  ExportToLocalRepository /var/cache/pacman/pkg-local

Then give the directory the necessary permissions so Yaourt can write to it as a regular user:

mkdir -p /var/cache/pacman/pkg-local
chmod 766 /var/cache/pacman/pkg-local

Copy these packages to your seperate medium.

Storing the Backup

After you have made up your tarred configurations, package lists, and (optionally) your install packages, you are going to need to store them on a seperate medium than your install partition/drive. Do not put your package lists and install packages in your tarred configurations. This is because all packages must be reinstalled first before you restore your configurations to prevent file conflicts (pacman will not install packages with file conflicts). If you have large enough USB Flash Drive these work well. Optionally you can burn them to a CD or use a partition utility like gparted to create an extra partition. If using CD's you can span large archives by using the split utility. To create a new partition consider using the Parted Magic CD which has gparted on it.

Restoring

Restoring will involve:

  1. Installing the base system through the AIF (Arch Installation Framework).
  2. Changing root.
  3. Reinstalling all your packages.
  4. Extracting your configurations.
  5. Adding a new user.

AIF Install

Install Arch Linux as you normally would through the AIF on the LiveCD.

Change Root

When finished, mount your USB Flash Drive (or whatever medium you choose to save your configurations... on).

mkdir /backup-files
mount /dev/<disk-drive-parition> /backup-files

Your Arch install will already be mounted on /mnt so now copy these files to your Arch install:

mkdir -p /mnt/opt/restore
cd /backup-files
cp -a * /mnt/opt/restore

Now you will need to chroot (Change Root) to your Arch install:

cd /mnt
cp /etc/resolv.conf /mnt/etc
mount -t proc none /mnt/arch/proc
mount -t sysfs none /mnt/arch/sys
mount -o bind /dev /mnt/arch/dev
chroot . /bin/bash

Reinstall your Packages

Reinstall packages from the official repositories, the AUR, and locally installed packages separately to better diagnose problems if they occur.

Official

First reinstall packages from the official repositories;

pacman -Sy
pacman -S --needed $(cat /opt/restore/pkglist-off.txt)

The AUR

Yaourt comes in handy here. To quickly install yaourt again:

wget http://aur.archlinux.org/packages/yaourt/yaourt.tar.gz
tar xvf yaourt.tar.gz && cd yaourt*
makepkg -s
pacman -U yaourt-*.pkg.tar.gz

Then to install AUR pakages from the list:

yaourt -S $(cat /opt/restore/pkglist-loc.txt | grep -vx "$(pacman -Qqm)")

grep -vx ... here is used to remove packages that are already installed. This comes in useful in case you have to restart the command because you had trouble installing one of the packages. If you have packages already built by yaourt and in your yaourt cache, you can avoid recompiling again by going to that cache and installing the packages manually (pacman -U ...).

Extract Configurations

Once all packages have been installed you can extract the configurations:

tar xvf /opt/restore/hostname-arch-configs-date-tar.gz -C /mnt

A couple things to look out for:

  • Be aware of any changes to your partition layout. If you changed your partition, you will need to edit both /etc/fstab and /boot/grub/menu.lst.
  • If you had special options for the kernel ram disk (initrd), then you will have to re-compile it before your reboot to get your expected behavior.

Final Details

Good time to add your user now before you reboot. When creating a user, consider giving the user a unique user id (UID). This will help prevent conflicts in the future with other users and programs having the same UID (UIDs for users generally start at 1000):

useradd -m -u 1050 -G audio,optical,power,storage,users,video -s /bin/bash user

If you have restored a user home directory (/home/user) from your backup configurations, the -m switch will give a warning about an already existing home directory but will not alter the directory. Do not forget to change permissions in your home directory if your UIDs differ:

chown -R username:users /home/user

Now, reboot. Expect a few unexpected things here. No re-install is perfect. ALSA may pop up a warning and may have to be configure again and there may be a few other things unconsidered. That's it. Congratulations on your reinstall.