Difference between revisions of "Full system backup with rsync"

From ArchWiki
Jump to: navigation, search
(Backup script: looks better on a single line (more compact this way). If someone knows what "date > $1/BACKUP" does, please add it to this section)
(Boot setup: "configuration in the destination's" ? wut? Changed the titles, major overhaul. Notes placed at the top in case someone stops reading at only one bootloader)
Line 79: Line 79:
 
{{Note|The trailing "{{ic|/}}" character at the end is necessary.}}
 
{{Note|The trailing "{{ic|/}}" character at the end is necessary.}}
  
==Boot setup==
+
==Boot requirements==
After the sync is finished, the backup destination's {{ic|/etc/fstab}} has to be modified, a boot loader needs to be installed on the backup destination, and configuration in the destination's {{ic|/boot/grub/menu.lst}} requires to reflect the new location.
+
  
===Modify fstab===
+
If you transferred the system to a different partition or HDD and you want to boot it, the process is as simple as updating the backup's {{ic|/etc/fstab}} and your bootloader's configuration file (e.g. {{ic|/boot/grub/menu.lst}}). But if you plan on replacing the current install (or replacing the HDD) and you know that you will no longer have a bootloader, you will also have to (''re'')install one.
Edit the backup destination's fstab:
+
 
 +
Here's how you can find out the device name or [[UUID]] of a certain partition:
 +
 
 +
$ blkid
 +
 
 +
# fdisk -l
 +
 
 +
===Update the fstab===
 +
Edit the backup's [[fstab]] to reflect the changes:
 
{{hc|# nano /media/backup/etc/fstab|
 
{{hc|# nano /media/backup/etc/fstab|
 
tmpfs        /tmp          tmpfs    nodev,nosuid,noexec      0  0
 
tmpfs        /tmp          tmpfs    nodev,nosuid,noexec      0  0
Line 93: Line 100:
 
}}
 
}}
  
Because rsync has performed a recursive copy of the ''entire'' root filesystem, all of the {{ic|sda}} mountpoints are problematic and will cause the backup boot to fail. In this example, all of the offending entries are replaced with a single one:
+
Because rsync has performed a recursive copy of the ''entire'' root filesystem, all of the {{ic|sda}} mountpoints are problematic and booting the backup will fail. In this example, all of the offending entries are replaced with a single one:
 
{{hc|# nano /media/backup/etc/fstab|
 
{{hc|# nano /media/backup/etc/fstab|
 
tmpfs        /tmp          tmpfs    nodev,nosuid,noexec      0  0
 
tmpfs        /tmp          tmpfs    nodev,nosuid,noexec      0  0
Line 102: Line 109:
 
Remember to use the proper device name and filesystem type.
 
Remember to use the proper device name and filesystem type.
  
===Install bootloader===
+
===Update the bootloader's configuration file===
While these instructions assume [[GRUB]] is being employed, they could easily be adapted to other bootloaders, such as [[LILO]].
+
  
Open the GRUB console:
+
This section assumes that you backed up the system to another drive or partition, that your current bootloader is working fine, and that you want to boot from the backup as well.
# grub
+
  
Direct the install towards the destination device:
+
{{Note|Instead of updating the configuration file, you can also edit the menu during boot. When the menu shows up, press ''e'' (for GRUB) or the ''Tab'' key (Syslinux) and change the relevant entries. To boot, press ''b'' (GRUB) or ''Enter'' (Syslinux). You should take this opportunity to consult their wiki pages to learn which ones start counting drives/partitions from zero and which ones do not. For example, GRUB Legacy sees "hd0,0" as the first partition of the first drive. Other bootloaders work differently.}}
root (hd'''1,0''')
+
setup (hd'''1''')
+
  
;root; {{ic|hd 1,0}}
+
For [[GRUB Legacy]]:
:This should point to where the GRUB files are located--in this case, "{{ic|hd 1}}" means the second storage device ({{ic|/dev/sdb}}) and "{{ic|0}}" is the first partition ({{ic|/dev/sdb''1''}}).
+
  
;setup; {{ic|hd 1}}
+
# nano /boot/grub/menu.lst
:The command specifies where the actual boot loader is to be installed. In this example it is installed to the [[MBR]] of the second storage device.
+
  
===Configure bootloader===
+
For [[Syslinux]]:
The problem here is that even though the boot loader installs correctly, its menu entries are for the main system's partitions, not the backup system's.
+
  
It's possible to fix this by creating a custom {{ic|/boot/grub/menu.lst}} for the backup destination. In order to do this, modify {{ic|rbackup.sh}} so that it copies a custom {{ic|menu.lst}}:
+
# nano /boot/syslinux/syslinux.cfg
{{hc|rbackup.sh|2=
+
#!/bin/sh
+
# rsync backup script
+
  
sudo sh -c "
+
For [[GRUB2]], it's recommended that you automatically re-generate a .cfg:
    rsync -av --delete-excluded --exclude-from=backup.lst / $1;
+
 
    '''cp ~/custom.menu.lst $1/boot/grub/menu.lst;'''
+
# grub-mkconfig -o /boot/grub/grub.cfg
    date > $1/BACKUP
+
 
"
+
===(Re)install the bootloader===
}}
+
 
 +
This section assumes that the system was backed up on a machine that no longer has a bootloader installed, or a machine that only has Windows on it (e.g. Windows has taken over the MBR after reinstalling it).
 +
 
 +
{{Box RED|Remember:|Don't forget to set the ''boot'' flag on {{ic|/dev/sda1}} (or whatever partition the boot files are located) using either {{ic|gparted}} or {{ic|cfdisk}}, depending on whichever the LiveCD comes with. Else you may get a ''"Missing operating system. FATAL: INT18: BOOT FAILURE"'' message.}}
 +
 
 +
 
 +
Using a "live" Linux distribution and from a [[chroot]] environment ''(this is important)'', run:
 +
 
 +
{{Box RED|Note:|Use {{ic|/dev/sdx}}, not {{ic|/dev/sdx1}}; you want to install to the MBR.}}
 +
 
 +
'''GRUB Legacy''':
 +
 
 +
# grub-install /dev/sda
 +
 
 +
'''GRUB2''':
 +
 
 +
# grub-install /dev/sda
 +
# grub-mkconfig -o /boot/grub/grub.cfg
 +
 
 +
'''Syslinux''':
  
{{tip|instead of replacing {{ic|menu.lst}} with a custom version solely for the backup, add a new GRUB entry pointing to the backup device or simply edit GRUB's menu during boot time.}}
+
# extlinux --install /boot/syslinux
 +
# dd bs=440 conv=notrunc count=1 if=/usr/lib/syslinux/mbr.bin of=/dev/sda
  
 
== External links ==
 
== External links ==

Revision as of 14:31, 8 July 2012

This rsync script allows creating a full backup copy across filesystems. It is setup so that the copy includes intact booting capabilities, optionally excluding selected files.

The approach has benefits over omitting system files by just copying personal data; if the system becomes corrupted in the main partition, overcoming the problem means booting into the backup as opposed to identifying and reinstalling affected programs.

Instructions were converted from this forum post.

With a single command

This operation can be done while the system is running. Since it's going to take a while, you may freely browse the web during this time. Worst case scenario you won't get the same opened tabs when you restore the backup (or boot from it) because they weren't saved. Not a big deal.

As root, run:

Warning: If you plan on backing up your system somewhere other than /mnt or /media, don't forget to add it to the list, to avoid a copying loop.
# rsync -av /* /media/Backup/backup --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found,$HOME/.gvfs}
Note: The contents of /dev, /proc, /sys, /tmp, /run were excluded because they are populated at boot (while the folders themselves are not created), lost+found is filesystem-specific, and $HOME/.gvfs should be added too, so that it won't complain at the end that "some files/attrs were not transferred".
Note: Additionally, you may also want to skip ~/.thumbnails/*, ~/.mozilla/firefox/*.default/Cache/* and ~/.cache/chromium/*

With a list

For this method, two files are needed: the backup script and a file stating which files to include/exclude from the backup source.

Backup script

The script is very simple; it rsyncs in archive mode, ensuring that symbolic links, devices, permissions and ownerships, among other file attributes are preserved, while excluding files that match the patterns from the include/exclude list.

Save it as rbackup.sh and make it executable:

rbackup.sh
#!/bin/sh

sudo sh -c "
    rsync -av --delete-excluded --exclude-from=backup.lst / $1;
    date > $1/BACKUP
"

Backup source: / — In this case it's performing a backup on the whole root.

Backup destination: $1 — Passed as an argument to the script. It will be a substitute for whatever backup destination you want later (e.g. /media/backup).

Exclude list: --exclude-from=backup.lst — This example uses a file called backup.lst.

Exclude list

As deciding which files should populate this list can be difficult, here's a typical example of files that do not need to be backed up.

Save the following as backup.lst:

backup.lst
# Include

# Commented out. See bellow.


# Exclude
- /dev/*
- /run/*
- /proc/*
- /sys/*
- /tmp/*
- lost+found/
- /media/*
- /mnt/*
Warning: don't forget to also exclude the mounted directory where you'll put the backup to avoid an infinite loop. You can use some folder in /media or /mnt, since they're already on the list.
Exclude
Content in system directories; /dev, /proc, /sys and /tmp are excluded because they are created by the system at runtime, while the directories themselves need to be preserved since they are not regenerated at boot. Lastly, all lost+found instances are skipped since they are partition-specific. For Archlinux /var/lib/pacman/sync/* can also be excluded. This can save a lot of time on every backup since the directory contains many small files that tend to change quite often. These are description files for every package from the repositories. These files can be regenerated with pacman -Syu.
Include
See the example at the bottom of this article if you explicitly want to keep something from the user folder or /media (although you could simply copy them after rsync is done, so this section is a bit redundant). Note that specifying every desired file or directory in Include is not needed; this section only acts as a filter for statements in Exclude.

Backing up

While the system is running, open a terminal and run (as root):

# /path/to/rbackup.sh /media/backup/
Note: Substitute /media/backup as appropriate.
Note: The trailing "/" character at the end is necessary.

Boot requirements

If you transferred the system to a different partition or HDD and you want to boot it, the process is as simple as updating the backup's /etc/fstab and your bootloader's configuration file (e.g. /boot/grub/menu.lst). But if you plan on replacing the current install (or replacing the HDD) and you know that you will no longer have a bootloader, you will also have to (re)install one.

Here's how you can find out the device name or UUID of a certain partition:

$ blkid
# fdisk -l

Update the fstab

Edit the backup's fstab to reflect the changes:

# nano /media/backup/etc/fstab
tmpfs        /tmp          tmpfs     nodev,nosuid,noexec      0   0

/dev/sda1    /boot         ext2      defaults                 0   2
/dev/sda5    none          swap      defaults                 0   0
/dev/sda6    /             ext4      defaults                 0   1
/dev/sda7    /home         ext4      defaults                 0   2

Because rsync has performed a recursive copy of the entire root filesystem, all of the sda mountpoints are problematic and booting the backup will fail. In this example, all of the offending entries are replaced with a single one:

# nano /media/backup/etc/fstab
tmpfs        /tmp          tmpfs     nodev,nosuid,noexec      0   0

/dev/sdb1    /             ext4      defaults                 0   1

Remember to use the proper device name and filesystem type.

Update the bootloader's configuration file

This section assumes that you backed up the system to another drive or partition, that your current bootloader is working fine, and that you want to boot from the backup as well.

Note: Instead of updating the configuration file, you can also edit the menu during boot. When the menu shows up, press e (for GRUB) or the Tab key (Syslinux) and change the relevant entries. To boot, press b (GRUB) or Enter (Syslinux). You should take this opportunity to consult their wiki pages to learn which ones start counting drives/partitions from zero and which ones do not. For example, GRUB Legacy sees "hd0,0" as the first partition of the first drive. Other bootloaders work differently.

For GRUB Legacy:

# nano /boot/grub/menu.lst

For Syslinux:

# nano /boot/syslinux/syslinux.cfg

For GRUB2, it's recommended that you automatically re-generate a .cfg:

# grub-mkconfig -o /boot/grub/grub.cfg

(Re)install the bootloader

This section assumes that the system was backed up on a machine that no longer has a bootloader installed, or a machine that only has Windows on it (e.g. Windows has taken over the MBR after reinstalling it).

Remember: Don't forget to set the boot flag on /dev/sda1 (or whatever partition the boot files are located) using either gparted or cfdisk, depending on whichever the LiveCD comes with. Else you may get a "Missing operating system. FATAL: INT18: BOOT FAILURE" message.


Using a "live" Linux distribution and from a chroot environment (this is important), run:

Note: Use /dev/sdx, not /dev/sdx1; you want to install to the MBR.

GRUB Legacy:

# grub-install /dev/sda

GRUB2:

# grub-install /dev/sda
# grub-mkconfig -o /boot/grub/grub.cfg

Syslinux:

# extlinux --install /boot/syslinux
# dd bs=440 conv=notrunc count=1 if=/usr/lib/syslinux/mbr.bin of=/dev/sda

External links

A sample implementation can be found here: