Difference between revisions of "Disk cloning"

From ArchWiki
Jump to: navigation, search
(Cloning an entire hard disk: - add note about reloading partition table)
m (Using dd: style)
 
(140 intermediate revisions by 41 users not shown)
Line 1: Line 1:
 
[[Category:Data compression and archiving]]
 
[[Category:Data compression and archiving]]
 
[[Category:System recovery]]
 
[[Category:System recovery]]
[[it:Disk Cloning]]
+
[[it:Disk cloning]]
[[ru:Disk Cloning]]
+
[[ja:ディスクのクローン]]
[[tr:Disk_klonlama]]
+
[[ru:Disk cloning]]
Disk cloning is the process of making an image of a partition or an entire hard drive. This can be useful both for copying the drive to other computers and for backup/recovery purposes. There is also a dedicated page [[File_Recovery|File Recovery]].
+
[[zh-hans:Disk cloning]]
 +
{{Related articles start}}
 +
{{Related|Synchronization and backup programs}}
 +
{{Related|System maintenance#Backup}}
 +
{{Related|System backup}}
 +
{{Related articles end}}
  
===Using dd===
+
Disk cloning is the process of making an image of a partition or of an entire hard drive. This can be useful for copying the drive to other computers and for [[backup]] and [[File recovery|recovery]] purposes.
The dd command is a simple, yet versatile and powerful tool. It can be used to copy from source to destination, block-by-block, regardless of their filesystem types or operating systems. A convenient method is to use dd from a live environment, as in a livecd.
 
{{Warning|As with any command of this type, you should be very cautious when using it; it can destroy data. Remember the order of input file <nowiki>(if=) and output file (of=) and do not reverse them! Always ensure that the destination drive or partition (of=) is of equal or greater size than the source (if=).</nowiki>}}
 
====Cloning a partition====
 
From physical disk /dev/sda, partition 1, to physical disk /dev/sdb, partition 1.
 
dd if=/dev/sda1 of=/dev/sdb1 bs=4096 conv=notrunc,noerror
 
If output file '''''of''''' (sdb1 in the example) does not exist, dd will start at the beginning of the disk and create it.  
 
  
====Cloning an entire hard disk====
+
== Using dd ==
From physical disk /dev/sda to physical disk /dev/sdb
 
dd if=/dev/sda of=/dev/sdb bs=4096 conv=notrunc,noerror
 
This will clone the entire drive, including MBR (and therefore bootloader), all partitions, UUIDs, and data.
 
*''notrunc'' or 'do not truncate' maintains data integrity by instructing dd not to truncate any data.
 
*''noerror'' instructs dd to continue operation, ignoring all input errors. Default behavior for dd is to halt at any error.
 
*''bs=4096'' sets the block size to 4k, an optimal size for hard disk read/write efficiency and therefore, cloning speed.
 
{{Note|To regain unique UUIDs, use "tune2fs /dev/sdbX -U random" on every partitions.}}
 
{{Note|Partition table changes from dd are not be registered by the kernel. To notify of changes without rebooting, use a utility like partprobe (part of GNU parted).}}
 
  
====Backing up the MBR====
+
See [[dd#Disk cloning and restore]].
  
The MBR is stored in the the first 512 bytes of the disk. It consist of 3 parts:
+
== Using ddrescue ==
#The first 446 bytes contain the boot loader.
+
''ddrescue'' is a tool designed for cloning and recovering data. It copies data from one file or block device (hard disc, cdrom, etc) to another, trying to rescue the good parts first in case of read errors, to maximize the recovered data.
#The next 64 bytes contain the partition table (4 entries of 16 bytes each, one entry for each primary partition).
 
#The last 2 bytes contain an identifier
 
  
To save the MBR into the file "mbr.img":
+
To clone a faulty or dying drive, run ddrescue twice. First round, copy every block without read error and log the errors to rescue.log.
  # dd if=/dev/hda of=/mnt/sda1/mbr.img bs=512 count=1
 
  
To restore (be careful : this could destroy your existing partition table and with it access to all data on the disk):
+
# ddrescue -f -n /dev/sdX /dev/sdY rescue.log
  # dd if=/mnt/sda1/mbr.img of=/dev/hda
 
  
If you only want to restore the boot loader, but not the primary partition table entries, just restore the first 446 bytes of the MBR:
+
Second round, copy only the bad blocks and try 3 times to read from the source before giving up.
  # dd if=/mnt/sda1/mbr.img of=/dev/hda bs=446 count=1
 
  
To restore only the partition table, one must use
+
# ddrescue -d -f -r3 /dev/sdX /dev/sdY rescue.log
  # dd if=/mnt/sda1/mbr.img of=/dev/hda bs=1 skip=446 count=64
 
  
You can also get the MBR from a full dd disk image.
+
Now you can check the file system for corruption and mount the new drive.
  #dd if=/path/to/disk.img of=/mnt/sda1/mbr.img bs=512 count=1
 
  
====Create disk image====
+
# fsck -f /dev/sdY
1. Boot from a liveCD or liveUSB.
 
  
2. Make sure no partitions are mounted from the source hard drive.
+
== Using e2image ==
  
3. Mount the external HD
+
''e2image'' is a tool included in {{Pkg|e2fsprogs}} for debugging purposes. It can be used to copy ext2, ext3, and ext4 partitions efficiently by only copying the used blocks. Note that this only works for ext2, ext3, and ext4 filesystems, and the unused blocks are not copied so this may not be a useful tool if one is hoping to recover deleted files.
  
4. Backup the drive.
+
To clone a partition from physical disk {{ic|/dev/sda}}, partition 1, to physical disk {{ic|/dev/sdb}}, partition 1 with e2image, run
  # dd if=/dev/hda conv=sync,noerror bs=64K | gzip -c  > /mnt/sda1/hda.img.gz
 
  
5. Save extra information about the drive geometry necessary in order to interpret the partition table stored within the image. The most important of which is the cylinder size.
+
# e2image -ra -p /dev/sda1 /dev/sdb1
  # fdisk -l /dev/hda > /mnt/sda1/hda_fdisk.info
 
  
'''NOTE:''' You may wish to use a block size (bs=) that is equal to the amount of cache on the HD you are backing up. For example, bs=8192K works for an 8MB cache. The 64K mentioned in this article is better than the default bs=512 bytes, but it will run faster with a larger bs=.
+
{{Note|gparted uses e2image to efficiently copy ext2/3/4 partitions.}}
  
====Restore system====
+
== Disk cloning software ==
  
To restore your system:
+
These applications allow easy backup of entire filesystems and recovery in case of failure, usually in the form of a Live CD or USB drive. They contain complete system images from one or more specific points in time and are frequently used to record known good configurations. See [[Wikipedia:Comparison of disk cloning software]] for their comparison.
  # gunzip -c /mnt/sda1/hda.img.gz | dd of=/dev/hda
 
===Using cp===
 
The cp program can be used to clone a disk, one partition at a time. An advantage to using cp is that the filesystem type of the destination partition(s) may be the same or different than the source. For safety, perform the process from a live environment.
 
  
The basic procedure from a live environment will be:
+
See also [[Synchronization and backup programs]] for other applications that can take full system snapshots, among other functionality.
* Create the new destination partition(s) using fdisk, cfdisk or other tools available in the live environment.
 
* Create a filesystem on each of the newly created partitions. Example:
 
mkfs -t ext3 /dev/sdb1
 
* Mount the source and destination partitions. Example:
 
mount -t ext3 /dev/sda1 /mnt/source
 
mount -t ext3 /dev/sdb1 /mnt/destination
 
* Copy the files from the source partition to the destination
 
cp -a /mnt/source/* /mnt/destination
 
'''-a''': preserve all attributes , never follow symbolic links and copy recursively
 
* Change the mount points of the newly cloned partitions in /etc/fstab accordingly
 
* Finally, install the GRUB bootloader if necessary. (See [[GRUB]])
 
  
==Disk cloning software==
+
* {{App|Arch Backup|A trivial backup script with simple configuration.
===Disk cloning in Arch===
+
** Configurable compression method.
 +
** Multiple backup targets.
 +
|https://github.com/p5n/archlinux-stuff/tree/master/arch-backup/|{{AUR|arch-backup}}}}
  
* [[Partclone|Partclone]] provides utilities to save and restore used blocks on a partition and supports ext2, ext3, ext4, hfs+, reiserfs, reiser4, btrfs, vmfs3, vmfs5, xfs, jfs, ufs, ntfs, fat(12/16/32) and exfat. Optionally, a ncurses interface can be used. Partclone is available in the community repository.
+
* {{App|[[Wikipedia:Clonezilla|Clonezilla]]|A disaster recovery, disk cloning, disk imaging and deployment solution.
* [http://www.partimage.org/ Partimage], an ncurses program, is available in the community repos. Partimage does not currently support ext4 or btrfs filesystems. NTFS is experimental.
+
** Boots from live CD, USB flash drive or PXE server.
 +
** Supports ext2, ext3, ext4, reiserfs, reiser4, xfs, jfs, btrfs, FAT32, NTFS, HFS+ and others.
 +
** Uses Partclone (default), Partimage (optional), ntfsclone (optional), or dd to image or clone a partition.
 +
** Multicasting server to restore to many machines at once.
 +
** Included on the Arch Linux installation media.
 +
|http://clonezilla.org/|{{Pkg|clonezilla}}}}
  
===Disk cloning outside of Arch===
+
* {{App|Deepin Clone|Tool by Deepin to backup and restore. It supports to clone, backup and restore disk or partition.|https://www.deepin.org/en/original/deepin-clone/|{{Pkg|deepin-clone}}}}
If you wish to backup or propagate your Arch install root, you are probably better off booting into something else and clone the partition from there. Some suggestions:
 
  
* [http://partedmagic.com/doku.php?id=start PartedMagic] has a very nice live cd/usb with PartImage and other recovery tools.
+
* {{App|FSArchiver|A safe and flexible file-system backup and deployment tool
* [http://www.mondorescue.org/ Mindi] is a linux distribution specifically for disk clone backup. It comes with its own cloning program, Mondo Rescue.
+
** Support for basic file attributes (permissions, owner, ...).
* [[wikipedia:Acronis_True_Image|Acronis True Image]] is a commercial disk cloner for Windows. It allows you to create a live cd (from within Windows), so you do not need a working Windows install on the actual machine to use it.
+
** Support for multiple file-systems per archive.
* [http://www.fsarchiver.org/Main_Page FSArchiver] allows you to save the contents of a file system to a compressed archive file. Can be found on the [http://www.sysresccd.org/Main_Page System Rescue CD].
+
** Support for extended attributes (they are used by SELinux).
* [http://clonezilla.org/ Clonezilla] is an enhanced partition imager which can also restore entire disks as well as partitions.
+
** Support the basic file-system attributes (label, uuid, block-size) for all linux file-systems.
* [http://redobackup.org/ Redo Backup and Recovery ] is a Live CD featuring a graphical front-end to partclone.
+
** Support for [http://www.fsarchiver.org/Cloning-ntfs ntfs filesystems] (ability to create flexible clones of a Windows partitions).
 +
** Checksumming of everything which is written in the archive (headers, data blocks, whole files).
 +
** Ability to restore an archive which is corrupt (it will just skip the current file).
 +
** Multi-threaded lzo, gzip, bzip2, lzma compression.
 +
** Support for splitting large archives into several files with a fixed maximum size.
 +
** Encryption of the archive using a password. Based on blowfish from libcrypto from [[OpenSSL]].
 +
** Support backup of a mounted root filesystem (-A option).
 +
** Can be found on the [http://www.sysresccd.org/Main_Page System Rescue CD].
 +
|http://www.fsarchiver.org/|{{Pkg|fsarchiver}}}}
 +
 
 +
* {{App|[[Wikipedia:Mondo Rescue|Mondo Rescue]]|A disaster recovery solution to create backup media that can be used to redeploy the damaged system.
 +
** Image-based backups, supporting Linux/Windows.
 +
** Compression rate is adjustable.
 +
** Can backup live systems (without having to halt it).
 +
** Can split image over many files.
 +
** Supports booting to a Live CD to perform a full restore.
 +
** Can backup/restore over NFS, from CDs, tape drives and and other media.
 +
** Can verify backups.
 +
|http://www.mondorescue.org/|{{AUR|mondo}}}}
 +
 
 +
* {{App|[[Partclone]]|A tool that can be used to back up and restore a partition while considering only used blocks.
 +
** Supports ''ext2'', ''ext3'', ''ext4'', ''hfs+'', ''reiserfs'', ''reiser4'', ''btrfs'', ''vmfs3'', ''vmfs5'', ''xfs'', ''jfs'', ''ufs'', ''ntfs'', ''fat(12/16/32)'', ''exfat''.
 +
** Supports compression.
 +
** Optionally, an ''ncurses'' interface can be used.
 +
|http://partclone.org/|{{Pkg|partclone}}}}
 +
 
 +
* {{App|[[Wikipedia:Partimage|Partimage]]|An ''ncurses'' disk cloning utility for Linux/UNIX environments.
 +
** Has a Live CD.
 +
** Supports the most popular filesystems on Linux, Windows and Mac OS.
 +
** Compression.
 +
** Saving to multiple CDs or DVDs or across a network using Samba/NFS.
 +
** Development stopped in favor of FSArchiver.
 +
|http://www.partimage.org|{{Pkg|partimage}}}}
 +
 
 +
* {{App|J7Z|GUI for Linux in java which attempts to simplify data compression and backup. It can create 7z, BZip2, Zip, GZip, Tar archives.
 +
** Updates existing archives quickly.
 +
** Backup multiple folders to a storage location.
 +
** Create or extract protected archives.
 +
** Lessen effort by using archiving profiles and lists.
 +
|http://j7z.xavion.name/|{{AUR|j7z}}}}
 +
 
 +
* {{App|[[Wikipedia:Redo Backup and Recovery|Redo Backup and Recovery]]|A backup and disaster recovery application that runs from a bootable Linux CD image.
 +
** Is capable of bare-metal backup and recovery of disk partitions.
 +
** Uses [http://www.xpud.org/ xPUD] and [[Partclone]] for the backend.
 +
|http://www.redobackup.org/|}}
 +
 
 +
* {{App|System Tar & Restore|Backup and Restore your system using tar or Transfer it with rsync
 +
** GUI and CLI interfaces
 +
** Creates ''.tar.gz'', ''.tar.bz2'', ''.tar.xz'' or ''.tar'' archives
 +
** Supports openssl / gpg encryption
 +
** Uses rsync to transfer a running system
 +
** Supports Grub2, Syslinux, EFISTUB/efibootmgr and Systemd/bootctl
 +
|https://github.com/tritonas00/system-tar-and-restore|{{AUR|system-tar-and-restore}}}}
 +
 
 +
=== dd spin-offs ===
 +
 
 +
; dcfldd : {{Pkg|dcfldd}} is a dd replacement with on-the-fly hashing capability helping to ensure integrity. It accepts most of dd's parameters and includes status output. A stable version of ''dcfldd'' was [http://dcfldd.sourceforge.net/ last released in 2006].
 +
 
 +
; ddrescue : GNU {{Pkg|ddrescue}} is a data recovery tool capable of ignoring read errors. ''ddrescue'' is not related to dd in any way except that both can be used for copying data from one device to another. The key difference is that ''ddrescue'' uses a sophisticated algorithm to copy data from failing drives causing them as little additional damage as possible. See the [http://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html ddrescue manual] for details.
 +
 
 +
== See also ==
  
==External Links==
 
 
* [[Wikipedia:List of disk cloning software]]
 
* [[Wikipedia:List of disk cloning software]]
 
* [https://bbs.archlinux.org/viewtopic.php?id=4329 Arch Linux forum thread]
 
* [https://bbs.archlinux.org/viewtopic.php?id=4329 Arch Linux forum thread]

Latest revision as of 09:43, 31 August 2018

Disk cloning is the process of making an image of a partition or of an entire hard drive. This can be useful for copying the drive to other computers and for backup and recovery purposes.

Using dd

See dd#Disk cloning and restore.

Using ddrescue

ddrescue is a tool designed for cloning and recovering data. It copies data from one file or block device (hard disc, cdrom, etc) to another, trying to rescue the good parts first in case of read errors, to maximize the recovered data.

To clone a faulty or dying drive, run ddrescue twice. First round, copy every block without read error and log the errors to rescue.log.

# ddrescue -f -n /dev/sdX /dev/sdY rescue.log

Second round, copy only the bad blocks and try 3 times to read from the source before giving up.

# ddrescue -d -f -r3 /dev/sdX /dev/sdY rescue.log

Now you can check the file system for corruption and mount the new drive.

# fsck -f /dev/sdY

Using e2image

e2image is a tool included in e2fsprogs for debugging purposes. It can be used to copy ext2, ext3, and ext4 partitions efficiently by only copying the used blocks. Note that this only works for ext2, ext3, and ext4 filesystems, and the unused blocks are not copied so this may not be a useful tool if one is hoping to recover deleted files.

To clone a partition from physical disk /dev/sda, partition 1, to physical disk /dev/sdb, partition 1 with e2image, run

# e2image -ra -p /dev/sda1 /dev/sdb1
Note: gparted uses e2image to efficiently copy ext2/3/4 partitions.

Disk cloning software

These applications allow easy backup of entire filesystems and recovery in case of failure, usually in the form of a Live CD or USB drive. They contain complete system images from one or more specific points in time and are frequently used to record known good configurations. See Wikipedia:Comparison of disk cloning software for their comparison.

See also Synchronization and backup programs for other applications that can take full system snapshots, among other functionality.

  • Arch Backup — A trivial backup script with simple configuration.
    • Configurable compression method.
    • Multiple backup targets.
https://github.com/p5n/archlinux-stuff/tree/master/arch-backup/ || arch-backupAUR
  • Clonezilla — A disaster recovery, disk cloning, disk imaging and deployment solution.
    • Boots from live CD, USB flash drive or PXE server.
    • Supports ext2, ext3, ext4, reiserfs, reiser4, xfs, jfs, btrfs, FAT32, NTFS, HFS+ and others.
    • Uses Partclone (default), Partimage (optional), ntfsclone (optional), or dd to image or clone a partition.
    • Multicasting server to restore to many machines at once.
    • Included on the Arch Linux installation media.
http://clonezilla.org/ || clonezilla
  • Deepin Clone — Tool by Deepin to backup and restore. It supports to clone, backup and restore disk or partition.
https://www.deepin.org/en/original/deepin-clone/ || deepin-clone
  • FSArchiver — A safe and flexible file-system backup and deployment tool
    • Support for basic file attributes (permissions, owner, ...).
    • Support for multiple file-systems per archive.
    • Support for extended attributes (they are used by SELinux).
    • Support the basic file-system attributes (label, uuid, block-size) for all linux file-systems.
    • Support for ntfs filesystems (ability to create flexible clones of a Windows partitions).
    • Checksumming of everything which is written in the archive (headers, data blocks, whole files).
    • Ability to restore an archive which is corrupt (it will just skip the current file).
    • Multi-threaded lzo, gzip, bzip2, lzma compression.
    • Support for splitting large archives into several files with a fixed maximum size.
    • Encryption of the archive using a password. Based on blowfish from libcrypto from OpenSSL.
    • Support backup of a mounted root filesystem (-A option).
    • Can be found on the System Rescue CD.
http://www.fsarchiver.org/ || fsarchiver
  • Mondo Rescue — A disaster recovery solution to create backup media that can be used to redeploy the damaged system.
    • Image-based backups, supporting Linux/Windows.
    • Compression rate is adjustable.
    • Can backup live systems (without having to halt it).
    • Can split image over many files.
    • Supports booting to a Live CD to perform a full restore.
    • Can backup/restore over NFS, from CDs, tape drives and and other media.
    • Can verify backups.
http://www.mondorescue.org/ || mondoAUR
  • Partclone — A tool that can be used to back up and restore a partition while considering only used blocks.
    • Supports ext2, ext3, ext4, hfs+, reiserfs, reiser4, btrfs, vmfs3, vmfs5, xfs, jfs, ufs, ntfs, fat(12/16/32), exfat.
    • Supports compression.
    • Optionally, an ncurses interface can be used.
http://partclone.org/ || partclone
  • Partimage — An ncurses disk cloning utility for Linux/UNIX environments.
    • Has a Live CD.
    • Supports the most popular filesystems on Linux, Windows and Mac OS.
    • Compression.
    • Saving to multiple CDs or DVDs or across a network using Samba/NFS.
    • Development stopped in favor of FSArchiver.
http://www.partimage.org || partimage
  • J7Z — GUI for Linux in java which attempts to simplify data compression and backup. It can create 7z, BZip2, Zip, GZip, Tar archives.
    • Updates existing archives quickly.
    • Backup multiple folders to a storage location.
    • Create or extract protected archives.
    • Lessen effort by using archiving profiles and lists.
http://j7z.xavion.name/ || j7zAUR
  • Redo Backup and Recovery — A backup and disaster recovery application that runs from a bootable Linux CD image.
    • Is capable of bare-metal backup and recovery of disk partitions.
    • Uses xPUD and Partclone for the backend.
http://www.redobackup.org/ ||
  • System Tar & Restore — Backup and Restore your system using tar or Transfer it with rsync
    • GUI and CLI interfaces
    • Creates .tar.gz, .tar.bz2, .tar.xz or .tar archives
    • Supports openssl / gpg encryption
    • Uses rsync to transfer a running system
    • Supports Grub2, Syslinux, EFISTUB/efibootmgr and Systemd/bootctl
https://github.com/tritonas00/system-tar-and-restore || system-tar-and-restoreAUR

dd spin-offs

dcfldd 
dcfldd is a dd replacement with on-the-fly hashing capability helping to ensure integrity. It accepts most of dd's parameters and includes status output. A stable version of dcfldd was last released in 2006.
ddrescue 
GNU ddrescue is a data recovery tool capable of ignoring read errors. ddrescue is not related to dd in any way except that both can be used for copying data from one device to another. The key difference is that ddrescue uses a sophisticated algorithm to copy data from failing drives causing them as little additional damage as possible. See the ddrescue manual for details.

See also