Difference between revisions of "Software RAID and LVM"

From ArchWiki
Jump to navigation Jump to search
(Update RAID configuration: Remove draft formatting)
m (Partition hard drives: - fb00 -> fd00)
Line 114: Line 114:
# Use the default for the first sector: {{Keypress|Enter}}
# Use the default for the first sector: {{Keypress|Enter}}
# For {{filename|sda1}} and {{filename|sda2}} type the appropriate size in MB (i.e. {{codeline|+100MB}} and {{codeline|+2048M}}). For {{filename|sda3}} just hit {{Keypress|Enter}} to select the remainder of the disk.
# For {{filename|sda1}} and {{filename|sda2}} type the appropriate size in MB (i.e. {{codeline|+100MB}} and {{codeline|+2048M}}). For {{filename|sda3}} just hit {{Keypress|Enter}} to select the remainder of the disk.
# Select {{codeline|Linux RAID}} as the partition type: {{Codeline|fb00}}
# Select {{codeline|Linux RAID}} as the partition type: {{Codeline|fd00}}
# Write the table to disk and exit: {{Keypress|w}}
# Write the table to disk and exit: {{Keypress|w}}

Revision as of 16:53, 19 September 2011

This template has only maintenance purposes. For linking to local translations please use interlanguage links, see Help:i18n#Interlanguage links.

Local languages: Català – Dansk – English – Español – Esperanto – Hrvatski – Indonesia – Italiano – Lietuviškai – Magyar – Nederlands – Norsk Bokmål – Polski – Português – Slovenský – Česky – Ελληνικά – Български – Русский – Српски – Українська – עברית – العربية – ไทย – 日本語 – 正體中文 – 简体中文 – 한국어

External languages (all articles in these languages should be moved to the external wiki): Deutsch – Français – Română – Suomi – Svenska – Tiếng Việt – Türkçe – فارسی

Template:Article summary start Template:Article summary text Template:Article summary heading Template:Article summary link Template:Article summary heading Template:Article summary wiki Template:Article summary wiki Template:Article summary wiki Template:Article summary wiki Template:Article summary end

The combination of RAID and LVM provides numerous features with few caveats compared to just using RAID.


Although RAID and LVM may seem like analogous technologies they each present unique features. This article uses an example with three similar 1TB SATA hard drives. The article assumes that the drives are accessible as Template:Filename, Template:Filename, and Template:Filename. If you are using IDE drives, for maximum performance make sure that each drive is a master on its own separate channel.

Tip: It is good practice to ensure that only the drives involved in the installation are attached while performing the installation.
LVM Logical Volumes Template:Codeline Template:Codeline Template:Codeline Template:Codeline
LVM Volume Groups Template:Filename
RAID Arrays Template:Filename Template:Filename
Physical Partitions Template:Filename Template:Filename Template:Filename Template:Filename Template:Filename Template:Filename
Hard Drives Template:Filename Template:Filename Template:Filename

Swap space

Note: If you want extra performance, just let the kernel use distinct swap partitions as it does striping by default.

Many tutorials treat the swap space differently, either by creating a separate RAID1 array or a LVM logical volume. Creating the swap space on a separate array is not intended to provide additional redundancy, but instead, to prevent a corrupt swap space from rendering the system inoperable, which is more likely to happen when the swap space is located on the same partition as the root directory.


Template:Wikipedia The widespread Master Boot Record (MBR) partitioning scheme, dating from the early 1980s, imposed limitations which affected the use of modern hardware. GUID Partition Table (GPT) is a new standard for the layout of the partition table based on the UEFI specification derived from Intel. Although GPT provides a significant improvement over a MBR, it does require the additional step of creating an additional partition at the beginning of each disk for GRUB2 (see: GPT specific instructions).

Boot loader

This tutorial will use SYSLINUX instead of GRUB2. GRUB2 when used in conjunction with GPT requires an additional BIOS Boot Partition. Additionally, the 2011.08.19 Arch Linux installer does not support GRUB2.

GRUB2 support the current default style of metadata created by mdadm (i.e. 1.2) when combined with an initramfs, which has replaced in Arch Linux with mkinitcpio. SYSLINUX only supports version 1.0, and therefore requires the Template:Codeline option.

Some boot loaders (e.g. GRUB, LILO) will not support any 1.x metadata versions, and instead require the older version, 0.90. If you would like to use one of those boot loaders make sure to add the option Template:Codeline to the Template:Codeline array during RAID installation.


Obtain the latest installation media and boot the Arch Linux installer as outlined in the Beginners' Guide, or alternatively, in the Official Arch Linux Install Guide. Follow the directions outlined there until you have configured your network.

Load kernel modules

Enter another TTY terminal by typing Template:Keypress+Template:Keypress. Load the appropriate RAID (e.g. Template:Filename, Template:Filename, Template:Filename, Template:Filename, Template:Filename) and LVM (i.e. Template:Filename) modules. The following example makes use of RAID1 and RAID5.

# modprobe raid1
# modprobe raid5
# modprobe dm-mod

Prepare the hard drives

Note: If your hard drives are already prepared and all you want to do is activate RAID and LVM jump to Activate existing RAID devices and LVM volumes. This can be achieved with alternative partitioning software (see: Article).

Each hard drive will have a 100MB Template:Codeline partition, 2048MB Template:Codeline partition, and a Template:Codeline partition that takes up the remainder of the disk.

The boot partition must be RAID1, because GRUB does not have RAID drivers. Any other level will prevent your system from booting. Additionally, if there is a problem with one boot partition, the boot loader can boot normally from the other two partitions in the Template:Codeline array. Finally, the partition you boot from must not be striped (i.e. RAID5, RAID0).

Install gdisk

Since most disk partitioning software does not support GPT (i.e. Template:Package Official, Template:Package Official) you will need to install Template:Package Official to set the partition type of the boot loader partitions.

Update the pacman database:

$ pacman-db-upgrade

Refresh the package list:

$ pacman -Syy

Install Template:Package Official:

$ pacman -S gdisk

Partition hard drives

We will use gdisk to create three partitions on each of the three hard drives (i.e. Template:Filename, Template:Filename, Template:Filename):

   Name        Flags      Part Type  FS Type          [Label]        Size (MB)
   sda1        Boot        Primary   linux_raid_m                       100.00  # /boot
   sda2                    Primary   linux_raid_m                      2000.00  # /swap
   sda3                    Primary   linux_raid_m                     97900.00  # /

Open Template:Codeline with the first hard drive:

$ gdisk /dev/sda

and type the following commands at the prompt:

  1. Add a new partition: Template:Keypress
  2. Select the default partition number: Template:Keypress
  3. Use the default for the first sector: Template:Keypress
  4. For Template:Filename and Template:Filename type the appropriate size in MB (i.e. Template:Codeline and Template:Codeline). For Template:Filename just hit Template:Keypress to select the remainder of the disk.
  5. Select Template:Codeline as the partition type: Template:Codeline
  6. Write the table to disk and exit: Template:Keypress

Repeat this process for Template:Filename and Template:Filename or use the alternate Template:Codeline method below. You may need to reboot to allow the kernel to recognize the new tables.

Note: Make sure to create the same exact partitions on each disk. If a group of partitions of different sizes are assembled to create a RAID partition, it will work, but the redundant partition will be in multiples of the size of the smallest partition, leaving the unallocated space to waste.

Clone partitions with sgdisk

If you are using GPT, then you can use Template:Codeline to clone the partition table from Template:Filename to the other two hard drives:

$ sgdisk --backup=table /dev/sda
$ sgdisk --load-backup=table /dev/sdb
$ sgdisk --load-backup=table /dev/sdc

RAID installation

After creating the physical partitions, you are ready to setup the Template:Codeline, Template:Codeline, and Template:Codeline arrays with Template:Codeline. It is an advanced tool for RAID management that will be used to create a Template:Filename within the installation environment.

Create the Template:Codeline array at Template:Filename:

# mdadm --create /dev/md0 --level=5 --raid-devices=3 /dev/sd[abc]3

Create the Template:Codeline array at Template:Filename:

# mdadm --create /dev/md1 --level=1 --raid-devices=3 /dev/sd[abc]2
Note: If you plan on installing a boot loader that does not support the 1.x version of RAID metadata make sure to add the Template:Codeline option to the following command.

Create the Template:Codeline array at Template:Filename:

# mdadm --create /dev/md2 --level=1 --raid-devices=3 --metadata=1.0 /dev/sd[abc]1


Tip: If you want to avoid the initial resync with new hard drives add the Template:Codeline flag.

After you create a RAID volume, it will synchronize the contents of the physical partitions within the array. You can monitor the progress by refreshing the output of Template:Filename ten times per second with:

# watch -n .1 cat /proc/mdstat
Tip: Follow the synchronization in another TTY terminal by typing Template:Keypress + Template:Keypress and then execute the above command.

Further information about the arrays is accessible with:

# mdadm --misc --detail /dev/md[012] | less

Once synchronization is complete the Template:Codeline line should read Template:Codeline. Each device in the table at the bottom of the output should read Template:Codeline or Template:Codeline in the Template:Codeline column. Template:Codeline means each device is actively in the array.

Note: Since the RAID synchronization is transparent to the file-system you can proceed with the installation and reboot your computer when necessary.

LVM installation

This section will convert the two RAIDs into physical volumes (PVs). Then combine those PVs into a volume group (VG). The VG will then be divided into logical volumes (LVs) that will act like physical partitions (e.g. Template:Codeline, Template:Codeline, Template:Codeline). If you did not understand that make sure you read the LVM Introduction section.

Create physical volumes

Make the RAIDs accessible to LVM by converting them into physical volumes (PVs):

# pvcreate /dev/md0
Note: This might fail if you are creating PVs on an existing Volume Group. If so you might want to add Template:Codeline option.

Confirm that LVM has added the PVs with:

# pvdisplay

Create the volume group

Next step is to create a volume group (VG) on the PVs.

Create a volume group (VG) with the first PV:

# vgcreate VolGroupArray /dev/md0

Confirm that LVM has added the VG with:

# vgdisplay

Create logical volumes

Now we need to create logical volumes (LVs) on the VG, much like we would normally prepare a hard drive. In this example we will create separate Template:Codeline, Template:Codeline, Template:Codeline, Template:Codeline LVs. The LVs will be accessible as Template:Filename or Template:Filename.

Create a Template:Codeline LV:

# lvcreate -L 20G VolGroupArray -n lvroot

Create a Template:Codeline LV:

# lvcreate -L 15G VolGroupArray -n lvvar
Note: If you would like to add the swap space to the LVM create a Template:Codeline LV with the Template:Codeline option, which creates a contiguous partition, so that your swap space does not get partitioned over one or more disks nor over non-contiguous physical extents:
# lvcreate -C y -L 2G VolGroupArray -n lvswap

Create a Template:Codeline LV that takes up the remainder of space in the VG:

# lvcreate -l +100%FREE VolGroupArray -n lvhome

Confirm that LVM has created the LVs with:

# lvdisplay
Tip: You can start out with relatively small logical volumes and expand them later if needed. For simplicity, leave some free space in the volume group so there is room for expansion.

Update RAID configuration

Since the installer builds the initrd using Template:Filename in the target system, you should update that file with your RAID configuration. The original file can simply be deleted because it contains comments on how to fill it correctly, and that is something mdadm can do automatically for you. So let us delete the original and have mdadm create you a new one with the current setup:

# mdadm --examine --scan > /mnt/etc/mdadm.conf

Prepare hard drive

Follow the directions outlined the Installation section until you reach the Prepare Hard Drive section. Skip the first two steps and navigate to the Manually Configure block devices, filesystems and mountpoints page. Remember to only configure the PVs (e.g. Template:Filename) and not the actual disks (e.g. Template:Filename).

Configure system

Warning: Follow the steps in the LVM Important section before proceeding with the installation.


mkinitcpio can use a hook to assemble the arrays on boot. For more information see mkinitpcio Using RAID.

  1. Add the Template:Codeline module to the Template:Codeline list in Template:Filename.
  2. Add the Template:Codeline and Template:Codeline hooks to the Template:Codeline list in Template:Filename after Template:Codeline.


Once it is complete you can safely reboot your machine:

# reboot

Install Grub on the Alternate Boot Drives

Once you have successfully booted your new system for the first time, you will want to install Grub onto the other two disks (or on the other disk if you have only 2 HDDs) so that, in the event of disk failure, the system can be booted from another drive. Log in to your new system as root and do:

# grub
grub> device (hd0) /dev/sdb
grub> root (hd0,0)
grub> setup (hd0)
grub> device (hd0) /dev/sdc
grub> root (hd0,0)
grub> setup (hd0)
grub> quit

Archive your Filesystem Partition Scheme

Now that you are done, it is worth taking a second to archive off the partition state of each of your drives. This guarantees that it will be trivially easy to replace/rebuild a disk in the event that one fails. You do this with the sfdisk tool and the following steps:

# mkdir /etc/partitions
# sfdisk --dump /dev/sda >/etc/partitions/disc0.partitions
# sfdisk --dump /dev/sdb >/etc/partitions/disc1.partitions
# sfdisk --dump /dev/sdc >/etc/partitions/disc2.partitions


For further information on how to maintain your software RAID or LVM review the RAID and LVM aritcles.

Additional Resources

Forum threads