User:M0p/LUKS Root on Btrfs

This article is not officially supported.

The Arch Linux community does not offer support for the information contained in this page; for installation procedures, the Installation guide is the only officially supported document. The content below is mainly maintained by User:M0p, who last reviewed it on 10 February 2021, and it may be out of date or inaccurate.

Notes

Fully encrypted Arch Linux setup. Everything except EFI system partition (contains a single grubx64.efi file) and BIOS boot sector is encrypted with LUKS 1.

Only single disk installation is supported. Consider Root on ZFS if you want to use a multi-disk setup.

Btrfs also has quite some pitfalls, gotchas and limitations. See [1], [2] and search reddit for posts involving crashed btrfs filesystems. If uncertain, use Root on ZFS. I use Root on ZFS on my laptop.

Existing data on target disk will be destroyed.

Password MUST BE interactively entered at boot and can not be automated. Efforts to automate this for the cloud exist, see [3].

For support and questions, leave a message on User_talk:M0p/LUKS_Root_on_Btrfs.

Data integrity

It should be noted that, with only one disk, data are not redundant and not protected from bitrot.

For best data integrity and redundancy, you can either:

minimum 2 disks: use Root on ZFS with mirror/raidz2/raidz3, or
minimum 3 disks: use single disk Root on Btrfs, but store important data on ZFS with mirror/raidz2/raidz3

Preinstallation

Follow official installation guide up to Installation_guide#Boot_the_live_environment.

Connect to the internet. If the target computer aquires IP address with DHCP, no further steps need to be taken. Otherwise, refer to Network Configuration wiki page.

SSH server

Interactively set root password with:

passwd

Start SSH server:

systemctl start sshd

Find the IP address of the target computer:

ip -4 address show scope global

On another computer, connect to the target computer with:

ssh root@192.168.1.10

Enter a bash shell:

bash

Select mirror

Kill reflector:

killall -9 reflector

Edit the following files:

/etc/pacman.d/mirrorlist

Uncomment and move mirrors to the beginning of the file.

Tools

pacman -Sy --needed cryptsetup btrfs-progs gdisk

Variables

In this part, we will set some variables to configure the system.

Timezone

List the available timezones with:

ls /usr/share/zoneinfo/

Store the target timezone in a variable:

INST_TZ=/usr/share/zoneinfo/Asia/Irkutsk

Host name

Store the host name in a variable:

INST_HOST='localhost'

Kernel variant

Store the kernel variant in a variable. Available variants in official repo are:

linux
linux-lts
linux-zen
linux-hardened

INST_LINVAR='linux'

Target disk

List the available disks with:

ls -d /dev/disk/by-id/* | grep -v part

If the disk is not in the command output, use /dev/disk/by-path.

Store the target disk in a variable:

DISK=/dev/disk/by-id/nvme-foo_NVMe_bar_512GB

Mountpoint

INST_MNT=$(mktemp -d)

Format and Partition

Clear the partition table:

sgdisk --zap-all $DISK

Create EFI system partition (for use now or in the future):

sgdisk -n1:1M:+1G -t1:EF00 $DISK

Create BIOS boot partition, skip if you don't use this:

sgdisk -a1 -n5:24K:+1000K -t5:EF02 $DISK

Create main partition:

sgdisk -n2:0:0   $DISK

cryptsetup mapper

This naming scheme is taken from Debian installer

boot_partuuid=`blkid -s PARTUUID -o value $DISK-part2`
boot_mapper_name=cryptroot-luks1-partuuid-$boot_partuuid
boot_mapper_path=/dev/mapper/$boot_mapper_name

Format and open LUKS container

cryptsetup luksFormat --type luks1 $DISK-part2
cryptsetup open $DISK-part2 $boot_mapper_name

Format the LUKS container as Btrfs

mkfs.btrfs $boot_mapper_path
mount $boot_mapper_path $INST_MNT

Create subvolumes

The idea is to separate persistent data from root file system rollbacks.

System is installed to snapshot 0.

cd $INST_MNT

btrfs subvolume create @
mkdir @/0
btrfs subvolume create @/0/snapshot

for i in {home,root,srv,usr,usr/local,swap,var};
do btrfs subvolume create @$i;
done

# exclude these dirs under /var from system snapshot
for i in {tmp,spool,log};
do btrfs subvolume create @var/$i;
done

Mount subvolumes

cd ~
umount $INST_MNT
mount $boot_mapper_path $INST_MNT -o subvol=/@/0/snapshot,compress-force=zstd,noatime,space_cache=v2

mkdir -p $INST_MNT/{.snapshots,home,root,srv,tmp,usr/local,swap}

mkdir -p $INST_MNT/var/{tmp,spool,log}
mount $boot_mapper_path $INST_MNT/.snapshots/ -o subvol=@,compress-force=zstd,noatime,space_cache=v2

# mount subvolumes
# separate /{home,root,srv,swap,usr/local} from root filesystem
for i in {home,root,srv,swap,usr/local};
do mount $boot_mapper_path $INST_MNT/$i -o subvol=@$i,compress-force=zstd,noatime,space_cache=v2;
done

# separate /var/{tmp,spool,log} from root filesystem
for i in {tmp,spool,log};
do mount $boot_mapper_path $INST_MNT/var/$i -o subvol=@var/$i,compress-force=zstd,noatime,space_cache=v2;
done

// WARNING: NOT USE /usr as dedicated mountpoint - it's road to FAIL at boot.

Disable Copy-on-Write

for i in {swap,};
do chattr +C $INST_MNT/$i;
done

Format and mount EFI partition

mkfs.vfat -n EFI $DISK-part1
mkdir -p $INST_MNT/boot/efi
mount $DISK-part1 $INST_MNT/boot/efi

Packages

Only essential packages given here

pacstrap $INST_MNT base vi mandoc grub cryptsetup btrfs-progs snapper snap-pac grub grub-btrfs
chmod 750 $INST_MNT/root
chmod 1777 $INST_MNT/var/tmp/

Install kernel

pacstrap $INST_MNT $INST_LINVAR

If your computer has hardware that requires firmware to run:

pacstrap $INST_MNT linux-firmware

If you boot your computer with EFI:

pacstrap $INST_MNT dosfstools efibootmgr

Microcode:

pacstrap $INST_MNT amd-ucode
pacstrap $INST_MNT intel-ucode

For other optional packages, see ArchWiki.

System Configuration

First, generate fstab

genfstab -U $INST_MNT >> $INST_MNT/etc/fstab

Remove hard-coded system subvolume. If not removed, system will ignore btrfs default-id setting, which is used by snapper when rolling back.

sed -i 's|,subvolid=258,subvol=/@/0/snapshot,subvol=@/0/snapshot||g' $INST_MNT/etc/fstab

Create LUKS key for initramfs. Without this, you will need to enter the password twice: once in GRUB, once in initramfs.

mkdir -p $INST_MNT/lukskey
dd bs=512 count=8 if=/dev/urandom of=$INST_MNT/lukskey/crypto_keyfile.bin
chmod 600 $INST_MNT/lukskey/crypto_keyfile.bin
cryptsetup luksAddKey $DISK-part2 $INST_MNT/lukskey/crypto_keyfile.bin
chmod 700 $INST_MNT/lukskey

Configure initramfs.

Embed keyfile.
Embed btrfs binary.
Add encrypt hook to decrypt LUKS at boot.
Add OverlayFS for Btrfs snapshots, see [4].
sd-encrypt hook has a few extra features, but depends on systemd, not documented here. See Dm-crypt/System_configuration#Using_sd-encrypt_hook
if you use NVME : in MODULES section set 'vmd' (otherwise - boot failure)

cryptkey=/lukskey/crypto_keyfile.bin
mv $INST_MNT/etc/mkinitcpio.conf $INST_MNT/etc/mkinitcpio.conf.original
tee $INST_MNT/etc/mkinitcpio.conf << EOF
BINARIES=(/usr/bin/btrfs)
FILES=($cryptkey)
HOOKS=(base udev autodetect modconf block encrypt filesystems keyboard fsck grub-btrfs-overlayfs)
EOF

Edit /etc/default/grub

Enable cryptodisk for GRUB
Add kernel command line to decrypt and map LUKS
Specify LUKS key
Specify root device

echo "GRUB_ENABLE_CRYPTODISK=y" >> $INST_MNT/etc/default/grub
echo "GRUB_CMDLINE_LINUX=\"cryptdevice=PARTUUID=$boot_partuuid:$boot_mapper_name root=$boot_mapper_path cryptkey=rootfs:$cryptkey\"" >> $INST_MNT/etc/default/grub

Optional: Create swapfile. Adjust the file size if needed.

touch $INST_MNT/swap/swapfile
truncate -s 0 $INST_MNT/swap/swapfile
chattr +C $INST_MNT/swap/swapfile
btrfs property set $INST_MNT/swap/swapfile compression none
dd if=/dev/zero of=$INST_MNT/swap/swapfile bs=1M count=8192 status=progress
chmod 700 $INST_MNT/swap
chmod 600 $INST_MNT/swap/swapfile
mkswap $INST_MNT/swap/swapfile
echo /swap/swapfile none swap defaults 0 0 >> $INST_MNT/etc/fstab

Host name:

echo $INST_HOST > $INST_MNT/etc/hostname

Configure the network interface: Find the interface name:

ip link

Store it in a variable:

INET=enp1s0

Create network configuration:

tee $INST_MNT/etc/systemd/network/20-default.network <<EOF

[Match]
Name=$INET

[Network]
DHCP=yes
EOF

Customize this file if the system is not a DHCP client. See Network Configuration.

Timezone:

ln -sf $INST_TZ $INST_MNT/etc/localtime
hwclock --systohc

Locale:

echo "en_US.UTF-8 UTF-8" >> $INST_MNT/etc/locale.gen
echo "LANG=en_US.UTF-8" >> $INST_MNT/etc/locale.conf

Other locales should be added after reboot.

Chroot:

arch-chroot $INST_MNT /usr/bin/env  DISK=$DISK \
  INST_UUID=$INST_UUID bash --login

Apply locales:

locale-gen

Enable networking:

systemctl enable systemd-networkd systemd-resolved

Set root password:

passwd

Generate initramfs:

mkinitcpio -P

Enable btrfs service

systemctl enable grub-btrfs.path

Enable snapper

umount /.snapshots/
rmdir /.snapshots/
snapper --no-dbus -c root create-config /
rmdir /.snapshots/
mkdir /.snapshots/
mount /.snapshots/
snapper --no-dbus -c home create-config /home/
systemctl enable /lib/systemd/system/snapper-*

Optionally add a normal user, use --btrfs-subvolume-home:

useradd -s /bin/bash -U -G wheel,video -m --btrfs-subvolume-home myuser
snapper --no-dbus -c myuser create-config /home/myuser

GRUB installation

EFI

grub-install

Some motherboards does not properly recognize GRUB boot entry, to ensure that your computer will boot, also install GRUB to fallback location with:

grub-install --removable

BIOS

grub-install $DISK

Generate GRUB menu

grub-mkconfig -o /boot/grub/grub.cfg

Finish Installation

exit
mount | grep "$INST_MNT/" | tac | cut -d' ' -f3 | xargs -i{} umount -lf {}
umount $INST_MNT
cryptsetup close $boot_mapper_name
reboot

Create root filesystem snapshot

snapper -c root create

Additional options for create, such as --description, see snapper help.

Rollback root filesystem

If the system is broken, reboot, select a bootable entry in GRUB snapshot list. Your computer will then boot with a read-only root filesystem with a writable OverlayFS on it.

Run snapper -c root list to find out which snapshot you want to rollback to.

 # | Type   | Pre # | Date                            | User | Cleanup | Description        | Userdata
---+--------+-------+---------------------------------+------+---------+--------------------+---------
0- | single |       |                                 | root |         | current            |
1  | single |       | Tue 09 Feb 2021 09:44:55 PM +08 | root | number  | boot               |
2  | single |       | Tue 09 Feb 2021 09:49:30 PM +08 | root | number  | boot               |
3  | pre    |       | Tue 09 Feb 2021 09:49:51 PM +08 | root | number  | pacman -S dropbear |
4  | post   |     3 | Tue 09 Feb 2021 09:49:52 PM +08 | root | number  | dropbear           |

If we want to rollback to 3, run snapper --ambit classic rollback 3 to rollback.

Ambit is classic.
Creating read-only snapshot of current system. (Snapshot 5.)
Creating read-write snapshot of snapshot 3. (Snapshot 6.)
Setting default subvolume to snapshot 6.

Remember the new default subvolume number 6. This will be the new / the computer will boot into. Now run grub-mkconfig -o /boot/grub/grub.cfg to let GRUB know about the new snapshots. Reboot, select snapshot 6 from GRUB menu.

After reboot, run

grub-install
grub-mkconfig -o /boot/grub/grub.cfg

to make snapshot 6 the default boot volume. Then you won't need to manually select it again on next reboot.