User:M0p/Root on ZFS Native Encryption with Boot Environment/Layout

From ArchWiki

For a Root on ZFS installation, creating datasets with a rollback-compatible layout is arguably the most important procedure. Here, we need to properly separate any directory which you want to exclude from a rollback of /, such as /home, /var/log and /var/lib/libvirt. If not excluded, upon entering a boot environment before the upgrade, your home directory will become inaccessible: you can get your data back only by mounting the previous broken system dataset.

We will create the following layout for system directories

rpool/ROOT/default    on    /
bpool/BOOT/default    on    /boot
rpool/DATA/default    on    / (canmount=off, container for persistent datasets)
rpool/DATA/default/home    on    /home

Common directories containing variable files also need to be excluded, including

/usr/local
/var/log
/var/spool
/var/tmp
/var/lib/libvirt
/var/lib/docker    # docker manages its own dataset
/var/lib/lxc       # same as above

Note that usr,var,var/lib themselves and many subfolders must not be excluded from root snapshot, therefore rpool/ROOT/default/{usr,var,var/lib} are created as placeholders with canmount=off, same as rpool/ROOT.

After a pacman transaction, a boot environment is created by creating a read-only snapshot of the current root and then cloning the snapshot to a writable dataset

rpool/ROOT/default@install-firefox    # readonly snapshot
bpool/BOOT/default@install-firefox    # readonly snapshot
rpool/ROOT/install-firefox     on     / (not mounted, canmount=noauto)
bpool/BOOT/install-firefox     on     /boot (not mounted, canmount=noauto)

After rebooting into the install-firefox boot environment, a properly configured layout should result in the following mount output

rpool/ROOT/install-firefox    on    /
bpool/BOOT/install-firefox    on    /boot
rpool/DATA/default/home/username   on    /home/username
rpool/DATA/default/var/log    on    /var/log

As you can see, nothing ever gets stored in rpool/ROOT or bpool/BOOT, they are merely placeholders for other boot environments. Therefore the properties canmount=off and mountpoint=none is set for both of them.

camount=noauto for root dataset

For

rpool/ROOT/default

and their clones, canmount=noauto must be set to prevent unexpected mounting behavior.

Suppose we boot from install-firefox boot environment, rpool/ROOT/install-firefox should be mounted at /, but if canmount=on is also set for rpool/ROOT/default, ZFS will automatically mount rpool/ROOT/default on / instead when importing rpool, resulting in

rpool/ROOT/default             on     /      (mounted when importing, if canmount=on)
rpool/ROOT/install-firefox     on     /      (mounted by systemd)

resulting a convoluted system. The same goes for /boot dataset.

mountpoint=legacy for boot dataset

For

bpool/BOOT/default

and their clones, mountpoint=legacy must be set to deterministically mount /boot upon entering another boot environment.

Mounting is handled by /etc/fstab and boot environment manager bieazAUR will edit the file when creating a boot environment.

Example

[yc@yc-eb820g3 ~]$ findmnt -t zfs -o TARGET,SOURCE
TARGET                   SOURCE
/                        rpool_13sijd/ROOT/default
├─/boot                  bpool_13sijd/BOOT/default
├─/root                  rpool_13sijd/DATA/default/root
├─/home                  rpool_13sijd/DATA/default/home
│ └─/home/yc             rpool_13sijd/DATA/default/home/yc
│   ├─/home/yc/.cache    rpool_13sijd/DATA/default/home/yc/.cache
│   ├─/home/yc/.mozilla  rpool_13sijd/DATA/default/home/yc/.mozilla
│   └─/home/yc/Downloads rpool_13sijd/DATA/default/home/yc/Downloads
├─/srv                   rpool_13sijd/DATA/default/srv
├─/var/lib/nfs           rpool_13sijd/DATA/default/var/lib/nfs
├─/var/lib/libvirt       rpool_13sijd/DATA/default/var/lib/libvirt
├─/usr/local             rpool_13sijd/DATA/default/usr/local
├─/var/lib/lxc           rpool_13sijd/DATA/default/var/lib/lxc
├─/var/spool             rpool_13sijd/DATA/default/var/spool
├─/var/log               rpool_13sijd/DATA/default/var/log
└─/var/tmp               rpool_13sijd/DATA/default/var/tmp
[yc@yc-eb820g3 ~]$ zfs list -o name,mounted,mountpoint,canmount
NAME                                         MOUNTED  MOUNTPOINT          CANMOUNT
bpool_13sijd                                 no       /boot               off
bpool_13sijd/BOOT                            no       none                off
bpool_13sijd/BOOT/default                    yes      legacy              noauto
bpool_13sijd/BOOT/pac-3edj06                 no       legacy              noauto
bpool_13sijd/BOOT/pac-7n4pwg                 no       legacy              noauto
bpool_13sijd/BOOT/pac-r65yck                 no       legacy              noauto
bpool_13sijd/BOOT/pac-rjw48h                 no       legacy              noauto
rpool_13sijd                                 no       /                   off
rpool_13sijd/DATA                            no       none                off
rpool_13sijd/DATA/default                    no       /                   off
rpool_13sijd/DATA/default/home               yes      /home               on
rpool_13sijd/DATA/default/home/yc            yes      /home/yc            on
rpool_13sijd/DATA/default/home/yc/.cache     yes      /home/yc/.cache     on
rpool_13sijd/DATA/default/home/yc/.mozilla   yes      /home/yc/.mozilla   on
rpool_13sijd/DATA/default/home/yc/Downloads  yes      /home/yc/Downloads  on
rpool_13sijd/DATA/default/root               yes      /root               on
rpool_13sijd/DATA/default/srv                yes      /srv                on
rpool_13sijd/DATA/default/usr                no       /usr                off
rpool_13sijd/DATA/default/usr/local          yes      /usr/local          on
rpool_13sijd/DATA/default/var                no       /var                off
rpool_13sijd/DATA/default/var/lib            no       /var/lib            off
rpool_13sijd/DATA/default/var/lib/libvirt    yes      /var/lib/libvirt    on
rpool_13sijd/DATA/default/var/lib/lxc        yes      /var/lib/lxc        on
rpool_13sijd/DATA/default/var/lib/nfs        yes      /var/lib/nfs        on
rpool_13sijd/DATA/default/var/log            yes      /var/log            on
rpool_13sijd/DATA/default/var/spool          yes      /var/spool          on
rpool_13sijd/DATA/default/var/tmp            yes      /var/tmp            on
rpool_13sijd/ROOT                            no       none                off
rpool_13sijd/ROOT/default                    yes      /                   noauto
rpool_13sijd/ROOT/pac-3edj06                 no       /                   noauto
rpool_13sijd/ROOT/pac-7n4pwg                 no       /                   noauto
rpool_13sijd/ROOT/pac-r65yck                 no       /                   noauto
rpool_13sijd/ROOT/pac-rjw48h                 no       /                   noauto
[yc@yc-eb820g3 ~]$ bieaz info
ZFS property for label:    org:bieaz
Current boot environment:  default
Root dataset container:    /ROOT/
Root dataset full path:    rpool_13sijd/ROOT/default
Boot dataset container:    /BOOT/
Boot dataset full path:    bpool_13sijd/BOOT/default
Root pool name:            rpool_13sijd
Boot pool name:            bpool_13sijd