Diskless system

From ArchWiki
Revision as of 02:41, 9 December 2012 by Buhman (talk | contribs) (DHCP: can't believe I forgot this)
Jump to: navigation, search

Template:Article summary start Template:Article summary text Template:Article summary heading Template:Article summary wiki Template:Article summary wiki Template:Article summary wiki Template:Article summary end

Merge-arrows-2.pngThis article or section is a candidate for merging with PXE.Merge-arrows-2.png

Notes: needs to be rewritten into sub-articles; duplicated information (Discuss in Talk:Diskless system#)

Contrast to the official installation guide, which concentrates on an arch installation on local storage, here the installation is to be placed on network storage.

Server configuration

You will need a DHCP server to setup networking, as well as a TFTP server to transfer the boot image (a requirement of all PXE option roms). Unlike PXE which is designed for quick-and-dirty/temporary setups to boot installation media, here you're actually doing a network installation, which is more-permanent, and thus calls for doing things properly.


Install ISC dhcp.

# pacman -Syu dhcp

Configure ISC DHCP.

# vim /etc/dhcpd.conf
allow booting;
allow bootp;


option domain-name-servers;

    filename "pxelinux.0";

    subnet netmask {
        option routers;
Note: next-server should be the address of the TFTP server; everything else should be changed to match your network


# systemctl start dhcpd4


The TFTP server will be used to transfer the kernel, initramfs, and pxelinux to the client.

Install tftp-hpa.

# pacman -Syu tftp-hpa
Note: Change /srv/arch/boot to /srv/boot if you're going to use NBD, because you'll be unable to mount your root filesystem while in use


Copy the tftp unit files in /usr/lib/systemd/system to /etc/systemd/system; the former gets overwritten when systemd is updated. The systemd article talks in more detail about customizing unit files.

# cp /usr/lib/systemd/system/tftpd.s* /etc/systemd/system/

Next we tell tftpd where the tftp root is: /srv/arch/boot.

# vim /etc/systemd/system/tftpd.service
Description=hpa's original TFTP daemon

ExecStart=/usr/sbin/in.tftpd -s /srv/arch/boot/

Start tftpd :

# systemctl enable tftpd.socket
# systemctl start tftpd.socket

Network storage

The primary difference between using NFS and NBD is while with both you can in fact have multiple clients using the same installation, with NBD (by the nature of manipulating a filesystem directly) you'll need to use the copyonwrite mode to do so, which ends up discarding all writes on client disconnected. In some situations however, this might be highly desirable.


Install nfs-utils on the server.

# pacman -Syu nfs-utils

You'll need to add the root of your arch installation to your NFS exports:

# vim /etc/exports
/srv/arch *(rw,fsid=0,no_root_squash,no_subtree_check,async)
Note: If you're worried about data loss in the event of network failure, replace async with sync--additional options can be found in the NFS article.

Next start NFS.

# systemctl start rpc-idmapd.service rpc-mountd.service


Install nbd.

# pacman -Syu nbd

Configure nbd.

# vim /etc/nbd-server/config
    exportname = /srv/arch.img
    copyonwrite = false
Note: Set copyonwrite to true if you want to have multiple clients using the same NBD share simultaneously; refer to man 5 nbd-server for more details.

Start nbd.

# systemctl start nbd

Client installation

Directory setup

Create a sparse file of at least 1 gigabyte, and create an ext4 filesystem on it (you can of course also use a real block device or LVM if you so desire).

# truncate -s 1G /srv/arch.img
# mkfs.ext4 /srv/arch.img
# export root=/srv/arch
# mkdir -p "$root"
# mount /srv/arch.img "$root"
Note: Creating a separate filesystem is required for NBD but optional for NFS and can be skipped/ignored.

Bootstrapping installation

Install devtools and arch-install-scripts, and run mkarchoot.

# pacman -Syu devtools arch-install-scripts
# mkarchroot -f --arch x86_64 "$root" base mkinitcpio-nfs-utils nfs-utils

Replace x86_64 with i686 as appropriate for your target hardware.

Note: If you're using NBD, you'll want to replace nfs-utils with mkinitcpio-nbdAUR. mkinitcpio-nfs-utils is still required--ipconfig used in early-boot is provided only by the latter.

Trivial modifications to the net hook are required in order for NFSv4 mounting to work (not supported by nfsmount--the default for the net hook).

# sed s/nfsmount/mount.nfs4/ "$root/usr/lib/initcpio/hooks/net" | tee "$root/usr/lib/initcpio/hooks/net_nfs4"
# cp "$root/usr/lib/initcpio/install/{net,net_nfs4}"

The copy of net is unfortunately needed so it does not get overwritten when mkinitcpio-nfs-utils is updated on the client installation.

The newly-created net_nfs4 hook needs to be added to the HOOKS array in mkinitcpio.conf, and mount.nfs4 to BINARIES.

# vim "$root/etc/mkinitcpio.conf"
HOOKS="base udev autodetect net_nfs4 filesystems"
Note: The upstream net hook is sufficient for NBD installations; you will also need to append nbd to your HOOKS array; mount.nfs4 is also not needed

The initramfs now needs to be rebuilt; the easiest way to do this is via arch-chroot.

# arch-chroot "$root" /bin/bash
(chroot) # mkinitcpio -p linux
(chroot) # exit

Client configuration

In addition to the setup mentioned here, you should also set up your hostname, timezone, locale, and keymap, and follow any other relevant parts of the Installation Guide.


Install syslinux.

# pacman -Syu syslinux

Copy the pxelinux bootloader (provided by the syslinux package) to the boot directory of the client.

# cp /usr/lib/syslinux/pxelinux.0 "$root/boot"
# mkdir "$root/boot/pxelinux.cfg"

We also created the pxelinux.cfg directory, which is where pxelinux searches for configuration files by default. Because we don't want to discriminate between different host MACs, we then create the default configuration.

# vim "$root/boot/pxelinux.cfg/default"
default linux

label linux
kernel vmlinuz-linux
append initrd=initramfs-linux.img ip=:::::eth0:dhcp nfsroot=

Or if you are using NBD, use the following append line:

append initrd=initramfs-linux.img ip=:::::eth0:dhcp nbd_host= nbd_name=arch
Note: You will need to change nbd_host and/or nfsroot, respectively, to match your network configuration (the address of the NFS/NBD server)

The pxelinux configuration syntax identical to syslinux; refer to the upstream documentation for more information.

The kernel and initramfs will be transferred via TFTP, so the paths to those are going to be relative to the TFTP root. Otherwise, the root filesystem is going to be the NFS mount itself, so those are relative to the root of the NFS server.

VFS mountpoints

Tango-inaccurate.pngThe factual accuracy of this article or section is disputed.Tango-inaccurate.png

Reason: Might no longer be necessary (Discuss in Talk:Diskless system#)

Add hacks to your fstab for the root filesystem and devpts.

# vim "$root/etc/fstab"
none    /           none      defaults                        0 0
none    /dev/pts    devpts    mode=0620,gid=5,nosuid,noexec   0 0

One might also find it beneficial to put /var/log in tmpfs, so that messages from different hosts can be distinguished when viewing logs locally.

tmpfs   /var/log    tmpfs     nodev,nosuid                    0 0

Client boot


Tango-inaccurate.pngThe factual accuracy of this article or section is disputed.Tango-inaccurate.png

Reason: When using COW on the server, the clients all effectively have read-only mounts of the original filesystem; it should theoretically be safe to do a read-write mount on the NBD server (Discuss in Talk:Diskless system#)

If you're using NBD, you'll need to umount the arch.img before/while you boot your client.

This makes things particularly interesting when it comes to kernel updates. You can't have your client filesystem mounted while you're booting a client, but that also means you need to use a kernel separate from your client filesystem in order to build it.

You'll need to first copy $root/boot from the client installation to your tftp root (i.e. /srv/boot).

# cp -r "$root/boot" /srv/boot

You'll then need to umount $root before you start the client.

# umount "$root"
Note: To update the kernel in this setup, you either need to mount /srv/boot using NFS in fstab on the client (prior to doing the kernel update) or mount your client filesystem after the client has disconnected from NBD


kernel.org: Mounting the root filesystem via NFS (nfsroot)

syslinux.org: pxelinux FAQ