Difference between revisions of "Diskless system"

From ArchWiki
Jump to: navigation, search
m (DHCP)
m (Bootstrapping installation: added accuracy flag)
 
(173 intermediate revisions by 19 users not shown)
Line 1: Line 1:
[[Category:Boot process]]
+
[[Category:Getting and installing Arch]]
[[Category:File systems]]
+
[[ja:ディスクレスシステム]]
[[Category:Networking]]
+
[[zh-cn:Diskless system]]
{{Article summary start}}
+
{{Related articles start}}
{{Article summary text|Detailed explanation of a PXE booted Arch installation with networked root filesystem.}}
+
{{Related|NFS}}
{{Article summary heading|Related}}
+
{{Related|NFS Troubleshooting}}
{{Article summary wiki|NFS}}
+
{{Related|PXE}}
{{Article summary wiki|PXE}}
+
{{Related|Mkinitcpio#Using_net}}
{{Article summary wiki|Mkinitcpio#Using_net}}
+
{{Related articles end}}
{{Article summary end}}
+
{{Merge|PXE|needs to be rewritten into sub-articles; duplicated information}}
+
  
Contrast to the [[Installation Guide|official installation guide]], which concentrates on an arch installation on local storage, here the installation is to be placed on network storage.
+
From [[Wikipedia:Diskless node]]
 +
:''A diskless node (or diskless workstation) is a workstation or personal computer without disk drives, which employs network booting to load its operating system from a server.''
  
==Server configuration==
+
== Server configuration ==
  
You will need a DHCP server to setup networking, as well as a TFTP server to transfer the boot image (a requirement of all PXE option roms). Unlike [[PXE]] which is designed for quick-and-dirty/temporary setups to boot installation media, here you're actually doing a ''network'' installation, which is more-permanent, and thus calls for doing things properly.
+
First of all, we must install the following components:
 +
* A [[Dhcpd|DHCP]] server to assign IP addresses to our diskless nodes.
 +
* A [[TFTP]] server to transfer the boot image (a requirement of all PXE option roms).
 +
* A form of network storage ([[NFS]] or NBD) to export the Arch installation to the diskless node.
  
===DHCP===
+
{{Note|{{pkg|dnsmasq}} is capable of simultaneously acting as both DHCP and TFTP server. For more information, see the [[dnsmasq]] article.}}
{{Note|{{pkg|dnsmasq}} can provide both DHCP and TFTP eliminating the need for a separate TFTP package. See the [[PXE#DHCP_.2B_TFTP]] for information on setting this up.}}
+
  
Install ISC {{pkg|dhcp}}.
+
=== DHCP ===
  
{{bc|# pacman -Syu dhcp}}
+
Install ISC {{Pkg|dhcp}} and configure it:
  
Configure ISC DHCP.
+
{{hc|/etc/dhcpd.conf|2=
 
+
allow booting;
{{hc|# vim /etc/dhcpd.conf|2=
+
<nowiki>allow booting;
+
 
allow bootp;
 
allow bootp;
  
Line 34: Line 33:
 
option domain-name-servers 10.0.0.1;
 
option domain-name-servers 10.0.0.1;
  
group
+
option architecture code 93 = unsigned integer 16;
{
+
 
 +
group {
 
     next-server 10.0.0.1;
 
     next-server 10.0.0.1;
     filename "pxelinux.0";
+
 
 +
     if option architecture = 00:07 {
 +
        filename "/grub/x86_64-efi/core.efi";
 +
    } else {
 +
        filename "/grub/i386-pc/core.0";
 +
    }
  
 
     subnet 10.0.0.0 netmask 255.255.255.0 {
 
     subnet 10.0.0.0 netmask 255.255.255.0 {
Line 43: Line 48:
 
         range 10.0.0.128 10.0.0.254;
 
         range 10.0.0.128 10.0.0.254;
 
     }
 
     }
}</nowiki>}}
+
}
 +
}}
  
 
{{Note|{{ic|next-server}} should be the address of the TFTP server; everything else should be changed to match your network}}
 
{{Note|{{ic|next-server}} should be the address of the TFTP server; everything else should be changed to match your network}}
  
Start ISC DHCP.
+
RFC 4578 defines the "Client System Architecture Type" dhcp option. In the above configuration, if the PXE client requests an x86_64-efi binary (type 0x7), we appropriately give them one, otherwise falling back to the legacy binary. This allows both UEFI and legacy BIOS clients to boot simultaneously on the same network segment.
  
{{bc|# systemctl start dhcpd}}
+
Start ISC DHCP [[systemd]] service.
  
===TFTP===
+
=== TFTP ===
  
The TFTP server will be used to transfer the kernel, initramfs, and pxelinux to the client.
+
The TFTP server will be used to transfer the bootloader, kernel, and initramfs to the client.
  
Install {{pkg|tftp-hpa}}.
+
Set the TFTP root to {{ic|/srv/arch/boot}}. See [[Tftpd server]] for installation and configuration.
  
{{bc|# pacman -Syu tftp-hpa}}
+
=== Network storage ===
  
{{Note|Change {{ic|/mnt/arch/boot}} to {{ic|/mnt/boot}} if you're going to use NBD, because you'll be unable to mount your root filesystem while in use}}
+
The primary difference between using NFS and NBD is while with both you can in fact have multiple clients using the same installation, with NBD (by the nature of manipulating a filesystem directly) you will need to use the {{ic|copyonwrite}} mode to do so, which ends up discarding all writes on client disconnect. In some situations however, this might be highly desirable.
  
====Configuration====
+
==== NFS ====
  
Copy the tftp unit files in {{ic|/usr/lib/systemd}} to {{ic|/etc/systemd}}; the former gets overwritten when systemd is updated. The [[systemd]] article talks in more detail about customizing unit files.
+
Install {{Pkg|nfs-utils}} on the server.
  
{{bc|# cp /usr/lib/systemd/system/tftpd.s* /etc/systemd/system/}}
+
You will need to add the root of your Arch installation to your [[NFS]] exports:
  
{{hc|# vim /etc/systemd/system/tftpd.service|2=
+
{{hc|/etc/exports|2=
<nowiki>
+
/srv      *(rw,fsid=0,no_root_squash,no_subtree_check)
[Unit]
+
/srv/arch  *(rw,no_root_squash,no_subtree_check)
Description=hpa's original TFTP daemon
+
}}
  
[Service]
+
Next, start NFS services: {{ic|rpc-idmapd}} {{ic|rpc-mountd}}.
ExecStart=/usr/sbin/in.tftpd -s /mnt/arch/boot/
+
StandardInput=socket
+
StandardOutput=inherit
+
StandardError=journal
+
</nowiki>}}
+
  
Start tftpd :
+
==== NBD ====
  
{{bc|1=
+
Install {{Pkg|nbd}} and configure it.
# systemctl enable tftpd.socket
+
# systemctl start tftpd.socket}}
+
  
===Network storage===
+
{{hc|# vim /etc/nbd-server/config|2=
 
+
[generic]
The primary difference between using NFS and NBD is while with both you can in fact have multiple clients using the same installation, with NBD (by the nature of manipulating a filesystem directly) you'll need to use the {{ic|copyonwrite}} mode to do so, which ends up discarding all writes on client disconnected. In some situations however, this might be highly desirable.
+
    user = nbd
 
+
    group = nbd
====NFS====
+
[arch]
 
+
    exportname = /srv/arch.img
Install {{pkg|nfs-utils}} on the server.
+
    copyonwrite = false
 
+
{{bc|# pacman -Syu nfs-utils}}
+
 
+
You'll need to add the root of your arch installation to your NFS exports.
+
 
+
{{hc|# vim /etc/exports|2=
+
/mnt/arch *(rw,fsid=0,no_root_squash,no_subtree_check,async)
+
 
}}
 
}}
  
Next start NFS.
+
{{Note|Set {{ic|copyonwrite}} to true if you want to have multiple clients using the same NBD share simultaneously; refer to {{ic|man 5 nbd-server}} for more details.}}
  
{{bc|# systemctl start nfsd.service rpc-idmapd.service rpc-mountd.service rpcbind.service}}
+
Start {{ic|nbd}} systemd service.
  
Additional information can be found in the [[NFS]] article.
+
== Client installation ==
  
====NBD====
+
Next we will create a full Arch Linux installation in a subdirectory on the server. During boot, the diskless client will get an IP address from the DHCP server, then boot from the host using PXE and mount this installation as its root.
  
Install {{pkg|nbd}}.
+
=== Directory setup ===
  
{{bc|# pacman -Syu nbd}}
+
Create a [[Wikipedia: Sparse file|sparse file]] of at least 1 gigabyte, and create a btrfs filesystem on it (you can of course also use a real block device or [[LVM]] if you so desire).
  
Configure nbd.
+
# truncate -s 1G /srv/arch.img
 +
# mkfs.btrfs /srv/arch.img
 +
# export root=/srv/arch
 +
# mkdir -p "$root"
 +
# mount -o loop,discard,compress=lzo /srv/arch.img "$root"
  
{{hc|# vim /etc/nbd-server/config|2=
+
{{Note|Creating a separate filesystem is required for NBD but optional for NFS and can be skipped/ignored.}}
[generic]
+
[arch]
+
    exportname = /mnt/arch.img
+
    copyonwrite = false}}
+
  
{{note|Set {{ic|copyonwrite}} to true if you want to have multiple clients using the same NBD share simultaneously; refer to man 5 nbd-server for more details.}}
+
=== Bootstrapping installation ===
  
Start nbd.
+
{{Accuracy|{{ic|mkarchroot}} is mentioned only in the following sentence.}}
  
{{bc|# systemctl start nbd}}
+
Install {{Pkg|devtools}} and {{Pkg|arch-install-scripts}}, and run {{ic|mkarchroot}}.
  
==Client installation==
+
# pacstrap -d "$root" base mkinitcpio-nfs-utils nfs-utils
  
===Directory setup===
+
{{Note|In all cases {{Pkg|mkinitcpio-nfs-utils}} is still required. {{ic|ipconfig}} used in early-boot is provided only by the latter.}}
  
Create a [[Wikipedia: Sparse file|sparse file]] of at least 5 gigabytes, and create an ext4 filesystem on it (you can of course also use a real block device or [[LVM]] if you so desire).
+
Now the initramfs needs to be constructed.
  
{{bc|1=
+
==== NFS ====
# truncate -s 5G /mnt/arch.img
+
# mkfs.ext4 /mnt/arch.img
+
# mkdir -p /mnt/arch
+
# mount /mnt/arch.img /mnt/arch}}
+
  
{{Note|Creating a separate filesystem is required for NBD but optional for NFS and can be skipped/ignored}}
+
Trivial modifications to the {{ic|net}} hook are required in order for NFSv4 mounting to work (not supported by {{ic|nfsmount}}--the default for the {{ic|net}} hook).
  
First, create a directory that will contain the Arch installation; replace {{ic|/mnt/arch}} with wherever you'd like to put your installation.
+
# sed s/nfsmount/mount.nfs4/ "$root/usr/lib/initcpio/hooks/net" > "$root/usr/lib/initcpio/hooks/net_nfs4"
 +
# cp $root/usr/lib/initcpio/install/net{,_nfs4}
  
{{bc|1=
+
The copy of {{ic|net}} is unfortunately needed so it does not get overwritten when {{pkg|mkinitcpio-nfs-utils}} is updated on the client installation.
<nowiki>
+
# export root=/mnt/arch
+
# mkdir -p $root/{proc,sys,run,tmp}
+
# mkdir -p $root/dev/{pts,shm}
+
</nowiki>}}
+
  
This also creates the directories that will be used for the API filesystem mountpoints later. Next we create the directory that pacman stores its database.
+
Edit {{ic|$root/etc/mkinitcpio.conf}} and add {{ic|nfsv4}} to {{ic|MODULES}}, {{ic|net_nfs4}} to {{ic|HOOKS}}, and {{ic|/usr/bin/mount.nfs4}} to {{ic|BINARIES}}.
  
{{bc|# mkdir -p "$root/var/lib/pacman"}}
+
Next, we [[chroot]] our installation and run ''mkinitcpio'':
  
===Bootstrapping installation===
+
# arch-chroot "$root" mkinitcpio -p linux
  
{{Note|You can also use {{ic|arch-chroot}} and {{ic|pacstrap}} from {{pkg|arch-install-scripts}} rather than {{ic|chroot}} and {{pkg|pacman}} directly}}
+
==== NBD ====
  
Mount the Linux API filesystems.
+
The {{AUR|mkinitcpio-nbd}} package needs to be installed on the client. Build it with ''makepkg'' and install it:
  
{{bc|1=
+
# pacman --root "$root" --dbpath "$root/var/lib/pacman" -U mkinitcpio-nbd-0.4-1-any.pkg.tar.xz
# mount -t proc proc "$root/proc" -o nosuid,noexec,nodev
+
# mount -t sysfs sys "$root/sys" -o nosuid,noexec,nodev
+
# mount -t devtmpfs udev "$root/dev" -o mode=0755,nosuid
+
# mount -t devpts devpts "$root/dev/pts" -o mode=0620,gid=5,nosuid,noexec}}
+
  
Install the essential packages needed.
+
You will then need to append {{ic|nbd}} to your {{ic|HOOKS}} array after {{ic|net}}; {{ic|net}} will configure your networking for you, but not attempt a NFS mount if {{ic|nfsroot}} is not specified in the kernel line.
  
{{bc|# pacman -Syu --root "$root" --dbpath "$root/var/lib/pacman" base base-devel --arch x86_64}}
+
== Client configuration ==
  
Replace {{ic|x86_64}} with {{ic|i686}} as appropriate for your target hardware.
+
In addition to the setup mentioned here, you should also set up your [[HOSTNAME#Set_the_host_name|hostname]], [[Timezone#Time_Zone|timezone]], [[Locale#Setting_system-wide_locale|locale]], and [[keymap]], and follow any other relevant parts of the [[Installation guide]].
  
You'll need to install either {{pkg|mkinitcpio-nfs-utils}} or {{aur|mkinitcpio-nbd}} depending on whether you are using NFS or NBD.
+
=== Bootloader ===
  
Next, edit {{ic|"$root/etc/mkinitcpio.conf"}} and add {{ic|nfsv3}} to the {{ic|MODULES}} array, and add {{ic|net}} after {{ic|udev}} to the {{ic|HOOKS}} array.
+
==== GRUB ====
  
{{hc|# vim "$root/etc/mkinitcpio.conf"|2=
+
{{Merge|GRUB|}}
MODULES="nfsv3"
+
HOOKS="base udev '''net''' autodetect filesystems"}}
+
  
{{Note|Add {{ic|nbd}} after the {{ic|net}} hook if you are using NBD.}}
+
Though poorly documented, GRUB supports being loaded via PXE.
  
The initramfs now needs to be rebuilt; the easiest way to do this is [[chroot]].
+
# pacman --root "$root" --dbpath "$root/var/lib/pacman" -S grub
  
{{bc|1=
+
Create a grub prefix on the target installation for both architectures using {{ic|grub-mknetdir}}.
# chroot "$root" /bin/bash
+
(chroot) # mkinitcpio -p linux
+
(chroot) # exit}}
+
  
Finally, cleanup by unmounting all of the virtual filesystems we mounted earlier.
+
# arch-chroot "$root" grub-mknetdir --net-directory=/boot --subdir=grub
  
{{bc|1=
+
Luckily for us, grub-mknetdir creates prefixes for all currently compiled/installed targets, and the {{Pkg|grub}} maintainers were nice enough to give us both in the same package, thus grub-mknetdir only needs to be run once.
# umount \
+
    "$root/dev/pts" \
+
    "$root/dev" \
+
    "$root/sys" \
+
    "$root/proc"}}
+
  
==Client configuration==
+
Now we create a trivial GRUB configuration:
  
In addition to the setup mentioned here, you should also set up your [[HOSTNAME#Set_the_host_name|hostname]], [[Timezone#Time_Zone|timezone]], [[Locale#Setting_system-wide_locale|locale]], and [[KEYMAP|keymap]].
+
{{hc|# vim "$root/boot/grub/grub.cfg"|2=
 +
menuentry "Arch Linux" {
 +
    linux /vmlinuz-linux quiet add_efi_memmap ip=:::::eth0:dhcp nfsroot=10.0.0.1:/arch
 +
    initrd /initramfs-linux.img
 +
}
 +
}}
  
===Pxelinux===
+
[[GRUB]] dark-magic will {{ic|1=set root=(tftp,10.0.0.1)}} automatically, so that the kernel and initramfs are transferred via TFTP without any additional configuration, though you might want to set it explicitly if you have any other non-tftp menuentries.
  
Install {{pkg|syslinux}}.
+
{{Note|Modify your kernel line as-necessary, refer to [[Syslinux#Pxelinux|Pxelinux]] for NBD-related options}}
  
{{bc|1=
+
==== Pxelinux====
# pacman -Syu syslinux}}
+
  
Copy the pxelinux bootloader (provided by the syslinux package) to the boot directory of the client.
+
[[Syslinux|Pxelinux]] is provided by {{Pkg|syslinux}}, see [[Syslinux#Pxelinux|here]] for detail.
  
{{bc|1=
+
=== Additional mountpoints ===
# cp /usr/lib/syslinux/pxelinux.0 "$root/boot"
+
# mkdir "$root/boot/pxelinux.cfg"}}
+
  
We also created the {{ic|pxelinux.cfg}} directory, which is where pxelinux searches for configuration files by default. Because we don't want to discriminate between different host MACs, we then create the {{ic|default}} configuration.
+
==== NBD root ====
  
{{hc|# vim "$root/boot/pxelinux.cfg/default"|<nowiki>
+
In late boot, you will want to switch your root filesystem mount to both {{ic|rw}}, and enable {{ic|1=compress=lzo}}, for much improved disk performance in comparison to [[NFS]].
default linux
+
  
label linux
+
{{hc|# vim "$root/etc/fstab"|2=
kernel vmlinuz-linux
+
/dev/nbd0  / btrfs  rw,noatime,discard,compress=lzo  0 0
append initrd=initramfs-linux.img rootfstype=nfs root=/dev/nfs nfsroot=10.0.0.1:/mnt/arch,v4,rsize=16384,wsize=16384 ip=:::::eth0:dhcp
+
}}
</nowiki>}}
+
  
Or if you are using NBD, use the following append line:
+
==== Program state directories ====
  
{{bc|<nowiki># append initrd=initramfs-linux.img root=/dev/nbd0 nbd_host=10.0.0.1 nbd_name=arch ip=:::::eth0:dhcp</nowiki>}}
+
{{Accuracy|systemd does not use persistent logging by default when /var/log/journal is in tmpfs and/or does not exist}}
  
{{Note|You will need to change {{ic|nbd_host}} and/or {{ic|nfsroot}}, respectively, to match your network configuration (the address of the NFS/NBD server)}}
+
You could mount {{ic|/var/log}}, for example, as tmpfs so that logs from multiple hosts do not mix unpredictably, and do the same with {{ic|/var/spool/cups}}, so the 20 instances of cups using the same spool do not fight with each other and make 1,498 print jobs and eat an entire ream of paper (or worse: toner cartridge) overnight.
 
+
The pxelinux configuration syntax identical to syslinux; refer to the upstream documentation for more information.
+
 
+
The kernel and initramfs will be transferred via TFTP, so the paths to those are going to be relative to the TFTP root. Otherwise, the root filesystem is going to be the NFS mount itself, so those are relative to the root of the NFS server.
+
 
+
===VFS mountpoints===
+
 
+
Add hacks to your [[fstab]] for the root filesystem and devpts.
+
  
 
{{hc|# vim "$root/etc/fstab"|2=
 
{{hc|# vim "$root/etc/fstab"|2=
none    /           none
+
tmpfs  /var/log        tmpfs    nodev,nosuid   0 0
none   /dev/pts    devpts    gid=5,mode=620   0 0}}
+
tmpfs  /var/spool/cups tmpfs    nodev,nosuid   0 0}}
  
===Late-boot networking===
+
It would be best to configure software that has some sort of state/database to use unique state/database storage directories for each host. If you wanted to run [http://puppetlabs.com/ puppet], for example, you could simply use the {{ic|%H}} specifier in the puppet unit file:
  
{{Out of date|Should no longer be necessary}}
+
{{hc|# vim "$root/etc/systemd/system/puppetagent.service"|2=
 +
[Unit]
 +
Description=Puppet agent
 +
Wants=basic.target
 +
After=basic.target network.target
  
This is to prevent the client from trying to reconnect the network and killing itself. Any disconnect of the network and your client will freeze.
+
[Service]
 +
Type=forking
 +
PIDFile=/run/puppet/agent.pid
 +
ExecStartPre=/usr/bin/install -d -o puppet -m 755 /run/puppet
 +
ExecStart=/usr/bin/puppet agent --vardir=/var/lib/puppet-%H --ssldir=/etc/puppet/ssl-%H
  
One way to fix this is by running {{pkg|dhcpcd}} with the {{ic|-s}} option to use the existing lease obtained in early-boot instead of requesting a new one.
+
[Install]
 +
WantedBy=multi-user.target
 +
}}
  
{{hc|# vim "$root/etc/conf.d/dhcpcd"|2=<nowiki>
+
Puppet-agent creates vardir and ssldir if they do not exist.
DHCPCD_ARGS=" -s $(ifconfig eth0 | grep -o '[0-9]*\.[0-9\.]*' | head -n1)"</nowiki>}}
+
  
{{hc|# vim "$root/etc/rc.conf"|2=<nowiki>
+
If neither of these approaches are appropriate, the last sane option would be to create a [http://www.freedesktop.org/wiki/Software/systemd/Generators systemd generator] that creates a mount unit specific to the current host (specifiers are not allowed in mount units, unfortunately).
NETWORK_PERSIST="yes"</nowiki>}}
+
  
==Client boot==
+
== Client boot ==
  
===NBD===
+
=== NBD ===
  
If you're using NBD, you'll need to umount the {{ic|arch.img}} before/while you boot your client.
+
{{Accuracy|When using COW on the server, the clients all effectively have read-only mounts of the original filesystem; it should theoretically be safe to do a read-write mount on the NBD server}}
  
This makes things particularly interesting when it comes to kernel updates. You can't have your client filesystem mounted while you're booting a client, but that also means you need to use a kernel separate from your client filesystem in order to build it.
+
If you are using NBD, you will need to umount the {{ic|arch.img}} before/while you boot your client.
  
You'll need to first copy $root/boot from the client installation to your tftp root (i.e. /mnt/boot).
+
This makes things particularly interesting when it comes to kernel updates. You cannot have your client filesystem mounted while you are booting a client, but that also means you need to use a kernel separate from your client filesystem in order to build it.
  
{{bc|# cp -r "$root/boot" /mnt/boot}}
+
You will need to first copy {{ic|$root/boot}} from the client installation to your tftp root (i.e. {{ic|/srv/boot}}).
  
You'll then need to umount $root before you start the client.
+
# cp -r "$root/boot" /srv/boot
  
{{bc|# umount "$root"}}
+
You will then need to umount {{ic|$root}} before you start the client.
  
{{Note|To update the kernel in this setup, you either need to mount {{ic|/mnt/boot}} using [[NFS]] in [[fstab]] on the client (prior to doing the kernel update) or mount your client filesystem after the client has disconnected from NBD}}
+
# umount "$root"
  
==References==
+
{{Note|To update the kernel in this setup, you either need to mount {{ic|/srv/boot}} using [[NFS]] in [[fstab]] on the client (prior to doing the kernel update) or mount your client filesystem after the client has disconnected from NBD}}
  
[https://www.kernel.org/doc/Documentation/filesystems/nfs/nfsroot.txt kernel.org: Mounting the root filesystem via NFS (nfsroot)]
+
== See also ==
  
[http://www.syslinux.org/wiki/index.php/PXELINUX syslinux.org: pxelinux FAQ]
+
* [https://www.kernel.org/doc/Documentation/filesystems/nfs/nfsroot.txt kernel.org: Mounting the root filesystem via NFS (nfsroot)]
 +
* [http://www.syslinux.org/wiki/index.php/PXELINUX syslinux.org: pxelinux FAQ]

Latest revision as of 08:05, 20 January 2016

From Wikipedia:Diskless node

A diskless node (or diskless workstation) is a workstation or personal computer without disk drives, which employs network booting to load its operating system from a server.

Server configuration

First of all, we must install the following components:

  • A DHCP server to assign IP addresses to our diskless nodes.
  • A TFTP server to transfer the boot image (a requirement of all PXE option roms).
  • A form of network storage (NFS or NBD) to export the Arch installation to the diskless node.
Note: dnsmasq is capable of simultaneously acting as both DHCP and TFTP server. For more information, see the dnsmasq article.

DHCP

Install ISC dhcp and configure it:

/etc/dhcpd.conf
allow booting;
allow bootp;

authoritative;

option domain-name-servers 10.0.0.1;

option architecture code 93 = unsigned integer 16;

group {
    next-server 10.0.0.1;

    if option architecture = 00:07 {
        filename "/grub/x86_64-efi/core.efi";
    } else {
        filename "/grub/i386-pc/core.0";
    }

    subnet 10.0.0.0 netmask 255.255.255.0 {
        option routers 10.0.0.1;
        range 10.0.0.128 10.0.0.254;
    }
}
Note: next-server should be the address of the TFTP server; everything else should be changed to match your network

RFC 4578 defines the "Client System Architecture Type" dhcp option. In the above configuration, if the PXE client requests an x86_64-efi binary (type 0x7), we appropriately give them one, otherwise falling back to the legacy binary. This allows both UEFI and legacy BIOS clients to boot simultaneously on the same network segment.

Start ISC DHCP systemd service.

TFTP

The TFTP server will be used to transfer the bootloader, kernel, and initramfs to the client.

Set the TFTP root to /srv/arch/boot. See Tftpd server for installation and configuration.

Network storage

The primary difference between using NFS and NBD is while with both you can in fact have multiple clients using the same installation, with NBD (by the nature of manipulating a filesystem directly) you will need to use the copyonwrite mode to do so, which ends up discarding all writes on client disconnect. In some situations however, this might be highly desirable.

NFS

Install nfs-utils on the server.

You will need to add the root of your Arch installation to your NFS exports:

/etc/exports
/srv       *(rw,fsid=0,no_root_squash,no_subtree_check)
/srv/arch  *(rw,no_root_squash,no_subtree_check)

Next, start NFS services: rpc-idmapd rpc-mountd.

NBD

Install nbd and configure it.

# vim /etc/nbd-server/config
[generic]
    user = nbd
    group = nbd
[arch]
    exportname = /srv/arch.img
    copyonwrite = false
Note: Set copyonwrite to true if you want to have multiple clients using the same NBD share simultaneously; refer to man 5 nbd-server for more details.

Start nbd systemd service.

Client installation

Next we will create a full Arch Linux installation in a subdirectory on the server. During boot, the diskless client will get an IP address from the DHCP server, then boot from the host using PXE and mount this installation as its root.

Directory setup

Create a sparse file of at least 1 gigabyte, and create a btrfs filesystem on it (you can of course also use a real block device or LVM if you so desire).

# truncate -s 1G /srv/arch.img
# mkfs.btrfs /srv/arch.img
# export root=/srv/arch
# mkdir -p "$root"
# mount -o loop,discard,compress=lzo /srv/arch.img "$root"
Note: Creating a separate filesystem is required for NBD but optional for NFS and can be skipped/ignored.

Bootstrapping installation

Tango-inaccurate.pngThe factual accuracy of this article or section is disputed.Tango-inaccurate.png

Reason: mkarchroot is mentioned only in the following sentence. (Discuss in Talk:Diskless system#)

Install devtools and arch-install-scripts, and run mkarchroot.

# pacstrap -d "$root" base mkinitcpio-nfs-utils nfs-utils
Note: In all cases mkinitcpio-nfs-utils is still required. ipconfig used in early-boot is provided only by the latter.

Now the initramfs needs to be constructed.

NFS

Trivial modifications to the net hook are required in order for NFSv4 mounting to work (not supported by nfsmount--the default for the net hook).

# sed s/nfsmount/mount.nfs4/ "$root/usr/lib/initcpio/hooks/net" > "$root/usr/lib/initcpio/hooks/net_nfs4"
# cp $root/usr/lib/initcpio/install/net{,_nfs4}

The copy of net is unfortunately needed so it does not get overwritten when mkinitcpio-nfs-utils is updated on the client installation.

Edit $root/etc/mkinitcpio.conf and add nfsv4 to MODULES, net_nfs4 to HOOKS, and /usr/bin/mount.nfs4 to BINARIES.

Next, we chroot our installation and run mkinitcpio:

# arch-chroot "$root" mkinitcpio -p linux

NBD

The mkinitcpio-nbdAUR package needs to be installed on the client. Build it with makepkg and install it:

# pacman --root "$root" --dbpath "$root/var/lib/pacman" -U mkinitcpio-nbd-0.4-1-any.pkg.tar.xz

You will then need to append nbd to your HOOKS array after net; net will configure your networking for you, but not attempt a NFS mount if nfsroot is not specified in the kernel line.

Client configuration

In addition to the setup mentioned here, you should also set up your hostname, timezone, locale, and keymap, and follow any other relevant parts of the Installation guide.

Bootloader

GRUB

Merge-arrows-2.pngThis article or section is a candidate for merging with GRUB.Merge-arrows-2.png

Notes: (Discuss in Talk:Diskless system#)

Though poorly documented, GRUB supports being loaded via PXE.

# pacman --root "$root" --dbpath "$root/var/lib/pacman" -S grub

Create a grub prefix on the target installation for both architectures using grub-mknetdir.

# arch-chroot "$root" grub-mknetdir --net-directory=/boot --subdir=grub

Luckily for us, grub-mknetdir creates prefixes for all currently compiled/installed targets, and the grub maintainers were nice enough to give us both in the same package, thus grub-mknetdir only needs to be run once.

Now we create a trivial GRUB configuration:

# vim "$root/boot/grub/grub.cfg"
menuentry "Arch Linux" {
    linux /vmlinuz-linux quiet add_efi_memmap ip=:::::eth0:dhcp nfsroot=10.0.0.1:/arch
    initrd /initramfs-linux.img
}

GRUB dark-magic will set root=(tftp,10.0.0.1) automatically, so that the kernel and initramfs are transferred via TFTP without any additional configuration, though you might want to set it explicitly if you have any other non-tftp menuentries.

Note: Modify your kernel line as-necessary, refer to Pxelinux for NBD-related options

Pxelinux

Pxelinux is provided by syslinux, see here for detail.

Additional mountpoints

NBD root

In late boot, you will want to switch your root filesystem mount to both rw, and enable compress=lzo, for much improved disk performance in comparison to NFS.

# vim "$root/etc/fstab"
/dev/nbd0  /  btrfs  rw,noatime,discard,compress=lzo  0 0

Program state directories

Tango-inaccurate.pngThe factual accuracy of this article or section is disputed.Tango-inaccurate.png

Reason: systemd does not use persistent logging by default when /var/log/journal is in tmpfs and/or does not exist (Discuss in Talk:Diskless system#)

You could mount /var/log, for example, as tmpfs so that logs from multiple hosts do not mix unpredictably, and do the same with /var/spool/cups, so the 20 instances of cups using the same spool do not fight with each other and make 1,498 print jobs and eat an entire ream of paper (or worse: toner cartridge) overnight.

# vim "$root/etc/fstab"
tmpfs   /var/log        tmpfs     nodev,nosuid    0 0
tmpfs   /var/spool/cups tmpfs     nodev,nosuid    0 0

It would be best to configure software that has some sort of state/database to use unique state/database storage directories for each host. If you wanted to run puppet, for example, you could simply use the %H specifier in the puppet unit file:

# vim "$root/etc/systemd/system/puppetagent.service"
[Unit]
Description=Puppet agent
Wants=basic.target
After=basic.target network.target

[Service]
Type=forking
PIDFile=/run/puppet/agent.pid
ExecStartPre=/usr/bin/install -d -o puppet -m 755 /run/puppet
ExecStart=/usr/bin/puppet agent --vardir=/var/lib/puppet-%H --ssldir=/etc/puppet/ssl-%H

[Install]
WantedBy=multi-user.target

Puppet-agent creates vardir and ssldir if they do not exist.

If neither of these approaches are appropriate, the last sane option would be to create a systemd generator that creates a mount unit specific to the current host (specifiers are not allowed in mount units, unfortunately).

Client boot

NBD

Tango-inaccurate.pngThe factual accuracy of this article or section is disputed.Tango-inaccurate.png

Reason: When using COW on the server, the clients all effectively have read-only mounts of the original filesystem; it should theoretically be safe to do a read-write mount on the NBD server (Discuss in Talk:Diskless system#)

If you are using NBD, you will need to umount the arch.img before/while you boot your client.

This makes things particularly interesting when it comes to kernel updates. You cannot have your client filesystem mounted while you are booting a client, but that also means you need to use a kernel separate from your client filesystem in order to build it.

You will need to first copy $root/boot from the client installation to your tftp root (i.e. /srv/boot).

# cp -r "$root/boot" /srv/boot

You will then need to umount $root before you start the client.

# umount "$root"
Note: To update the kernel in this setup, you either need to mount /srv/boot using NFS in fstab on the client (prior to doing the kernel update) or mount your client filesystem after the client has disconnected from NBD

See also