Diskless network boot NFS root
From ArchWiki
Contents |
[edit] Root over NFS (diskless boot)
This HOWTO will explain the step-by-step method for booting an Archlinux installation over an NFS share. The first half will primarily cover configuration of the server, and the second will cover client side configuration.
This text will contain errors, please report any problems you have with error messages on the talk page.
For more information on Network File Sharing, see Nfs.
[edit] Hardware Considerations
Server: Any standard computer should work, but for performance reasons it should have fast disk access and preferably gigabit ethernet.
Client: For a completely diskless boot, the client computer must have a network-bootable (PXE) network card. Wireless will not work, with exception of a wireless-to-ethernet external bridge. Many motherboards have a PXE-compatible ethernet card built in, but you will need to enable support in the BIOS.
If you don't have a PXE-compatible network card, you can setup a semi-diskless client. This involves building a flashdrive or CD-based boot partition with syslinux or grub. The configuration is the same, but skipping the tftp steps. Creating boot-CDs or flashdrives is beyond the scope of this article.
[edit] Server-Side Setup
The server will have to run a tftp daemon to share the kernel/bootsector along with an NFS share to share the root directory. PXE requires the DHCP server to announce the location of the tftp server to any clients that connect. Most routers will not support this, and this article will show how to modify an existing DNSMasq or DHCPD server. See NAT'ing_firewall_-_Share_your_broadband_connection for setting up a simple router replacement, including DHCP via DNSMasq, a good place to start for this project. If you do have a router that supports PXE, then you will need to tell it to send all tftp requests to the server.
At this point I will assume you have a server running with DNSMasq or DHCPd providing DHCP services, along with some space for the client's root.
First, install nfs-utils, syslinux (for the pxelinux bootloader, you can substitute for GRUB, see [1]).
pacman -S nfs-utils syslinux
[edit] Create Client Root Directory
First, create a directory that will hold the root filesystem, this can be anywhere. For this example, I will use /disklessroot.
mkdir /disklessroot
Now you need to populate that root with an installation of Archlinux.
pacman -S --root /disklessroot --dbpath /disklessroot/var/lib/pacman base
Edit /disklessroot/etc/mkinitcpio.conf, add 'nfs' to the MODULE section, and add 'net' as the second entry under HOOKS..
MODULES="nfs" HOOKS="base net udev autodetect pata scsi sata filesystems"
Now, chroot into /disklessroot and recreate the kernel image.
chroot /disklessroot mkinitcpio -p kernel26 exit
You should be back to the normal root, the exit command should leave the chroot. Check before continuing.
[edit] PXE/TFTP Setup
Now to setup tftp and PXE.
Within DNSMasq, there is an easy-to-use tftp server, so this article will use that. First, edit /etc/dnsmasq.conf and uncomment these lines, changing them to reflect your file paths.
# Set the boot filename for BOOTP. You will only need # this is you want to boot machines over the network and you will need # a TFTP server; either dnsmasq's built in TFTP server or an # external one. (See below for how to enable the TFTP server.) dhcp-boot=pxelinux.0 #change this if you are using GRUB. # Enable dnsmasq's built-in TFTP server enable-tftp # Set the root directory for files available via FTP. tftp-root=/disklessroot/boot/
This has the additional feature of allowing you to update the kernel within the client OS.
Make sure the firewall on your server does not block UDP port 69 (UDP ports 67-68 and 4011 are also used, but should not need to be opened).
[edit] NFS Setup
Edit /etc/exports, add this line.
/disklessroot *(rw,no_root_squash,no_subtree_check)
If security is an issue, you can replace * with an IP address or IP range to restrict connections. This is semi-redundant with the /etc/hosts.allow configuration below.
Edit /etc/hosts.allow
nfsd: 10.0.0. portmap: 10.0.0. mountd: 10.0.0.
This partial IP will match everything in the 10.0.0.x range, you can specify a single address, IP range, or simply "ALL" (Insecure!).
Add "portmap nfslock nfsd" to the server's /etc/rc.conf under DAEMONS, in that specific order. (Assuming dnsmasq was already installed and on the daemons list.)
Refer to Nfs for any configuration/security issues that are not covered here.
[edit] Client Configuration
These files are to be edited from the server, as the client is not ready yet.
[edit] Boot Configuration
Copy pxelinux.0 to /diskless/boot/
cp /usr/lib/syslinux/pxelinux.0 /disklessroot/boot/
Create and edit /diskless/boot/pxelinux.cfg/default (Replacing IP addresses and file-paths as needed.)
default linux label linux kernel vmlinuz26 append initrd=kernel26.img rootfstype=nfs root=/dev/nfs nfsroot=10.0.0.1:/disklessroot,v3,rsize=16384,wsize=16384 ip=dhcp
The syntax for the pxelinux bootloader is the same as Syslinux, here[2] under "CONFIGURATION FILE". Remember that any filenames will be relative to the tftp server, check the setting in /etc/dnsmasq.conf named "tftp-root=". This example places tftp-root at /disklessroot/boot/. Thus the kernel should be at /disklessroot/boot/vmlinuz26 on the server, and will show up as /vmlinuz26 for the first stage of client boot. Exactly like a separate /boot partition.
As there is no entry for / in the clients /etc/fstab file, the client may display the error "mount: can't find / in /etc/fstab or /etc/mtab" on startup. This error can be safely ignored. However, if you want to suppress the error you can add a hack to /etc/fstab. This involves adding the following to /disklessroot/etc/fstab:
none / none
[edit] DHCP/Network Daemon Workaround
This is to prevent the client from trying to reconnect the network and killing itself, also prevents the client from prematurely killing itself while shutting down. Any disconnect of the network and your client will freeze.
The following workaround involves running the dhcpcd DHCP client on startup with the -s option to use the existing kernel DHCP auto-configured IP address instead of requesting a new one. The -p option is used to prevent dhcpcd from deconfiguring eth0 when it exits (which causes problems when shutting down). Notes: This workaround assumes that eth0 is the network interface used for DHCP.
Create a new file /etc/rc.d/network-nfsroot (/disklessroot/etc/rc.d/network-nfsroot on the server) with the following contents:
#!/bin/bash
. /etc/rc.conf
. /etc/rc.d/functions
case "$1" in
start)
if ! ck_daemon network-nfsroot; then
echo "Network is already running. Try 'network restart'"
exit
fi
stat_busy "Starting Network"
if dhcpcd -p -s $(ifconfig eth0 | grep -o '[0-9]*\.[0-9\.]*' | head -n1) eth0; then
add_daemon network-nfsroot
stat_done
else
stat_fail
fi
;;
stop)
stat_busy "Stopping Network"
if [ -f /var/run/dhcpcd-eth0.pid ]; then
/bin/kill $(cat /var/run/dhcpcd-eth0.pid)
rm_daemon network-nfsroot
stat_done
else
stat_fail
fi
;;
restart)
$0 stop
/bin/sleep 2
$0 start
;;
*)
echo "usage: $0 {start|stop|restart}"
esac
# vim: set ts=2 noet:
Edit the client's rc.conf and change to use network-nfsroot daemon instead of network daemon (/disklessroot/etc/rc.conf, or just /etc/rc.conf once you boot into the client).
Before:
DAEMONS=(syslog-ng network ...
After:
DAEMONS=(syslog-ng network-nfsroot ...
Another workaround is presented below which configures the DNS server manually.
[edit] DHCP/Network Daemon Workaround Number 2
Edit the client's rc.conf and disable the network daemon (/disklessroot/etc/rc.conf, or just /etc/rc.conf once you boot into the client).
DAEMONS=(syslog-ng !network ...
The main issue with this method is the fact that your resolv.conf will not auto-update (since this bypasses DHCP). To work around this, edit /etc/resolv.conf and add this line (/disklessroot/etc/resolv.conf on the server).
nameserver 10.0.0.1 #change this IP for whatever DHCP/DNS server you have, the server's IP usually works if you configured DNS correctly.
[edit] Testing/Debugging the Client
Now reboot the client computer, making sure the network card is first in the boot-order.
If all goes right, you should see the network card get an IP address from the server, then connect and boot the kernel. After the initial kernel messages, you should either see ArchLinux boot, or a "killed init" message.
If init was killed, check your configuration, it usually means it could not mount the NFS folder. Refer to Nfs for more help.
If it freezes during a mounting operation, check the client's /etc/fstab or the /boot/pxelinux.cfg/default settings.
If it freezes duing the "Network" section, either disable it using the previously mentioned workaround, or specify IP settings in /etc/rc.conf. (You can try setting eth0="dhcp")
[edit] Performance Notes
I am currently using a diskless client as my primary internet/development OS due to a hard-drive restriction on my PC. My server is an older machine is running Archlinux with a software-based raid5 array. My client can read files from the NFS share almost as fast as my server can, but there is a limitation with latency and caching. It slows down during "pacman -Sy" and package installs that involve many small files. Otherwise, it runs fine. It also seems that NFS runs synchronously by default, meaning that all writes have to be committed before continuing, no caching in free memory like it would on a local hard drive. This is actually a good thing, from the data integrity standpoint, as fewer files are lost during a random power outage. I have not tried the "async" NFS option because the nfs man page does not suggest it will work for NFSv3.
Mostly Complete.
Thank you, Net147, for your excellent edits.
Please edit/update as needed.