Difference between revisions of "Xen"

From ArchWiki
Jump to: navigation, search
(Lots of updates.)
m (Dedicate a cpu)
Line 51: Line 51:
# /etc/rc.d/xend start
# /etc/rc.d/xend start
Allocating a fixed amount of memory is recommended when using xen. Also, if you're running IO intensive guests it might be a good idea to dedicate (pin) a CPU core only for dom0 use. Please see the external [http://wiki.xensource.com/xenwiki/XenCommonProblems XenCommonProblems] wiki page section "Can I dedicate a cpu core (or cores) only for dom0?" for more information.
===Configuring GRUB2===
===Configuring GRUB2===

Revision as of 00:42, 8 May 2010

This document explains how to setup Xen for Arch Linux.

What is Xen?

According to the Xen development team: "The Xen hypervisor, the powerful open source industry standard for virtualization, offers a powerful, efficient, and secure feature set for virtualization of x86, x86_64, IA64, PowerPC, and other CPU architectures. It supports a wide range of guest operating systems including Windows®, Linux®, Solaris®, and various versions of the BSD operating systems."

The Xen hypervisor is a thin layer of software which emulates a computer architecture. It is started by the boot loader and allows several operating systems to run simultaneously on top of it. Once the Xen hypervisor is loaded, is starts the "dom0" (for "domain 0"), or privileged domain, which in our case runs a modified Linux kernel (other possible dom0 operating systems are NetBSD and OpenSolaris). The dom0 kernel in the AUR is currently based on Linux kernel; hardware must be supported by this kernel to run Xen. Once the dom0 has started, one or more "domU" (for "user") or unprivileged domains can be started and controlled from the dom0.

Setting up Xen

Installing the necessary packages

Before building xen4, you need to build the required package lib32-glibc-devel from the AUR first. After installing it, create a temporary symlink:

ln -s /opt/lib32/usr/include/gnu/stubs-32.h /usr/include/gnu

The next step is to install xen from the AUR. You can either install xen version 3 or 4 by choosing either the xen or xen4 package. This wiki focuses on xen4, to prevent confusion :)

If you need only the xen-tools, version 4 can be installed by using the xen-hv-tools package.

The next step is to build and install the dom0 kernel. To do so, build the kernel26-xen-dom0 package from the AUR.

Please note: At the time of this writing the current version uses a patched kernel version 2.6.31, and it won't compile when you use gcc 4.5. I worked around this problem by temporarily downgrading gcc and gcc-libs to version 4.4.3. When I found out that it wasn't enough because of a missing library, I consequently built the gmp4 package.

..and there you go: the building part has been finished. Now you can configure Grub and boot into the kernel that's just been built.

Configuring GRUB

Grub must be configured so that the Xen hypervisor is booted followed by the dom0 kernel. Add the following entry to /boot/grub/menu.lst:

title Xen with Arch Linux
root (hd0,X)
kernel /xen.gz dom0_mem=524288
module /vmlinuz26-xen-dom0 root=/dev/sdaY ro console=tty0 vga=gfx-1024x768x8
module /kernel26-xen-dom0.img

where X and Y are the appropriate numbers for your disk configuration; and dom0_mem, console, and vga are optional, customizable parameters. Nice little detail: you can use LVM volumes too. So instead of /dev/sdaY you can also fill in /dev/mapper/somelvm. Also, notice that the vga-parameter works a bit differently from usual kernel configuration lines. It has been discussed on the Xen-devel list: [1]

The standard arch kernel can be use to boot the domUs. In order for this to work one must add 'xen-blkfront' to the modules array in /etc/mkinitcpio.conf:

MODULES="... xen-blkfront ..."

So, next step is to reboot into the xen kernel.

Next step: start xend:

# /etc/rc.d/xend start

Allocating a fixed amount of memory is recommended when using xen. Also, if you're running IO intensive guests it might be a good idea to dedicate (pin) a CPU core only for dom0 use. Please see the external XenCommonProblems wiki page section "Can I dedicate a cpu core (or cores) only for dom0?" for more information.

Configuring GRUB2

This works just like with Grub, but here you need to use the command 'multiboot' instead of using 'kernel'. So it becomes:

# (2) Arch Linux(XEN)
menuentry "Arch Linux(XEN)" {
    set root=(hd0,X)
    multiboot /boot/xen.gz dom0_mem=2048M
    module /boot/vmlinuz26-xen-dom0 root=/dev/sdaY ro
    module /boot/kernel26-xen-dom0.gz

If you had success when booting up into the dom0 kernel, we can continue.

Add domU instances

The basic idea behind adding a domU is as follows. We must get the domU kernels, allocate space for the virtual hard disk, create a configuration file for the domU, and finally start the domU with xm.

$ mkfs.ext4 /dev/sdb1    ## format partition
$ mkdir /tmp/install
$ mount /dev/sdb1 /tmp/install
$ mkdir /tmp/install/var/lib/pacman
$ mkdir /tmp/install/var/cache/pacman/pkg
$ pacman -Sy base -r /tmp/install --cache-dir /tmp/install/var/cache/pacman/pkg
$ mount -o bind /dev /tmp/install/dev
$ mount -t proc none /tmp/install/proc
$ mount -o bind /sys /tmp/install/sys
$ chroot /tmp/install /bin/bash
$ vi /etc/resolv.conf
$ vi /etc/fstab
    /dev/xvda               /           ext4    defaults                0       1

$ vi /etc/inittab
    c1:2345:respawn:/sbin/agetty -8 38400 hvc0 linux
    #c1:2345:respawn:/sbin/agetty -8 38400 tty1 linux
    #c2:2345:respawn:/sbin/agetty -8 38400 tty2 linux
    #c3:2345:respawn:/sbin/agetty -8 38400 tty3 linux
    #c4:2345:respawn:/sbin/agetty -8 38400 tty4 linux
    #c5:2345:respawn:/sbin/agetty -8 38400 tty5 linux
    #c6:2345:respawn:/sbin/agetty -8 38400 tty6 linux

    ## maybe
    #c1:2345:respawn:/sbin/agetty -8 38400 hvc0 rxvt-256color

$ exit  ## exit chroot
$ umount /tmp/install/dev
$ umount /tmp/install/proc
$ umount /tmp/install/sys
$ umount /tmp/install

If not starting from a fresh install and one wants to rsync from an existing system:

$ mkfs.ext4 /dev/sdb1    ## format lv partition
$ mkdir /tmp/install
$ mount /dev/sdb1 /tmp/install
$ mkdir /tmp/install/{proc,sys}
$ chmod 555 /tmp/install/proc
$ rsync -avzH --delete --exclude=proc/ --exclude=sys/ old_ooga:/ /tmp/install/

$ vi /etc/xen/dom01     ## create config file
    #  -*- mode: python; -*-
    kernel = "/boot/vmlinuz26"
    ramdisk = "/boot/kernel26.img"
    memory = 1024
    name = "dom01"
    vif = [ 'mac=00:16:3e:00:01:01' ]
    disk = [ 'phy:/dev/sdb1,xvda,w' ]
    hostname = "ooga"
    root = "/dev/xvda ro"

$ xm create -c dom01

Alternative method

To set up Xen by hand, see Xen_Install. This page is very out of date, so be careful. It details installing a custom xen kernel and the xen userland tools by hand, rather than by taking advantage of packages in the AUR, as described above.

Arch as Xen guest (PV mode)

To get paravirtualization you need to install:

Change mode to PV with commands (on dom0):

 xe vm-param-set uuid=<vm uuid> HVM-boot-policy=""
 xe vm-param-set uuid=<vm uuid> PV-bootloader=pygrub

Edit /boot/grub/menu.lst and add kernel26-xen:

 # (1) Arch Linux (domU)
 title  Arch Linux (domU)
 root   (hd0,0)
 kernel /boot/vmlinuz26-xen root=/dev/xvda1 ro console=hvc0
 initrd /boot/kernel26-xen.img


To use xe-guest-utilities, add xenfs mount point into /etc/fstab:

 xenfs                  /proc/xen     xenfs     defaults            0      0

and add xe-linux-distribution into /etc/rc.conf:DAEMONS array.


  • pygrub does not show boot menu.
  • To avoid hwclock error messages, set HARDWARECLOCK="xen" in /etc/rc.conf (actually you can use any value here except "UTC" and "localtime")
  • If you want to return to hardware VM, set HVM-boot-policy="BIOS order"

Xen management tools

You can use following tools:

After installing xvp, you need to generate /etc/xvp.conf with xvpdiscover tool and adjust your web server for using /srv/http/xvpweb/.


  • Xen's homepage: [2]
  • The Xen Wiki: [3]