From ArchWiki
Revision as of 23:06, 17 July 2013 by Lahwaacz (Talk | contribs) (notes have been added to related sections)

Jump to: navigation, search

KVM, Kernel-based Virtual Machine, is a hypervisor built right into the Linux kernel. It is similar to Xen in purpose but much simpler to get running. To start using the hypervisor, just load the appropriate kvm kernel modules and the hypervisor is up. As with Xen's full virtualization, in order for KVM to work, you must have a processor that supports Intel's VT-x extensions or AMD's AMD-V extensions.

Using KVM, one can run multiple virtual machines running unmodified GNU/Linux, Windows, or any other operating system. (See Guest Support Status). Each virtual machine has private virtualized hardware: a network card, disk, graphics adapter, etc. See KVM Howto.

Differences among KVM, Xen, VMware, and QEMU can be found at the KVM FAQ. KVM is used alongside Libvirt for better management with "virsh" command set. Check Libvirt for more details.

Get the packages

Arch Linux kernels provide the appropriate kernel modules to support KVM. You can check if your kernel supports KVM with the following command (assuming your kernel is built with CONFIG_IKCONFIG_PROC):

$ zgrep KVM /proc/config.gz

KVM requires that the virtual machine host's processor has virtualization support (named VT-x for Intel processors and AMD-V for AMD processors). You can check whether your processor supports hardware virtualization with the following command:

$ lscpu

You processor supports virtualization only if there is a line telling you so.

You can also run:

$ grep -E "(vmx|svm)" --color=always /proc/cpuinfo

If nothing is displayed after running that command, then your processor does not support hardware virtualization, and you will not be able to use KVM.

Set up kernel modules

First, you need to add your user account into the kvm group to use the /dev/kvm device.

# gpasswd -a <Your_Login_Name> kvm
Note: If you use systemd and are a local user, this is not necessary, as access is now granted by systemd/udev.

Secondly, you have to choose one of the following depending on the manufacturer of the VM host's CPU.

  • If you have Intel's VT-x extensions, modprobe the kvm_intel module.
# modprobe kvm_intel
  • If you have AMD's AMD-V (code name "Pacifica") extensions, modprobe the kvm-amd module.
# modprobe kvm_amd

If modprobing kvm_intel or kvm_amd fails but modprobing kvm succeeds, (and lscpu claims that hardware acceleration is supported), check your BIOS settings. Some vendors (especially laptop vendors) disable these processor extensions by default. To determine whether there's no hardware support or there is but the extensions are disabled in BIOS, the output from dmesg after having failed to modprobe will tell.

If you want these modules to persist, see Kernel_modules#Loading.

How to use KVM

As qemu-kvm package is merged into qemu, see the main article QEMU, and especially the section Enabling KVM.

Enable HugePages

Tango-inaccurate.pngThe factual accuracy of this article or section is disputed.Tango-inaccurate.png

Reason: With systemd, hugetlbfs is mounted on /dev/hugepages by default, but with mode 0755 and root's uid and gid. (Discuss in Talk:KVM#)

Merge-arrows-2.pngThis article or section is a candidate for merging with QEMU.Merge-arrows-2.png

Notes: qemu-kvm no longer exists as all of its features have been merged into qemu. After the above issue is cleared, I suggest merging this section into QEMU. (Discuss in Talk:KVM#)

You may also want to enable hugepages to improve the performance of your virtual machine. With an up to date Arch Linux and a running KVM you probably already have everything you need. Check if you have the directory /dev/hugepages. If not create it. Now we need the right permissions to use this directory. Check if the group kvm exist and if you are member of this group. This should be the case if you already have a running virtual machine.

$ getent group kvm

Add to your /etc/fstab:

hugetlbfs       /dev/hugepages  hugetlbfs       mode=1770,gid=78        0 0

Of course the gid must match that of the kvm group. The mode of 1770 allows anyone in the group to create files but not unlink or rename each other's files. Make sure /dev/hugepages is mounted properly:

# umount /dev/hugepages
# mount /dev/hugepages
$ mount | grep huge
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,mode=1770,gid=78)

Now you can calculate how many hugepages you need. Check how large your hugepages are:

$ cat /proc/meminfo | grep Hugepagesize

Normally that should be 2048 kB ≙ 2 MB. Let's say you want to run your virtual machine with 1024 MB. 1024 / 2 = 512. Add a few extra so we can round this up to 550. Now tell your machine how many hugepages you want:

# echo 550 > /proc/sys/vm/nr_hugepages

If you had enough free memory you should see:

$ cat /proc/meminfo | grep HugePages_Total
HugesPages_Total:  550

If the number is smaller, close some applications or start your virtual machine with less memory (number_of_pages x 2):

$ qemu-system-x86_64 -enable-kvm -m 1024 -mem-path /dev/hugepages -hda <disk_image> [...]

Note the -mem-path parameter. This will make use of the hugepages.

Now you can check, while your virtual machine is running, how many pages are used:

$ cat /proc/meminfo | grep HugePages
HugePages_Total:     550
HugePages_Free:       48
HugePages_Rsvd:        6
HugePages_Surp:        0

Now that everything seems to work you can enable hugepages by default if you like. Add to your /etc/sysctl.conf:

vm.nr_hugepages = 550

See also:

Bridged Networking

Using netcfg

Tango-view-refresh-red.pngThis article or section is out of date.Tango-view-refresh-red.png

Reason: Netcfg has been superseded by netctl. (Discuss in Talk:KVM#)

Bridged networking is used when you want your VM to be on the same network as your host machine. This will allow it to get a static or DHCP IP address on your network, and then you can access it using that IP address from anywhere on your LAN. The preferred method for setting up bridged networking for KVM is to use the netcfg package. You will also need to install bridge-utils.

For more information, see: Netcfg Tips#Configuring a bridge for use with virtual machines (VMs)

You can follow this page to configure the bridge: Libvirt#Bridged Networking

Additional notes

Merge-arrows-2.pngThis article or section is a candidate for merging with QEMU#Tap networking with QEMU.Merge-arrows-2.png

Notes: qemu-kvm no longer exists as all of its features have been merged into qemu. This section also duplicates part of QEMU's content. (Discuss in Talk:KVM#)

Other information can be found here: QEMU#Tap Networking with QEMU and QEMU#Networking with VDE2.

If you are using iptables, it is recommended for performance and security reasons to disable the firewall on the bridge:

# cat >> /etc/sysctl.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0
# sysctl -p /etc/sysctl.conf

See the libvirt wiki and Fedora bug 512206. If you get errors by sysctl during init (boot) about non-existing files, make the bridge module load at boot. See Kernel_modules#Loading.

Alternatively, you can configure iptables to allow all traffic to be forwarded across the bridge by adding a rule like this:

-I FORWARD -m physdev --physdev-is-bridged -j ACCEPT

Tips and tricks

Note: See QEMU#Tips and tricks and QEMU#Troubleshooting for more tricks.

Live snapshots

Merge-arrows-2.pngThis article or section is a candidate for merging with libvirt.Merge-arrows-2.png

Notes: virsh is part of libvirt (Discuss in Talk:KVM#)

A feature called external snapshotting allows one to take a live snapshot of a virtual machine without turning it off. Currently it only works with qcow2 and raw file based images.

Once a snapshot is created, KVM attaches that new snapshotted image to virtual machine that is used as its new block device, storing any new data directly to it while the original disk image is taken offline which you can easily copy or backup. After that you can merge the snapshotted image to the original image, again without shutting down your virtual machine.

Here's how it works.

Current running vm

# virsh list --all
Id    Name                           State
3     archey                            running

List all its current images

# virsh domblklist archey 
Target     Source
vda        /vms/archey.img

Notice the image file properties

# qemu-img info /vms/archey.img
image: /vms/archey.img
file format: qcow2
virtual size: 50G (53687091200 bytes)
disk size: 2.1G
cluster_size: 65536

Create a disk-only snapshot. The switch --atomic makes sure that the VM is not modified if snapshot creation fails.

# virsh snapshot-create-as archey snapshot1 --disk-only --atomic

List if you want to see the snapshots

# virsh snapshot-list archey
Name                 Creation Time             State
snapshot1           2012-10-21 17:12:57 -0700 disk-snapshot

Notice the new snapshot image created by virsh and its image properties. It weighs just a few MiBs and is linked to its original "backing image/chain".

# qemu-img info /vms/archey.snapshot1
image: /vms/archey.snapshot1
file format: qcow2
virtual size: 50G (53687091200 bytes)
disk size: 18M
cluster_size: 65536
backing file: /vms/archey.img

At this point, you can go ahead and copy the original image with cp -sparse=true or rsync -S. Then you can merge the original image back into the snapshot.

# virsh blockpull --domain archey --path /vms/archey.snapshot1

Now that you have pulled the blocks out of original image, the file /vms/archey.snapshot1 becomes the new disk image. Check its disk size to see what it means. After that is done, the original image /vms/archey.img and the snapshot metadata can be deleted safely. The virsh blockcommit would work opposite to blockpull but it seems to be currently under development in qemu-kvm 1.3 (including snapshot-revert feature), scheduled to be released sometime next year.

This new feature of KVM will certainly come handy to the people who like to take frequent live backups without risking corruption of the file system.

Poor Man's Networking

Merge-arrows-2.pngThis article or section is a candidate for merging with QEMU.Merge-arrows-2.png

Notes: This section is not KVM-specific, it's generally applicable to all QEMU VMs. (Discuss in Talk:KVM#)

Setting up bridged networking can be a bit of a hassle sometimes. If the sole purpose of the VM is experimentation, one strategy to connect the host and the guests is to use SSH tunneling.

The basic steps are as follows:

  • Setup an SSH server in the host OS
  • (optional) Create a designated user used for the tunneling (e.g. tunneluser)
  • Install SSH in the VM
  • Setup authentication

See: SSH for the setup of SSH, especially SSH#Forwarding_Other_Ports.

When using the default user network stack, the host is reachable at address

Tango-edit-clear.pngThis article or section needs language, wiki syntax or style improvements.Tango-edit-clear.png

Reason: Usage of /etc/rc.local is discouraged. This should be a proper systemd service file. (Discuss in Talk:KVM#)

If everything works and you can SSH into the host, simply add something like the following to your /etc/rc.local

# Local SSH Server
echo "Starting SSH tunnel"
sudo -u vmuser ssh tunneluser@ -N -R 2213: -f
# Random remote port (e.g. from another VM)
echo "Starting random tunnel"
sudo -u vmuser ssh tunneluser@ -N -L 2345: -f

In this example a tunnel is created to the SSH server of the VM and an arbitrary port of the host is pulled into the VM.

This is a quite basic strategy to do networking with VMs. However, it is very robust and should be quite sufficient most of the time.

Note: Isn't this option enough? I think it should have the same effect: -redir tcp:2222: (it redirects port 2222 from host to, where is guest's IP address.

Nested virtualization

Tango-edit-clear.pngThis article or section needs language, wiki syntax or style improvements.Tango-edit-clear.png

Reason: The wrapper part is really terrible, and maybe unnecessary. (Discuss in Talk:KVM#)

Enable nested feature for kvm_intel:

 modprobe -r kvm_intel
 modprobe kvm_intel nested=1

Verify that feature is activated:

 # systool -m kvm_intel -v | grep nested
   nested              = "Y"

Create wrapper around qemu-kvm:

 # cat /usr/bin/qemu-kvm-nested 
 /usr/bin/qemu-system-x86_64 -cpu host "$@"
 # chmod a+x /usr/bin/qemu-kvm-nested
 # ls -la /usr/bin/qemu-kvm
 lrwxrwxrwx 1 root root 18 29 oct.  15:38 /usr/bin/qemu-kvm -> qemu-system-x86_64
 # ln -s /usr/bin/qemu-kvm-nested /usr/bin/qemu-kvm
 # ls -la /usr/bin/qemu-kvm
 lrwxrwxrwx 1 root root 24 12 nov.  13:09 /usr/bin/qemu-kvm -> /usr/bin/qemu-kvm-nested

Boot VM and check if vmx flag is present:

 grep vmx /proc/cpuinfo