Difference between revisions of "Linux Containers"

From ArchWiki
Jump to: navigation, search
(Starting a container on boot with Systemd: forking services need to actually fork)
m (fix broken link)
 
(163 intermediate revisions by 47 users not shown)
Line 1: Line 1:
 
[[Category:Security]]
 
[[Category:Security]]
 
[[Category:Virtualization]]
 
[[Category:Virtualization]]
{{Stub|Currently just a rough draft... I think I will need to restructure this a bit and I have also noticed I have become a bit too verbose -_-;; I will be along shortly to complete this as well as clean it up.}}
+
[[ja:Linux Containers]]
 +
[[pt:Linux Containers]]
 +
{{Related articles start}}
 +
{{Related|LXD}}
 +
{{Related|AirVPN}}
 +
{{Related|Cgroups}}
 +
{{Related|Docker}}
 +
{{Related|OpenVPN}}
 +
{{Related|OpenVPN in Linux containers}}
 +
{{Related|PeerGuardian Linux}}
 +
{{Related|systemd-nspawn}}
 +
{{Related articles end}}
  
==Introduction==
+
Linux Containers (LXC) is an operating-system-level virtualization method for running multiple isolated Linux systems (containers) on a single control host (LXC host). It does not provide a virtual machine, but rather provides a virtual environment that has its own CPU, memory, block I/O, network, etc. space and the resource control mechanism. This is provided by  [[Wikipedia:Linux namespaces|namespaces]] and [[cgroups]] features in Linux kernel on LXC host. It is similar to a chroot, but offers much more isolation.
  
===Synopsis===
+
Alternatives for using containers are [[systemd-nspawn]] and [[docker]].
  
Linux Containers (LXC) are an operating system-level virtualization method for running multiple isolated server installs (containers) on a single control host. LXC does not provide a virtual machine, but rather provides a virtual environment that has its own process and network space. It is similar to a chroot, but offers much more isolation.
+
== Setup ==
 +
=== Required software ===
 +
Install the {{Pkg|lxc}} and {{Pkg|arch-install-scripts}}  packages.
  
===About this HowTo===
+
Verify that the running kernel is properly configured to run a container:
 +
$ lxc-checkconfig
  
This document is intended as an overview on setting up and deploying containers, and is not an in depth detailed instruction by instruction guide. A certain amount of prerequisite knowledge and skills are assumed (running commands as root, kernel configuration, mounting filesystems, shell scripting, chroot type environments, networking setup, etc).
+
Due to security concerns, the default Arch kernel does '''not''' ship with the ability to run containers as an unprivileged user; therefore, it is normal to see a '''missing''' status for "User namespaces" when running the check. See {{Bug|36969}} for this feature request.
  
Much of this was taken verbatim from [http://lxc.teegra.net/ Dwight Schauer], [http://tuxce.selfip.org/informatique/conteneurs-linux-lxc Tuxce] and [http://artisan.karma-lab.net/node/1749 Ulhume]. It has been copied here both to enable to community to share their collective wisdom and to expand on a few points.
+
=== Host network configuration ===
 +
LXCs support different virtual network types and devices (see [https://linuxcontainers.org/lxc/manpages//man5/lxc.container.conf.5.html lxc.container.conf(5)]). A bridge device on the host is required for most types of virtual networking.  The examples of creating a bridge provided below are not meant to be limiting, but illustrative. Users may use other programs to achieve the same results.  A wired and wireless example is provided below, but other setups are possible. Users are referred to the [[Network bridge]] article for additional options.
  
===Less verbose tutorial===
+
{{Merge|Network bridge|Duplication; this is not a guide on Linux networking.}}
  
[[User:Delerious010|Delerious010]] 21:43, 1 December 2009 (EST) I have come to realize I have added a lot of text to this HowTo. If you would like something more streamlined, please head on over to [http://lxc.teegra.net/ http://lxc.teegra.net/] for Dwight's excellent guide.
+
==== Example for a wired network ====
  
===Testing capabilities===
+
This example uses [[netctl]]: a bridge template can be found in {{ic|/etc/netctl/examples}} which needs to be edited to match the host network hardware specs and IP ranges of the host network.  Below are two example bridge configs, one using a dhcp setup and the other using a static IP setup.
  
Once the lxc package is installed, running lxc-checkconfig will print out a list of your system's capabilities
+
{{hc|1=/etc/netctl/lxcbridge|2=
 +
Description="LXC bridge"
 +
Interface=br0
 +
Connection=bridge
 +
BindsToInterfaces=('eno1')
 +
IP=dhcp
 +
SkipForwardingDelay=yes}}
  
==Host configuration==
+
{{hc|1=/etc/netctl/lxcbridge|2=
 +
Description="LXC bridge"
 +
Interface=br0
 +
Connection=bridge
 +
BindsToInterfaces=('eno1')
 +
IP=static
 +
Address=192.168.0.2/24
 +
Gateway='192.168.0.1'
 +
DNS=('192.168.0.1')}}
  
===Control group filesystem===
+
Before attempting to start the bridge, [[disable]] the running network interface on the host as the bridge will replace it; this depends on how the host network is configured, see [[Network configuration]].
  
LXC depends on the control group filesystem being mounted. The standard location for it is {{ic|/sys/fs/cgroup}}. If you use systemd, the cgroup filesystem will be mounted automatically, including the default controllers, but with other initsystems you might have to do it yourself:
+
For users already using netctl to manage an adapter, simply switch-to it:
 +
# netctl switch-to lxcbridge
 +
# netctl enable lxcbridge
  
  mount -t tmpfs none /sys/fs/cgroup
+
Verify network connectivity on the host before continuing. This can be accomplished with a simple ping:
 +
  $ ping -c 1 www.google.com
  
===Userspace tools===
+
==== Example for a wireless network ====
  
Install {{Pkg|lxc}} from [community].
+
Wireless networks cannot be bridged directly; a different method must be used in this case.  First, a bridge must be created similar to the previous examples, but it should not have any interface defined to it (other than the virtual interface of the container itself, which is done automatically).  Assign a static IP address to the bridge, but do not assign a gateway.
  
===Bridge device setup===
+
The host must be configured to perform NAT using [[iptables]]:
  
The package {{pkg|bridge-utils}}, which provides the {{ic|brctl}} command, provides the ability to set up one or more network bridges to be used by LXC. You can use {{ic|brctl}} directly to set up the bridges, but the process is archaic and probably best avoided.
+
# iptables -t nat -A POSTROUTING -o ''wlp3s0'' -j MASQUERADE
  
If using netcfg, you could try to refer to the [[netcfg]] wiki page, but the bridge related documentation has been removed. So when using netcfg, you will be mostly on your own.
+
where {{ic|''wlp3s0''}} is the name of the wireless interface.  [[Internet_sharing#Enable_packet_forwarding|Enable packet forwarding]], which is disabled by default.
  
OpenVPN has complete instructions for setting up a bridge, so it is probably your best option at the current. [[OpenVPN Bridge]]
+
The remaining steps are similar, except for one thing: for the container, the gateway must be configured to be the IP address of the host (in this example, it was 192.168.0.2).  This is specified in {{ic|/var/lib/lxc/''container_name''/config}} (see the following sections).
  
===Starting a container on boot with [[Systemd]]===
+
=== Container creation ===
 +
Select a template from {{ic|/usr/share/lxc/templates}} that matches the target distro to containerize.  Users wishing to containerize non-Arch distros will need additional packages on the host depending on the target distro:
 +
* Debian-based: {{Pkg|debootstrap}}
 +
* Fedora-based: {{AUR|yum}}
  
If you completed a container, starting it when the host boots is possible with the following systemd service template:
+
Run {{ic|lxc-create}} to create the container, which installs the root filesystem of the LXC to {{ic|/var/lib/lxc/CONTAINER_NAME/rootfs}} by default.  Example creating an Arch Linux LXC named "playtime":
 +
# lxc-create -n playtime -t /usr/share/lxc/templates/lxc-archlinux
  
{{bc|1=
+
{{Tip|Users may optionally install {{Pkg|haveged}} and [[start]] {{ic|haveged.service}} to avoid a perceived hang during the setup process while waiting for system entropy to be seeded.  Without it, the generation of private/GPG keys can add a lengthy wait to the process.}}
[Unit]
+
Description=Linux Container
+
After=network.target
+
  
[Service]
+
{{Tip|Users of [[Btrfs]] can append {{ic|-B btrfs}} to create a Btrfs subvolume for storing containerized rootfs. This comes in handy if cloning containers with the help of {{ic|lxc-clone}} command. [[ZFS]] users may use {{ic|-B zfs}}, correspondingly.}}
Type=forking
+
ExecStartPre=/bin/mount --make-rprivate /
+
ExecStart=/usr/bin/lxc-start -dn CONTAINER_NAME
+
ExecStop=/usr/bin/lxc-stop -n CONTAINER_NAME
+
  
[Install]
+
{{Tip|As of July 2015, creating an empty container using {{ic|-t none}} does not work, see the [https://bugs.launchpad.net/bugs/1466458 bug report]. As a workaround one can use {{ic|-t /bin/true}}.}}
WantedBy=multi-user.target
+
}}
+
Replace CONTAINER_NAME with the name of your container and save this file as /etc/systemd/system/lxc-CONTAINER_NAME.service
+
  
==Container setup==
+
=== Container configuration ===
 +
==== Basic config with networking ====
 +
System resources to be virtualized/isolated when a process is using the container are defined in {{ic|/var/lib/lxc/CONTAINER_NAME/config}}. By default, the creation process will make a minimum setup without networking support.  Below is an example config with networking:
  
There are various different means to do this
+
{{hc|/var/lib/lxc/playtime/config|<nowiki>
 +
# Template used to create this container: /usr/share/lxc/templates/lxc-archlinux
 +
# Parameters passed to the template:
 +
# For additional config options, please look at lxc.container.conf(5)
  
===Creating the filesystem===
+
## default values
 +
lxc.rootfs = /var/lib/lxc/playtime/rootfs
 +
lxc.utsname = playtime
 +
lxc.arch = x86_64
 +
lxc.include = /usr/share/lxc/config/archlinux.common.conf
  
====Bootstrap====
+
## network
Bootstrap an install ( [http://blog.mudy.info/tag/mkarchroot/ mkarchroot], [http://wiki.debian.org/Debootstrap debootstrap], [http://www.xen-tools.org/software/rinse/faq.html rinse], [[Install From Existing Linux]] ). You can also just copy/use an existing installation’s complete root filesystem.
+
lxc.network.type = veth
 +
lxc.network.link = br0
 +
lxc.network.flags = up
 +
lxc.network.ipv4 = 192.168.0.3/24
 +
lxc.network.ipv4.gateway = 192.168.0.1
 +
lxc.network.name = eth0
  
For example, install a small debian to /home/lxc/debianfs
+
## mounts
 +
## specify shared filesystem paths in the format below
 +
## make sure that the mount point exists on the lxc
 +
#lxc.mount.entry = /mnt/data/share mnt/data none bind 0 0
 +
#
 +
# if running the same Arch linux on the same architecture it may be
 +
# adventitious to share the package cache directory
 +
#lxc.mount.entry = /var/cache/pacman/pkg var/cache/pacman/pkg none bind 0 0
 +
</nowiki>}}
  
  yaourt -S debootstrap # install debootstrap from AUR
+
==== Xorg program considerations (optional) ====
 +
In order to run programs on the host's display, some bind mounts need to be defined so that the containerized programs can access the host's resources.  Add the following section to {{ic|/var/lib/lxc/playtime/config}}:
 +
  ## for xorg
 +
## fix overmounting see: https://github.com/lxc/lxc/issues/434
 +
lxc.mount.entry = tmpfs tmp tmpfs defaults
 +
lxc.mount.entry = /dev/dri dev/dri none bind,optional,create=dir
 +
lxc.mount.entry = /dev/snd dev/snd none bind,optional,create=dir
 +
lxc.mount.entry = /tmp/.X11-unix tmp/.X11-unix none bind,optional,create=dir
 +
lxc.mount.entry = /dev/video0 dev/video0 none bind,optional,create=file
  
# method 1:
+
If you still get a permission denied error in your LXC guest, then you may need to call {{ic|xhost +}} in your host to allow the guest to connect to the host's display server. Take note of the security concerns of opening up your display server by doing this.
sudo debootstrap wheezy /home/lxc/debianfst http://ftp.us.debian.org/debian  # use us mirror site install wheezy version
+
# or, method 2:  use faster tar ball method
+
sudo debootstrap --make-tarball wheezy.packages.tgz sid http://debian.osuosl.org/debian/
+
sudo debootstrap --unpack-tarball wheezy.packages.tgz wheezy debianfs
+
  
====Download existing====
+
==== OpenVPN considerations ====
You can download a base install tar ball. OpenVZ templates work just fine.
+
  
====Using the lxc tools====
+
Users wishing to run [[OpenVPN]] within the container should read the [[OpenVPN in Linux containers]] article.
/usr/bin/lxc-debian {create|destroy|purge|help}
+
/usr/bin/lxc-fedora {create|destroy|purge|help}
+
  
===Creating the device nodes===
+
== Managing Containers ==
Since [[udev]] does not work within the container, you will want to make sure that a certain minimum amount of devices is created for it. This may be done with the following script:
+
#!/bin/bash
+
ROOT=$(pwd)
+
DEV=${ROOT}/dev
+
mv ${DEV} ${DEV}.old
+
mkdir -p ${DEV}
+
mknod -m 666 ${DEV}/null c 1 3
+
mknod -m 666 ${DEV}/zero c 1 5
+
mknod -m 666 ${DEV}/random c 1 8
+
mknod -m 666 ${DEV}/urandom c 1 9
+
mkdir -m 755 ${DEV}/pts
+
mkdir -m 1777 ${DEV}/shm
+
mknod -m 666 ${DEV}/tty c 5 0
+
mknod -m 600 ${DEV}/console c 5 1
+
mknod -m 666 ${DEV}/tty0 c 4 0
+
mknod -m 666 ${DEV}/full c 1 7
+
mknod -m 600 ${DEV}/initctl p
+
mknod -m 666 ${DEV}/ptmx c 5 2
+
  
==Container configuration==
+
To list all installed LXC containers:
 +
# lxc-ls -f
  
===Configuration file===
+
Systemd can be used to [[start]] and to [[stop]] LXCs via {{ic|lxc@CONTAINER_NAME.service}}.  [[Enable]] {{ic|lxc@CONTAINER_NAME.service}} to have it start when the host system boots.
  
The main configuration files are used to describe how to originally create a container. Though these files may be located anywhere, /etc/lxc is probably a good place.
+
Users can also start/stop LXCs without systemd.
 +
Start a container:
 +
# lxc-start -n CONTAINER_NAME
  
'''23/Aug/2010: Be aware that the kernel may not handle additional whitespace in the configuration file. This has been experienced on "lxc.cgroup.devices.allow" settings but may also be true on other settings. If in doubt use only one space wherever whitespace is required.'''
+
Stop a container:
 +
# lxc-stop -n CONTAINER_NAME
  
====Basic settings====
+
To login into a container:
 +
# lxc-console -n CONTAINER_NAME
  
lxc.utsname = $CONTAINER_NAME<br>
+
Once logged, treat the container like any other linux system, set the root password, create users, install packages, etc.
lxc.mount = $CONTAINER_FSTAB
+
lxc.rootfs = $CONTAINER_ROOTFS<br>
+
lxc.network.type = veth
+
lxc.network.flags = up
+
lxc.network.link = br0
+
lxc.network.hwaddr = $CONTAINER_MACADDR
+
lxc.network.ipv4 = $CONTAINER_IPADDR
+
lxc.network.name = $CONTAINER_DEVICENAME
+
  
=====Basic settings explained=====
+
To attach to a container:
 +
# lxc-attach -n CONTAINER_NAME
  
'''lxc.utsname''' : This will be the name of the cgroup for the container. Once the container is started, you should be able to see a new folder named ''/cgroup/$CONTAINER_NAME''.
+
It works nearly the same as lxc-console, but you are automatically accessing root prompt inside the container, bypassing login.
  
Furthermore, this will also be the value returned by ''hostname'' from within the container. Assuming you have not removed access, the container may overwrite this with it's init script.
+
== Running Xorg programs ==
 +
Either attach to or [[SSH]] into the target container and prefix the call to the program with the DISPLAY ID of the host's X session.  For most simple setups, the display is always 0.
  
'''lxc.mount''' : This points to an fstab formatted file that is a listing of the mount points used when ''lxc-start'' is called. This file is further explained [[#Configuring fstab|further]]
+
An example of running Firefox from the container in the host's display:
 +
$ DISPLAY=:0 firefox
  
====Terminal settings====
+
Alternatively, to avoid directly attaching to or connecting to the container, the following can be used on the host to automate the process:
 +
# lxc-attach -n playtime --clear-env -- sudo -u YOURUSER env DISPLAY=:0 firefox
  
The following configuration is optional. You may add them to your main configuration file if you wish to login via lxc-console, or through a terminal ( e.g.: {{Keypress|Ctrl+Alt+F1}} ).
+
== Troubleshooting ==
  
The container can be configured with virtual consoles (tty devices). These may be devices from the host that the container is given permission to use (by its configuration file) or they may be devices created locally within the container.
+
=== root login fails ===
  
The host's virtual consoles are accessed using the key sequence {{Keypress|Alt+Fn}} (or {{Keypress|Ctrl+Alt+Fn}} from within an X11 session). The left {{Keypress|Alt}} key reaches consoles 1 through 12 and the right {{Keypress|Alt}} key reaches consoles 13 through 24. Further virtual consoles may be reached by the {{Keypress|Alt+→}} key sequence which steps to the next virtual console.
+
If you get the following error when you try to login using lxc-console:
  
The container's local virtual consoles may be accessed using the "lxc-console" command.
+
login: root
 +
Login incorrect
  
===== Host Virtual Consoles =====
+
And the container's {{ic|journalctl}} shows:
  
The container may access the host's virtual consoles if the host is not using them and the container's configuration allows it. Typical container configuration would deny access to all devices and then allow access to specific devices like this:
+
pam_securetty(login:auth): access denied: tty 'pts/0' is not secure !
  
  lxc.cgroup.devices.deny = a          # Deny all access to devices
+
Add {{ic|pts/0}} to the list of terminal names in {{ic|/etc/securetty}} on the '''container''' filesystem, see [http://unix.stackexchange.com/questions/41840/effect-of-entries-in-etc-securetty/41939#41939]. You can also opt to delete {{ic|/etc/securetty}} on the '''container''' to allow always root to login, see [https://github.com/systemd/systemd/issues/852].
  lxc.cgroup.devices.allow = c 4:0 rwm # /dev/tty0
+
  lxc.cgroup.devices.allow = c 4:1 rwm # /dev/tty1
+
  lxc.cgroup.devices.allow = c 4:2 rwm # /dev/tty2
+
  
For a container to be able to use a host's virtual console it must not be in use by the host. This will most likely require the host's {{ic|/etc/inittab}} to be modified to ensure no getty or other process runs on any virtual console that is to be used by the container.
+
Alternatively, create a new user in lxc-attach and use it for logging in to the system, then switch to root.
  
After editing the host's {{ic|/etc/inittab}} file, issung a {{ic|killall -HUP init}} will terminate any getty processes that are no longer configured and this will free up the virtual conosole for use by the container.
+
# lxc-attach -n playtime
 +
[root@playtime]# useradd -m -Gwheel newuser
 +
[root@playtime]# passwd newuser
 +
[root@playtime]# passwd root
 +
[root@playtime]# exit
 +
# lxc-console -n playtime
 +
[newuser@playtime]$ su
  
Note that local virtual consoles take precedence over host virtual consoles. This is described in the next section.
+
===no network-connection with veth in container config===
  
===== Local Virtual Consoles =====
+
If you can't access your LAN or WAN with a networking interface configured as '''veth''' and setup through {{ic|/etc/lxc/''containername''/config}}.
 +
If the virtual interface gets the ip assigned and should be connected to the network correctly.
 +
ip addr show veth0
 +
inet 192.168.1.111/24
 +
You may disable all the relevant static ip formulas and try setting the ip through the booted container-os like you would normaly do.
  
The number of local virtual consoles that the container has is defined in the container's configuration file (normally on the host in {{ic|/etc/lxc}}). It is defined thus:
+
Example {{ic|''container''/config}}
  
  lxc.tty = n
+
  ...
 
+
  lxc.network.type = veth
where {{ic|n}} is the number of local virtual consoles required.
+
  lxc.network.name = veth0
 
+
  lxc.network.flags = up
The local virtual consoles are numbered starting at tty1 and take precedence over any of the host's virtual consoles that the container might be entitled to use. This means that, for example, if n = 2 then the container will not be able to use the host's tty1 and tty2 devices even entitled to do so by its configuration file. Setting n to 0 will prevent local virtual consoles from being created thus allowing full access to any of host's virtual consoles that the container might be entitled to use.
+
  lxc.network.link = {{ic|bridge}}
 
+
  ...
===== /dev/tty Device Files =====
+
The container must have a tty device file (e.g. {{ic|/dev/tty1}}) for each virtual console (host or local). These can be created thus:
+
  # mknod -m 666 /dev/tty1 c 4 1
+
# mknod -m 666 /dev/tty2 c 4 2
+
 
+
and so on...
+
 
+
In the above, {{ic|c}} means character device, {{ic|4}} is the major device number (tty devices) and {{ic|1}}, {{ic|2}}, {{ic|3}}, etc., is the minor device number (specific tty device). Note that {{ic|/dev/tty0}} is special and always refers to the current virtual console.
+
 
+
For further info on tty devices, read this: http://www.kernel.org/pub/linux/docs/device-list/devices.txt
+
 
+
'''If a virtual console's device file does not exist in the container, then the container cannot use the virtual console.'''
+
 
+
===== Configuring Log-In Ability =====
+
 
+
The container's virtual consoles may be used for login sessions if the container runs "getty" services on their tty devices. This is normally done by the container's "init" process and is configured in the container's {{ic|/etc/inittab}} file using lines like this:
+
 
+
  c1:2345:respawn:/sbin/agetty -8 38400 tty1 linux
+
 
+
There is one line per device. The first part {{ic|c1}} is just a unique label, the second part defines applicable run levels, the third part tells init to start a new getty when the current one terminates and the last part gives the command line for the getty. For further information refer to {{ic|man init}}.
+
 
+
If there is no getty process on a virtual console it will not be possible to log in via that virtual console. A getty is not required on a virtual console unless it is to be used to log in.
+
 
+
If a virtual console is to allow root logins it also needs to be listed in the container's {{ic|/etc/securetty}} file.
+
 
+
===== Troubleshooting virtual consoles =====
+
 
+
If lxc.tty is set to a number, n, then no host devices numbered n or below will be accessible even if the above configuration is present because they will be replaced with local virtual consoles instead.
+
 
+
A tty device file's major number will change from 4 to 136 if it is a local virtual console. This change is visible within the container but not when viewing the container's devices from the host's filesystem. This information is useful when troubleshooting.
+
 
+
This can be checked from within a container thus:
+
 
+
  # ls -Al /dev/tty*
+
  crw------- 1 root root 136, 10 Aug 21 21:28 /dev/tty1
+
  crw------- 1 root root  4, 2 Aug 21 21:28 /dev/tty2
+
 
+
===== Pseudo Terminals =====
+
 
+
  lxc.pseudo = 1024
+
 
+
Maximum amount of pseudo terminals that may be created in {{ic|/dev/pts}}. Currently, assuming the kernel was compiled with {{ic|CONFIG_DEVPTS_MULTIPLE_INSTANCES}}, this tells lxc-start to mount the devpts filesystem with the newinstance flag.
+
 
+
====Host device access settings====
+
 
+
  lxc.cgroup.devices.deny = a # Deny all access to devices<br>
+
  lxc.cgroup.devices.allow = c 1:3 rwm # dev/null
+
  lxc.cgroup.devices.allow = c 1:5 rwm # dev/zero<br>
+
lxc.cgroup.devices.allow = c 5:1 rwm # dev/console
+
lxc.cgroup.devices.allow = c 5:0 rwm # dev/tty
+
lxc.cgroup.devices.allow = c 4:0 rwm # dev/tty0<br>
+
lxc.cgroup.devices.allow = c 1:9 rwm # dev/urandom
+
lxc.cgroup.devices.allow = c 1:8 rwm # dev/random
+
lxc.cgroup.devices.allow = c 136:* rwm # dev/pts/*
+
lxc.cgroup.devices.allow = c 5:2 rwm # dev/pts/ptmx<br>
+
# No idea what this is .. dev/bsg/0:0:0:0 ???
+
lxc.cgroup.devices.allow = c 254:0 rwm
+
 
+
=====Host device access settings explained=====
+
 
+
'''lxc.cgroup.devices.deny''' : By settings this to ''a'', we are stating that the container has access to no devices unless explicitely defined within the configuration file.
+
 
+
===Configuration file notes===
+
====At runtime /dev/ttyX devices are recreated====
+
If you have enabled multiple DevPTS instances in your kernel, lxc-start will recreate ''lxc.tty'' amount of {{ic|/dev/ttyX}} devices when it is executed.
+
 
+
This means that you will have ''lxc.tty'' amount of pseudo ttys. If you are planning on accessing the container via a "real" terminal ({{Keypress|Ctrl+Alt+FX}}), make sure that it is a number that is inferior to ''lxc.tty''.
+
 
+
To tell whether it has been re-created, just log in to the container via either lxc-console or SSH and perform a {{ic|ls -Al}} command on the tty. Devices with a major number of 4 are "real" tty devices whereas a major number of 136 indicates a pts.
+
 
+
Be aware that this is only visible from within the container itself and not from the host.
+
 
+
====Containers have access to host's TTY nodes====
+
 
+
If you do not properly restrict the container's access to the /dev/tty nodes, the container may have access to the host's.
+
 
+
Taking into consideration that, as previously mentioned, lxc-start recreates ''lxc.tty'' amount of /dev/tty devices, any tty nodes present in the container that are of a greater minor number than ''lxc.tty'' will be linked to the host's.
+
 
+
=====To access the container from a host TTY=====
+
 
+
# On the host, verify no getty is started for that tty by checking ''/etc/inittab''.
+
# In the container, start a getty for that tty.
+
 
+
=====To prevent access to the host TTY=====
+
 
+
Please have a look at the configuration statements found in [[#Host device access settings|host device access settings]].
+
 
+
Via the ''lxc.cgroup.devices.deny = a'' we are preventing access to all host level devices. And then, throuh ''lxc.cgroup.devices.allow = c 4:'''1''' rwm'' we are allowing access to the host's /dev/tty'''1'''. In the above example, simply removing all allow statements for major number 4 and minor > 1 should be sufficient.
+
 
+
=====To test this access=====
+
 
+
I may be off here, but looking at the output of the ''ls'' command below should show you both the ''major'' and ''minor'' device numbers. These are located after the user and group and represented as : 4, 2
+
 
+
# Set lxc.tty to 1
+
# Make there that the container has dev/tty1 and /dev/tty2
+
# ''lxc-start'' the container
+
# ''lxc-console'' into the container
+
# ''ls -Al /dev/tty''<br>crw------- 1 root root 4, 2 Dec 2 00:20 /dev/tty2
+
# ''echo "test output" > /dev/tty2''
+
# ''Ctrl+Alt+F2'' to view the host's second terminal
+
# You should see "test output" printed on the screen
+
 
+
====Configuration troubleshooting====
+
 
+
=====console access denied: Permission denied=====
+
 
+
If, when executing lxc-console, you receive the error ''lxc-console: console access denied: Permission denied'' you have most likely either omitted lxc.tty or set it to 0.
+
 
+
=====lxc-console does not provide a login prompt=====
+
 
+
Though you are reaching a tty on the container, it most likely is not running a getty. You will want to double check that you have a getty defined in the container's ''/etc/inittab'' for the specific tty.
+
 
+
===Configuring fstab===
+
none $CONTAINER_ROOTFS/dev/pts devpts defaults 0 0
+
none $CONTAINER_ROOTFS/proc    proc  defaults 0 0
+
none $CONTAINER_ROOTFS/sys    sysfs  defaults 0 0
+
none $CONTAINER_ROOTFS/dev/shm tmpfs  defaults 0 0
+
 
+
This fstab is used by lxc-start when mounting the container. As such, you can define any mount that would be possible on the host such as bind mounting to the host's own filesystem. However, please be aware of any and all security implications that this may have.
+
 
+
'''Warning''' : You certainly do not want to bind mount the host's /dev to the container as this would allow it to, amongst other things, reboot the host.
+
 
+
==Container Creation and Destruction==
+
 
+
===Creation===
+
lxc-create -f $CONTAINER_CONFIGPATH -n $CONTAINER_NAME
+
 
+
''lxc-create'' will create /var/lib/lxc/$CONTAINER_NAME with a new copy of the container configuration file found in $CONTAINER_CONFIGPATH.
+
 
+
As such, if you need to make modifications to the container's configuration file, it's advisable to modify only the original file and then perform ''lxc-destroy'' and ''lxc-create'' operations afterwards. No data will be lost by doing this.
+
 
+
'''Note''' : When copying the file over, lxc-create will strip all comments from the file.
+
 
+
'''Note''' : As of lxc-git from atleast ''2009-12-01'', performing lxc-create no longer splits the config file into multiple files and folders. Therefore, we only have the configuration file to worry about.
+
 
+
===Destruction===
+
lxc-destroy -n $CONTAINER_NAME
+
 
+
This will delete /var/lib/lxc/$CONTAINER_NAME which only contains configuration files. No data will be lost.
+
 
+
==Readying the host for virtualization==
+
===/etc/inittab===
+
# Comment out any getty that are not required
+
 
+
===/etc/rc.sysinit replacement===
+
Since we are running in a virtual environment, a number of steps undertaken by rc.sysinit are superfluous and may even flat out fail or stall. As such, until the initscripts are made virtualization aware, this will take some hack and slash.
+
  
For now, simply replace the file :
+
And then assign your IP through your preferred method '''inside''' the container, see also [[Network configuration#Configure the IP address]].
#!/bin/bash
+
# Whatever is needed to clean out old daemon/service pids from your container
+
rm -f $(find /var/run -name '*pid')
+
rm -f /var/lock/subsys/*<br>
+
# Configure network settings
+
## You can either use dhcp here, manually configure your
+
## interfaces or try to get the rc.d/network script working.
+
## There have been reports that network failed in this
+
## environment.
+
route add default gw 192.168.10.1
+
echo > /etc/resolv.conf search your-domain
+
echo >> /etc/resolv.conf nameserver 192.168.10.1<br>
+
# Initally we do not have any container originated mounts
+
rm -f /etc/mtab
+
touch /etc/mtab
+
  
===/etc/rc.conf cleanup===
+
== See also ==
You may want to remove any and all hardware related daemons from the DAEMONS line. Furthermore, depending on your situation, you may also want to remove the ''network'' daemon.
+
  
===TBC===
+
* [https://www.stgraber.org/2013/12/20/lxc-1-0-blog-post-series/ LXC 1.0 Blog Post Series]
 +
* [http://www.ibm.com/developerworks/linux/library/l-lxc-containers/ LXC@developerWorks]
 +
* [http://docs.docker.io/en/latest/installation/archlinux/ Docker Installation on ArchLinux]

Latest revision as of 10:37, 24 August 2016

Linux Containers (LXC) is an operating-system-level virtualization method for running multiple isolated Linux systems (containers) on a single control host (LXC host). It does not provide a virtual machine, but rather provides a virtual environment that has its own CPU, memory, block I/O, network, etc. space and the resource control mechanism. This is provided by namespaces and cgroups features in Linux kernel on LXC host. It is similar to a chroot, but offers much more isolation.

Alternatives for using containers are systemd-nspawn and docker.

Setup

Required software

Install the lxc and arch-install-scripts packages.

Verify that the running kernel is properly configured to run a container:

$ lxc-checkconfig

Due to security concerns, the default Arch kernel does not ship with the ability to run containers as an unprivileged user; therefore, it is normal to see a missing status for "User namespaces" when running the check. See FS#36969 for this feature request.

Host network configuration

LXCs support different virtual network types and devices (see lxc.container.conf(5)). A bridge device on the host is required for most types of virtual networking. The examples of creating a bridge provided below are not meant to be limiting, but illustrative. Users may use other programs to achieve the same results. A wired and wireless example is provided below, but other setups are possible. Users are referred to the Network bridge article for additional options.

Merge-arrows-2.pngThis article or section is a candidate for merging with Network bridge.Merge-arrows-2.png

Notes: Duplication; this is not a guide on Linux networking. (Discuss in Talk:Linux Containers#)

Example for a wired network

This example uses netctl: a bridge template can be found in /etc/netctl/examples which needs to be edited to match the host network hardware specs and IP ranges of the host network. Below are two example bridge configs, one using a dhcp setup and the other using a static IP setup.

/etc/netctl/lxcbridge
Description="LXC bridge"
Interface=br0
Connection=bridge
BindsToInterfaces=('eno1')
IP=dhcp
SkipForwardingDelay=yes
/etc/netctl/lxcbridge
Description="LXC bridge"
Interface=br0
Connection=bridge
BindsToInterfaces=('eno1')
IP=static
Address=192.168.0.2/24
Gateway='192.168.0.1'
DNS=('192.168.0.1')

Before attempting to start the bridge, disable the running network interface on the host as the bridge will replace it; this depends on how the host network is configured, see Network configuration.

For users already using netctl to manage an adapter, simply switch-to it:

# netctl switch-to lxcbridge
# netctl enable lxcbridge

Verify network connectivity on the host before continuing. This can be accomplished with a simple ping:

$ ping -c 1 www.google.com

Example for a wireless network

Wireless networks cannot be bridged directly; a different method must be used in this case. First, a bridge must be created similar to the previous examples, but it should not have any interface defined to it (other than the virtual interface of the container itself, which is done automatically). Assign a static IP address to the bridge, but do not assign a gateway.

The host must be configured to perform NAT using iptables:

# iptables -t nat -A POSTROUTING -o wlp3s0 -j MASQUERADE

where wlp3s0 is the name of the wireless interface. Enable packet forwarding, which is disabled by default.

The remaining steps are similar, except for one thing: for the container, the gateway must be configured to be the IP address of the host (in this example, it was 192.168.0.2). This is specified in /var/lib/lxc/container_name/config (see the following sections).

Container creation

Select a template from /usr/share/lxc/templates that matches the target distro to containerize. Users wishing to containerize non-Arch distros will need additional packages on the host depending on the target distro:

Run lxc-create to create the container, which installs the root filesystem of the LXC to /var/lib/lxc/CONTAINER_NAME/rootfs by default. Example creating an Arch Linux LXC named "playtime":

# lxc-create -n playtime -t /usr/share/lxc/templates/lxc-archlinux
Tip: Users may optionally install haveged and start haveged.service to avoid a perceived hang during the setup process while waiting for system entropy to be seeded. Without it, the generation of private/GPG keys can add a lengthy wait to the process.
Tip: Users of Btrfs can append -B btrfs to create a Btrfs subvolume for storing containerized rootfs. This comes in handy if cloning containers with the help of lxc-clone command. ZFS users may use -B zfs, correspondingly.
Tip: As of July 2015, creating an empty container using -t none does not work, see the bug report. As a workaround one can use -t /bin/true.

Container configuration

Basic config with networking

System resources to be virtualized/isolated when a process is using the container are defined in /var/lib/lxc/CONTAINER_NAME/config. By default, the creation process will make a minimum setup without networking support. Below is an example config with networking:

/var/lib/lxc/playtime/config
# Template used to create this container: /usr/share/lxc/templates/lxc-archlinux
# Parameters passed to the template:
# For additional config options, please look at lxc.container.conf(5)

## default values
lxc.rootfs = /var/lib/lxc/playtime/rootfs
lxc.utsname = playtime
lxc.arch = x86_64
lxc.include = /usr/share/lxc/config/archlinux.common.conf

## network
lxc.network.type = veth
lxc.network.link = br0
lxc.network.flags = up
lxc.network.ipv4 = 192.168.0.3/24
lxc.network.ipv4.gateway = 192.168.0.1
lxc.network.name = eth0

## mounts
## specify shared filesystem paths in the format below
## make sure that the mount point exists on the lxc
#lxc.mount.entry = /mnt/data/share mnt/data none bind 0 0
#
# if running the same Arch linux on the same architecture it may be
# adventitious to share the package cache directory
#lxc.mount.entry = /var/cache/pacman/pkg var/cache/pacman/pkg none bind 0 0

Xorg program considerations (optional)

In order to run programs on the host's display, some bind mounts need to be defined so that the containerized programs can access the host's resources. Add the following section to /var/lib/lxc/playtime/config:

## for xorg
## fix overmounting see: https://github.com/lxc/lxc/issues/434
lxc.mount.entry = tmpfs tmp tmpfs defaults
lxc.mount.entry = /dev/dri dev/dri none bind,optional,create=dir
lxc.mount.entry = /dev/snd dev/snd none bind,optional,create=dir
lxc.mount.entry = /tmp/.X11-unix tmp/.X11-unix none bind,optional,create=dir
lxc.mount.entry = /dev/video0 dev/video0 none bind,optional,create=file

If you still get a permission denied error in your LXC guest, then you may need to call xhost + in your host to allow the guest to connect to the host's display server. Take note of the security concerns of opening up your display server by doing this.

OpenVPN considerations

Users wishing to run OpenVPN within the container should read the OpenVPN in Linux containers article.

Managing Containers

To list all installed LXC containers:

# lxc-ls -f

Systemd can be used to start and to stop LXCs via lxc@CONTAINER_NAME.service. Enable lxc@CONTAINER_NAME.service to have it start when the host system boots.

Users can also start/stop LXCs without systemd. Start a container:

# lxc-start -n CONTAINER_NAME

Stop a container:

# lxc-stop -n CONTAINER_NAME

To login into a container:

# lxc-console -n CONTAINER_NAME

Once logged, treat the container like any other linux system, set the root password, create users, install packages, etc.

To attach to a container:

# lxc-attach -n CONTAINER_NAME

It works nearly the same as lxc-console, but you are automatically accessing root prompt inside the container, bypassing login.

Running Xorg programs

Either attach to or SSH into the target container and prefix the call to the program with the DISPLAY ID of the host's X session. For most simple setups, the display is always 0.

An example of running Firefox from the container in the host's display:

$ DISPLAY=:0 firefox

Alternatively, to avoid directly attaching to or connecting to the container, the following can be used on the host to automate the process:

# lxc-attach -n playtime --clear-env -- sudo -u YOURUSER env DISPLAY=:0 firefox

Troubleshooting

root login fails

If you get the following error when you try to login using lxc-console:

login: root
Login incorrect

And the container's journalctl shows:

pam_securetty(login:auth): access denied: tty 'pts/0' is not secure !

Add pts/0 to the list of terminal names in /etc/securetty on the container filesystem, see [1]. You can also opt to delete /etc/securetty on the container to allow always root to login, see [2].

Alternatively, create a new user in lxc-attach and use it for logging in to the system, then switch to root.

# lxc-attach -n playtime
[root@playtime]# useradd -m -Gwheel newuser
[root@playtime]# passwd newuser
[root@playtime]# passwd root
[root@playtime]# exit
# lxc-console -n playtime
[newuser@playtime]$ su

no network-connection with veth in container config

If you can't access your LAN or WAN with a networking interface configured as veth and setup through /etc/lxc/containername/config. If the virtual interface gets the ip assigned and should be connected to the network correctly.

ip addr show veth0 
inet 192.168.1.111/24

You may disable all the relevant static ip formulas and try setting the ip through the booted container-os like you would normaly do.

Example container/config

...
lxc.network.type = veth
lxc.network.name = veth0
lxc.network.flags = up
lxc.network.link = bridge
...

And then assign your IP through your preferred method inside the container, see also Network configuration#Configure the IP address.

See also