Difference between revisions of "Arch VServer"

From ArchWiki
Jump to: navigation, search
(Network via dummy adapters: Remove ifconfig info.)
(29 intermediate revisions by 10 users not shown)
Line 1: Line 1:
[[Category:Emulators (English)]][[Category: HOWTOs (English)]]
+
[[Category:Virtualization]]
{{ stub }}
+
This article aims to provide all necessary information regarding the creation of a vserver host as well as vserver guests running Arch Linux. This will enable you to setup virtual servers that provide different services as if they were on different machines, with a very little overhead. You can get more information about virtual servers [http://en.wikipedia.org/wiki/Virtual_private_server here].
{{ translateme }}
+
 
+
=Introduction=
+
 
+
This article aims to provide all necessary information regarding the creation of a vserver host and it's related guests on Arch Linux. This will enable you to setup virtual servers that provide different services as if they were on different machines, with a very little overhead. You can get more information about virtual servers [http://en.wikipedia.org/wiki/Virtual_private_server here].
+
 
+
I've just started playing around with vserver and, since this page was pretty empty, figured I'd jot down some notes. I'll come back to clean things up / clarify things soon(ish) ( [[User:Delerious010|Delerious010]] 11:30, 15 November 2009 (EST) ).
+
  
 
=Preparing the Host=
 
=Preparing the Host=
  
To prepare the vserver host environment, you will need to install both a patch kernel and the vserver utilities which are located in [http://wiki.archlinux.org/index.php/AUR AUR]. In the following steps, the instructions are provided using [http://wiki.archlinux.org/index.php/Yaourt yaourt] to fetch them automatically. This has the aded benefit of pulling down the required dependencies ( ''dietlibc'' and ''beecrypt'' ) automagically :
+
To prepare the vserver host environment, you will need to install both a vserver patched kernel, the vserver utilities and their dependencies which are located in the [[AUR]]. The required packages are {{AUR|dietlibc}}, {{AUR|beecrypt}}, {{AUR|kernel26-vserver}} (not found or {{AUR|linux-vserver}} orphan), and {{AUR|util-vserver}}
# yaourt -S kernel26-vserver util-vserver
+
 
+
'''Please note''' : Since vserver requires a patched kernel, it's possible that your vserver kernel may lag behind the standard kernel26 package release.
+
  
 
=Paths of Interest=
 
=Paths of Interest=
Line 25: Line 15:
 
=Preparing the Guests=
 
=Preparing the Guests=
  
Vserver will launch guests from subfolders of ''/etc/vservers/.defaults/vdirbase'' ( which defaults to /vservers ). As such, creating a new guest system is as simple as installing the required packages in a folder. Furthermore, there's nothing stopping you ( and quite a few things encouraging you ) to mount filesystems to the subfolders of ''vdirbase'' and install your guest in there.
+
Vserver will launch guests from subfolders of ''/etc/vservers/.defaults/vdirbase''. As such, creating a new guest system is as simple as installing the required packages in a folder of the host. Furthermore, there's nothing stopping you ( and quite a few things encouraging you ) to mount filesystems to the subfolders of ''vdirbase'' and installing your guest in there.
  
The following steps are going to be very brief so as not to duplicate information from [http://wiki.archlinux.org/index.php/Install_From_Existing_Linux Install From Existing Linux]. For the most part, this can all be easily automated via a few simple bash scripts.
+
If you plan on doing this often, I highly recommend that you write yourself a little batch script since most of these steps can be automated quite easily.
  
In looking at ''/etc/vservers/.distribution'', i'm getting the fealing that we could just create a template for ArchLinux to fully automate the below steps, however I've not had the time to investigate in more detail.
+
==Preparing the guest installation media==
  
# (optional) Create LVM volume via lvcreate.
+
'''NOTE''' : This is all heavily inspired from [[Install From Existing Linux]] and will therefore be quite brief when not mentioning Vserver specific steps.
# (optional) Create filesystem on lvm volume / disk partition.
+
 
# (optional) Mount newly create filesystem to /vserver/<guest>
+
===Optional: Base variables to follow along with the steps===
# (optional) Link new guests pacman package cache to the host's
+
GuestName= # Name of the guest
## mkdir -p /vserver/<guest>/var/cache/pacman/pkg
+
GuestRoot=/etc/vservers/.defaults/vdirbase/$GuestName
## mount -o bind /var/cache/pacman/pkg /vserver/<guest>/var/cache/pacman/pkg
+
GuestPackages= # Listing of packages to install via pacman
# Prepare guest filesystem for pacman db and install base packages
+
GuestDisk= # Installation target device
 +
GuestNetDevice= # ex.: eth0, dummy0, etc...
 +
GuestIP= # I think you get it
 +
GuestContext= # Unique identifier for the guest, I go with the last part of the IP
 +
 
 +
===Optional: Preparing the guest disk===
 +
# Create a LVM Physical Volume, a Volume Group and a Logical Volume ( [https://wiki.archlinux.org/index.php/Lvm#Create_Logical_Volumes wiki:LVM] )
 +
# Create a filesystem on the lvm volume
 +
# mkdir $GuestRoot
 +
# mount /dev/$GuestDisk $GuestRoot
 +
 
 +
===Optional: Link the host and guest pacman cache===
 +
# mkdir $GuestRoot/var/cache/pacman/pkg
 +
# mount -o bind /var/cache/pacman/pkg $GuestRoot/var/cache/pacman/pkg
 +
 
 +
===Prepare Vserver===
 +
# vserver $GuestName -m skeleton --context $GuestContext --interface $GuestNetDevice:$GuestIP --flags lock,virt_mem,virt_uptime,virt_cpu,virt_load,sched_hard,hide_netif --initstyle plain
 +
# (optional) cd /etc/vservers/$GuestName/interfaces
 +
# (optional) cp -r 0 1
 +
# (optional) echo 'lo' > dev
 +
# (optional) echo '127.0.0.$GuestContext' > ip
 +
 
 +
===Prepare the guest's filesystem===
 +
# Prepare guest filesystem for the pacman db
 
## mkdir -p /newarch/var/lib/pacman  
 
## mkdir -p /newarch/var/lib/pacman  
## pacman -Sy base -r /vserver/<guest><br>'''Note : ''' To save some time, it's probably a good idea to create a text file containing all the packages to install and call it via ''pacman -Sy `cat <file>` -r /vserver/<guest>
+
 
# (optional) If you want to chroot into the newly created guest so as to install new packages, it might be a good idea to mount a few filesystems required by some packages.
+
===Install the base system===
## Bind /dev, /proc, /sys to the corresponding directories in /vserver/<guest>
+
'''NOTE : ''' To save some time, it's probably a good idea to create a text file containing all the packages to install and call it via "pacman -S `cat $GuestPackages` -r $GuestRoot" instead of the following :
 +
# pacman -S base -r $GuestRoot
 +
# Optional: If you want to chroot into the newly created guest so as to install new packages, it might be a good idea to mount a few filesystems required by some packages.
 +
## Bind /dev, /proc, /sys to the corresponding directories in $GuestRoot
 
# Modify guest configuration files to enable a smoother boot process
 
# Modify guest configuration files to enable a smoother boot process
## '''/etc/inittab''' : Delete all lines that create the consoles ( agetty )
+
## Modify '''/etc/rc.shutdown''' by removing anything hardware/clock/mount related. This includes most everything under ''Saving Random Seed'.
## '''/etc/rc.shutdown''' : Still modifying this, but basically you want to remove anything hardware related. This includes most everything under ''Saving Random Seed'.
+
## Modify '''/etc/rc.sysinit''' by anything hardware/clock/mount related.
## Create '''/etc/init.d/rc''' which vserver will call rather than ''/etc/rc.sysinit''.
+
## Modify '''/etc/syslog-ng.conf''' by removing file("/proc/kmsg")
# Create a vserver instance for our newly create filesystem
+
## vserver <guest name> build -m skeleton \<br> --context <unique context number ... 1 through ???> \<br> --interface <device:ip/netmask> \<br> --flags lock,virt_mem,virt_uptime,virt_cpu,virt_load,sched_hard,hide_netif \<br> --initstyle plain
+
## To start on boot : echo default > /etc/vservers/<guest>/apps/init/mark
+
 
+
'''/etc/init.d/rc'''
+
if [ $1 -eq 3 ]; then
+
echo "entering runlevel 3:multi";
+
/etc/rc.multi
+
fi
+
if [ $1 -eq 6 ]; then
+
echo "entering runlevel 6:reboot";
+
/etc/rc.shutdown
+
fi
+
if [ $1 -eq 0 ]; then
+
echo "entering runlevel 0: shutdown";
+
/etc/rc.shutdown
+
fi
+
if [ $1 -eq 4 ]; then
+
echo "entering runlevel 4";
+
fi
+
if [ $1 -eq 5 ]; then
+
echo "entering runlevel 5";
+
fi
+
if [ $1 -eq 1 ]; then
+
echo "entering runlevel 1:single";
+
/etc/rc.single
+
fi
+
if [ $1 -eq 2 ]; then
+
echo "entering runlevel 2:multi";
+
/etc/rc.multi
+
fi
+
  
 
=Troubleshooting=
 
=Troubleshooting=
 +
==Viewing output from vserver $GuestName start / stop==
 +
 +
'''NOTE : ''' For me, this only worked in the actual consoles, not in X.
 +
 +
# Make sure that the device /dev/console exists in the guest
 +
## If it does not, cp -a /dev/console $GuestRoot/dev/
 +
 +
==SSH will not start==
 +
I noticed that /dev/null did not always get created properly in my first experimentations. Therefore I did a quick :
 +
# cp -a /dev/null $GuestRoot/dev
 +
# cp -a /dev/zero $GuestRoot/dev
 +
 +
Furthermore, if you're not using the dummy network driver and are attaching to the host's network interface, you'll want to configure the ListenAddress statement of /etc/ssh/sshd_config so that it binds only to the guest's IP address as opposed to 127.0.0.1.
 +
 +
==SSH immediately terminates the connection==
 +
On my machine, SSH used to authenticate me correctly and log me in, but then immediately drop the connection without an explanation. Consulting /var/log/auth.log revealed the following:
 +
 +
sshd[17899]: pam_limits(sshd:session): Could not set limit for 'nice': Operation not permitted
 +
 +
This is easily fixed by commenting all '''nice''' related lines in /etc/security/limits.conf.
 +
 +
=Tips & Trick=
 +
==Network via dummy adapters==
 +
Here, you're either using the dummy module to create virtual network adapters or created interface aliases via /usr/sbin/ip. I went for the former and configured the host as such :
 +
# ''/etc/sysctl.conf'' : net.ipv4.ip_forward=1<br>
 +
Modify or add that statement to enable routing on the host
 +
# ''/etc/rc.local''<br>
 +
modprobe dummy numdummies=$NumberOfGuests<br>
 +
ip link set dev dummy$GuestContext name $GuestName
  
=Tips & Tricks=
+
This provides me with dummy interfaces that I can route / firewall that are all named the same as my guests... yay.
  
 
=More Resources=
 
=More Resources=
[http://linux-vserver.org/Problematic_Programs Problematic Programs]
+
[http://linux-vserver.org/Problematic_Programs Problematic Programs]<br>
[http://linux-vserver.org/Howto_make_bsd-style_init_sysv-compatible Make BSD style init SYSV compatible]
+
[http://linux-vserver.org/Howto_make_bsd-style_init_sysv-compatible Make BSD style init SYSV compatible]<br>
[http://www.cedarcreeksoftware.com/an-even-easier-linux-vserver-tutorial.html Vserver tutorial]
+
[http://www.cedarcreeksoftware.com/an-even-easier-linux-vserver-tutorial.html Vserver tutorial]<br>
[http://linux-vserver.org/Installation_on_ArchLinux linux-vserver.org's Installation on ArchLinux]
+
[http://linux-vserver.org/Installation_on_ArchLinux linux-vserver.org's Installation on ArchLinux]<br>
[http://wiki.linux-vserver.org/Networking_vserver_guests linux-verserver.org's networking tutorial]
+
[http://wiki.linux-vserver.org/Networking_vserver_guests linux-verserver.org's networking tutorial]<br>
 
<!-- vim: set ft=Wikipedia: -->
 
<!-- vim: set ft=Wikipedia: -->

Revision as of 08:46, 10 April 2013

This article aims to provide all necessary information regarding the creation of a vserver host as well as vserver guests running Arch Linux. This will enable you to setup virtual servers that provide different services as if they were on different machines, with a very little overhead. You can get more information about virtual servers here.

Preparing the Host

To prepare the vserver host environment, you will need to install both a vserver patched kernel, the vserver utilities and their dependencies which are located in the AUR. The required packages are dietlibcAUR, beecryptAUR, kernel26-vserverAUR (not found or linux-vserverAUR orphan), and util-vserverAUR

Paths of Interest

/etc/vservers : configuration root ( reference )

/etc/vservers/.defaults : configuration skeleton used when building new guests
/etc/vservers/.defaults/vdirbase : symlink to the folder containing vserver guests. This defaults to /vservers.
/etc/vservers/<guest name> : guest specific configurations

Preparing the Guests

Vserver will launch guests from subfolders of /etc/vservers/.defaults/vdirbase. As such, creating a new guest system is as simple as installing the required packages in a folder of the host. Furthermore, there's nothing stopping you ( and quite a few things encouraging you ) to mount filesystems to the subfolders of vdirbase and installing your guest in there.

If you plan on doing this often, I highly recommend that you write yourself a little batch script since most of these steps can be automated quite easily.

Preparing the guest installation media

NOTE : This is all heavily inspired from Install From Existing Linux and will therefore be quite brief when not mentioning Vserver specific steps.

Optional: Base variables to follow along with the steps

GuestName= # Name of the guest
GuestRoot=/etc/vservers/.defaults/vdirbase/$GuestName
GuestPackages= # Listing of packages to install via pacman
GuestDisk= # Installation target device
GuestNetDevice= # ex.: eth0, dummy0, etc...
GuestIP= # I think you get it
GuestContext= # Unique identifier for the guest, I go with the last part of the IP

Optional: Preparing the guest disk

  1. Create a LVM Physical Volume, a Volume Group and a Logical Volume ( wiki:LVM )
  2. Create a filesystem on the lvm volume
  3. mkdir $GuestRoot
  4. mount /dev/$GuestDisk $GuestRoot

Optional: Link the host and guest pacman cache

  1. mkdir $GuestRoot/var/cache/pacman/pkg
  2. mount -o bind /var/cache/pacman/pkg $GuestRoot/var/cache/pacman/pkg

Prepare Vserver

  1. vserver $GuestName -m skeleton --context $GuestContext --interface $GuestNetDevice:$GuestIP --flags lock,virt_mem,virt_uptime,virt_cpu,virt_load,sched_hard,hide_netif --initstyle plain
  2. (optional) cd /etc/vservers/$GuestName/interfaces
  3. (optional) cp -r 0 1
  4. (optional) echo 'lo' > dev
  5. (optional) echo '127.0.0.$GuestContext' > ip

Prepare the guest's filesystem

  1. Prepare guest filesystem for the pacman db
    1. mkdir -p /newarch/var/lib/pacman

Install the base system

NOTE : To save some time, it's probably a good idea to create a text file containing all the packages to install and call it via "pacman -S `cat $GuestPackages` -r $GuestRoot" instead of the following :

  1. pacman -S base -r $GuestRoot
  2. Optional: If you want to chroot into the newly created guest so as to install new packages, it might be a good idea to mount a few filesystems required by some packages.
    1. Bind /dev, /proc, /sys to the corresponding directories in $GuestRoot
  3. Modify guest configuration files to enable a smoother boot process
    1. Modify /etc/rc.shutdown by removing anything hardware/clock/mount related. This includes most everything under Saving Random Seed'.
    2. Modify /etc/rc.sysinit by anything hardware/clock/mount related.
    3. Modify /etc/syslog-ng.conf by removing file("/proc/kmsg")

Troubleshooting

Viewing output from vserver $GuestName start / stop

NOTE : For me, this only worked in the actual consoles, not in X.

  1. Make sure that the device /dev/console exists in the guest
    1. If it does not, cp -a /dev/console $GuestRoot/dev/

SSH will not start

I noticed that /dev/null did not always get created properly in my first experimentations. Therefore I did a quick :

  1. cp -a /dev/null $GuestRoot/dev
  2. cp -a /dev/zero $GuestRoot/dev

Furthermore, if you're not using the dummy network driver and are attaching to the host's network interface, you'll want to configure the ListenAddress statement of /etc/ssh/sshd_config so that it binds only to the guest's IP address as opposed to 127.0.0.1.

SSH immediately terminates the connection

On my machine, SSH used to authenticate me correctly and log me in, but then immediately drop the connection without an explanation. Consulting /var/log/auth.log revealed the following:

sshd[17899]: pam_limits(sshd:session): Could not set limit for 'nice': Operation not permitted

This is easily fixed by commenting all nice related lines in /etc/security/limits.conf.

Tips & Trick

Network via dummy adapters

Here, you're either using the dummy module to create virtual network adapters or created interface aliases via /usr/sbin/ip. I went for the former and configured the host as such :

  1. /etc/sysctl.conf : net.ipv4.ip_forward=1

Modify or add that statement to enable routing on the host

  1. /etc/rc.local
modprobe dummy numdummies=$NumberOfGuests
ip link set dev dummy$GuestContext name $GuestName

This provides me with dummy interfaces that I can route / firewall that are all named the same as my guests... yay.

More Resources

Problematic Programs
Make BSD style init SYSV compatible
Vserver tutorial
linux-vserver.org's Installation on ArchLinux
linux-verserver.org's networking tutorial