Difference between revisions of "Kernels/Traditional compilation"

From ArchWiki
Jump to: navigation, search
(Compilation and installation: Duplicated info removed, users at this page are aware of the other options stated at the introduction.)
m (Bootloader configuration: correction to install.sh path)
(34 intermediate revisions by 10 users not shown)
Line 1: Line 1:
[[Category:Kernel (English)]]
+
[[Category:Kernel]]
{{i18n|Kernel Compilation/Traditional}}
+
[[it:Kernels/Compilation/Traditional]]
 
+
[[ru:Kernel Compilation without ABS]]
 +
[[zh-CN:Kernels/Compilation/Traditional]]
 
The summary below is helpful for building custom kernels from '''kernel.org sources'''.  This method of compiling kernels is the traditional method common to all distros; however, an excellent method of cleanly installing the custom kernel with makepkg and pacman is also included.  
 
The summary below is helpful for building custom kernels from '''kernel.org sources'''.  This method of compiling kernels is the traditional method common to all distros; however, an excellent method of cleanly installing the custom kernel with makepkg and pacman is also included.  
  
Alternatively, you can use ABS to build and install your kernel; see: [[Kernel Compilation]]. Using the existing {{Pkg|linux}} PKGBUILD will automate most of the process and will result in a package. However, some Arch users prefer the {{ic|traditional}} way.
+
Alternatively, you can use ABS to build and install your kernel; see: [[Kernels#Compilation]]. Using the existing {{Pkg|linux}} PKGBUILD will automate most of the process and will result in a package. However, some Arch users prefer the traditional way.
 
+
New users who are not fully familiar with compiling software should have a look at [[Kernel Compilation without ABS for New Users]].
+
  
 
== Fetching source ==
 
== Fetching source ==
* Fetch the kernel source from {{ic|ftp.xx.kernel.org/pub/linux/kernel/}}, where xx is your country key (f.e. 'us', 'uk', 'de', ... - Check [http://www.kernel.org] for a complete list of mirrors). If you have no ftp gui, you can use {{ic|wget}}. For this example, we will fetch and compile 2.6.32.8; you should need to change only the version to get a different kernel.
+
* Fetch the kernel source from {{ic|ftp.xx.kernel.org/pub/linux/kernel/}}, where xx is your country key (e.g. 'us', 'uk', 'de', ... - Check [http://www.kernel.org/mirrors/] for a complete list of mirrors). If you have no ftp gui, you can use {{ic|wget}}. For this example, we will fetch and compile 3.2.9; you should need to change only the version to get a different kernel.
 
For instance:
 
For instance:
  $ wget -c http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.32.8.tar.bz2
+
  $ wget -c http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.2.9.tar.bz2
* It is always a good idea to verify the signature for any downloaded tarball. See http://kernel.org/signature.html for how this works.  Basically:
+
* It is always a good idea to verify the signature for any downloaded tarball. See [http://kernel.org/signature.html#using-gnupg-to-verify-kernel-signatures kernel.org/signature] for how this works and other details.   
$ gpg --keyserver pgp.keyserver.domain --recv-keys 0x12345678
+
$ gpg --verify linux-2.6.32.8.tar.sign linux-2.6.32.8.tar
+
{{Note|Since kernel.org security was compromised over the summer of 2011, the old public key 517D0F0E is considered obsoleteNewer kernel tarballs (>3.0) are currently being signed with key: 00411886.}}
+
  
 
* Copy the kernel source to your build directory, e.g.:
 
* Copy the kernel source to your build directory, e.g.:
  $ cp linux-2.6.32.8.tar.bz2 ~/kernelbuild/
+
  $ cp linux-3.2.9.tar.bz2 ~/kernelbuild/
  
 
* Unpack it and enter the source directory:
 
* Unpack it and enter the source directory:
 
  $ cd ~/kernelbuild
 
  $ cd ~/kernelbuild
  $ tar -xvjf linux-2.6.32.8.tar.bz2
+
  $ tar -xvjf linux-3.2.9.tar.bz2
  $ cd linux-2.6.32.8
+
  $ cd linux-3.2.9
 
Prepare for compilation by running the following command:
 
Prepare for compilation by running the following command:
 
  make mrproper
 
  make mrproper
 
This ensures that the kernel tree is absolutely clean. The kernel team recommends that this command be issued prior to each kernel compilation. Do not rely on the source tree being clean after un-tarring.
 
This ensures that the kernel tree is absolutely clean. The kernel team recommends that this command be issued prior to each kernel compilation. Do not rely on the source tree being clean after un-tarring.
 +
 +
=== What about /usr/src/ ? ===
 +
Using the /usr/src/ directory for kernel compilation as root, along with the creation of the corresponding symlink, is considered poor practice by some kernel hackers. They consider the cleanest method to simply use your home directory. If you subscribe to this point of view, build and configure your kernel as normal user, and install as root, or [[Kernels/Compilation/Arch Build System|with makepkg and pacman]].
 +
 +
However, this concept has been the target of debate, and other very experienced hackers consider the practice of compiling as root under /usr/src/ to be completely safe, acceptable and even preferable.
 +
 +
Use whichever method you feel more comfortable with.
  
 
== Build configuration ==
 
== Build configuration ==
 
This is the most crucial step in customizing the kernel to reflect your computer's precise specifications. By setting the configurations in 'menuconfig' properly, your kernel and computer will function most efficiently.
 
This is the most crucial step in customizing the kernel to reflect your computer's precise specifications. By setting the configurations in 'menuconfig' properly, your kernel and computer will function most efficiently.
  
=== Get .config file ===
+
=== Pre-configuration ===
Optional. Copy the .config file from the running kernel, if you want to modify default Arch settings.
+
Optional, but strongly recommended for first-timers:
 +
*Copy the .config file from the running kernel, if you want to modify default Arch settings.
 
   $ zcat /proc/config.gz > .config
 
   $ zcat /proc/config.gz > .config
 +
*Note the output of currently loaded modules with {{ic|lsmod}}.  This will be specific to each system.
  
 
=== Configure your kernel ===
 
=== Configure your kernel ===
There are two choice:
+
{{Warning| If compiling the ''radeon'' driver into the kernel(>3.3.3) with a newer video card, you '''must''' include the firmware files for your card.  Otherwise acceleration will be crippled. See [http://wiki.x.org/wiki/radeonBuildHowTo#Missing_Extra_Firmware here]}}
* Traditional menuconfig
+
 
 +
There are two main choices:
 +
==== Traditional menuconfig ====
 
  $ make menuconfig        (Will start with a fresh '.config'.  Option dependencies are usually automatically selected.)
 
  $ make menuconfig        (Will start with a fresh '.config'.  Option dependencies are usually automatically selected.)
  
Make your changes to the kernel and save your config file. It is a good idea to make a backup copy, since you will likely be doing this multiple times until you get all the options right.
+
Make your changes to the kernel and save your config file. It is a good idea to make a backup copy, since you could be doing this multiple times until you get all the options right. If unsure, only change a few options between compiles.  If you cannot boot your newly built kernel, see the list of necessary config items [https://www.archlinux.org/news/users-of-unofficial-kernels-must-enable-devtmpfs-support/ here]. Running {{ic|$ lspci -k #}}  from liveCD lists names of kernel modules in use.  Most importantly, you must maintain {{ic|cgroup}} support.  This is necessary for [[systemd]].
  
* localmodconfig  
+
==== localmodconfig ====
Since kernel 2.6.32, localmodconfig is provided to ease kernel configuration:
+
Since kernel 2.6.32, localmodconfig is provided to ease kernel configuration. To use this option:
 
+
# Boot the distribution kernel, and plug in any devices that you expect to use on the system, which will load the kernel drivers for them. # Go into your kernel source directory, and run {{bc|$make localmodconfig}}
$ make oldconfig        (Only works with the old '.config' file, copied into the build directory.  Also marks previously unused options as 'NEW'.)
+
# That option will dig through your system and find the kernel configuration for the running kernel (which is usually at /proc/config.gz, but can sometimes be located in the boot partition, depending on the distribution).  
$ make localmodconfig   (Tries to extract /proc/config.gz from running kernel. Pre-selecting options/modules in use.)
+
# Then, the script will remove all options for kernel modules that are not currently loaded, stripping down the number of drivers that will be built significantly.  
$ make localyesconfig    (Same as above, except that as many modules as possible compiled into the kernel.)
+
# The resulting configuration file will be written to the .config file, and then you can build the kernel and install it as normal.
$ make xconfig          (Depends on Qt. A nicer interface. Dependency checking not verified.)
+
$ make gconfig          (Depends on GTK. Otherwise same as xconfig.)
+
$ make help              (Lists ALL targets available.)
+
 
+
For more information about the build target "localmodconfig" refer to the [http://kernelnewbies.org/Linux_2_6_32#head-11f54cdac41ad6150ef817fd68597554d9d05a5f 2.6.32 release notes].''
+
  
 
==== Local version ====
 
==== Local version ====
Line 60: Line 62:
 
  General setup  --->
 
  General setup  --->
 
   (-ARCH) Local version - append to kernel release '3.n.n-RCn'
 
   (-ARCH) Local version - append to kernel release '3.n.n-RCn'
 
=== What about /usr/src/ ? ===
 
Using the /usr/src/ directory for kernel compilation as root, along with the creation of the corresponding symlink, is considered poor practice by some kernel hackers. They consider the cleanest method to simply use your home directory. If you subscribe to this point of view, build and configure your kernel as normal user, and install as root, or with makepkg and pacman (covered below).
 
 
However, this concept has been the target of debate, and other very experienced hackers consider the practice of compiling as root under /usr/src/ to be completely safe, acceptable and even preferable.
 
 
Use whichever method you feel more comfortable with.
 
  
 
== Compilation and installation ==
 
== Compilation and installation ==
Line 72: Line 67:
  
 
=== Compile ===
 
=== Compile ===
$ make
 
(Same as make vmlinux && make modules && make bzImage - see make help for more information on this.)
 
 
{{Warning | Do not run {{ic|make all}} if you use GRUB and still have LILO installed; it will configure LILO in the end, and you may no longer be able to boot your machine! Remove LILO (pacman -R lilo) before running {{ic|make all}} if you use GRUB!}}
 
{{Warning | Do not run {{ic|make all}} if you use GRUB and still have LILO installed; it will configure LILO in the end, and you may no longer be able to boot your machine! Remove LILO (pacman -R lilo) before running {{ic|make all}} if you use GRUB!}}
 
+
$ make      (Same as make vmlinux && make modules && make bzImage - see 'make help' for more information on this.)
If you have a multi-core computer (i.e Dual Core, Quad Core, etc), you can compile the kernel faster by adding the -j flag.  This will use all processors at 100%
+
or
  $ make -j[# of processors + 1]
+
  $ make -jN  (N = # of processors + 1) (This utilizes all CPUs at 100%.)
 
+
Compilation time will vary from 15 minutes to over an hour. This is largely based on how many options/modules are selected, as well as processor capability.
{{Note|It is recommended to put one more than how many processors you have. For example: on a dual core machine you could compile the kernel with make -j3.  A Dual-core 2.8Ghz compiled in only 10-15 minutes.}}
+
  
 
=== Install modules ===
 
=== Install modules ===
Line 85: Line 77:
 
  # make modules_install
 
  # make modules_install
  
This copies the compiled modules into a directory in /lib/modules named by the kernel version and appended string you set in menuconfig. This way, modules are kept separate from those used by other kernels on your machine.
+
This copies the compiled modules into <code>/lib/modules/</code>[kernel version + CONFIG_LOCALVERSION]. This way, modules can be kept separate from those used by other kernels on your machine.
  
 
=== Copy kernel to /boot directory ===
 
=== Copy kernel to /boot directory ===
Line 93: Line 85:
 
The initial RAM disk (initrd option in the GRUB menu, or, the file "initramfs-YourKernelName.img") is an initial root file system that is mounted prior to when the real root file system is available. The initrd is bound to the kernel and loaded as part of the kernel boot procedure. The kernel then mounts this initrd as part of the two-stage boot process to load the modules to make the real file systems available and get at the real root file system. The initrd contains a minimal set of directories and executables to achieve this, such as the insmod tool to install kernel modules into the kernel. In the case of desktop or server Linux systems, the initrd is a transient file system. Its lifetime is short, only serving as a bridge to the real root file system. In embedded systems with no mutable storage, the initrd is the permanent root file system.
 
The initial RAM disk (initrd option in the GRUB menu, or, the file "initramfs-YourKernelName.img") is an initial root file system that is mounted prior to when the real root file system is available. The initrd is bound to the kernel and loaded as part of the kernel boot procedure. The kernel then mounts this initrd as part of the two-stage boot process to load the modules to make the real file systems available and get at the real root file system. The initrd contains a minimal set of directories and executables to achieve this, such as the insmod tool to install kernel modules into the kernel. In the case of desktop or server Linux systems, the initrd is a transient file system. Its lifetime is short, only serving as a bridge to the real root file system. In embedded systems with no mutable storage, the initrd is the permanent root file system.
  
If you need any modules loaded in order to mount the root filesystem, build a ramdisk (most users need this). The -k parameter accepts the kernel version and appended string you set in menuconfig and is used to locate the modules in /lib/modules:
+
If you need any modules loaded in order to mount the root filesystem, build a ramdisk (most users need this). The -k parameter accepts the kernel version and appended string you set in menuconfig and is used to locate the corresponding modules directory in '/usr/lib/modules':
  
 
  # mkinitcpio -k FullKernelName -g /boot/initramfs-YourKernelName.img
 
  # mkinitcpio -k FullKernelName -g /boot/initramfs-YourKernelName.img
  
 
You are free to name the /boot files anything you want.  However, using the [kernel-major-minor-revision] naming scheme helps to keep order if you: Keep multiple kernels/ Use mkinitcpio often/ Build third-party modules.
 
You are free to name the /boot files anything you want.  However, using the [kernel-major-minor-revision] naming scheme helps to keep order if you: Keep multiple kernels/ Use mkinitcpio often/ Build third-party modules.
 +
{{Tip| If rebuilding images often, it might be helpful to create a separate preset file resulting in the command being something like:<code># mkinitcpio -p custom</code>. See [[mkinitcpio#Image_creation_and_activation| here]]}}
  
 
If you are using LILO and it cannot communicate with the kernel device-mapper driver, you have to run {{ic|modprobe dm-mod}} first.
 
If you are using LILO and it cannot communicate with the kernel device-mapper driver, you have to run {{ic|modprobe dm-mod}} first.
Line 117: Line 110:
 
== Bootloader configuration ==
 
== Bootloader configuration ==
  
Add an entry for your amazing new kernel in your bootloader's configuration file - see [[GRUB]] or [[LILO]] for examples. Note that if you use LILO, the kernel sources include a script to automate the process:
+
Add an entry for your amazing new kernel in your bootloader's configuration file - see [[GRUB]], [[LILO]], [[GRUB2]] or [[Syslinux]] for examples. Note that if you use LILO, the kernel sources include a script to automate the process:
  
  $ arch/i386/boot/install.sh
+
  $ arch/x86/boot/install.sh
  
 
If you use LILO, remember to type {{ic|lilo}} as root at the prompt to update it.
 
If you use LILO, remember to type {{ic|lilo}} as root at the prompt to update it.
 +
 +
If you use Grub 2, you will probably want to add a menuentry to '/etc/grub.d/40_custom'. The simplest way is to copy an existing menuentry found in your '/boot/grub/grub.cfg' file, modify the entry to include the name of the kernel and image you created, and update grub ('grub-mkconfig -o /boot/grub/grub.cfg').
  
 
== Using the NVIDIA video driver with your custom kernel ==
 
== Using the NVIDIA video driver with your custom kernel ==
To use the NVIDIA driver with your new custom kernel, see: [[NVIDIA#Alternate_install:_custom_kernel|Installing the driver for a custom kernel]]. You can also install nvidia drivers from AUR.
+
To use the NVIDIA driver with your new custom kernel, see: [[NVIDIA#Alternate install: custom kernel|How to install nVIDIA driver with custom kernel]]. You can also install nvidia drivers from AUR.

Revision as of 19:53, 4 January 2013

The summary below is helpful for building custom kernels from kernel.org sources. This method of compiling kernels is the traditional method common to all distros; however, an excellent method of cleanly installing the custom kernel with makepkg and pacman is also included.

Alternatively, you can use ABS to build and install your kernel; see: Kernels#Compilation. Using the existing linux PKGBUILD will automate most of the process and will result in a package. However, some Arch users prefer the traditional way.

Fetching source

  • Fetch the kernel source from ftp.xx.kernel.org/pub/linux/kernel/, where xx is your country key (e.g. 'us', 'uk', 'de', ... - Check [1] for a complete list of mirrors). If you have no ftp gui, you can use wget. For this example, we will fetch and compile 3.2.9; you should need to change only the version to get a different kernel.

For instance:

$ wget -c http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.2.9.tar.bz2
  • It is always a good idea to verify the signature for any downloaded tarball. See kernel.org/signature for how this works and other details.
  • Copy the kernel source to your build directory, e.g.:
$ cp linux-3.2.9.tar.bz2 ~/kernelbuild/
  • Unpack it and enter the source directory:
$ cd ~/kernelbuild
$ tar -xvjf linux-3.2.9.tar.bz2
$ cd linux-3.2.9

Prepare for compilation by running the following command:

make mrproper

This ensures that the kernel tree is absolutely clean. The kernel team recommends that this command be issued prior to each kernel compilation. Do not rely on the source tree being clean after un-tarring.

What about /usr/src/ ?

Using the /usr/src/ directory for kernel compilation as root, along with the creation of the corresponding symlink, is considered poor practice by some kernel hackers. They consider the cleanest method to simply use your home directory. If you subscribe to this point of view, build and configure your kernel as normal user, and install as root, or with makepkg and pacman.

However, this concept has been the target of debate, and other very experienced hackers consider the practice of compiling as root under /usr/src/ to be completely safe, acceptable and even preferable.

Use whichever method you feel more comfortable with.

Build configuration

This is the most crucial step in customizing the kernel to reflect your computer's precise specifications. By setting the configurations in 'menuconfig' properly, your kernel and computer will function most efficiently.

Pre-configuration

Optional, but strongly recommended for first-timers:

  • Copy the .config file from the running kernel, if you want to modify default Arch settings.
 $ zcat /proc/config.gz > .config
  • Note the output of currently loaded modules with lsmod. This will be specific to each system.

Configure your kernel

Warning: If compiling the radeon driver into the kernel(>3.3.3) with a newer video card, you must include the firmware files for your card. Otherwise acceleration will be crippled. See here

There are two main choices:

Traditional menuconfig

$ make menuconfig        (Will start with a fresh '.config'.  Option dependencies are usually automatically selected.)

Make your changes to the kernel and save your config file. It is a good idea to make a backup copy, since you could be doing this multiple times until you get all the options right. If unsure, only change a few options between compiles. If you cannot boot your newly built kernel, see the list of necessary config items here. Running $ lspci -k # from liveCD lists names of kernel modules in use. Most importantly, you must maintain cgroup support. This is necessary for systemd.

localmodconfig

Since kernel 2.6.32, localmodconfig is provided to ease kernel configuration. To use this option:

  1. Boot the distribution kernel, and plug in any devices that you expect to use on the system, which will load the kernel drivers for them. # Go into your kernel source directory, and run
    $make localmodconfig
  2. That option will dig through your system and find the kernel configuration for the running kernel (which is usually at /proc/config.gz, but can sometimes be located in the boot partition, depending on the distribution).
  3. Then, the script will remove all options for kernel modules that are not currently loaded, stripping down the number of drivers that will be built significantly.
  4. The resulting configuration file will be written to the .config file, and then you can build the kernel and install it as normal.

Local version

If you are compiling a kernel using your current config file, do not forget to rename your kernel version, or you may replace your existing one by mistake.

$ make menuconfig
General setup  --->
 (-ARCH) Local version - append to kernel release '3.n.n-RCn'

Compilation and installation

To compile kernel manually, follow these steps:

Compile

Warning: Do not run make all if you use GRUB and still have LILO installed; it will configure LILO in the end, and you may no longer be able to boot your machine! Remove LILO (pacman -R lilo) before running make all if you use GRUB!
$ make      (Same as make vmlinux && make modules && make bzImage - see 'make help' for more information on this.)

or

$ make -jN   (N = # of processors + 1) (This utilizes all CPUs at 100%.)

Compilation time will vary from 15 minutes to over an hour. This is largely based on how many options/modules are selected, as well as processor capability.

Install modules

This needs to be done as root.

# make modules_install

This copies the compiled modules into /lib/modules/[kernel version + CONFIG_LOCALVERSION]. This way, modules can be kept separate from those used by other kernels on your machine.

Copy kernel to /boot directory

# cp -v arch/x86/boot/bzImage /boot/vmlinuz-YourKernelName

Make initial RAM disk

The initial RAM disk (initrd option in the GRUB menu, or, the file "initramfs-YourKernelName.img") is an initial root file system that is mounted prior to when the real root file system is available. The initrd is bound to the kernel and loaded as part of the kernel boot procedure. The kernel then mounts this initrd as part of the two-stage boot process to load the modules to make the real file systems available and get at the real root file system. The initrd contains a minimal set of directories and executables to achieve this, such as the insmod tool to install kernel modules into the kernel. In the case of desktop or server Linux systems, the initrd is a transient file system. Its lifetime is short, only serving as a bridge to the real root file system. In embedded systems with no mutable storage, the initrd is the permanent root file system.

If you need any modules loaded in order to mount the root filesystem, build a ramdisk (most users need this). The -k parameter accepts the kernel version and appended string you set in menuconfig and is used to locate the corresponding modules directory in '/usr/lib/modules':

# mkinitcpio -k FullKernelName -g /boot/initramfs-YourKernelName.img

You are free to name the /boot files anything you want. However, using the [kernel-major-minor-revision] naming scheme helps to keep order if you: Keep multiple kernels/ Use mkinitcpio often/ Build third-party modules.

Tip: If rebuilding images often, it might be helpful to create a separate preset file resulting in the command being something like:# mkinitcpio -p custom. See here

If you are using LILO and it cannot communicate with the kernel device-mapper driver, you have to run modprobe dm-mod first.

Copy System.map

The System.map file is not required for booting Linux. It is a type of "phone directory" list of functions in a particular build of a kernel. The System.map contains a list of kernel symbols (i.e function names, variable names etc) and their corresponding addresses. This "symbol-name to address mapping" is used by:

  • Some processes like klogd, ksymoops etc
  • By OOPS handler when information has to be dumped to the screen during a kernel crash (i.e info like in which function it has crashed).

Copy System.map to /boot and create symlink

# cp System.map /boot/System.map-YourKernelName

After completing all steps above, you should have the following 3 files and 1 soft symlink in your /boot directory along with any other previously existing files:

vmlinuz-YourKernelName          (Kernel)
initramfs-YourKernelName.img    (Ramdisk)
System.map-YourKernelName       (System Map)

Bootloader configuration

Add an entry for your amazing new kernel in your bootloader's configuration file - see GRUB, LILO, GRUB2 or Syslinux for examples. Note that if you use LILO, the kernel sources include a script to automate the process:

$ arch/x86/boot/install.sh

If you use LILO, remember to type lilo as root at the prompt to update it.

If you use Grub 2, you will probably want to add a menuentry to '/etc/grub.d/40_custom'. The simplest way is to copy an existing menuentry found in your '/boot/grub/grub.cfg' file, modify the entry to include the name of the kernel and image you created, and update grub ('grub-mkconfig -o /boot/grub/grub.cfg').

Using the NVIDIA video driver with your custom kernel

To use the NVIDIA driver with your new custom kernel, see: How to install nVIDIA driver with custom kernel. You can also install nvidia drivers from AUR.