Kernel/Arch build system: Difference between revisions

From ArchWiki
(update interlanguage links)
m (fix broken section link)
 
(30 intermediate revisions by 13 users not shown)
Line 1: Line 1:
[[Category:Kernel]]
[[Category:Kernel]]
[[de:Eigenen Kernel erstellen]]
[[de:Eigenen Kernel erstellen]]
[[ja:カーネル/コンパイル/Arch Build System]]
[[ja:カーネル/Arch build system]]
[[pt:Kernel (Português)/Arch Build System]]
[[ru:Kernel (Русский)/Arch Build System]]
[[ru:Kernel (Русский)/Arch Build System]]
[[zh-hans:Kernel (简体中文)/Arch Build System]]
[[zh-hans:Kernel/Arch Build System]]
See [[Kernels]] for the main article.
The [[Arch build system]] can be used to build a custom kernel based on the official {{Pkg|linux}} package. This compilation method can automate the entire process, and is based on a very well tested package. You can edit the PKGBUILD to use a custom kernel configuration or add additional [[patch]]es.
 
The [[Arch Build System]] can be used to build a custom kernel based on the official {{Pkg|linux}} package. This compilation method can automate the entire process, and is based on a very well tested package. You can edit the PKGBUILD to use a custom kernel configuration or add additional [[patch]]es.


== Getting the ingredients ==
== Getting the ingredients ==


Since you will be using [[makepkg]], follow the best practices outlined there first. For example, you cannot run makepkg as root/sudo. Therefore, create a {{ic|build}} directory in your user home first.
Since you will be using [[makepkg]], follow the best practices outlined there first. For example, you cannot run makepkg as the [[root user]]. Therefore, create a {{ic|build}} directory in your user home first.


  $ mkdir ~/build/
  $ mkdir ~/build/
  $ cd ~/build/
  $ cd ~/build/


[[Install]] the {{Pkg|asp}} package and the {{Grp|base-devel}} package group.
[[Install]] the {{Pkg|devtools}} and {{Pkg|base-devel}} package.


You need a clean kernel to start your customization from. [[ABS#Retrieve PKGBUILD source using Git|Retrieve PKGBUILD source using Git]] and few other files into your build directory by running:
You need a clean kernel to start your customization from. [[Arch build system#Retrieve PKGBUILD source|Retrieve PKGBUILD source]] and few other files into your build directory by running:


  $ asp update linux
  $ pkgctl repo clone --protocol=https linux
$ asp export linux


At this point, the directory tree looks like (there may be a few other files):
At this point, the directory tree looks like (there may be a few other files):
Line 43: Line 39:
=== Avoid creating the doc ===
=== Avoid creating the doc ===


A large portion of the lengthy [[#Compiling|compiling]] effort is devoted to creating the documentation. As of June 16. 2021, the following patch to PKGBUILD avoids its creation:
A large portion of the lengthy [[#Compiling|compiling]] effort is devoted to creating the documentation. As of 25 August 2022, the following patch to PKGBUILD avoids its creation:


{{bc|1=
{{bc|1=
63d63
63c63
<  make htmldocs
<  make htmldocs all
194c194
---
>  make all
195c195
< pkgname=("$pkgbase" "$pkgbase-headers" "$pkgbase-docs")
< pkgname=("$pkgbase" "$pkgbase-headers" "$pkgbase-docs")
---
---
Line 54: Line 52:
}}
}}


This patch deletes line #63, and changes line #194. You might have to edit the PKGBUILD file manually if it does not apply cleanly.
This patch changes lines #63 and #195. You might have to edit the PKGBUILD file manually if it does not apply cleanly.


=== Changing prepare() ===
=== Changing prepare() ===


In {{ic|prepare()}} function, you can [[Patching_packages#Applying_patches|apply needed kernel patches]] or change kernel build configuration.  
In {{ic|prepare()}} function, you can [[Patching packages#Applying patches|apply needed kernel patches]] or change kernel build configuration. Since [https://gitlab.archlinux.org/archlinux/packaging/packages/linux/-/commit/fbf5b3b8adaf4274e53f35634124e1c3d2e4d328 2018-08-01], the PKGBUILD automatically applies all {{ic|*.patch}} files in source.


If you need to change a few config options you can edit config file in the source.
If you need to change a few configuration options you can edit {{ic|config}} in the source.


Or you can use a GUI tool to tweak the options. Comment {{ic|make olddefconfig}} in the prepare() function of the PKGBUILD, and add your favorite tool:
Or you can use a GUI tool to tweak the options. Comment {{ic|make olddefconfig}} in the prepare() function of the PKGBUILD, and add your favorite tool (run {{ic|make help}} to list all of the possible configuration targets):


{{hc|PKGBUILD|
{{hc|PKGBUILD|
Line 70: Line 68:
   #make olddefconfig
   #make olddefconfig


   make nconfig # new CLI menu for configuration
   make nconfig     # new CLI menu for configuration
   #make menuconfig # CLI menu for configuration
   #make menuconfig # CLI menu for configuration
   #make xconfig # X-based configuration
   #make xconfig   # X-based configuration
   #make oldconfig # using old config from previous kernel version
   #make oldconfig # using old config from previous kernel version
   # ... or manually edit .config
   # ... or manually edit .config
   make prepare
   make prepare
Line 85: Line 83:
[[#Changing prepare()]] suggests a possible modification to {{ic|$_srcname/.config}}. Since this path is not where downloading the package files ended, its checksum was not checked by makepkg (which actually checked {{ic|$_srcname/../../config}}).
[[#Changing prepare()]] suggests a possible modification to {{ic|$_srcname/.config}}. Since this path is not where downloading the package files ended, its checksum was not checked by makepkg (which actually checked {{ic|$_srcname/../../config}}).


If you replaced the downloaded {{ic|config}} with another config file before running makepkg, [[install]] the {{Pkg|pacman-contrib}} package and generate new checksums by running:
If you replaced the downloaded {{ic|config}} with another one before running makepkg, [[install]] the {{Pkg|pacman-contrib}} package and generate new checksums by running:


  $ updpkgsums
  $ updpkgsums
Line 102: Line 100:
* Kernel sources are [https://www.kernel.org/signature.html#kernel-org-web-of-trust PGP signed], and makepkg will attempt to verify them. See [[makepkg#Signature checking]] for details.
* Kernel sources are [https://www.kernel.org/signature.html#kernel-org-web-of-trust PGP signed], and makepkg will attempt to verify them. See [[makepkg#Signature checking]] for details.
* The compilation can take up to several hours to complete depending on the hardware performance. [[makepkg#Parallel compilation|Running compilation jobs simultaneously]] can reduce compilation time significantly on multi-core systems.
* The compilation can take up to several hours to complete depending on the hardware performance. [[makepkg#Parallel compilation|Running compilation jobs simultaneously]] can reduce compilation time significantly on multi-core systems.
* It can be informative to run the above {{ic|makepkg}} using the {{ic|time}} command to know how long your system took to perform the compilation.
}}
}}


Line 119: Line 118:
== Boot loader ==
== Boot loader ==


If you have modified {{ic|pkgbase}} in order to have your new kernel installed alongside the default kernel you will need to update your bootloader configuration file and add new entries ('default' and 'fallback') for your custom kernel and the associated initramfs images.
If you have modified {{ic|pkgbase}} in order to have your new kernel installed alongside the default kernel you will need to update your [[boot loader]] configuration file and add new entries ('default' and 'fallback') for your custom kernel and the associated [[initramfs]] images.


== Updating ==
== Updating ==
Line 164: Line 163:
This shows few specific archlinux patches between {{ic|Arch Linux kernel v5.2.7-arch1}} and {{ic|Linux 5.2.7}}.
This shows few specific archlinux patches between {{ic|Arch Linux kernel v5.2.7-arch1}} and {{ic|Linux 5.2.7}}.


The up to date PKGBUILD, as well archlinux kernel configuration file, can be pulled in by the {{ic|asp}} command:
The up to date PKGBUILD, as well archlinux kernel configuration file, can be pulled in using {{ic|git}} in the package directory:


  $ cd ~/build/linux/
  $ cd ~/build/linux/
  $ asp update linux
  $ git pull
$ asp export linux
 
{{Note|Sometimes the {{ic|asp}} command does not update linux files even though there is a newer archlinux source tag. A possible reason is that archlinux linux files lag behind archlinux {{Pkg|linux}} source.}}


Now you should [[Vim#Merging files|merge files]] located in {{ic|~/build/linux/linux/*}} into {{ic|~/build/linux/}}. Merging can also done manually, or with [[list of applications#Comparison, diff, merge|specific utilities]]. Review [[#Changing prepare()]], and run manually most, if not all, the shell commands of PKGBUILD::prepare().
Now you should [[Vim#Merging files|merge files]] located in {{ic|~/build/linux/linux/*}} into {{ic|~/build/linux/}}. Merging can also done manually, or with [[List of applications/Utilities#Comparison, diff, merge|specific utilities]]. Review [[#Changing prepare()]], and run manually most, if not all, the shell commands of PKGBUILD::prepare().


At this point, {{ic|makepkg --verifysource}} should succeed. While [[#Compiling]], make sure to also add {{ic|--noextract}} option to the {{ic|makepkg}} command, since it should be able to build the packages as if the source was extracted by {{ic|makepkg --nobuild}}. And you are back to [[#Installing]].
At this point, {{ic|makepkg --verifysource}} should succeed. While [[#Compiling]], make sure to also add {{ic|--noextract}} option to the {{ic|makepkg}} command, since it should be able to build the packages as if the source was extracted by {{ic|makepkg --nobuild}}. And you are back to [[#Installing]].
Line 185: Line 181:
== See also ==
== See also ==


* https://www.kernel.org/doc/html/latest/kbuild/kconfig.html and the parent directory
* https://docs.kernel.org/kbuild/kconfig.html and the parent directory

Latest revision as of 13:29, 28 January 2024

The Arch build system can be used to build a custom kernel based on the official linux package. This compilation method can automate the entire process, and is based on a very well tested package. You can edit the PKGBUILD to use a custom kernel configuration or add additional patches.

Getting the ingredients

Since you will be using makepkg, follow the best practices outlined there first. For example, you cannot run makepkg as the root user. Therefore, create a build directory in your user home first.

$ mkdir ~/build/
$ cd ~/build/

Install the devtools and base-devel package.

You need a clean kernel to start your customization from. Retrieve PKGBUILD source and few other files into your build directory by running:

$ pkgctl repo clone --protocol=https linux

At this point, the directory tree looks like (there may be a few other files):

~/build/linux/-+
               +--config
               \__PKGBUILD

Then, get any other file you need (e.g. custom configuration files, patches, etc.) from the respective sources.

Modifying the PKGBUILD

Edit PKGBUILD and look for the pkgbase parameter. Change this to your custom package name, e.g.:

PKGBUILD
pkgbase=linux-custom
Warning: Do not add linux to the provides array. Your custom kernel will not be compatible with binary modules built against that kernel, so it cannot satisfy that dependency. Similarly, do not add linux-headers to the provides array of the headers package, for similar reasons.

Avoid creating the doc

A large portion of the lengthy compiling effort is devoted to creating the documentation. As of 25 August 2022, the following patch to PKGBUILD avoids its creation:

63c63
<   make htmldocs all
---
>   make all
195c195
< pkgname=("$pkgbase" "$pkgbase-headers" "$pkgbase-docs")
---
> pkgname=("$pkgbase" "$pkgbase-headers")

This patch changes lines #63 and #195. You might have to edit the PKGBUILD file manually if it does not apply cleanly.

Changing prepare()

In prepare() function, you can apply needed kernel patches or change kernel build configuration. Since 2018-08-01, the PKGBUILD automatically applies all *.patch files in source.

If you need to change a few configuration options you can edit config in the source.

Or you can use a GUI tool to tweak the options. Comment make olddefconfig in the prepare() function of the PKGBUILD, and add your favorite tool (run make help to list all of the possible configuration targets):

PKGBUILD
...
  msg2 "Setting config..."
  cp ../config .config
  #make olddefconfig

  make nconfig     # new CLI menu for configuration
  #make menuconfig # CLI menu for configuration
  #make xconfig    # X-based configuration
  #make oldconfig  # using old config from previous kernel version
  # ... or manually edit .config
  make prepare
...
Warning: systemd has a number of kernel configuration requirements for general use, for specific usecases (e.g., UEFI) and for specific systemd functionality (e.g., bootchart). Failure to meet these requirements can result in your system being degraded or unusable. The list of required and recommended kernel CONFIGs can be found in /usr/share/doc/systemd/README. Check them before you compile. These requirements also change over time. Because Arch assumes you are using the official kernel, there will be no announcement of these changes. Before you install a new version of systemd, check the version release notes to make sure your current custom kernel meets any new systemd requirements.

Generate new checksums

#Changing prepare() suggests a possible modification to $_srcname/.config. Since this path is not where downloading the package files ended, its checksum was not checked by makepkg (which actually checked $_srcname/../../config).

If you replaced the downloaded config with another one before running makepkg, install the pacman-contrib package and generate new checksums by running:

$ updpkgsums

Compiling

You can now proceed to compile your kernel by the usual command makepkg.

If you have chosen an interactive program for configuring the kernel parameters (like menuconfig), you need to be there during the compilation.

$ makepkg -s

The -s parameter will download any additional dependencies used by recent kernels such as xml and docs.

Note:
  • Kernel sources are PGP signed, and makepkg will attempt to verify them. See makepkg#Signature checking for details.
  • The compilation can take up to several hours to complete depending on the hardware performance. Running compilation jobs simultaneously can reduce compilation time significantly on multi-core systems.
  • It can be informative to run the above makepkg using the time command to know how long your system took to perform the compilation.

Installing

The compile step will leave two packages in the ~/build/linux folder, one for the kernel and one for the kernel headers. They might have names like:

linux-custom-5.8.12-x86_64.pkg.tar.zst
linux-custom-headers-5.8.12-x86_64.pkg.tar.zst

Best practice is to install both packages together as they might be both needed (e.g. DKMS):

# pacman -U linux-custom-headers-5.8.12-x86_64.pkg.tar.zst linux-custom-5.8.12-x86_64.pkg.tar.zst

(substitute the actual names of the files you have in the folder)

Boot loader

If you have modified pkgbase in order to have your new kernel installed alongside the default kernel you will need to update your boot loader configuration file and add new entries ('default' and 'fallback') for your custom kernel and the associated initramfs images.

Updating

Assuming one has an arch kernel source that they want to update, one method to do that is with https://github.com/archlinux/linux. In what follows, the top kernel source directory is assumed at ~/build/linux/.

In general, arch sets an arch kernel source with two local git repositories. The one at archlinux-linux/ is a local bare git repository pointing to https://github.com/archlinux/linux.git. The other one is at src/archlinux-linux/, pulling from the bare repository. Possible local patches, and building, are expected at src/archlinux-linux/.

For this example, the HEAD of the locally installed bare git repository source at archlinux-linux/ was initially pointing to

$ cd ~/build/linux/archlinux-linux/
$ git log --oneline --max-count 1 HEAD
4010b622f1d2 Merge branch 'dax-fix-5.3-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/nvdimm/nvdimm

which is somewhere between v5.2.5-arch1 and v5.2.6-arch1.

$ git fetch --verbose

One can see it fetched v5.2.7-arch1, which was the newest archlinux tag, because it prints what new tags were obtained. If no new tags were obtained then there is no newer archlinux source available.

Now the source can be updated where the actual build will take place.

$ cd ~/build/linux/src/archlinux-linux/
$ git checkout master
$ git pull
$ git fetch --tags --verbose
$ git branch --verbose 5.2.7-arch1 v5.2.7-arch1
$ git checkout 5.2.7-arch1

You can verify you are on track with something like

$ git log --oneline 5.2.7-arch1 --max-count=7
13193bfc03d4 Arch Linux kernel v5.2.7-arch1
9475c6772d05 netfilter: nf_tabf676926c7f60les: fix module autoload for redir
498d650048f6 iwlwifi: Add support for SAR South Korea limitation
bb7293abdbc7 iwlwifi: mvm: disable TX-AMSDU on older NICs
f676926c7f60 ZEN: Add CONFIG for unprivileged_userns_clone
5e4e503f4f28 add sysctl to disallow unprivileged CLONE_NEWUSER by default
5697a9d3d55f Linux 5.2.7

This shows few specific archlinux patches between Arch Linux kernel v5.2.7-arch1 and Linux 5.2.7.

The up to date PKGBUILD, as well archlinux kernel configuration file, can be pulled in using git in the package directory:

$ cd ~/build/linux/
$ git pull

Now you should merge files located in ~/build/linux/linux/* into ~/build/linux/. Merging can also done manually, or with specific utilities. Review #Changing prepare(), and run manually most, if not all, the shell commands of PKGBUILD::prepare().

At this point, makepkg --verifysource should succeed. While #Compiling, make sure to also add --noextract option to the makepkg command, since it should be able to build the packages as if the source was extracted by makepkg --nobuild. And you are back to #Installing.

Cleanup

One will probably want to remove ~/build/linux/linux/ after merging. In addition, ~/build/linux/src/archlinux will accumulate branches in the form of 5.2.7-arch1 if more recent updates are done in this fashion. These can be deleted with

$ cd ~/build/linux/src/archlinux
$ git branch --delete --force --verbose 5.2.7-arch1

See also