User:Davezerave/Python Bootstrapping
This article describes the process of bootstrapping packages for the purpose of upgrading to a new interpreter version of Python.
Packages required for bootstrapping
The following subsections describe the relationship between the various packages that are required to build and install other packages. Makedepends that require sphinx are purposely left out, as they increase the required bootstrap cycle significantly. Checkdepends and optdepends are left out on purpose, as they are not necessary for bootstrapping.
Python-build
Makedepends
Depends
Python-flit
Makedepends
Depends
Python-installer
Makedepends
Python-pdm
Makedepends
Depends
- python-blinker
- python-cachecontrol
- python-click
- python-dotenv
- python-findpython
- python-installer
- python-packaging
- python-pdm-pep517
- python-pep517
- python-pip
- python-platformdirs
- python-requests-toolbelt
- python-resolvelib
- python-rich
- python-shellingham
- python-tomli
- python-tomlkit
- python-unearth
- python-virtualenv
- python-wheel
Python-poetry
Makedepends
Depends
- python-cachecontrol
- python-cachy
- python-cleo
- python-html5lib
- python-keyring
- python-lockfile
- python-packaging
- python-pexpect
- python-pkginfo
- python-poetry-core
- python-requests
- python-requests-toolbelt
- python-shellingham
- python-tomlkit
- python-virtualenv
Python-setuptools
Makedepends
Depends
- python-appdirs
- python-jaraco.text
- python-more-itertools
- python-nspektr
- python-ordered-set
- python-packaging
- python-pyparsing
- python-tomli
- python-validate-pyproject
Python-wheel
Makedepends
Depends
Bootstrapping
In the following subsections different bootstrapping mechanism our outlined.
Each can be hidden behind a custom $_bootstrap variable in a PKGBUILD and thus be added to all existing packages that require bootstrapping at some point.
Egg-based
When relying on egg distributions (e.g. via source tarballs from pypi.org), we can manually install files (excluding entry_point and sphinx-based documentation) to the filesystem and afterwards byte-compile the files.
In the below example it is assumed that the Python package is referred to as $_name and that it is located top-level in the extracted source tarball (the assignment of _src may need to be altered if the package is located e.g. in a src/ subdirectory).
PKGBUILD
package(){
local _site_packages=$(python -c "import site; print(site.getsitepackages()[0])")
local _pyver=$(python -c 'import sys; print(".".join(map(str, sys.version_info[:2])))')
local _src=$_name-$pkgver
(
cd $_src
find $_name \
-type f \
\( \
-iname "*.py" \
-o -iname "*.typed" \
\) \
-exec install -vDm 644 {} "$pkgdir$_site_packages/"{} \;
install -vDm 644 $_name.egg-info/* -t "$pkgdir$_site_packages/$_name-$pkgver-py$_pyver.egg-info/"
python -m compileall --invalidation-mode=checked-hash "$pkgdir"
)
}
Python-bootstrap based
When relying on python-bootstrap we would create initial wheel distributions using a script which vendors specific upstream versions of all required Python packages for bootstrapping the ecosystem. These initial wheel distributions may be vendored (as they are built as upstream intends). python-bootstrap would need to be available as makedepends for all packages that need boostrapping and would then be used to build all wheels and install the required ones.
In the below example $_name is the Python package which is the bootstrap target.
PKGBUILD
prepare() {
mkdir -p bootstrap
}
build() {
cd bootstrap
python -m bootstrap.build
}
package() {
cd bootstrap
python -m bootstrap.install -s $_name*.whl
}