Difference between revisions of "Python package guidelines"

From ArchWiki
Jump to: navigation, search
m (sentence case in headings)
(fix word repetition)
 
(21 intermediate revisions by 12 users not shown)
Line 1: Line 1:
 
[[Category:Package development]]
 
[[Category:Package development]]
[[it:Python Package Guidelines]]
+
[[it:Python package guidelines]]
{{Package Guidelines}}
+
[[ja:Python パッケージガイドライン]]
 
+
{{Package guidelines}}
Writing [[PKGBUILD]]s for software written in [[Python]].
+
This document covers standards and guidelines on writing [[PKGBUILD]]s for [[Python]] software.
  
 
== Package naming ==
 
== Package naming ==
For libraries, use {{Ic|python-''modulename''}}. For applications, use the program name. In either case, the package name should be entirely lowercase.
+
 
 +
For Python 3 libraries, use {{Ic|python-''modulename''}}. For applications, use the program name. In either case, the package name should be entirely lowercase.
  
 
Python 2 libraries should instead be named {{Ic|python2-''modulename''}}.
 
Python 2 libraries should instead be named {{Ic|python2-''modulename''}}.
  
== File placement ==
+
===Versioned packages===
Most Python packages are installed with the [http://docs.python.org/library/distutils.html distutils] system using '''setup.py''', which installs files under {{Ic|/usr/lib/python''<python version>''/site-packages/''pkgname''}} directory.
+
 
 +
If you need to add a versioned package then use {{Ic|python-''modulename''-''version''}}, e.g. {{Ic|python-colorama-0.2.5}}. So python dependency {{Ic|colorama&#61;&#61;0.2.5}} will turn into {{Ic|python-colorama-0.2.5}} Arch package.
 +
 
 +
== Installation methods ==
 +
 
 +
Python packages are generally installed using language-specific tools, such as [https://pip.pypa.io/ pip] or [https://setuptools.readthedocs.io/en/latest/easy_install.html easy_install], which are comparable to dedicated package managers in that they are designed to fetch the source files from an online repository (usually [https://pypi.python.org/ PyPI], the Python Package Index) and track the relevant files (for a detailed comparison between the two, see [https://packaging.python.org/pip_easy_install/#pip-vs-easy-install pip vs easy_install]).
 +
 
 +
However, for managing Python packages from within PKGBUILDs, the standard-provided [http://docs.python.org/library/distutils.html distutils] proves to be the most convenient solution since it uses the downloaded source package's {{ic|setup.py}} and easily installs files under {{ic|''$pkgdir''/usr/lib/python''<python version>''/site-packages/''$pkgname''}} directory.
 +
 
 +
=== distutils ===
 +
 
 +
A ''distutils'' example PKGBUILD can be found [https://projects.archlinux.org/abs.git/tree/prototypes/PKGBUILD-python.proto here] or at {{ic|/usr/share/pacman/PKGBUILD-python.proto}}, provided by the {{Pkg|abs}} package. It follows the form:
 +
   
 +
''<python version>'' setup.py install --root="$pkgdir/" --prefix=/usr --optimize=1
 +
 
 +
where:
 +
 
 +
* ''<python version>'' is replaced with the proper binary, {{ic|python}} or {{ic|python2}}
 +
* {{ic|1=--root="$pkgdir/" }} prevents trying to directly install in the host system instead of inside the package file, which would result in a permission error
 +
* {{ic|1=--optimize=1}} compiles {{ic|.pyo}} files so they can be tracked by [[pacman]].
 +
 
 +
=== setuptools ===
 +
 
 +
The Python packaging scene has largely migrated from ''distutils'' to ''setuptools'', which is actively developed and functions as a drop-in replacement import in {{ic|setup.py}}. The main difference for packagers is that ''setuptools'' is packaged separately from Python itself, and must be specified as a {{ic|makedepends}}.
 +
 
 +
If the resulting package includes executables which [https://setuptools.readthedocs.io/en/latest/setuptools.html#automatic-script-creation import the {{ic|pkg_resources}} module], then ''setuptools'' must be additionally specified as a {{ic|depends}} in the split {{ic|package_*()}} functions; alternatively, if the PKGBUILD only installs the Python package for a single version of Python, ''setuptools'' should be moved from {{ic|makedepends}} to {{ic|depends}}.
 +
 
 +
=== pip ===
 +
 
 +
If you need to use ''pip'' (provided by {{Pkg|python-pip}} and {{Pkg|python2-pip}}), ''e.g.'' for installing [https://bitbucket.org/pypa/wheel/ wheel] packages, remember to pass the following flags:
 +
 
 +
PIP_CONFIG_FILE=/dev/null pip install --isolated --root="$pkgdir" --ignore-installed --no-deps *.whl
 +
 
 +
* {{ic|PIP_CONFIG_FILE&#61;/dev/null}} ignores {{ic|{/etc,~/.config}/pip.conf}} that may be appending arbitrary flags to '''pip'''.
 +
* {{ic|--isolated}} ignores environment variables (and again {{ic|{/etc,~/.config}/pip/pip.conf}}) that may otherwise also be appending arbitrary flags to '''pip'''.
 +
* {{ic|--ignore-installed}} is necessary until https://github.com/pypa/pip/issues/3063 is resolved (otherwise '''pip''' skips the install in the presence of an earlier {{ic|--user}} install).
 +
* {{ic|--no-deps}} ensures, that dependencies do not get packaged together with the main package.
  
 
== Notes ==
 
== Notes ==
The {{Ic|1=--optimize=1}} parameter compiles {{Ic|.pyo}} files so they can be tracked by [[pacman]].
 
  
 
In most cases, you should put {{Ic|any}} in the {{Ic|arch}} array since most Python packages are architecture independent.
 
In most cases, you should put {{Ic|any}} in the {{Ic|arch}} array since most Python packages are architecture independent.
Line 20: Line 56:
 
Please do not install a directory named just {{Ic|tests}}, as it easily conflicts with other Python packages (for example, {{Ic|/usr/lib/python2.7/site-packages/tests/}}).
 
Please do not install a directory named just {{Ic|tests}}, as it easily conflicts with other Python packages (for example, {{Ic|/usr/lib/python2.7/site-packages/tests/}}).
  
== Example ==
+
=== PyPI download URLs ===
An example PKGBUILD can be found at {{Ic|/usr/share/pacman/PKGBUILD-python.proto}}, which is in the {{Pkg|abs}} package.
+
 
 +
PyPI URLs of the form {{ic|<nowiki>https://pypi.python.org/packages/source/${pkgname:0:1}/${pkgname}/${pkgname}-${pkgver}.tar.gz</nowiki>}} were silently abandoned for new package versions in the course of 2016, replaced by a scheme using an unpredictable hash that needs to be fetched from the PyPI website each time a package must be updated[https://bitbucket.org/pypa/pypi/issues/438/backwards-compatible-un-hashed-package#comment-27230605].
 +
 
 +
As downstream packagers voiced their concerns to PyPI maintainers[https://bitbucket.org/pypa/pypi/issues/438/backwards-compatible-un-hashed-package], a new stable scheme was provided[https://bitbucket.org/pypa/pypi/issues/438/backwards-compatible-un-hashed-package#comment-27606213]: [[PKGBUILD#source]] {{ic|1=source=()}} array should now use {{ic|<nowiki>https://files.pythonhosted.org/packages/source/${pkgname:0:1}/${pkgname}/${pkgname}-${pkgver}.tar.gz</nowiki>}} as their URL.

Latest revision as of 01:25, 5 December 2016

Package creation guidelines

CLRCrossEclipseFree PascalGNOMEGoHaskellJavaKDEKernelLispMinGWNode.jsNonfreeOCamlPerlPHPPythonRubyVCSWebWine

This document covers standards and guidelines on writing PKGBUILDs for Python software.

Package naming

For Python 3 libraries, use python-modulename. For applications, use the program name. In either case, the package name should be entirely lowercase.

Python 2 libraries should instead be named python2-modulename.

Versioned packages

If you need to add a versioned package then use python-modulename-version, e.g. python-colorama-0.2.5. So python dependency colorama==0.2.5 will turn into python-colorama-0.2.5 Arch package.

Installation methods

Python packages are generally installed using language-specific tools, such as pip or easy_install, which are comparable to dedicated package managers in that they are designed to fetch the source files from an online repository (usually PyPI, the Python Package Index) and track the relevant files (for a detailed comparison between the two, see pip vs easy_install).

However, for managing Python packages from within PKGBUILDs, the standard-provided distutils proves to be the most convenient solution since it uses the downloaded source package's setup.py and easily installs files under $pkgdir/usr/lib/python<python version>/site-packages/$pkgname directory.

distutils

A distutils example PKGBUILD can be found here or at /usr/share/pacman/PKGBUILD-python.proto, provided by the abs package. It follows the form:

<python version> setup.py install --root="$pkgdir/" --prefix=/usr --optimize=1

where:

  • <python version> is replaced with the proper binary, python or python2
  • --root="$pkgdir/" prevents trying to directly install in the host system instead of inside the package file, which would result in a permission error
  • --optimize=1 compiles .pyo files so they can be tracked by pacman.

setuptools

The Python packaging scene has largely migrated from distutils to setuptools, which is actively developed and functions as a drop-in replacement import in setup.py. The main difference for packagers is that setuptools is packaged separately from Python itself, and must be specified as a makedepends.

If the resulting package includes executables which import the pkg_resources module, then setuptools must be additionally specified as a depends in the split package_*() functions; alternatively, if the PKGBUILD only installs the Python package for a single version of Python, setuptools should be moved from makedepends to depends.

pip

If you need to use pip (provided by python-pip and python2-pip), e.g. for installing wheel packages, remember to pass the following flags:

PIP_CONFIG_FILE=/dev/null pip install --isolated --root="$pkgdir" --ignore-installed --no-deps *.whl
  • PIP_CONFIG_FILE=/dev/null ignores {/etc,~/.config}/pip.conf that may be appending arbitrary flags to pip.
  • --isolated ignores environment variables (and again {/etc,~/.config}/pip/pip.conf) that may otherwise also be appending arbitrary flags to pip.
  • --ignore-installed is necessary until https://github.com/pypa/pip/issues/3063 is resolved (otherwise pip skips the install in the presence of an earlier --user install).
  • --no-deps ensures, that dependencies do not get packaged together with the main package.

Notes

In most cases, you should put any in the arch array since most Python packages are architecture independent.

Please do not install a directory named just tests, as it easily conflicts with other Python packages (for example, /usr/lib/python2.7/site-packages/tests/).

PyPI download URLs

PyPI URLs of the form https://pypi.python.org/packages/source/${pkgname:0:1}/${pkgname}/${pkgname}-${pkgver}.tar.gz were silently abandoned for new package versions in the course of 2016, replaced by a scheme using an unpredictable hash that needs to be fetched from the PyPI website each time a package must be updated[1].

As downstream packagers voiced their concerns to PyPI maintainers[2], a new stable scheme was provided[3]: PKGBUILD#source source=() array should now use https://files.pythonhosted.org/packages/source/${pkgname:0:1}/${pkgname}/${pkgname}-${pkgver}.tar.gz as their URL.