Python package guidelines

This document covers standards and guidelines on writing PKGBUILDs for Python software.

Arch package guidelines

32-bit – CLR – CMake – Cross – DKMS – Eclipse – Electron – Font – Free Pascal – GNOME – Go – Haskell – Java – KDE – Kernel – Lisp – Meson – MinGW – Node.js – Nonfree – OCaml – Perl – PHP – Python – R – Ruby – Rust – Shell – VCS – Web – Wine

Package naming

For Python 3 library modules, use python-modulename. Also use the prefix if the package provides a program that is strongly coupled to the Python ecosystem (e.g. pip or tox). For other applications, use only the program name.

The same applies to Python 2 except that the prefix (if needed) is python2-.

Note: The package name should be entirely lowercase.
Warning: Python 2 has reached end-of-life and is no longer maintained or used by Arch.

Architecture

See PKGBUILD#arch.

A Python package that contains C extensions is architecture-dependent. Otherwise it is most likely architecture-independent.

Packages built using setuptools define their C extensions using the ext_modules keyword in setup.py.

Source

Download URLs linked from the PyPI website include an unpredictable hash that needs to be fetched from the PyPI website each time a package must be updated. This makes them unsuitable for use in a PKGBUILD. PyPI provides an alternative stable scheme: PKGBUILD#source source=() array should use the following URL templates:

Source package
https://files.pythonhosted.org/packages/source/${_name::1}/$_name/$_name-$pkgver.tar.gz
Pure Python wheel package
https://files.pythonhosted.org/packages/py2.py3/${_name::1}/$_name/${_name//-/_}-$pkgver-py2.py3-none-any.whl (Bilingual – Python 2 and Python 3 compatible)
https://files.pythonhosted.org/packages/py3/${_name::1}/$_name/${_name//-/_}-$pkgver-py3-none-any.whl (Python 3 only)
Note that the distribution name can contain dashes, while its representation in a wheel filename cannot (they are converted to underscores).
Architecture specific wheel package
Additional architecture-specific arrays can be added by appending an underscore and the architecture name, e.g. . Also can be used to not repeat the Python version:
https://files.pythonhosted.org/packages/$_py/${_name::1}/$_name/${_name//-/_}-$pkgver-$_py-${_py}m-manylinux1_x86_64.whl

Note that a custom variable is used instead of since Python packages are generally prefixed with . This variable can generically be defined as follows:

_name=${pkgname#python-}

Installation methods

Python packages are generally installed using language-specific package manager such as pip, which fetches packages from an online repository (usually PyPI, the Python Package Index) and tracks the relevant files.

However, for managing Python packages from within PKGBUILDs, one needs to "install" the Python package to the temporary location .

For Python packages using standard metadata to specify their build backend in , this can most easily achieved using and . Old packages might fail to specify that they use setuptools, and only offer a setup.py that has to be invoked manually.

Standards based (PEP 517)

A standards based workflow is straightforward: Build a wheel using and install it to using :

where

  • --wheel results in only a wheel file to be built, no source distribution.
  • --no-isolation means that the package is built using what is installed on your system (which includes packages you specified in ), by default the tool creates an isolated virtual environment and performs the build there.
  • prevents trying to directly install in the host system instead of inside the package file, which would result in a permission error
  • or can be passed to , but the default is sensibly picked, so this should not be necessary.

setuptools or distutils

If no can be found or it fails to contain a table, it means the project is using the old legacy format, which uses a setup.py file which invokes setuptools or distutils. Note that while distutils is included in Python's standardlib, having setuptools installed means that you use a patched version of distutils.

where:

  • --root="$pkgdir" works like --destdir above
  • compiles optimized bytecode files (.opt-1.pyc) so they can be tracked by pacman instead of being created on the host system on demand.
  • Adding optimizes away the unnecessary attempt to re-run the build steps already run in the function, if that is the case.

If the resulting package includes executables which import the deprecated pkg_resources module, then setuptools must be additionally specified as a in the split functions; alternatively, if the PKGBUILD only installs the Python package for a single version of Python, setuptools should be moved from to .

Some packages try to use setuptools and fall back to distutils if setuptools could not be imported. In this case, setuptools should be added as a , so that the resulting Python metadata is better.

If a package needs setuptools to be built due to including executables (which is not supported by distutils), but only imports distutils, then building will raise a warning, and the resulting package will be broken (it will not contain the executables):

An upstream bug should be reported. To work around the problem, an undocumented setuptools feature can be used:

If a package uses , the package most likely will not build with an error such as:

To get it building SETUPTOOLS_SCM_PRETEND_VERSION has to be exported as an environment variable with as the value:

export SETUPTOOLS_SCM_PRETEND_VERSION=$pkgver

Check

Most Python projects providing a testsuite use nosetests or pytest to run tests with test in the name of the file or directory containing the testsuite. In general, simply running or is enough to run the testsuite.

If there is a compiled C extension, the tests need to be run using a , that reflects the current major and minor version of Python in order to find and load it.

Some projects provide setup.py entry points for running the test. This works for both and .

Tips and tricks

Discovering detached PGP signatures on PyPI

If detached PGP signatures for a given Python sdist tarball exist, they should be used to verify the tarball. However, the signature files do not show up directly in the files download section of any given project on pypi.org. To discover the sdist tarballs and their potential signature files, it is possible to use this service to get an overview per project: https://pypi.debian.net/

For , this would be https://pypi.debian.net/requests.

Using python version

Sometimes during preparing, building, testing or installation it is required to refer to the system's major and minor Python version. Do not hardcode this (e.g. or 3.10) and instead use a call to the Python interpreter to retrieve the information and store it in a local variable:

Using site-packages

Sometimes during building, testing or installation it is required to refer to the system's site-packages directory. Do not hardcode this directory and use a call to the Python interpreter instead to retrieve the path and store it in a local variable:

Test directory in site-package

Make sure to not install a directory named just into site-packages (i.e. ). Python package projects using setuptools are sometimes misconfigured to include the directory containing its tests as a top level Python package. If you encounter this, you can help by filing an issue with the package project and ask them to fix this, e.g. like this.

This article is issued from Archlinux. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.