//python/extensions:python.bzl

Python toolchain module extensions for use with bzlmod.

Basic usage

The simplest way to configure the toolchain with rules_python is as follows.

python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.defaults(python_version = "3.11")
python.toolchain(python_version = "3.11")
use_repo(python, "python_3_11")

See also

For more in-depth documentation see the python.toolchain.

Overrides

Overrides can be done at 3 different levels:

• Overrides affecting all python toolchain versions on all platforms - python.override.

• Overrides affecting a single toolchain versions on all platforms - python.single_version_override.

• Overrides affecting a single toolchain versions on a single platforms - python.single_version_platform_override.

See also

The main documentation page on registering toolchains.

module ext python

Bzlmod extension that is used to register Python toolchains.

python.defaults(python_version='', python_version_env='', python_version_file=None)

Tag class to specify the default Python version.

Attributes:

• python_version – (str) (default “”)

  String saying what the default Python version should be. If the string matches the python_version attribute of a toolchain, this toolchain is the default version. If this attribute is set, the is_default attribute of the toolchain is ignored.

  Added in version 1.4.0.

  optional

• python_version_env – (str) (default “”)

  Environment variable saying what the default Python version should be. If the string matches the python_version attribute of a toolchain, this toolchain is the default version. If this attribute is set, the is_default attribute of the toolchain is ignored.

  Added in version 1.4.0.

  optional

• python_version_file – (label) (default None)

  File saying what the default Python version should be. If the contents of the file match the python_version attribute of a toolchain, this toolchain is the default version. If this attribute is set, the is_default attribute of the toolchain is ignored.

  Added in version 1.4.0.

  optional

python.override(auth_patterns={}, available_python_versions=[], base_url='https://github.com/astral-sh/python-build-standalone/releases/download', ignore_root_user_error=True, minor_mapping={}, netrc='', register_all_versions=False)

Tag class used to override defaults and behaviour of the module extension.

Added in version 0.36.0.

Attributes:

• auth_patterns – (dict[str, str]) (default {})

  An optional dict mapping host names to custom authorization patterns.

  If a URL’s host name is present in this dict the value will be used as a pattern when generating the authorization header for the http request. This enables the use of custom authorization schemes used in a lot of common cloud storage providers.

  The pattern currently supports 2 tokens: <login> and <password>, which are replaced with their equivalent value in the netrc file for the same host name. After formatting, the result is set as the value for the Authorization field of the HTTP request.

  Example attribute and netrc for a http download to an oauth2 enabled API using a bearer token:

  auth_patterns = {
      "storage.cloudprovider.com": "Bearer <password>"
  }
  

  netrc:

  machine storage.cloudprovider.com
          password RANDOM-TOKEN
  

  The final HTTP request would have the following header:

  Authorization: Bearer RANDOM-TOKEN
  

  optional

• available_python_versions – (list[str]) (default [])

  The list of available python tool versions to use. Must be in X.Y.Z format. If the unknown version given the processing of the extension will fail - all of the versions in the list have to be defined with python.single_version_override or python.single_version_platform_override before they are used in this list.

  This attribute is usually used in order to ensure that no unexpected transitive dependencies are introduced.

  optional

• base_url – (str) (default “https://github.com/astral-sh/python-build-standalone/releases/download”)

  The base URL to be used when downloading toolchains.

  optional

• ignore_root_user_error – (bool) (default True)

  Deprecated; do not use. This attribute has no effect.

  optional

• minor_mapping – (dict[str, str]) (default {})

  The mapping between X.Y to X.Y.Z versions to be used when setting up toolchains. It defaults to the interpreter with the highest available patch version for each minor version. For example if one registers 3.10.3, 3.10.4 and 3.11.4 then the default for the minor_mapping dict will be:

  {
  "3.10": "3.10.4",
  "3.11": "3.11.4",
  }
  

  Changed in version 0.37.0: The values in this mapping override the default values and do not replace them.

  optional

• netrc – (str) (default “”)

  Location of the .netrc file to use for authentication

  optional

• register_all_versions – (bool) (default False)

  Add all versions

  optional

python.single_version_override(python_version, distutils=None, distutils_content='', patch_strip=0, patches=[], sha256={}, strip_prefix='python', urls=[])

Override single python version URLs and patches for all platforms.

Note

This will replace any existing configuration for the given python version.

Tip

If you would like to modify the configuration for a specific (version, platform), please use the single_version_platform_override tag class.

Added in version 0.36.0.

Attributes:

• python_version – (str)

  The python version to override URLs for. Must be in X.Y.Z format.

  mandatory

• distutils – (label) (default None)

  A distutils.cfg file to be included in the Python installation. Either distutils or distutils_content can be specified, but not both.

  optional

• distutils_content – (str) (default “”)

  A distutils.cfg file content to be included in the Python installation. Either distutils or distutils_content can be specified, but not both.

  optional

• patch_strip – (int) (default 0)

  Same as the –strip argument of Unix patch.

  optional

• patches – (list[label]) (default [])

  A list of labels pointing to patch files to apply for the interpreter repository. They are applied in the list order and are applied before any platform-specific patches are applied.

  optional

• sha256 – (dict[str, str]) (default {})

  The python platform to sha256 dict. See python.single_version_platform_override.platform for allowed key values.

  optional

• strip_prefix – (str) (default “python”)

  The ‘strip_prefix’ for the archive, defaults to ‘python’.

  optional

• urls – (list[str]) (default [])

  The URL template to fetch releases for this Python version. See python.single_version_platform_override.urls for documentation.

  optional

python.single_version_platform_override(platform, python_version, arch='', coverage_tool=None, os_name='', patch_strip=0, patches=[], sha256='', strip_prefix='python', target_compatible_with=[], target_settings=[], urls=[])

Override single python version for a single existing platform.

If the (version, platform) is new, we will add it to the existing versions and will use the same url template.

Tip

If you would like to add or remove platforms to a single python version toolchain configuration, please use single_version_override.

Added in version 0.36.0.

Attributes:

• platform – (str)

  The platform to override the values for, typically one of:

  • aarch64-apple-darwin-freethreaded

  • aarch64-apple-darwin

  • aarch64-pc-windows-msvc-freethreaded

  • aarch64-pc-windows-msvc

  • aarch64-unknown-linux-gnu-freethreaded

  • aarch64-unknown-linux-gnu

  • armv7-unknown-linux-gnu-freethreaded

  • armv7-unknown-linux-gnu

  • i386-unknown-linux-gnu-freethreaded

  • i386-unknown-linux-gnu

  • ppc64le-unknown-linux-gnu-freethreaded

  • ppc64le-unknown-linux-gnu

  • riscv64-unknown-linux-gnu-freethreaded

  • riscv64-unknown-linux-gnu

  • s390x-unknown-linux-gnu-freethreaded

  • s390x-unknown-linux-gnu

  • x86_64-apple-darwin-freethreaded

  • x86_64-apple-darwin

  • x86_64-pc-windows-msvc-freethreaded

  • x86_64-pc-windows-msvc

  • x86_64-unknown-linux-gnu-freethreaded

  • x86_64-unknown-linux-gnu

  • x86_64-unknown-linux-musl-freethreaded

  • x86_64-unknown-linux-musl

  Other values are allowed, in which case, target_compatible_with, target_settings, os_name, and arch should be specified so the toolchain is only used when appropriate.

  Changed in version 1.5.0: Arbitrary platform strings allowed.

  mandatory

• python_version – (str)

  The python version to override URLs for. Must be in X.Y.Z format.

  mandatory

• arch – (str) (default “”)

  The arch (cpu) the runtime is compatible with.

  If not set, then the runtime cannot be used as a python_X_Y_host runtime.

  If set, the os_name, target_compatible_with and target_settings attributes should also be set.

  The values should be one of the values in @platforms//cpu

  See also

  Docs for [Registering custom runtimes]

  optional

• coverage_tool – (label) (default None)

  The coverage tool to be used for a particular Python interpreter. This can override rules_python defaults.

  optional

• os_name – (str) (default “”)

  The host OS the runtime is compatible with.

  If not set, then the runtime cannot be used as a python_X_Y_host runtime.

  If set, the os_name, target_compatible_with and target_settings attributes should also be set.

  The values should be one of the values in @platforms//os

  See also

  Docs for [Registering custom runtimes]

  optional

• patch_strip – (int) (default 0)

  Same as the –strip argument of Unix patch.

  optional

• patches – (list[label]) (default [])

  A list of labels pointing to patch files to apply for the interpreter repository. They are applied in the list order and are applied after the common patches are applied.

  optional

• sha256 – (str) (default “”)

  The sha256 for the archive

  optional

• strip_prefix – (str) (default “python”)

  The ‘strip_prefix’ for the archive, defaults to ‘python’.

  optional

• target_compatible_with – (list[str]) (default [])

  The target_compatible_with values to use for the toolchain definition.

  If not set, then os_name and arch will be used to populate it.

  If set, target_settings, os_name, and arch should also be set.

  See also

  Docs for [Registering custom runtimes]

  optional

• target_settings – (list[str]) (default [])

  The target_setings values to use for the toolchain definition.

  If set, target_compatible_with, os_name, and arch should also be set.

  See also

  Docs for [Registering custom runtimes]

  optional

• urls – (list[str]) (default [])

  The URL template to fetch releases for this Python version. If the URL template results in a relative fragment, default base URL is going to be used. Occurrences of {python_version}, {platform} and {build} will be interpolated based on the contents in the override and the known platform values.

  optional

python.toolchain(python_version, configure_coverage_tool=False, ignore_root_user_error=True, is_default=False)

Tag class used to register Python toolchains. Use this tag class to register one or more Python toolchains. This class is also potentially called by sub modules. The following covers different business rules and use cases.

Toolchains in the Root Module

This class registers all toolchains in the root module.

Toolchains in Sub Modules

It will create a toolchain that is in a sub module, if the toolchain of the same name does not exist in the root module. The extension stops name clashing between toolchains in the root module and toolchains in sub modules. You cannot configure more than one toolchain as the default toolchain.

Toolchain set as the default version

This extension will not create a toolchain that exists in a sub module, if the sub module toolchain is marked as the default version. If you have more than one toolchain in your root module, you need to set one of the toolchains as the default version. If there is only one toolchain it is set as the default toolchain.

Toolchain repository name

A toolchain’s repository name uses the format python_{major}_{minor}, e.g. python_3_10. The major and minor components are major and minor are the Python version from the python_version attribute.

If a toolchain is registered in X.Y.Z, then similarly the toolchain name will be python_{major}_{minor}_{patch}, e.g. python_3_10_19.

Toolchain detection

The definition of the first toolchain wins, which means that the root module can override settings for any python toolchain available. This relies on the documented module traversal from the module_ctx.modules.

Tip

In order to use a different name than the above, you can use the following MODULE.bazel syntax:

python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.defaults(python_version = "3.11")
python.toolchain(python_version = "3.11")

use_repo(python, my_python_name = "python_3_11")

Then the python interpreter will be available as my_python_name.

Attributes:

• python_version – (str)

  The Python version, in major.minor or major.minor.patch format, e.g 3.12 (or 3.12.3), to create a toolchain for.

  mandatory

• configure_coverage_tool – (bool) (default False)

  Whether or not to configure the default coverage tool provided by rules_python for the compatible toolchains.

  optional

• ignore_root_user_error – (bool) (default True)

  Changed in version VERSION_NEXT_FEATURE: Noop, will be removed in the next major release.

  optional

• is_default – (bool) (default False)

  Whether the toolchain is the default version.

  Changed in version 1.4.0: This setting is ignored if the default version is set using the defaults tag class (encouraged).

  optional