|
| 1 | +.. _c-info.limited-api: |
| 2 | + |
| 3 | +***************************************** |
| 4 | +Using NumPy with the Python Limited API |
| 5 | +***************************************** |
| 6 | + |
| 7 | +The `Python Limited API <https://docs.python.org/3/c-api/stable.html>`_ |
| 8 | +(also called the **Stable ABI**) is a subset of Python's C API that is |
| 9 | +guaranteed to remain binary-compatible across Python versions. Extension |
| 10 | +modules built against the Limited API produce ``.abi3`` wheels that run on |
| 11 | +any Python version equal to or newer than the one they were compiled against, |
| 12 | +without recompilation. |
| 13 | + |
| 14 | +NumPy has supported building against the Python Limited API since **NumPy |
| 15 | +2.0**. |
| 16 | + |
| 17 | +.. note:: |
| 18 | + |
| 19 | + Free-threaded (GIL-free) abi3 builds (``abi3t``) are being added in a |
| 20 | + later NumPy release. This page currently covers the standard ``abi3`` |
| 21 | + workflow only. |
| 22 | + |
| 23 | + |
| 24 | +Why use the Limited API? |
| 25 | +======================== |
| 26 | + |
| 27 | +Normally, a compiled extension wheel is tied to a specific Python version |
| 28 | +(e.g., ``my_ext-1.0-cp313-cp313-linux_x86_64.whl`` only runs on CPython |
| 29 | +3.13). With the Limited API you can instead ship a single |
| 30 | +``my_ext-1.0-cp39-abi3-linux_x86_64.whl`` that runs on Python 3.9, 3.10, |
| 31 | +3.11, 3.12, 3.13 and beyond. |
| 32 | + |
| 33 | +Benefits: |
| 34 | + |
| 35 | +- **Fewer wheels to build and upload** — one wheel per platform instead of |
| 36 | + one per (platform × Python version). |
| 37 | +- **Future-proof** — your wheel continues to work on new Python releases |
| 38 | + without a rebuild. |
| 39 | +- **Smaller CI matrix** — no need to build on every Python minor version. |
| 40 | + |
| 41 | +Constraints: |
| 42 | + |
| 43 | +- You can only use the C API symbols that are part of the stable subset. |
| 44 | + Some NumPy types and struct members are **not** available when |
| 45 | + ``Py_LIMITED_API`` is defined (see :ref:`limited-api-restrictions` below). |
| 46 | +- The minimum Python version is locked at compile time. Choosing ``3.9`` |
| 47 | + means you cannot use Limited-API symbols added after 3.9. |
| 48 | + |
| 49 | + |
| 50 | +How it works at the C level |
| 51 | +============================ |
| 52 | + |
| 53 | +When you define ``Py_LIMITED_API`` before including ``<Python.h>``, the |
| 54 | +CPython headers hide everything that is not part of the stable ABI. NumPy |
| 55 | +respects this macro: its public headers (``numpy/arrayobject.h``, |
| 56 | +``numpy/ndarraytypes.h``, etc.) detect ``Py_LIMITED_API`` and guard out |
| 57 | +struct members and types that are not safe to expose. |
| 58 | + |
| 59 | +You do **not** need to define ``Py_LIMITED_API`` manually when using Meson — |
| 60 | +the build system does it for you when you pass ``limited_api:`` to |
| 61 | +``py.extension_module()``. |
| 62 | + |
| 63 | + |
| 64 | +Building with Meson |
| 65 | +=================== |
| 66 | + |
| 67 | +`Meson <https://mesonbuild.com/>`_ is the recommended build system for new |
| 68 | +NumPy extension modules. Pass the ``limited_api`` keyword to |
| 69 | +``py.extension_module()`` with the minimum Python version you want to |
| 70 | +support:: |
| 71 | + |
| 72 | + project('my_extension', 'c') |
| 73 | + |
| 74 | + py = import('python').find_installation(pure: false) |
| 75 | + |
| 76 | + # Get the NumPy include directory |
| 77 | + npy_include = run_command( |
| 78 | + py, ['-c', 'import numpy; print(numpy.get_include())'], |
| 79 | + check: true, |
| 80 | + ).stdout().strip() |
| 81 | + |
| 82 | + py.extension_module( |
| 83 | + 'my_ext', |
| 84 | + 'my_ext.c', |
| 85 | + include_directories: [npy_include], |
| 86 | + c_args: ['-DNPY_NO_DEPRECATED_API=NPY_1_21_API_VERSION'], |
| 87 | + limited_api: '3.9', # minimum Python version |
| 88 | + install: true, |
| 89 | + ) |
| 90 | + |
| 91 | +Meson automatically: |
| 92 | + |
| 93 | +- Defines ``Py_LIMITED_API`` to the correct hex version (e.g. ``0x03090000`` |
| 94 | + for Python 3.9). |
| 95 | +- Names the output file with the ``.abi3.so`` suffix on Linux/macOS or |
| 96 | + ``.pyd`` on Windows. |
| 97 | +- Links against the stable ``python3.dll`` import library on Windows. |
| 98 | + |
| 99 | +The corresponding C source file needs no special changes — just include the |
| 100 | +NumPy headers as usual:: |
| 101 | + |
| 102 | + #include <Python.h> |
| 103 | + #include <numpy/arrayobject.h> |
| 104 | + |
| 105 | + static PyModuleDef moduledef = { |
| 106 | + .m_base = PyModuleDef_HEAD_INIT, |
| 107 | + .m_name = "my_ext", |
| 108 | + }; |
| 109 | + |
| 110 | + PyMODINIT_FUNC PyInit_my_ext(void) |
| 111 | + { |
| 112 | + import_array(); |
| 113 | + return PyModule_Create(&moduledef); |
| 114 | + } |
| 115 | + |
| 116 | + |
| 117 | +Building with setuptools |
| 118 | +======================== |
| 119 | + |
| 120 | +If you use ``setuptools``, define ``Py_LIMITED_API`` manually via |
| 121 | +``define_macros`` and set ``py_limited_api=True`` on the ``Extension``:: |
| 122 | + |
| 123 | + import numpy as np |
| 124 | + from setuptools import Extension, setup |
| 125 | + |
| 126 | + ext = Extension( |
| 127 | + "my_ext", |
| 128 | + sources=["my_ext.c"], |
| 129 | + include_dirs=[np.get_include()], |
| 130 | + define_macros=[ |
| 131 | + ("Py_LIMITED_API", "0x03090000"), # Python 3.9+ |
| 132 | + ("NPY_NO_DEPRECATED_API", "NPY_1_21_API_VERSION"), |
| 133 | + ], |
| 134 | + py_limited_api=True, # tags wheel as .abi3 |
| 135 | + ) |
| 136 | + |
| 137 | + setup(ext_modules=[ext]) |
| 138 | + |
| 139 | +.. note:: |
| 140 | + |
| 141 | + ``py_limited_api=True`` only affects the wheel tag generated by |
| 142 | + ``bdist_wheel``. You still need to pass ``Py_LIMITED_API`` as a macro so |
| 143 | + the headers hide non-stable symbols. |
| 144 | + |
| 145 | + |
| 146 | +Building with Cython |
| 147 | +==================== |
| 148 | + |
| 149 | +`Cython <https://cython.org/>`_ 3.0+ supports the Limited API. In your |
| 150 | +``meson.build``:: |
| 151 | + |
| 152 | + py.extension_module( |
| 153 | + 'my_cython_ext', |
| 154 | + 'my_cython_ext.pyx', |
| 155 | + c_args: [ |
| 156 | + '-DNPY_NO_DEPRECATED_API=0', |
| 157 | + '-DCYTHON_LIMITED_API=1', |
| 158 | + ], |
| 159 | + include_directories: [npy_include], |
| 160 | + limited_api: '3.9', |
| 161 | + install: true, |
| 162 | + ) |
| 163 | + |
| 164 | +Set ``CYTHON_LIMITED_API=1`` (not ``Py_LIMITED_API`` directly) so that Cython |
| 165 | +can generate code compatible with the stable ABI. |
| 166 | + |
| 167 | +.. _limited-api-restrictions: |
| 168 | + |
| 169 | +Restrictions and known limitations |
| 170 | +==================================== |
| 171 | + |
| 172 | +Not all NumPy C API features are available when ``Py_LIMITED_API`` is defined. |
| 173 | + |
| 174 | +**Types hidden under ``Py_LIMITED_API``:** |
| 175 | + |
| 176 | +- :c:type:`PyArrayScalar_VAL` and :c:type:`PyUnicodeScalarObject` are not |
| 177 | + accessible. Code that reads scalar values directly from the struct must be |
| 178 | + rewritten to use the NumPy Python API (e.g. :func:`numpy.asarray`) or the |
| 179 | + iterator API instead. |
| 180 | + |
| 181 | +- :c:type:`PyArray_DTypeMeta` is a ``typedef`` for ``PyTypeObject`` when |
| 182 | + ``Py_LIMITED_API`` is set, because it contains ``PyHeapTypeObject`` which |
| 183 | + is not part of the stable ABI. Custom DType implementations cannot use |
| 184 | + the full ``PyArray_DTypeMeta`` struct layout in Limited API builds. |
| 185 | + |
| 186 | +**General guidelines:** |
| 187 | + |
| 188 | +- Avoid direct struct member access on ``PyArrayObject``. Prefer the |
| 189 | + accessor macros (``PyArray_DATA``, ``PyArray_NDIM``, ``PyArray_DIMS``, |
| 190 | + etc.) which remain available in the Limited API. |
| 191 | +- Check the ``numpy/ndarraytypes.h`` and ``numpy/arrayobject.h`` headers for |
| 192 | + ``#ifndef Py_LIMITED_API`` guards to identify what is unavailable. |
| 193 | +- Build and test your extension against a Python version *different* from the |
| 194 | + one you compiled against (e.g. compile on 3.9, test on 3.12) to catch ABI |
| 195 | + issues early. |
| 196 | + |
| 197 | + |
| 198 | +Distributing abi3 wheels |
| 199 | +========================= |
| 200 | + |
| 201 | +When building wheels for distribution, set the minimum Python version tag to |
| 202 | +match the ``Py_LIMITED_API`` version you compiled against. Tools like |
| 203 | +`cibuildwheel <https://cibuildwheel.pypa.io/>`_ handle this automatically |
| 204 | +when configured correctly — see the |
| 205 | +`cibuildwheel abi3 docs <https://cibuildwheel.pypa.io/en/stable/faq/#abi3>`__ |
| 206 | +for details. |
| 207 | + |
| 208 | +For a project supporting Python 3.9+, the resulting wheel filename will look |
| 209 | +like:: |
| 210 | + |
| 211 | + my_ext-1.0-cp39-abi3-manylinux_2_17_x86_64.whl |
| 212 | + |
| 213 | + |
| 214 | +See also |
| 215 | +======== |
| 216 | + |
| 217 | +- :ref:`CPython stable ABI documentation <python:stable-abi>` |
| 218 | +- :ref:`for-downstream-package-authors` — NumPy ABI compatibility overview |
| 219 | +- `cibuildwheel abi3 guide <https://cibuildwheel.pypa.io/en/stable/faq/#abi3>`__ |
| 220 | +- `Meson Python extension_module docs |
| 221 | + <https://mesonbuild.com/Python-module.html#extension_module>`__ |
0 commit comments