Skip to content

Commit 2f4cd07

Browse files
DOC: Add documentation for using NumPy with the Python Limited API
Add a new page c-info.limited-api.rst covering: - What the Python Limited API / Stable ABI is and why to use it - Build examples using Meson, setuptools, and Cython - Known restrictions (hidden types, struct access) - Wheel distribution guidance Update c-info.rst toctree and add a cross-reference from depending_on_numpy.rst to the new page. Closes numpy#31259
1 parent 927c7bf commit 2f4cd07

3 files changed

Lines changed: 225 additions & 1 deletion

File tree

‎doc/source/dev/depending_on_numpy.rst‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,9 @@ Modules can also be safely built against NumPy 2.0 or later in
3939
building against a single (minimum-supported) version of Python but be
4040
forward compatible higher versions in the same series (e.g., ``3.x``).
4141
This can greatly reduce the number of wheels that need to be built and
42-
distributed. For more information and examples, see the
42+
distributed. For a full guide including build system examples and known
43+
limitations, see :ref:`c-info.limited-api`. For wheel distribution details,
44+
see the
4345
`cibuildwheel docs <https://cibuildwheel.pypa.io/en/stable/faq/#abi3>`__.
4446

4547
.. _testing-prereleases:
Lines changed: 221 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,221 @@
1+
.. _c-info.limited-api:
2+
3+
*****************************************
4+
Using NumPy with the Python Limited API
5+
*****************************************
6+
7+
The `Python Limited API <https://docs.python.org/3/c-api/stable.html>`_
8+
(also called the **Stable ABI**) is a subset of Python's C API that is
9+
guaranteed to remain binary-compatible across Python versions. Extension
10+
modules built against the Limited API produce ``.abi3`` wheels that run on
11+
any Python version equal to or newer than the one they were compiled against,
12+
without recompilation.
13+
14+
NumPy has supported building against the Python Limited API since **NumPy
15+
2.0**.
16+
17+
.. note::
18+
19+
Free-threaded (GIL-free) abi3 builds (``abi3t``) are being added in a
20+
later NumPy release. This page currently covers the standard ``abi3``
21+
workflow only.
22+
23+
24+
Why use the Limited API?
25+
========================
26+
27+
Normally, a compiled extension wheel is tied to a specific Python version
28+
(e.g., ``my_ext-1.0-cp313-cp313-linux_x86_64.whl`` only runs on CPython
29+
3.13). With the Limited API you can instead ship a single
30+
``my_ext-1.0-cp39-abi3-linux_x86_64.whl`` that runs on Python 3.9, 3.10,
31+
3.11, 3.12, 3.13 and beyond.
32+
33+
Benefits:
34+
35+
- **Fewer wheels to build and upload** — one wheel per platform instead of
36+
one per (platform × Python version).
37+
- **Future-proof** — your wheel continues to work on new Python releases
38+
without a rebuild.
39+
- **Smaller CI matrix** — no need to build on every Python minor version.
40+
41+
Constraints:
42+
43+
- You can only use the C API symbols that are part of the stable subset.
44+
Some NumPy types and struct members are **not** available when
45+
``Py_LIMITED_API`` is defined (see :ref:`limited-api-restrictions` below).
46+
- The minimum Python version is locked at compile time. Choosing ``3.9``
47+
means you cannot use Limited-API symbols added after 3.9.
48+
49+
50+
How it works at the C level
51+
============================
52+
53+
When you define ``Py_LIMITED_API`` before including ``<Python.h>``, the
54+
CPython headers hide everything that is not part of the stable ABI. NumPy
55+
respects this macro: its public headers (``numpy/arrayobject.h``,
56+
``numpy/ndarraytypes.h``, etc.) detect ``Py_LIMITED_API`` and guard out
57+
struct members and types that are not safe to expose.
58+
59+
You do **not** need to define ``Py_LIMITED_API`` manually when using Meson —
60+
the build system does it for you when you pass ``limited_api:`` to
61+
``py.extension_module()``.
62+
63+
64+
Building with Meson
65+
===================
66+
67+
`Meson <https://mesonbuild.com/>`_ is the recommended build system for new
68+
NumPy extension modules. Pass the ``limited_api`` keyword to
69+
``py.extension_module()`` with the minimum Python version you want to
70+
support::
71+
72+
project('my_extension', 'c')
73+
74+
py = import('python').find_installation(pure: false)
75+
76+
# Get the NumPy include directory
77+
npy_include = run_command(
78+
py, ['-c', 'import numpy; print(numpy.get_include())'],
79+
check: true,
80+
).stdout().strip()
81+
82+
py.extension_module(
83+
'my_ext',
84+
'my_ext.c',
85+
include_directories: [npy_include],
86+
c_args: ['-DNPY_NO_DEPRECATED_API=NPY_1_21_API_VERSION'],
87+
limited_api: '3.9', # minimum Python version
88+
install: true,
89+
)
90+
91+
Meson automatically:
92+
93+
- Defines ``Py_LIMITED_API`` to the correct hex version (e.g. ``0x03090000``
94+
for Python 3.9).
95+
- Names the output file with the ``.abi3.so`` suffix on Linux/macOS or
96+
``.pyd`` on Windows.
97+
- Links against the stable ``python3.dll`` import library on Windows.
98+
99+
The corresponding C source file needs no special changes — just include the
100+
NumPy headers as usual::
101+
102+
#include <Python.h>
103+
#include <numpy/arrayobject.h>
104+
105+
static PyModuleDef moduledef = {
106+
.m_base = PyModuleDef_HEAD_INIT,
107+
.m_name = "my_ext",
108+
};
109+
110+
PyMODINIT_FUNC PyInit_my_ext(void)
111+
{
112+
import_array();
113+
return PyModule_Create(&moduledef);
114+
}
115+
116+
117+
Building with setuptools
118+
========================
119+
120+
If you use ``setuptools``, define ``Py_LIMITED_API`` manually via
121+
``define_macros`` and set ``py_limited_api=True`` on the ``Extension``::
122+
123+
import numpy as np
124+
from setuptools import Extension, setup
125+
126+
ext = Extension(
127+
"my_ext",
128+
sources=["my_ext.c"],
129+
include_dirs=[np.get_include()],
130+
define_macros=[
131+
("Py_LIMITED_API", "0x03090000"), # Python 3.9+
132+
("NPY_NO_DEPRECATED_API", "NPY_1_21_API_VERSION"),
133+
],
134+
py_limited_api=True, # tags wheel as .abi3
135+
)
136+
137+
setup(ext_modules=[ext])
138+
139+
.. note::
140+
141+
``py_limited_api=True`` only affects the wheel tag generated by
142+
``bdist_wheel``. You still need to pass ``Py_LIMITED_API`` as a macro so
143+
the headers hide non-stable symbols.
144+
145+
146+
Building with Cython
147+
====================
148+
149+
`Cython <https://cython.org/>`_ 3.0+ supports the Limited API. In your
150+
``meson.build``::
151+
152+
py.extension_module(
153+
'my_cython_ext',
154+
'my_cython_ext.pyx',
155+
c_args: [
156+
'-DNPY_NO_DEPRECATED_API=0',
157+
'-DCYTHON_LIMITED_API=1',
158+
],
159+
include_directories: [npy_include],
160+
limited_api: '3.9',
161+
install: true,
162+
)
163+
164+
Set ``CYTHON_LIMITED_API=1`` (not ``Py_LIMITED_API`` directly) so that Cython
165+
can generate code compatible with the stable ABI.
166+
167+
.. _limited-api-restrictions:
168+
169+
Restrictions and known limitations
170+
====================================
171+
172+
Not all NumPy C API features are available when ``Py_LIMITED_API`` is defined.
173+
174+
**Types hidden under ``Py_LIMITED_API``:**
175+
176+
- :c:type:`PyArrayScalar_VAL` and :c:type:`PyUnicodeScalarObject` are not
177+
accessible. Code that reads scalar values directly from the struct must be
178+
rewritten to use the NumPy Python API (e.g. :func:`numpy.asarray`) or the
179+
iterator API instead.
180+
181+
- :c:type:`PyArray_DTypeMeta` is a ``typedef`` for ``PyTypeObject`` when
182+
``Py_LIMITED_API`` is set, because it contains ``PyHeapTypeObject`` which
183+
is not part of the stable ABI. Custom DType implementations cannot use
184+
the full ``PyArray_DTypeMeta`` struct layout in Limited API builds.
185+
186+
**General guidelines:**
187+
188+
- Avoid direct struct member access on ``PyArrayObject``. Prefer the
189+
accessor macros (``PyArray_DATA``, ``PyArray_NDIM``, ``PyArray_DIMS``,
190+
etc.) which remain available in the Limited API.
191+
- Check the ``numpy/ndarraytypes.h`` and ``numpy/arrayobject.h`` headers for
192+
``#ifndef Py_LIMITED_API`` guards to identify what is unavailable.
193+
- Build and test your extension against a Python version *different* from the
194+
one you compiled against (e.g. compile on 3.9, test on 3.12) to catch ABI
195+
issues early.
196+
197+
198+
Distributing abi3 wheels
199+
=========================
200+
201+
When building wheels for distribution, set the minimum Python version tag to
202+
match the ``Py_LIMITED_API`` version you compiled against. Tools like
203+
`cibuildwheel <https://cibuildwheel.pypa.io/>`_ handle this automatically
204+
when configured correctly — see the
205+
`cibuildwheel abi3 docs <https://cibuildwheel.pypa.io/en/stable/faq/#abi3>`__
206+
for details.
207+
208+
For a project supporting Python 3.9+, the resulting wheel filename will look
209+
like::
210+
211+
my_ext-1.0-cp39-abi3-manylinux_2_17_x86_64.whl
212+
213+
214+
See also
215+
========
216+
217+
- :ref:`CPython stable ABI documentation <python:stable-abi>`
218+
- :ref:`for-downstream-package-authors` — NumPy ABI compatibility overview
219+
- `cibuildwheel abi3 guide <https://cibuildwheel.pypa.io/en/stable/faq/#abi3>`__
220+
- `Meson Python extension_module docs
221+
<https://mesonbuild.com/Python-module.html#extension_module>`__

‎doc/source/user/c-info.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,3 +8,4 @@ Using NumPy C-API
88
c-info.python-as-glue
99
c-info.ufunc-tutorial
1010
c-info.beyond-basics
11+
c-info.limited-api

0 commit comments

Comments
 (0)