numpy-abi-check

Layout checker for NumPy's opaque-PyObject stable ABI (abi3t, Python 3.15+).

NumPy 2.5 lets extensions target CPython's free-threading-compatible stable ABI ("abi3t"), where PyObject is an opaque type. NumPy's public headers handle this by compiling the exposed object structs in two shapes: NumPy itself (and every regular extension) sees them with PyObject_HEAD, while an abi3t extension sees headerless *_fields structs and reaches them through _Py*_GET_ITEM_DATA accessor functions exported in the API table.

Those two views describe the same object in memory, so they must agree on where every member lives. That holds only if the fields region starts at a multiple of the headerless struct's alignment. sizeof(PyObject) is 8, 16, and 32 on 32-bit-GIL, 64-bit-GIL, and 64-bit free-threaded builds — all multiples of 8 — but 20 on 32-bit free-threaded builds. There, any struct member with 8-byte alignment (for example the npy_uint64 flags in PyArray_Descr) is padded differently in the two views, and every member after it is silently read 4 bytes off. That is the crash fixed by numpy/numpy#31771 (issues #31762, #31764).

No single translation unit can check this invariant, because it relates two preprocessor configurations of the same headers. This tool therefore compiles the headers more than once and compares:

• src/npy_layout_members.inc — the single source of truth: every struct exposed through the opaque-ABI mechanism, with its complete member list.
• src/npy_layout_table.c — emits each struct's offsetof/sizeof/ _Alignof table as JSON. Built three times:
  • with-head — NumPy's own view;
  • abi3t (-DPy_TARGET_ABI3T) — the abi3t extension's view;
  • ft32sim (-DFAKE_FT32_HEAD) — the with-head view under a simulated 20-byte object header, reproducing the one problematic configuration (32-bit free-threaded) on any development machine. The program only evaluates offsetof/sizeof; no Python objects are ever created, so replacing the header with a char[20] blob is safe.
• scripts/compare_layouts.py — verifies, member by member, that a with-head build and the abi3t build agree on relative offsets, fields-region size, and the alignment guarantees NumPy promises (headerless alignment ≤ 8, fields region suitably aligned).
• scripts/check_header_coverage.py — scans the public headers and fails if any struct with an object head (direct, conditional, or via by-value embedding of another object struct) is visible to abi3t builds without being in the layout-check list. This is what keeps the checker honest: a new struct cannot be added to the public API without either getting the layout check or being explicitly gated out of abi3t.

Requirements

• meson >= 1.2 and a C11 compiler
• NumPy >= 2.5 installed in the interpreter under test via a regular install, e.g. pip install numpy (the opaque-ABI machinery does not exist before 2.5, and editable installs lack the generated headers)
• Python >= 3.15 headers for the layout comparisons (on older Pythons the abi3t target does not exist and NumPy intentionally applies no padding, so those two tests report SKIP; the header-coverage test still runs)

Usage

The tool checks the numpy installed in the selected interpreter (numpy.get_include()):

meson setup build -Dpython=/path/to/python3.15
meson test -C build --print-errorlogs

To check an unreleased NumPy, install it from a checkout first:

python3.15 -m pip install /path/to/numpy   # or: uv pip install /path/to/numpy
meson setup build -Dpython=python3.15
meson test -C build --print-errorlogs

The three tests:

test	meaning of failure
layout-real-head-vs-abi3t	On this platform, abi3t extensions read these members at the wrong offsets. Trivially green on 64-bit; on a real 32-bit free-threaded interpreter this is the true end-to-end check.
layout-ft32-simulated-head-vs-abi3t	The invariant breaks under a 20-byte object header, i.e. the code would be broken on 32-bit free-threaded builds — catchable from any 64-bit machine.
header-coverage	A struct with object-head layout is reachable from abi3t builds but not layout-checked (or the checked list went stale).

Note that layout-ft32-simulated-head-vs-abi3t fails by design against NumPy main without the numpy/numpy#31771 fix — that is the bug it exists to catch — and passes once the _NPY_OPAQUE_FIRST_FIELD padding is in place.

Maintaining the struct list

When NumPy exposes a new struct through the opaque-ABI mechanism, header-coverage fails and names it. Add a STRUCT_BEGIN/MEMBER/ STRUCT_END block for it in src/npy_layout_members.inc; the first MEMBER row must be the first member of the fields region (for structs that embed another fields struct by value, use a nested designator such as base.typeobj). If a member is added to an existing struct, the compiler will not complain (offsets are simply not compared for it) — the member lists should be refreshed against the headers when NumPy's API version bumps. Structs whose size is intentionally unstable (flexible array member, like PyArrayMultiIterObject_fields) set the second STRUCT_BEGIN argument to 0 to skip the size comparison.

License

Distributed under the same BSD 3-Clause license as NumPy; see LICENSE.txt.