Skip to content

BUG: descriptor accessors should account for struct padding on 32bit abi3t builds #31762

Closed

Description

@ngoldbaum

This was first seen in PyO3/rust-numpy#556

The flags field of the PyArray_Descr_fields struct is explicitly 64 bits:

/* Space for dtype instance specific flags. */
npy_uint64 flags;

On the 32-bit free-threaded build, sizeof(PyObject) is 20 bytes. On the 32 GIL-enabled build it's 8 bytes.

The combination of these facts have annoying implications for 32-bit Python abi3t builds, given our current field access implementation.

On the free-threaded build, type_num is at 28 bytes, so flags is at 32 bytes and no padding is necessary. On the 32 bit GIL-enabled build, the type_num starts at offset 16, so flags must be padded from 20 to 24 bytes to stay 8-byte aligned.

This throws off the way we implemented access for flags and all fields after flags on abi3t, since all accesses go through the _PyDataType_GET_ITEM_DATA function and there are now two different offsets we need to deal with for flags and every field after.

I think at a minimum we need to add at least one new API function to account for whether or not the padding bytes exist for accesses on flags and all fields after flags. On abi3t, the offset calculation needs to happen in NumPy internals, and it's the only place that can know at runtime if the interpreter is free-threaded or not.

On 64 bit, the PyObject sizes on free-threaded and GIL-enabled builds are 32 bytes and 16 bytes, respectively. These are both 8-byte aligned and we don't notice this issue.

I had an attempt at a fix for this in #31759. The fix is wrong (it breaks the GIL-enabled 32 bit build) but the test I added might be helpful.

Activity

  1. added
    39 - free-threadingPRs and issues related to support for free-threading CPython (a.k.a. no-GIL, PEP 703)
    on Jun 25, 2026
  2. changed the title [-]BUG: descriptor struct has aligned access issues on 32bit abi3t builds[/-] [+]BUG: descriptor struct has to account for struct padding on 32bit abi3t builds[/+] on Jun 26, 2026
  3. changed the title [-]BUG: descriptor struct has to account for struct padding on 32bit abi3t builds[/-] [+]BUG: descriptor accessors should account for struct padding on 32bit abi3t builds[/+] on Jun 26, 2026
  4. sahil-mangla commented on Jun 26, 2026

    @sahil-mangla

    @ngoldbaum Can i try working on this and try it?

  5. Aniketsy commented on Jun 26, 2026

    @Aniketsy
    Contributor

    @ngoldbaum Can i try working on this and try it?

    @sahil-mangla I was giving it try 😀, need some work before testing. Could you please wait from some time to avoid duplicate efforts. I'll let you know if there is no progress from my end. Thanks!

  6. ngoldbaum commented on Jun 26, 2026

    @ngoldbaum
    MemberAuthor

    There are some design decisions that need to be resolved in #31764 before a fix for this should come in. I'd hold off on sending in PRs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    00 - Bug39 - free-threadingPRs and issues related to support for free-threading CPython (a.k.a. no-GIL, PEP 703)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions