BUG: aligned access for the descriptor fields struct on 32bit abi3t - #31759

I'm going to close this PR because the approach is unsound and open a followup issue.

On a 32 bit free-threaded build, sizeof(PyObject) is 20 bytes. That places the flags field at a 4-byte aligned address instead of an 8-byte aligned address, the compiler inserts some padding bytes, and suddenly the fields access function is returning addresses at the wrong offset for all fields after the flags entry.

All of this sounds like a Python bug though? Python has to guarantee max_align_t alignment of the actual fields.

EDIT: OK, it does guarantee for the whole object, but if PyObject isn't a multiple of that it seems to me we are in deep trouble. At that point the whole PyObject_GetTypeData() function becomes useless for many objects. And I think the whole thing is broken, because .tp_basicsize = -sizeof(my_struct) is unclear that it'll do the right thing either, because it'll presumably not ensure padding to max_align_t there.

Or in other words: The only real solution to me seems for Python to ensure max_align_t size in a sense. That may not sound great, but anything else means that these patterns becomes unusable without extreme care in 32bit.

And I think the whole thing is broken, because .tp_basicsize = -sizeof(my_struct) is unclear that it'll do the right thing either, because it'll presumably not ensure padding to max_align_t there.

I believe that with will align to max_align_t (https://github.com/python/cpython/blob/285d96dd784620fe8afb02688abe72a916555061/Objects/typeobject.c#L5133 and https://github.com/python/cpython/blob/285d96dd784620fe8afb02688abe72a916555061/Objects/typeobject.c#L6088).

That makes PyObject_GetTypeData correct on an object declared with negative size (which is how it's intended to be used).

But I think it makes the assumption that

struct S{
   PyObject_HEAD
   int data1;
   int data2;
};

is safe to access through a

struct T{
    int data1;
    int data2;
};

pointer dodgy. (I think I did claim it was dodgy at the time, but failed to identify this as the reason).

So it really means that it's only safe if you consistently treat a PyObject* and opaque or non-opaque.

Ah, thanks for the note @da-woods, right. If Python does insert the extra alignment for such types, then a type defined via the opaque API can talk to others safely.

But it is still super annoying :(. Sure, using accessors functions works, but it means that we are far worse off then stable ABI objects here. I don't find it very enticing.

Part of me wonders if the right thing wouldn't be to just add the extra padding in NumPy if running on Python 3.15+. That breaks all existing Python 3.15 downstream that are already build against NumPy 2.5 which are also:

• 32bit
• free-threaded
• (It doesn't break other builds, because for them nothing changes.)

But 🤷, waiting for 3.16+ wouldn't be better, it's not like NumPy has a release for 3.15+ available yet, so in a sense we can do that?

Of course... We can also rip this all out and introduce a giant family of getter functions... In theory this affects pretty much all our objects (in practice it may not but for those others adding padding would also do nothing).

(EDIT: I dunno, maybe that is crazy and Cython might be a bit annoying about it as well if the size is Python dependent? And I just really need to warm up to "accessors all the way" even if everything is designed to not require this.
But I of course already find a per-object GetFields function an unnecessary hedge for a future where ABI flexibility reasons that I really don't begin to see being useful as it seems to assume opaque API as the only API and even then seems rather pointless to me.)

pointer dodgy. (I think I did claim it was dodgy at the time, but failed to identify this as the reason).

We were discussing it being dodgy because of aliasing rules mostly, IIRC. If you still think that, then we would have to just stop this all. For the alignment issue, manual alignment is at least conceivable (i.e. align the normal ABI with what an opaque API would do in practice for its objects as I said above).

pointer dodgy. (I think I did claim it was dodgy at the time, but failed to identify this as the reason).

We were discussing it being dodgy because of aliasing rules mostly, IIRC.

Yes indeed. I can't claim to have had an really useful advanced insight here.

Cython might be a bit annoying about it as well if the size is Python dependent?

For the moment I'm assuming that it's hard to mix opaque and non-opaque modules useful (or at least, they can only mix one way because we can always make things opaque). The only complication is the memoryview internal classes. I'll have to go back and check if they have this alignment issue on 32 bits. If not, it'll be luck rather than judgment.