Repository navigation
SubTracker: Move static variables into a struct and access them via PyModule_GetState. #31930
Description
Long Term Effort Tracker: #31913
| Module | Status | Link |
|---|---|---|
lapack_lite |
Done | #31928 |
_multiarray_umath |
State moved, still accessed through the global _npy_module_state pointer |
#31992, #32544 |
_pocketfft_umath _simd, _multiarray_tests _umath_linalg |
No change needed | — |
random (9 Cython modules) |
Out of scope | — |
_rational_tests |
Done | #32456 |
TODO
- State propagation in
_multiarray_umathENH: Migrate_multiarray_umathglobal variables to per-module state #31992 requires separate PRs - Remove
_npy_module_stateby passing the module or state pointer to every access site
Activity
ngoldbaum commentedon Jul 9, 2026
For these subtracker issues I'd like to see a bit more detail about what you're planning to do in the description.
da-woods commentedon Jul 12, 2026
You probably already know this (in that I'm sure you've thought about it more than me), but I think you have some "fun" to come with
import_array().I think you need to fix both sides of it, in that
static void**as your array API table (rather than a pointer in the module state).prathamhole14 commentedon Jul 13, 2026
Hi everyone!
This is a high level overview of how I plan to migrate to module-level state for the
_multiarray_umathmodule.Infrastructure set up containing these 5 structs and other scattered global variables (zero functional changes).
A : Add to the bottom of
npy_static_data.h- No change header file importB : Create a new module_state.h in multiarray - Clean in one single
I prefer option B but please let me know what you think.
Migration PR for each struct
npy_interned_str_struct,npy_static_pydata_struct,npy_static_cdata_struct,npy_runtime_imports_struct,npy_global_state_struct.To keep the PRs for the larger structs easy to review, I plan to temporarily use a global pointer
_npy_module_stateto access the module state. This avoids having to thread the state pointer through dozens of deep call chains in a single PR. We can remove the transitional global pointer once all migrations are complete.Temporary global pointer like
multiarray_umath_state *_npy_module_state = NULLMigration of other scattered globals just like structs.
This is high level of how I am planning to proceed, please let me know what you think then I will start creating PRs one by one.
CC: @mattip @ngoldbaum @rgommers @da-woods
ngoldbaum commentedon Jul 13, 2026
Are there still mutable globals scattered in the NumPy implementation? I thought I got all of them for free-threading support.
prathamhole14 commentedon Jul 14, 2026
Yeah, there are still a few scattered globals left
PyDataMem_DefaultHandler,current_handler,uo_indexandtypeDict(I took help from AI to find them). I think they were probably left as process-globals during the free-threading work because they're read-only and thread-safengoldbaum commentedon Jul 14, 2026
Ah it wasn’t clear to me you need to do this for immutable globals too. Thanks.
prathamhole14 commentedon Jul 14, 2026
Should the infrastructure PR include all 5 struct fields upfront (with FIXME notes), or should I add each field only in the PR that migrates it? If we add all 5 in one, the benefit is other PRs can be made without worrying about merge conflicts, but the disadvantage will be: until everything is shifted, "dead" code will be present in numpy for some time
ngoldbaum commentedon Jul 14, 2026
In that case I’d prefer to do it all in one PR. Others may disagree with me though.
mattip commentedon Jul 14, 2026
I think this is small enough to do in one PR. I am worried about the viability of moving the API table into a module struct, if we discover that it is not straight-forward we may have to revisit the idea of a single PR. If we decide to give up on some of the global data, we can clean up the dead code later on.
prathamhole14 commentedon Jul 14, 2026
@ngoldbaum and @mattip thank you for helping with decision making. Please checkout #31992