ENH: Implement per-module state for lapack_lite to avoid global static states by prathamhole14 · Pull Request #31928 · numpy/numpy

prathamhole14 · 2026-07-09T08:30:37Z

PR summary

Related to: #31930 and #32451

AI Disclosure

Only for some research and improving comments.

jorenham · 2026-07-09T08:40:34Z

Please ensure everything passes locally before making a PR. Running CI isn't free; and the maintainers get a notification for each commit regardless of whether it's a draft.

Also; AI disclosure

prathamhole14 · 2026-07-09T09:59:44Z

Sorry I should have provided context from the start, I did compile and ran tests locally and they passed but since this PR tries to adopt to limited api I was afraid that it might fail on other systems. This PR is related to #31913.

jorenham · 2026-07-09T10:01:26Z

Ah actually the CI failure isn't your fault; see #31932

ngoldbaum · 2026-07-09T18:55:06Z

Out of curiosity, why is this needed? I don't think per-module state is needed for limited API support. Or are you also working on subinterpreter support?

mattip · 2026-07-09T20:35:54Z

That would be my "fault".Yes, the goal of the internship is both tasks, as a modernization of the internal NumPy code base,and to see if these types of changes affect performance.

prathamhole14 · 2026-07-10T06:24:08Z

Out of curiosity, why is this needed? I don't think per-module state is needed for limited API support. Or are you also working on subinterpreter support?

Yes it's not "requirement" for limited api but since CPython is moving towards subinterpreters, @mattip and I discussed to start this effort with moving to per-module state.

prathamhole14 · 2026-07-10T06:28:33Z

| Change   | Before [dd4339c4] <main>   | After [902a8e82] <lapack_litemodule_tostruct>   |   Ratio | Benchmark (Parameter)                                                                      |
|----------|----------------------------|-------------------------------------------------|---------|--------------------------------------------------------------------------------------------|
| +        | 40.5±1μs                   | 77.2±3μs                                        |    1.91 | bench_core.StatsMethods.time_var('bool_', 10000)                                           |
| +        | 577±20μs                   | 802±100μs                                       |    1.39 | bench_core.CountNonzero.time_count_nonzero_axis(3, 10000, <class 'object'>)                |
| +        | 99.1±10μs                  | 128±8μs                                         |    1.3  | bench_core.CountNonzero.time_count_nonzero_multi_axis(3, 10000, <class 'str'>)             |
| +        | 603±40ns                   | 688±30ns                                        |    1.14 | bench_core.CountNonzero.time_count_nonzero(1, 100, <class 'numpy.float32'>)                |
| +        | 1.05±0.02ms                | 1.17±0.08ms                                     |    1.11 | bench_core.Indices.time_indices                                                            |
| -        | 651±7ns                    | 619±40ns                                        |    0.95 | bench_core.CountNonzero.time_count_nonzero(3, 100, <class 'numpy.int16'>)                  |
| -        | 34.5±0.5μs                 | 32.5±0.9μs                                      |    0.94 | bench_core.PackBits.time_packbits_little(<class 'numpy.uint64'>)                           |
| -        | 3.29±0.04μs                | 3.09±0.2μs                                      |    0.94 | bench_core.StatsMethods.time_min('int64', 10000)                                           |
| -        | 5.26±0.04μs                | 4.90±0.3μs                                      |    0.93 | bench_core.CountNonzero.time_count_nonzero_axis(1, 100, <class 'numpy.int64'>)             |
| -        | 1.21±0.01μs                | 1.12±0.04μs                                     |    0.92 | bench_core.StatsMethods.time_min('float32', 100)                                           |
| -        | 1.87±0.06μs                | 1.69±0.07μs                                     |    0.91 | bench_core.CountNonzero.time_count_nonzero(2, 10000, <class 'numpy.int8'>)                 |
| -        | 5.61±0.2μs                 | 5.08±0.2μs                                      |    0.91 | bench_core.CountNonzero.time_count_nonzero_axis(2, 100, <class 'numpy.float32'>)           |
| -        | 2.70±0.2μs                 | 2.46±0.1μs                                      |    0.91 | bench_core.NumPyChar.time_isalpha_small_list_big_string                                    |
| -        | 18.2±0.3μs                 | 16.4±0.7μs                                      |    0.9  | bench_core.StatsMethods.time_var('float32', 100)                                           |
| -        | 1.23±0.05μs                | 1.10±0.02μs                                     |    0.89 | bench_core.CountNonzero.time_count_nonzero(1, 10000, <class 'numpy.int8'>)                 |
| -        | 6.79±0.1μs                 | 6.03±0.3μs                                      |    0.89 | bench_core.CountNonzero.time_count_nonzero_axis(2, 100, <class 'str'>)                     |
| -        | 1.25±0.04μs                | 1.11±0.06μs                                     |    0.89 | bench_core.StatsMethods.time_min('uint64', 100)                                            |
| -        | 11.6±0.3μs                 | 10.4±0.7μs                                      |    0.89 | bench_core.StatsMethods.time_prod('float32', 10000)                                        |
| -        | 2.43±0.1μs                 | 2.16±0.1μs                                      |    0.89 | bench_core.StatsMethods.time_sum('bool_', 100)                                             |
| -        | 15.5±0.05μs                | 13.9±1μs                                        |    0.89 | bench_core.StatsMethods.time_var('uint64', 100)                                            |
| -        | 625±30ns                   | 549±10ns                                        |    0.88 | bench_core.CountNonzero.time_count_nonzero(1, 100, <class 'numpy.int32'>)                  |
| -        | 2.07±0.04μs                | 1.83±0.03μs                                     |    0.88 | bench_core.CountNonzero.time_count_nonzero(2, 10000, <class 'numpy.int16'>)                |
| -        | 2.50±0.04μs                | 2.20±0.2μs                                      |    0.88 | bench_core.CountNonzero.time_count_nonzero(3, 10000, <class 'numpy.int8'>)                 |
| -        | 5.31±0.2μs                 | 4.65±0.3μs                                      |    0.88 | bench_core.CountNonzero.time_count_nonzero_axis(1, 100, <class 'numpy.int32'>)             |
| -        | 14.5±0.3μs                 | 12.7±0.8μs                                      |    0.88 | bench_core.NumPyChar.time_startswith_big_list_small_string                                 |
| -        | 1.25±0.06μs                | 1.10±0.09μs                                     |    0.88 | bench_core.StatsMethods.time_max('int64', 100)                                             |
| -        | 675±40ns                   | 586±20ns                                        |    0.87 | bench_core.CountNonzero.time_count_nonzero(1, 100, <class 'numpy.float64'>)                |
| -        | 199±10μs                   | 173±10μs                                        |    0.87 | bench_core.CountNonzero.time_count_nonzero(3, 1000000, <class 'numpy.int8'>)               |
| -        | 3.42±0.02μs                | 2.98±0.3μs                                      |    0.87 | bench_core.StatsMethods.time_min('uint64', 10000)                                          |
| -        | 120±5μs                    | 103±4μs                                         |    0.86 | bench_core.CountNonzero.time_count_nonzero(2, 1000000, <class 'bool'>)                     |
| -        | 4.55±0.1μs                 | 3.91±0.2μs                                      |    0.86 | bench_core.CountNonzero.time_count_nonzero_axis(1, 100, <class 'bool'>)                    |
| -        | 4.04±0.02μs                | 3.49±0.3μs                                      |    0.86 | bench_core.NumPyChar.time_startswith_small_list_big_string                                 |
| -        | 40.3±0.5μs                 | 34.8±3μs                                        |    0.86 | bench_core.StatsMethods.time_std('int64', 10000)                                           |
| -        | 5.40±0.3μs                 | 4.62±0.09μs                                     |    0.85 | bench_core.CountNonzero.time_count_nonzero(2, 10000, <class 'numpy.int64'>)                |
| -        | 31.3±0.4μs                 | 26.5±2μs                                        |    0.85 | bench_core.StatsMethods.time_max('complex64', 10000)                                       |
| -        | 1.98±0.02μs                | 1.67±0.1μs                                      |    0.84 | bench_core.CountNonzero.time_count_nonzero(2, 10000, <class 'bool'>)                       |
| -        | 699±30ns                   | 590±60ns                                        |    0.84 | bench_core.CountNonzero.time_count_nonzero(3, 100, <class 'numpy.int8'>)                   |
| -        | 1.58±0.4μs                 | 1.32±0.04μs                                     |    0.84 | bench_core.Nonzero.time_nonzero_sparse(<class 'numpy.int64'>, (100,))                      |
| -        | 1.43±0.07μs                | 1.21±0.05μs                                     |    0.84 | bench_core.StatsMethods.time_prod('int64', 100)                                            |
| -        | 1.10±0.1ms                 | 918±30μs                                        |    0.83 | bench_core.CountNonzero.time_count_nonzero(3, 1000000, <class 'numpy.int32'>)              |
| -        | 1.22±0.02ms                | 1.01±0.09ms                                     |    0.83 | bench_core.CountNonzero.time_count_nonzero_axis(2, 1000000, <class 'numpy.int8'>)          |
| -        | 1.24±0.02μs                | 1.03±0.03μs                                     |    0.83 | bench_core.StatsMethods.time_min('float64', 100)                                           |
| -        | 16.7±0.08μs                | 13.8±1μs                                        |    0.83 | bench_core.StatsMethods.time_prod('bool_', 10000)                                          |
| -        | 663±60ns                   | 541±30ns                                        |    0.82 | bench_core.CountNonzero.time_count_nonzero(2, 100, <class 'bool'>)                         |
| -        | 6.79±2ms                   | 5.56±0.3ms                                      |    0.82 | bench_core.Nonzero.time_nonzero_sparse(<class 'numpy.int64'>, (1000, 1000))                |
| -        | 11.8±0.2μs                 | 9.63±1μs                                        |    0.82 | bench_core.StatsMethods.time_prod('uint64', 10000)                                         |
| -        | 2.40±0.7ms                 | 1.93±0.05ms                                     |    0.81 | bench_core.CountNonzero.time_count_nonzero(3, 1000000, <class 'numpy.int64'>)              |
| -        | 62.8±3μs                   | 50.9±5μs                                        |    0.81 | bench_core.StatsMethods.time_prod('complex64', 10000)                                      |
| -        | 33.9±2μs                   | 27.0±2μs                                        |    0.8  | bench_core.CountNonzero.time_count_nonzero_axis(3, 10000, <class 'numpy.float64'>)         |
| -        | 5.70±0.6ms                 | 4.53±0.2ms                                      |    0.8  | bench_core.Nonzero.time_nonzero_sparse(<class 'numpy.float32'>, (1000000,))                |
| -        | 12.5±0.2μs                 | 10.0±0.8μs                                      |    0.8  | bench_core.StatsMethods.time_prod('int64', 10000)                                          |
| -        | 35.1±8μs                   | 27.8±0.9μs                                      |    0.79 | bench_core.CountNonzero.time_count_nonzero(1, 10000, <class 'str'>)                        |
| -        | 939±40μs                   | 740±70μs                                        |    0.79 | bench_core.CountNonzero.time_count_nonzero_multi_axis(1, 1000000, <class 'numpy.float32'>) |
| -        | 11.2±1μs                   | 8.77±0.2μs                                      |    0.78 | bench_core.CountNonzero.time_count_nonzero(1, 10000, <class 'numpy.float32'>)              |
| -        | 3.51±0.6ms                 | 2.70±0.03ms                                     |    0.77 | bench_core.CountNonzero.time_count_nonzero(3, 1000000, <class 'numpy.float32'>)            |
| -        | 19.6±0.4μs                 | 14.8±1μs                                        |    0.76 | bench_core.CountNonzero.time_count_nonzero_axis(2, 10000, <class 'numpy.int8'>)            |
| -        | 1.40±0.04μs                | 1.06±0.01μs                                     |    0.76 | bench_core.StatsMethods.time_max('float64', 100)                                           |
| -        | 1.73±0.3μs                 | 1.28±0.04μs                                     |    0.74 | bench_core.CountNonzero.time_count_nonzero(1, 10000, <class 'numpy.int16'>)                |
| -        | 3.63±0.6μs                 | 2.67±0.08μs                                     |    0.74 | bench_core.CountNonzero.time_count_nonzero(1, 10000, <class 'numpy.int64'>)                |
| -        | 3.95±2ms                   | 2.91±0.1ms                                      |    0.74 | bench_core.CountNonzero.time_count_nonzero(3, 1000000, <class 'numpy.float64'>)            |
| -        | 6.72±2μs                   | 4.82±0.4μs                                      |    0.72 | bench_core.CountNonzero.time_count_nonzero_axis(3, 100, <class 'numpy.int16'>)             |
| -        | 1.82±0.2μs                 | 1.29±0.1μs                                      |    0.71 | bench_core.StatsMethods.time_min('bool_', 10000)                                           |
| -        | 4.86±1μs                   | 3.40±0.3μs                                      |    0.7  | bench_core.StatsMethods.time_max('uint64', 10000)                                          |
| -        | 73.7±20μs                  | 51.0±1μs                                        |    0.69 | bench_core.CountNonzero.time_count_nonzero(1, 1000000, <class 'numpy.int8'>)               |
| -        | 13.4±3μs                   | 9.07±0.3μs                                      |    0.68 | bench_core.CountNonzero.time_count_nonzero(1, 10000, <class 'numpy.float64'>)              |
| -        | 1.25±0.2μs                 | 843±60ns                                        |    0.67 | bench_core.Core.time_arange_100                                                            |
| -        | 7.28±2μs                   | 4.88±0.4μs                                      |    0.67 | bench_core.CorrConv.time_correlate(1000, 10, 'full')                                       |
| -        | 1.48±0.3μs                 | 987±40ns                                        |    0.67 | bench_core.CountNonzero.time_count_nonzero(2, 100, <class 'str'>)                          |
| -        | 1.03±0.3μs                 | 679±30ns                                        |    0.66 | bench_core.CountNonzero.time_count_nonzero(2, 100, <class 'numpy.float64'>)                |
| -        | 1.45±0.2μs                 | 963±70ns                                        |    0.66 | bench_core.Nonzero.time_nonzero_sparse(<class 'numpy.float32'>, (2,))                      |
| -        | 2.66±0.3μs                 | 1.77±0.2μs                                      |    0.66 | bench_core.PackBits.time_packbits_little(<class 'bool'>)                                   |
| -        | 1.59±0.5μs                 | 1.05±0.03μs                                     |    0.66 | bench_core.StatsMethods.time_max('float32', 100)                                           |
| -        | 2.24±0.3μs                 | 1.46±0.06μs                                     |    0.65 | bench_core.CorrConv.time_correlate(1000, 1000, 'valid')                                    |
| -        | 52.0±10μs                  | 33.3±2μs                                        |    0.64 | bench_core.CorrConv.time_correlate(1000, 100, 'valid')                                     |
| -        | 322±70μs                   | 205±10μs                                        |    0.64 | bench_core.CountNonzero.time_count_nonzero(1, 1000000, <class 'numpy.int32'>)              |
| -        | 83.8±30μs                  | 51.9±3μs                                        |    0.62 | bench_core.CountNonzero.time_count_nonzero(1, 1000000, <class 'bool'>)                     |
| -        | 637±200μs                  | 390±20μs                                        |    0.61 | bench_core.CorrConv.time_correlate(100000, 10, 'valid')                                    |
| -        | 709±20ns                   | 429±80ns                                        |    0.6  | bench_core.Core.time_array_1                                                               |
| -        | 2.95±0.7μs                 | 1.77±0.04μs                                     |    0.6  | bench_core.CountNonzero.time_count_nonzero(1, 10000, <class 'numpy.int32'>)                |
| -        | 6.74±3μs                   | 4.04±0.2μs                                      |    0.6  | bench_core.CountNonzero.time_count_nonzero_axis(3, 100, <class 'bool'>)                    |
| -        | 12.5±2μs                   | 7.53±0.5μs                                      |    0.6  | bench_core.StatsMethods.time_sum('complex64', 10000)                                       |
| -        | 265±8μs                    | 153±40μs                                        |    0.58 | bench_core.CorrConv.time_correlate(1000, 1000, 'same')                                     |
| -        | 766±100ns                  | 434±40ns                                        |    0.57 | bench_core.Core.time_array_empty                                                           |
| -        | 2.75±0.5ms                 | 1.56±0.2ms                                      |    0.57 | bench_core.CorrConv.time_correlate(1000, 10000, 'full')                                    |
| -        | 2.80±0.2ms                 | 1.55±0.2ms                                      |    0.55 | bench_core.CorrConv.time_correlate(1000, 10000, 'same')                                    |
| -        | 387±20μs                   | 211±50μs                                        |    0.54 | bench_core.CorrConv.time_correlate(1000, 1000, 'full')                                     |
| -        | 112±40μs                   | 60.3±0.9μs                                      |    0.54 | bench_core.CountNonzero.time_count_nonzero(1, 1000000, <class 'numpy.int16'>)              |
| -        | 15.6±1μs                   | 7.97±3μs                                        |    0.51 | bench_core.CorrConv.time_convolve(1000, 10, 'same')                                        |
| -        | 17.9±6μs                   | 9.09±2μs                                        |    0.51 | bench_core.CorrConv.time_convolve(50, 100, 'full')                                         |
| -        | 544±90μs                   | 279±50μs                                        |    0.51 | bench_core.CorrConv.time_convolve(50, 10000, 'same')                                       |
| -        | 3.02±0.2ms                 | 1.55±0.3ms                                      |    0.51 | bench_core.CorrConv.time_correlate(1000, 10000, 'valid')                                   |
| -        | 2.97±0.4μs                 | 1.48±0.08μs                                     |    0.5  | bench_core.Nonzero.time_nonzero_sparse(<class 'numpy.float32'>, (100,))                    |
| -        | 81.1±10μs                  | 40.1±5μs                                        |    0.49 | bench_core.CorrConv.time_correlate(1000, 100, 'full')                                      |
| -        | 71.6±6μs                   | 34.8±4μs                                        |    0.49 | bench_core.CorrConv.time_correlate(1000, 100, 'same')                                      |
| -        | 14.3±2μs                   | 6.77±2μs                                        |    0.47 | bench_core.CorrConv.time_convolve(1000, 10, 'valid')                                       |
| -        | 661±200μs                  | 305±60μs                                        |    0.46 | bench_core.CorrConv.time_convolve(50, 10000, 'valid')                                      |
| -        | 8.06±2μs                   | 3.41±0.6μs                                      |    0.42 | bench_core.CorrConv.time_convolve(1000, 1000, 'valid')                                     |
| -        | 91.5±30μs                  | 37.9±10μs                                       |    0.41 | bench_core.CorrConv.time_convolve(50, 1000, 'same')                                        |
| -        | 109±20μs                   | 43.4±9μs                                        |    0.4  | bench_core.CorrConv.time_convolve(1000, 100, 'full')                                       |
| -        | 766±200μs                  | 308±70μs                                        |    0.4  | bench_core.CorrConv.time_convolve(50, 10000, 'full')                                       |
| -        | 96.4±2μs                   | 34.2±5μs                                        |    0.35 | bench_core.CorrConv.time_convolve(1000, 100, 'valid')                                      |
| -        | 176±60μs                   | 46.5±5μs                                        |    0.26 | bench_core.CorrConv.time_convolve(1000, 100, 'same')                                       |

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
PERFORMANCE DECREASED.

I ran the benchmarks (spin bench -c main -t bench_core) and noticed a decrease in performance, is this expected?

da-woods · 2026-07-10T18:40:32Z

My basic (untested) intuition about performance is that looking things up in the module state will almost certainly be slower than looking things up from a static global. But in this case it's only an exception that will almost never be used so it shouldn't make a huge difference

seberg · 2026-07-16T07:27:43Z

This looks good and happy if you just move forward, thanks.

But, maybe for for Matti to decide: Now that I see this, it seems to me that this whole module can probably be just deleted? The one confusing thing might be the xerbla test in #12472 but probably we can just delete the np.linalg.lapack_lite.xerbla()...

mattip · 2026-07-16T07:46:42Z

The module is useful when there is no BLAS at all.

mattip · 2026-07-16T07:46:53Z

Thanks @prathamhole14

seberg · 2026-07-16T07:51:55Z

The module is useful when there is no BLAS at all.

That's the thing, though! lapack-lite is of course used, but it is used via _umath_linalg. The only actual import of lapack_lite is that one test calling xerbla().

seberg · 2026-07-16T07:55:03Z

That said, I guess technically import numpy.linalg.lapack_lite is "public" and that is probably the reason why it still exists...

mattip · 2026-07-16T07:55:31Z

Ahh, so the code is used but the module is not. Maybe we should deprecate the import and rename the module to _test_lapack-lite

seberg · 2026-07-16T07:56:50Z

Yeah, agreed. Since it is in principle public, let's just make sure to tag a deprecation warning on for now... Let me open an issue, it's a nice small one.

prathamhole14 · 2026-07-16T08:02:56Z

Thanks! I’ll put a PR for it.

prathamhole14 commented Jul 9, 2026

View reviewed changes

Comment thread numpy/linalg/lapack_litemodule.c Resolved

prathamhole14 marked this pull request as ready for review July 9, 2026 10:04

da-woods reviewed Jul 10, 2026

View reviewed changes

Comment thread numpy/linalg/lapack_litemodule.c Resolved

charris added the 01 - Enhancement label Jul 10, 2026

prathamhole14 added 2 commits July 12, 2026 12:06

implement per-module state for lapack_lite to avoid global state issues

a6052e6

Lazily fetch module state in lapack_lite check_object

070942a

prathamhole14 force-pushed the lapack_litemodule_tostruct branch from 902a8e8 to 070942a Compare July 12, 2026 06:36

prathamhole14 mentioned this pull request Jul 12, 2026

SubTracker: Move static variables into a struct and access them via PyModule_GetState. #31930

Open

1 of 2 tasks

prathamhole14 commented Jul 12, 2026

View reviewed changes

Comment thread numpy/linalg/lapack_litemodule.c Resolved

mattip merged commit 565c164 into numpy:main Jul 16, 2026
91 of 92 checks passed

prathamhole14 deleted the lapack_litemodule_tostruct branch July 16, 2026 07:49

prathamhole14 mentioned this pull request Jul 16, 2026

DEP: Deprecate numpy.linalg.lapack_lite module #32015

Merged

Conversation

prathamhole14 commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR summary

AI Disclosure

Uh oh!

jorenham commented Jul 9, 2026

Uh oh!

prathamhole14 commented Jul 9, 2026

Uh oh!

jorenham commented Jul 9, 2026

Uh oh!

Uh oh!

ngoldbaum commented Jul 9, 2026

Uh oh!

mattip commented Jul 9, 2026

Uh oh!

prathamhole14 commented Jul 10, 2026

Uh oh!

prathamhole14 commented Jul 10, 2026

Uh oh!

Uh oh!

da-woods commented Jul 10, 2026

Uh oh!

Uh oh!

seberg commented Jul 16, 2026

Uh oh!

mattip commented Jul 16, 2026

Uh oh!

Uh oh!

mattip commented Jul 16, 2026

Uh oh!

seberg commented Jul 16, 2026

Uh oh!

seberg commented Jul 16, 2026

Uh oh!

mattip commented Jul 16, 2026

Uh oh!

seberg commented Jul 16, 2026

Uh oh!

prathamhole14 commented Jul 16, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

prathamhole14 commented Jul 9, 2026 •

edited

Loading