Skip to content

BUG: ensure lock is held when accessing or writing to RNG state - #32061

Merged
charris merged 2 commits into
numpy:mainfrom
ngoldbaum:fix-rng-locking
Jul 22, 2026
Merged

charris merged 2 commits into
numpy:mainfrom
ngoldbaum:fix-rng-locking

Conversation

@ngoldbaum

@ngoldbaum ngoldbaum commented Jul 22, 2026 •

Copy link
Copy Markdown
Member

PR summary

Fixes #32059.

#30360 was supposed to fix these issues but clearly this is harder than I thought.

With this PR the lock is being held everywhere we access the RNG state. Also adds new tests for the case from the issue as well as some other spots I noticed while auditing the RNG codebase for similar issues.

I was able to reproduce the race in a slightly modified version of the script in the issue and it no longer triggers after this PR.

AI Disclosure

I used an AI model to find places where we access RNG state without holding the lock. It also helped with the new tests.

Comment thread numpy/random/mtrand.pyx
Comment on lines 303 to 307
if st['bit_generator'] != 'MT19937' and legacy:
warnings.warn('get_state and legacy can only be used with the '
'MT19937 BitGenerator. To silence this warning, '
'set `legacy` to False.', RuntimeWarning)
legacy = False

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might as well move the state access and the warning into the lock's critical section

@rkern rkern left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thank you!

@charris
charris merged commit c6b36d9 into numpy:main Jul 22, 2026
89 of 91 checks passed
@charris

charris commented Jul 22, 2026

Copy link
Copy Markdown
Member

The errors are unrelated. Thanks Nathan.

@kumaraditya303

Copy link
Copy Markdown
Contributor

The fix is correct and it covers all places. In future I would like PRs to be left open for atleast a day so that I get time to review it before merge. Thanks.

@charris charris removed the 09 - Backport-Candidate PRs tagged should be backported label Jul 27, 2026
charris added a commit that referenced this pull request Jul 27, 2026
BUG: ensure lock is held when accessing or writing to RNG state (#32061)
bashtage added a commit to vigna/randomgen that referenced this pull request Nov 3, 2026
Port NumPy's lock changes (numpy/numpy#30360, numpy/numpy#32061) so
bit generators are safe to share across threads on free-threaded Python.

- Always use an RLock, since NumPy < 2.5 provides a plain Lock that
  deadlocks when locked calls nest; RDRAND's RaisingLock wraps an RLock
- Lock state get/set, seeding (including key=/counter= paths), advance
  and jump in every bit generator and in UserBitGenerator
- Validate state before writing in the AES, Philox and ThreeFry setters
  so a bad state no longer leaves the generator half updated
- Lock the scalar path of ExtendedGenerator.complex_normal
- Declare the extensions free-threading compatible when built with
  Cython >= 3.1 so importing randomgen does not re-enable the GIL
- Add threaded tests covering seed/state races, reentrancy and the GIL

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: Data race reading and writing np.random.seed

5 participants