Repository navigation
Commit 30cd596
Stop a GIL-releasing __del__ from segfaulting the standard C extension build (#1491)
<!-- Thank you for your contribution! -->
## What do these changes do?
Extends the deferred-decref technique from #1487 (free-threaded builds
only) to the standard, GIL-enabled C extension build. On that build
`Py_BEGIN_CRITICAL_SECTION` compiles to a no-op, so a decref whose
`__del__` (or a weakref callback) itself releases the GIL was the only
thing standing between a mutation and a second thread running
concurrently
on the very same `MultiDict`; nothing caught that. `update()`,
`merge()`,
`__setitem__()`, `__delitem__()`, `pop()`, `popone()`, and `popall()`
now
defer every such decref until the mutation has fully finished, the same
way the free-threaded build already does. `md_clear()` needed no change:
it already swaps `md->keys` to the empty table before releasing any
reference, so a suspended decref there can only ever be observed against
a fully detached old table.
Also bumped the deferred-decref accumulator's inline buffer from 8 to
1024 entries so the allocator is essentially never reached in practice.
A related, pre-existing gap was found but is intentionally **not** fixed
here (see the follow-up note below): `md_del()`/`popall()`'s own
multi-entry loops cache the entries array and iterator once at loop
start
and never refresh them the way `_md_replace()`/`_md_update()` do, so a
decref-triggered suspension mid-loop can leave them pointing at a table
a
concurrent resize already freed. This reproduces identically on
unmodified upstream `master`, on both builds, so it predates this PR.
C extension only; the pure-Python implementation has no locking of its
own to regress.
## Are there changes in behavior for the user?
Yes: a value type whose `__del__` (or a weakref callback) releases the
GIL while another thread concurrently mutates the same `MultiDict` no
longer risks a segfault or table corruption on the standard build.
Single-threaded behavior is unchanged.
## Is it a substantial burden for the maintainers to support this?
No. This removes conditional compilation rather than adding it: the
`Py_GIL_DISABLED`-only deferred-decref machinery from #1487 becomes the
one, unconditional implementation for both builds, and two now-redundant
functions (the non-deferred half-deletion helpers) are deleted rather
than kept alongside their deferred counterparts.
## Related issue number
Fixes #1489
## Checklist
- [x] I think the code is well written
- [x] Unit tests for the changes exist
- [x] Documentation reflects the changes
- [ ] If you provide code modification, please add yourself to
`CONTRIBUTORS.txt`
- [x] Add a new news fragment into the `CHANGES/` folder
- [x] `make doc-spelling` passes and any new technical words are added
to `docs/spelling_wordlist.txt`
<details>
<summary>Agent run details (optional, for reviewers)</summary>
Tests: `pytest -q` on a standard CPython 3.13.2 build (2062 passed, 19
skipped), on a free-threaded CPython 3.14.7 build (2062 passed, 19
skipped), and with `--no-c-extensions` (1037 passed, 15 skipped, C-only
tests deselected).
Confirmed the fix is real, not a no-op: the issue's own repro script,
plus an extended version mixing `pop()`/`popall()`/`__delitem__()` in,
segfault reliably (`exit 139`, several runs each) against unmodified
`upstream/master` (242e5bd) and run clean against this branch.
`MULTIDICT_DEBUG_BUILD=1 MULTIDICT_ASAN_BUILD=1`: clean, 2051 passed, no
sanitizer reports.
`MULTIDICT_TSAN_BUILD=1` against the free-threaded build was started but
not completed (stopped partway through a slow run) -- not run to
completion for this submission, so no TSan coverage claim is made here.
`make doc-spelling`: passes with no changes to
`docs/spelling_wordlist.txt`.
`pre-commit run` (ruff, clang-format, mypy for 3.11/3.13, changelog
filename/role checks) on the changed files: all passed.
**Follow-up issue filed:** #1492. `md_del()` (`__delitem__`) and
`md_pop_all()` (`popall()`) compute their hash-chain `entries`/iterator
once at the top of their loop and never refresh them against a
concurrent resize, unlike `_md_replace()`/`_md_update()`, which
recompute from `md->keys` every pass specifically to survive one.
Reproduces a segfault reliably on unmodified upstream `master` on both
builds. Fixing it properly needs the same
recompute-and-retry-on-staleness
shape `_md_replace()`/`_md_update()` already have, which felt like its
own PR rather than a drive-by here.
</details>
---------
Co-authored-by: Claude Sonnet 5 <[email protected]>1 parent d1c331a commit 30cd596
4 files changed
Lines changed: 252 additions & 428 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
0 commit comments