Skip to content

Commit 4f7363d

Browse files
authored
Fix Meta(pattern=..) memory leak / corruption (msgspec#1256)
`MS_CONSTR_STR_REGEX` slot clear was missing. It was not properly traversed and freed. Now it is. It used to fail with memory sanitized with: ``` ==2637==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x50e000e8e350 in thread T0 #0 0x7f07a10fc4d8 in free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:52 msgspec#1 0x7f079c42ddf4 in TypeNode_Free src/msgspec/_core.c:3474 msgspec#2 0x7f079c42dbdf in TypeNode_Free src/msgspec/_core.c:3468 msgspec#3 0x7f079c55e2eb in JSONDecoder_dealloc src/msgspec/_core.c:17235 msgspec#4 0x7f07a0999132 in _PyEval_EvalFrameDefault Python/bytecodes.c:1398 msgspec#5 0x7f07a0a15f73 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92 msgspec#6 0x7f07a0a15f73 in method_vectorcall Objects/classobject.c:69 msgspec#7 0x7f07a098d82f in _PyEval_EvalFrameDefault Python/bytecodes.c:3263 msgspec#8 0x7f07a0a14e6c in _PyFunction_Vectorcall Objects/call.c:419 msgspec#9 0x7f07a0a14e6c in _PyObject_FastCallDictTstate Objects/call.c:144 msgspec#10 0x7f07a0a14e6c in _PyObject_Call_Prepend Objects/call.c:508 msgspec#11 0x7f07a0ac3381 in slot_tp_call Objects/typeobject.c:8791 msgspec#12 0x7f07a0a14206 in _PyObject_MakeTpCall Objects/call.c:240 msgspec#13 0x7f07a098d20a in _PyEval_EvalFrameDefault Python/bytecodes.c:2715 msgspec#14 0x7f07a0a14e6c in _PyFunction_Vectorcall Objects/call.c:419 msgspec#15 0x7f07a0a14e6c in _PyObject_FastCallDictTstate Objects/call.c:144 msgspec#16 0x7f07a0a14e6c in _PyObject_Call_Prepend Objects/call.c:508 msgspec#17 0x7f07a0ac3381 in slot_tp_call Objects/typeobject.c:8791 msgspec#18 0x7f07a0a1550e in PyObject_Call (/opt/hostedtoolcache/Python/3.12.15/x64/lib/libpython3.12.so.1.0+0x21550e) (BuildId: d5ce433e66c26342bfaf6b82ab1a95541a8ebade) msgspec#19 0x7f07a098d82f in _PyEval_EvalFrameDefault Python/bytecodes.c:3263 msgspec#20 0x7f07a0a14e6c in _PyFunction_Vectorcall Objects/call.c:419 msgspec#21 0x7f07a0a14e6c in _PyObject_FastCallDictTstate Objects/call.c:144 msgspec#22 0x7f07a0a14e6c in _PyObject_Call_Prepend Objects/call.c:508 msgspec#23 0x7f07a0ac3381 in slot_tp_call Objects/typeobject.c:8791 msgspec#24 0x7f07a0a14206 in _PyObject_MakeTpCall Objects/call.c:240 msgspec#25 0x7f07a098d20a in _PyEval_EvalFrameDefault Python/bytecodes.c:2715 msgspec#26 0x7f07a0a14e6c in _PyFunction_Vectorcall Objects/call.c:419 msgspec#27 0x7f07a0a14e6c in _PyObject_FastCallDictTstate Objects/call.c:144 msgspec#28 0x7f07a0a14e6c in _PyObject_Call_Prepend Objects/call.c:508 msgspec#29 0x7f07a0ac3381 in slot_tp_call Objects/typeobject.c:8791 msgspec#30 0x7f07a0a14206 in _PyObject_MakeTpCall Objects/call.c:240 msgspec#31 0x7f07a098d20a in _PyEval_EvalFrameDefault Python/bytecodes.c:2715 msgspec#32 0x7f07a0a14e6c in _PyFunction_Vectorcall Objects/call.c:419 msgspec#33 0x7f07a0a14e6c in _PyObject_FastCallDictTstate Objects/call.c:144 msgspec#34 0x7f07a0a14e6c in _PyObject_Call_Prepend Objects/call.c:508 msgspec#35 0x7f07a0ac3381 in slot_tp_call Objects/typeobject.c:8791 msgspec#36 0x7f07a0a14206 in _PyObject_MakeTpCall Objects/call.c:240 msgspec#37 0x7f07a098d20a in _PyEval_EvalFrameDefault Python/bytecodes.c:2715 msgspec#38 0x7f07a0ae8f35 in PyEval_EvalCode Python/ceval.c:580 msgspec#39 0x7f07a0ae7a3c in builtin_exec_impl Python/bltinmodule.c:1093 msgspec#40 0x7f07a0ae7a3c in builtin_exec Python/clinic/bltinmodule.c.h:543 msgspec#41 0x7f07a0a345a2 in cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:438 msgspec#42 0x7f07a0a14532 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92 msgspec#43 0x7f07a0a14532 in PyObject_Vectorcall Objects/call.c:325 msgspec#44 0x7f07a098d20a in _PyEval_EvalFrameDefault Python/bytecodes.c:2715 msgspec#45 0x7f07a0ae8f35 in PyEval_EvalCode Python/ceval.c:580 msgspec#46 0x7f07a0b09f76 in run_eval_code_obj Python/pythonrun.c:1757 msgspec#47 0x7f07a0b09aaa in run_mod Python/pythonrun.c:1778 msgspec#48 0x7f07a0b099f1 in pyrun_file Python/pythonrun.c:1674 msgspec#49 0x7f07a0b09852 in _PyRun_SimpleFileObject Python/pythonrun.c:459 msgspec#50 0x7f07a0b09667 in _PyRun_AnyFileObject Python/pythonrun.c:78 msgspec#51 0x7f07a0b12943 in pymain_run_file_obj Modules/main.c:361 msgspec#52 0x7f07a0b12943 in pymain_run_file Modules/main.c:380 msgspec#53 0x7f07a0b12943 in pymain_run_python Modules/main.c:634 msgspec#54 0x7f07a0b12943 in Py_RunMain Modules/main.c:714 msgspec#55 0x7f07a0b1243a in Py_BytesMain Modules/main.c:768 msgspec#56 0x7f07a042a1c9 (/lib/x86_64-linux-gnu/libc.so.6+0x2a1c9) (BuildId: a4a7992a8e66555c8141ab2a08a8465ff6e0ea65) msgspec#57 0x7f07a042a28a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2a28a) (BuildId: a4a7992a8e66555c8141ab2a08a8465ff6e0ea65) msgspec#58 0x558976af7094 in _start (/opt/hostedtoolcache/Python/3.12.15/x64/bin/python3.12+0x1094) (BuildId: d664fd041bd9df9902c326eb80717fa04325aaaa) 0x50e000e8e350 is located 16 bytes inside of 152-byte region [0x50e000e8e340,0x50e000e8e3d8) allocated by thread T0 here: #0 0x7f07a10fd9c7 in malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69 msgspec#1 0x7f07a0a7c2fc in gc_alloc Modules/gcmodule.c:2330 msgspec#2 0x7f07a0a7c2fc in _PyObject_GC_NewVar Modules/gcmodule.c:2364 SUMMARY: AddressSanitizer: bad-free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:52 in free ==2637==ABORTING ``` See https://github.com/msgspec/msgspec/actions/runs/38243375356/job/114791328508 Now it is properly cleaned.
1 parent dac6320 commit 4f7363d

3 files changed

Lines changed: 18 additions & 4 deletions

File tree

‎docs/changelog.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,7 @@
55
- Fix the error state left set when looking up `ClassVar` on a module-level
66
`typing` name that is not the real `typing` module. On CPython builds with
77
assertions enabled this terminated the interpreter. The lookup error now
8-
propagates as intended ({issue}`1221`).
9-
8+
propagates as intended ({issue}`1221`)
109
- Fix a `NameError` when resolving a generic `TypedDict` defined with PEP 695
1110
type parameter syntax (`class Foo[T](TypedDict)`) under
1211
`from __future__ import annotations` on Python 3.12/3.13. Also prevent a
@@ -16,6 +15,8 @@
1615
to `READONLY` and C level getters ({pr}`1244`)
1716
- Support `TypeForm[]` type instead of `Type[]` in decoders,
1817
requires `mypy>=2.1` ({pr}`1245`)
18+
- Fix a possible crash when `Meta(pattern='...')`
19+
was used in loops, ({pr}`1256`)
1920

2021
## Version 0.22.0 (2026-09-29)
2122

‎src/msgspec/_core.c‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3433,7 +3433,7 @@ TypeNode_get_traverse_ranges(
34333433
MS_TYPE_INTENUM | MS_TYPE_INTLITERAL |
34343434
MS_TYPE_ENUM | MS_TYPE_STRLITERAL |
34353435
MS_TYPE_TYPEDDICT | MS_TYPE_DATACLASS |
3436-
MS_TYPE_NAMEDTUPLE
3436+
MS_TYPE_NAMEDTUPLE | MS_CONSTR_STR_REGEX
34373437
)
34383438
);
34393439
/* Number of typenode details */

‎tests/unit/test_constraints.py‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
11
import datetime
2+
import gc
23
import math
34
import re
45
import sys
5-
from typing import Annotated
6+
from typing import Annotated, Union
67

78
import pytest
89

@@ -231,6 +232,18 @@ def test_mixing_numeric_and_nonnumeric_constraints_errors(self):
231232
with pytest.raises(ValueError, match="Cannot mix numeric constraints"):
232233
Meta(gt=0, pattern="foo")
233234

235+
def test_pattern_constraint_leak(self):
236+
# See https://github.com/msgspec/msgspec/pull/1256 for the context.
237+
for _ in range(5):
238+
d = msgspec.json.Decoder(
239+
Union[Annotated[str, Meta(pattern="^a")], list[int]],
240+
)
241+
d.decode(b'"abc"')
242+
d.decode(b"[1,2]")
243+
# Must not crash with the sanitizers:
244+
del d
245+
gc.collect()
246+
234247

235248
class TestInvalidConstraintAnnotations:
236249
"""Constraint validity is applied in two places:

0 commit comments

Comments
 (0)