Repository navigation
BUG: Segfault/stack-overflow from initializing numpy.random._mt19937.MT19937 with recursive list #28822
Description
Describe the issue:
Initializing numpy.random._mt19937.MT19937 with a recursive list in an ASAN Python build leads to a segfault, which ASAN says comes from a stack-overflow. Confirmed to also affect PCG64, Philox etc.
Reproduce the code example:
import numpy.random._mt19937
t = []
t.append(t)
numpy.random._mt19937.MT19937(t)Error message:
Thread 1 "python" received signal SIGSEGV, Segmentation fault.
0x00007ffff75febeb in __interceptor_memset (dst=0x7fffb5dbd9a8, v=205, size=48) at ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:799
799 ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc: No such file or directory.
(gdb) bt
#0 0x00007ffff75febeb in __interceptor_memset (dst=0x7fffb5dbd9a8, v=205, size=48)
at ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:799
#1 0x0000555555ade1bf in memset (__len=<optimized out>, __ch=205, __dest=<optimized out>)
at /usr/include/x86_64-linux-gnu/bits/string_fortified.h:59
#2 mi_debug_fill (size=<optimized out>, c=205, block=0x7fffb5dbd980, page=0x7fffb400d0d8) at Objects/mimalloc/alloc.c:33
#3 _mi_page_malloc (heap=heap@entry=0x5555566118a0 <_PyRuntime+333024>, page=0x7fffb400d0d8, size=size@entry=88,
zero=zero@entry=false) at Objects/mimalloc/alloc.c:77
#4 0x0000555555ade98f in mi_heap_malloc_small_zero (heap=0x5555566118a0 <_PyRuntime+333024>, size=80, zero=<optimized out>)
at Objects/mimalloc/alloc.c:127
#5 0x0000555555adfb6a in _mi_heap_malloc_zero_ex (huge_alignment=0, zero=false, size=80, heap=0x5555566118a0 <_PyRuntime+333024>)
at Objects/mimalloc/alloc.c:156
#6 _mi_heap_malloc_zero (zero=false, size=80, heap=0x5555566118a0 <_PyRuntime+333024>) at Objects/mimalloc/alloc.c:179
#7 mi_heap_malloc (size=80, heap=0x5555566118a0 <_PyRuntime+333024>) at Objects/mimalloc/alloc.c:183
#8 _PyObject_MiMalloc (ctx=<optimized out>, nbytes=80) at Objects/obmalloc.c:250
#9 0x0000555555ab7404 in _PyMem_DebugRawAlloc (use_calloc=use_calloc@entry=0, ctx=ctx@entry=0x5555565c0758 <_PyRuntime+920>,
nbytes=nbytes@entry=56) at Objects/obmalloc.c:2708
#10 0x0000555555ab746c in _PyMem_DebugRawMalloc (ctx=ctx@entry=0x5555565c0758 <_PyRuntime+920>, nbytes=nbytes@entry=56)
at Objects/obmalloc.c:2741
#11 0x0000555555ab7493 in _PyMem_DebugMalloc (ctx=0x5555565c0758 <_PyRuntime+920>, nbytes=56) at Objects/obmalloc.c:2898
#12 0x0000555555ae5dc0 in PyObject_Malloc (size=<optimized out>) at Objects/obmalloc.c:1316
#13 0x0000555555dd4048 in _PyObject_MallocWithType (size=<optimized out>, tp=0x0) at ./Include/internal/pycore_object_alloc.h:46
#14 gc_alloc (tp=tp@entry=0x555556459e20 <PyList_Type>, basicsize=<optimized out>, presize=0) at Python/gc_free_threading.c:1695
#15 0x0000555555dd41c0 in _PyObject_GC_New (tp=tp@entry=0x555556459e20 <PyList_Type>) at Python/gc_free_threading.c:1716
#16 0x0000555555a1ac71 in PyList_New (size=0) at Objects/listobject.c:279
#17 0x00007fffb1bb1247 in __pyx_pw_5numpy_6random_13bit_generator_3_coerce_to_uint32_array ()
from /home/danzin/venvs/3.13_upstream_venv/lib/python3.13t/site-packages/numpy/random/bit_generator.cpython-313t-x86_64-linux-gnu.so
#18 0x00007fffb1bb1351 in __pyx_pw_5numpy_6random_13bit_generator_3_coerce_to_uint32_array ()
from /home/danzin/venvs/3.13_upstream_venv/lib/python3.13t/site-packages/numpy/random/bit_generator.cpython-313t-x86_64-linux-gnu.so
#19 0x00007fffb1bb1351 in __pyx_pw_5numpy_6random_13bit_generator_3_coerce_to_uint32_array ()
from /home/danzin/venvs/3.13_upstream_venv/lib/python3.13t/site-packages/numpy/random/bit_generator.cpython-313t-x86_64-linux-gnu.so
#20 0x00007fffb1bb1351 in __pyx_pw_5numpy_6random_13bit_generator_3_coerce_to_uint32_array ()
--Type <RET> for more, q to quit, c to continue without paging--c
from /home/danzin/venvs/3.13_upstream_venv/lib/python3.13t/site-packages/numpy/random/bit_generator.cpython-313t-x86_64-linux-gnu.so
#21 0x00007fffb1bb1351 in __pyx_pw_5numpy_6random_13bit_generator_3_coerce_to_uint32_array () from /home/danzin/venvs/3.13_upstream_venv/lib/python3.13t/site-packages/numpy/random/bit_generator.cpython-313t-x86_64-linux-gnu.so
#22 0x00007fffb1bb1351 in __pyx_pw_5numpy_6random_13bit_generator_3_coerce_to_uint32_array () from /home/danzin/venvs/3.13_upstream_venv/lib/python3.13t/site-packages/numpy/random/bit_generator.cpython-313t-x86_64-linux-gnu.so
#23 0x00007fffb1bb1351 in __pyx_pw_5numpy_6random_13bit_generator_3_coerce_to_uint32_array () from /home/danzin/venvs/3.13_upstream_venv/lib/python3.13t/site-packages/numpy/random/bit_generator.cpython-313t-x86_64-linux-gnu.so
#24 0x00007fffb1bb1351 in __pyx_pw_5numpy_6random_13bit_generator_3_coerce_to_uint32_array () from /home/danzin/venvs/3.13_upstream_venv/lib/python3.13t/site-packages/numpy/random/bit_generator.cpython-313t-x86_64-linux-gnu.so
#25 0x00007fffb1bb1351 in __pyx_pw_5numpy_6random_13bit_generator_3_coerce_to_uint32_array () from /home/danzin/venvs/3.13_upstream_venv/lib/python3.13t/site-packages/numpy/random/bit_generator.cpython-313t-x86_64-linux-gnu.so
#26 0x00007fffb1bb1351 in __pyx_pw_5numpy_6random_13bit_generator_3_coerce_to_uint32_array () from /home/danzin/venvs/3.13_upstream_venv/lib/python3.13t/site-packages/numpy/random/bit_generator.cpython-313t-x86_64-linux-gnu.so
#27 0x00007fffb1bb1351 in __pyx_pw_5numpy_6random_13bit_generator_3_coerce_to_uint32_array () from /home/danzin/venvs/3.13_upstream_venv/lib/python3.13t/site-packages/numpy/random/bit_generator.cpython-313t-x86_64-linux-gnu.so
[over 2000 repetitions of this line, not sure what comes after]
ASAN error:
AddressSanitizer:DEADLYSIGNAL
=================================================================
==3741772==ERROR: AddressSanitizer: stack-overflow on address 0x7fffff7fefe8 (pc 0x7ffff75febeb bp 0x7fffff7ff860 sp 0x7fffff7feff0 T0)
#0 0x7ffff75febeb in __interceptor_memset ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:799
#1 0x555555ade1be in memset /usr/include/x86_64-linux-gnu/bits/string_fortified.h:59
#2 0x555555ade1be in mi_debug_fill Objects/mimalloc/alloc.c:33
#3 0x555555ade1be in _mi_page_malloc Objects/mimalloc/alloc.c:77
#4 0x555555ade98e in mi_heap_malloc_small_zero Objects/mimalloc/alloc.c:127
#5 0x555555adfb69 in _mi_heap_malloc_zero_ex Objects/mimalloc/alloc.c:156
#6 0x555555adfb69 in _mi_heap_malloc_zero Objects/mimalloc/alloc.c:179
#7 0x555555adfb69 in mi_heap_malloc Objects/mimalloc/alloc.c:183
#8 0x555555adfb69 in _PyObject_MiMalloc Objects/obmalloc.c:250
#9 0x555555ab7403 in _PyMem_DebugRawAlloc Objects/obmalloc.c:2708
#10 0x555555ab746b in _PyMem_DebugRawMalloc Objects/obmalloc.c:2741
#11 0x555555ab7492 in _PyMem_DebugMalloc Objects/obmalloc.c:2898
#12 0x555555ae5dbf in PyObject_Malloc Objects/obmalloc.c:1316
#13 0x555555dd4047 in _PyObject_MallocWithType Include/internal/pycore_object_alloc.h:46
#14 0x555555dd4047 in gc_alloc Python/gc_free_threading.c:1695
#15 0x555555dd41bf in _PyObject_GC_New Python/gc_free_threading.c:1716
#16 0x555555a1ac70 in PyList_New Objects/listobject.c:279
#17 0x7fffb1bb1246 in __pyx_pw_5numpy_6random_13bit_generator_3_coerce_to_uint32_array (/home/danzin/venvs/3.13_upstream_venv/lib/python3.13t/site-packages/numpy/random/bit_generator.cpython-313t-x86_64-linux-gnu.so+0x17246)
#18 0x7fffb1bb1350 in __pyx_pw_5numpy_6random_13bit_generator_3_coerce_to_uint32_array (/home/danzin/venvs/3.13_upstream_venv/lib/python3.13t/site-packages/numpy/random/bit_generator.cpython-313t-x86_64-linux-gnu.so+0x17350)Python and NumPy Versions:
2.2.5
3.13.3+ experimental free-threading build (heads/3.13:d8b90117024, Apr 21 2025, 15:20:00) [GCC 11.4.0]
Runtime Environment:
[{'numpy_version': '2.2.5',
'python': '3.13.3+ experimental free-threading build '
'(heads/3.13:d8b90117024, Apr 21 2025, 15:20:00) [GCC 11.4.0]',
'uname': uname_result(system='Linux', node='LAPTOP-CS6PE5KB', release='5.15.167.4-microsoft-standard-WSL2', version='#1 SMP Tue Nov 5 00:21:55 UTC 2024', machine='x86_64')},
{'simd_extensions': {'baseline': ['SSE', 'SSE2', 'SSE3'],
'found': ['SSSE3',
'SSE41',
'POPCNT',
'SSE42',
'AVX',
'F16C',
'FMA3',
'AVX2'],
'not_found': ['AVX512F',
'AVX512CD',
'AVX512_KNL',
'AVX512_KNM',
'AVX512_SKX',
'AVX512_CLX',
'AVX512_CNL',
'AVX512_ICL']}},
{'architecture': 'Haswell',
'filepath': '/home/danzin/venvs/3.13_upstream_venv/lib/python3.13t/site-packages/numpy.libs/libscipy_openblas64_-6bb31eeb.so',
'internal_api': 'openblas',
'num_threads': 8,
'prefix': 'libscipy_openblas',
'threading_layer': 'pthreads',
'user_api': 'blas',
'version': '0.3.28'}]
Context for the issue:
I have been fuzzing Numpy using fusil by @vstinner. I realize these crashes are unlikely to be triggered in normal usage and therefore might be of low priority.
The fuzzing was done with an ASAN free-threading clang build and not confirmed on a GILfull non-sanitizer GCC build yet.
Activity
devdanzin commentedon Apr 25, 2025
Maybe related to #28784?
seberg commentedon Apr 25, 2025
No, but IMO, please do start to bunch up these clearly miniscule issues. FWIW, while this can be worked around in NumPy but it is probably again a Cython or Python issue.
Not that I would consider it a real issue. It's easy to crash the interpreter with malicious recursions.
devdanzin commentedon Apr 25, 2025
Sure, will do. Thanks for the guidance!