NumPy test failure after upgrading to 3.14.1t on MacOS x86_64

Bug report

Bug description:

Intially reported at numpy/numpy#30370 and https://discuss.python.org/t/python-3-14-1-is-now-available/105163/6

At NumPy there's a test in CI that started failing on MacOS x86_64 after upgrading to 3.14.1t:

actions log: https://github.com/numpy/numpy/actions/runs/19909510040/job/57074363533?pr=30367
failing test: https://github.com/numpy/numpy/blob/4a6bb2d843ba271f09a4115fd2c2c9dc82c78820/numpy/_core/tests/test_dtype.py#L988-L995

def test_tuple_recursion(self):
    d = np.int32
    for i in range(100000):
        d = (d, (1,))
    with pytest.raises(RecursionError):  # <--
        np.dtype(d)

CPython versions tested on:

3.14

Operating systems tested on:

macOS

Happy to help if there are NumPy internals questions that come up.

Doesn't trigger on ARM Macs. Unfortunately that means I can't reproduce it or help bisect it on hardware I can easily use :(

This doesn't look free threading related. It looks like you don't have a 3.14 job in your matrix, just 3.14t.

Is this a bug in Python. It doesn't raise a RecursionError, but it doesn't crash. What's the problem? Does it crash if you raise the iterations?

The recursion check changed between 3.14.0 and 3.14.1:

cpython/Include/internal/pycore_ceval.h

Lines 199 to 203 in ebf955d

    
           static inline int _Py_MakeRecCheck(PyThreadState *tstate)  { 
        
               uintptr_t here_addr = _Py_get_machine_stack_pointer(); 
        
               _PyThreadStateImpl *_tstate = (_PyThreadStateImpl *)tstate; 
        
               return here_addr < _tstate->c_stack_soft_limit; 
        
           }

cpython/Include/internal/pycore_ceval.h

Lines 201 to 212 in 57e0d17

    
           static inline int _Py_MakeRecCheck(PyThreadState *tstate)  { 
        
               uintptr_t here_addr = _Py_get_machine_stack_pointer(); 
        
               _PyThreadStateImpl *_tstate = (_PyThreadStateImpl *)tstate; 
        
               // Overflow if stack pointer is between soft limit and the base of the hardware stack. 
        
               // If it is below the hardware stack base, assume that we have the wrong stack limits, and do nothing. 
        
               // We could have the wrong stack limits because of limited platform support, or user-space threads. 
        
           #if _Py_STACK_GROWS_DOWN 
        
               return here_addr < _tstate->c_stack_soft_limit && here_addr >= _tstate->c_stack_soft_limit - 2 * _PyOS_STACK_MARGIN_BYTES; 
        
           #else 
        
               return here_addr > _tstate->c_stack_soft_limit && here_addr <= _tstate->c_stack_soft_limit + 2 * _PyOS_STACK_MARGIN_BYTES; 
        
           #endif 
        
           }

We can just skip this test in numpy. It's testing observed behavior in Python rather than anything that's being relied on in NumPy. See numpy/numpy@bb76bff for context on why the test was added.

I opened numpy/numpy#30375. IMO you can close this issue.

@colesbury

IMO you can close this issue.

Alright, then let's.

Thanks for investigating @colesbury, @ngoldbaum!

@ngoldbaum

@ngoldbaum do you want to re-open this? I dunno it's worth digging into, but this is now failing on NumPy main even on non x86 MacOS. And it seems very unlikely to be a NumPy issue as such.

See also numpy/numpy#31733 and numpy/numpy#31364 (comment)

Sorry, I've been meaning to spend some time finding the root cause of this. Let's go ahead and re-open the issue.

Do you need help with the investigation?

Hi all, I think I understand what's happening now.

CPython is working as intended after the 3.14.1 stack-limit changes: with a genuinely large stack, deep C recursion now correctly completes instead of raising at an arbitrary depth. That invalidated NumPy's "must raise RecursionError at depth N" tests.
The segfaults were a NumPy bug that the same change exposed: arraydescr_dealloc recurses through nested subarray dtypes, so any deep dtype that construction is now allowed to finish became a possible C stack overflow. I have a fix for this that replaces the Py_DECREF(base) (which may trigger a deep C recursion) with a check that detaches the subarray chain to avoid recursion.

However, I think there are maybe some issues with the stack overflow detection in CPython. If a C extension like NumPy indicates via the C API that it's about to do some recursive operations, I think that violates assumptions in CPython's stack overflow protection code. If (for some reason) the recorded bounds overstate the real stack, recursion crashes with no diagnostics. See the reproducer below: claim 64 MiB on the (real 16 MiB / 8 MiB) main thread via ctypes, run nested json.loads — SIGSEGV on both 3.14.6 and 3.14.6t, with a recursion check passing at every single level.

stack_wrongbounds.py — overstated bounds ⇒ SIGSEGV despite a check at every level

import ctypes, json

libc = ctypes.CDLL(None)
libc.pthread_self.restype = ctypes.c_void_p
libc.pthread_get_stackaddr_np.restype = ctypes.c_void_p
libc.pthread_get_stackaddr_np.argtypes = [ctypes.c_void_p]

api = ctypes.pythonapi
api.PyThreadState_Get.restype = ctypes.c_void_p
api.PyUnstable_ThreadState_SetStackProtection.restype = ctypes.c_int
api.PyUnstable_ThreadState_SetStackProtection.argtypes = [
    ctypes.c_void_p, ctypes.c_void_p, ctypes.c_size_t]

top = libc.pthread_get_stackaddr_np(libc.pthread_self())
CLAIMED = 64 * 2**20  # real main-thread stack is 8-16 MiB
assert api.PyUnstable_ThreadState_SetStackProtection(
    api.PyThreadState_Get(), top - CLAIMED, CLAIMED) == 0
print("bounds set: claiming 64 MiB", flush=True)
try:
    json.loads("[" * 1_000_000 + "]" * 1_000_000)
    print("completed", flush=True)
except RecursionError as e:
    print("RecursionError:", e, flush=True)
# observed: neither line prints -- SIGSEGV (exit 139) on 3.14.6 and 3.14.6t

I think the issue is that Numpy dtype object is not a GC type so the trashcan mechanism and stack overflow protection doesn't kicks in. The object must be GC type so that CPython can delay the deallocation and avoid C stack overflow.

Closing as this is fixed in numpy.

Navigation Menu

Repository navigation

NumPy test failure after upgrading to 3.14.1t on MacOS x86_64 #142253

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	static inline int _Py_MakeRecCheck(PyThreadState *tstate) {
	uintptr_t here_addr = _Py_get_machine_stack_pointer();
	_PyThreadStateImpl _tstate = (_PyThreadStateImpl )tstate;
	return here_addr < _tstate->c_stack_soft_limit;
	}

NumPy test failure after upgrading to 3.14.1t on MacOS x86_64 #142253

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Activity

ngoldbaum commented on Dec 4, 2025

ngoldbaum commented on Dec 4, 2025

colesbury commented on Dec 4, 2025

ngoldbaum commented on Dec 4, 2025

ngoldbaum commented on Dec 4, 2025

jorenham commented on Dec 4, 2025

seberg commented on Jun 24, 2026

ngoldbaum commented on Jul 3, 2026

encukou commented on Jul 28, 2026

ngoldbaum commented on Jul 28, 2026

kumaraditya303 commented on Jul 29, 2026

kumaraditya303 commented on Jul 31, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions