Repository navigation
Stack Management Issue(s?) #5987
Description
🐛 Bug
I am using pyodide to run python code in a web worker.
The worker is scoped to the user session, so I'm invoking pyodide.runPython multiple times on the same pyodide instance, although the globals PyProxy object is re-created anew at every execution, as we don't want names from the previous code execution to be available in subsequent executions.
There seems to be an issue with regards to the stack management that lead to a stack overflow in pyodide in different scenarios.
I spent some time in the debugger, but I wasn't familiar enough with the pyodide codebase to figure out exactly the problem lies.
(Might be related to #5959).
To Reproduce
Let me provide you with some minimal code to reproduce a setup similar to our code-base and some exemplar python programs that trigger the issue:
In this repo you will find 3 files: the HTML page (index.html), its associated JS code (index.js) and the pyodide web worker code (pyodideWorker.js).
Simply serve the directory using a web server and load the html file (e.g., python3 -m http.server 8000).
Once pyodide (0.29.0) has loaded, you may run python code written in the input text area (see the console of the browser to see when pyodide is ready).
Program 1
Also included in the aforementioned repository as example1.py:
import sys
n = 3_000
sys.setrecursionlimit(n + 10)
def f(x):
if x == 0:
print('Base case')
raise ValueError('Boom!')
else:
f(x - 1)
try:
f(n)
except ValueError as e:
print(e)
print('Done!')When running the program:
- The recursion successfully reaches the base case
- 'Base case' is printed, and the ValueError is raised
- The exception is caught by the try-except and it is printed
- A stack overflow exception is thrown
('RangeError: Maximum call stack size exceeded' / 'too much recursion')
and the very last statement (theprint("Done!")) is never executed
Example 2
Also included in the aforementioned repository as example2.py:
import sys
n = 3_000
sys.setrecursionlimit(n + 10)
def f(x):
if x == 0:
print('Base case')
raise ValueError('Boom!')
else:
f(x - 1)
f(n)(Same code as Example 1, without the top-level try-except)
- Execution 1
- Recursion successfully reaches the base case
- 'Base case' is printed, and the ValueError is raised
- The exception let uncaught, the program crashes (as expected), but pyodide is still alive
- Execution 2 (run the same code again without refreshing the page)
- Recursion successfully reaches the base case
- 'Base case' is printed
- A stack overflow exception is thrown
('RangeError: Maximum call stack size exceeded' / 'too much recursion')
rather than theValueErrorone that follows theprint('Base case')statement
Note: On Firefox, the second execution may take significantly more time than the first one to complete.
Expected behavior
- For program 1, the program should not crash
- For program 2, the second execution should work exactly like the first one
Environment
- Pyodide Version: 0.29.0, 0.28.3
- Browser Version: Chrome 139, Firefox 144
Activity
hoodmane commentedon Nov 6, 2025
If you don't want to get range errors, don't increase the recursion limit.
bvlj commentedon Nov 6, 2025
Hi,
Thank you for the quick reply!
Could you please elaborate on that? I'd like to have a better understanding of this situation and the constraints at play here.
I know that there is a problem with regards to the JavaScript stack size being limited (and that limit varying between devices and browsers), but I am not 100% convinced that the recursion limit is the (only) problem here.
In both examples, the python interpreter is able to reach the deepest recursion level.
Moreover, in the first example it also seems able to unwind[1] the recursion and execute the print statement inside the expect, and only then it goes out of stack.
The second example makes me question that claim more because the program is able to execute as the normal/native python interpreter would, but only the first time. A second re-execution immediately goes out of stack.
My feeling after observing these behaviors is that something is being "leaked" / not being properly cleaned up.
It might be that the high recursion limit is just exposing an issue by forcing the interpreter into a state that was not accounted for during its development.
That state may or may not be something you actually want to account for, but nevertheless I'd like to understand what is going on so that I can properly plan around the limitations if there's nothing that we can be do about it.
Thanks for your time!
[1] Unless the unwinding part is somehow skipped when an error is raised
hoodmane commentedon Nov 6, 2025
The interpreter needs a bit of stack headroom to raise an error. On normal architectures it calculates the system stack space and reserves some head room for error handling. Then even if you set the recursion limit very large, it checks if it's passing into that reserved space and raises a stack overflow error. If another error is raised while handling the stack overflow error it will abort rather than propagating the second error since it doesn't have enough stack for it.
None of this works correctly on Emscripten because the call stack isn't in addressable memory so we can't perform stack checks. Thus, this reserved error handling headroom is not available and raising the exception blows the stack.
bvlj commentedon Nov 7, 2025
Thank you very much for the reply!
That indeed explains the first example then: when the
ValueError("Boom")exception is raised, it's going to fill the head room space reserved for error handling (since we should have filled the stack space at this point). Then, the "body" ofexpect ValueError as eis going to be executed, but here it's theprint(e)that triggers the stack overflow error, and since we were already handling another error, we don't have any more space in the stack and pyodide dies.I'm still not 100% sure how this fits the second example where it takes 2 executions to raise the fatal exception though.
Is it that the first execution leaves pyodide in the "handling the error" state and then the next execution is going to act just like the
print(e)of the first example (raising a stack overflow error that cannot be handled because we're out of space)?If that's the case, shouldn't it be possible / desirable for pyodide to (automatically) leave this error-handling state "in between" multiple
runPythonexecutions?