-
-
Notifications
You must be signed in to change notification settings - Fork 12.2k
BUG: Drastic memory usage increase in recent builds #22233
Description
Describe the issue:
Recently in MNE-Python our CIs started dying due to memory overconsumption on both
- Linux: https://github.com/mne-tools/mne-python/runs/8253676336?check_suite_focus=true
- Windows: https://dev.azure.com/mne-tools/mne-python/_build/results?buildId=21722&view=logs&j=305851a9-a7bb-55db-0042-7e2b6f48aa1c&t=4d37777d-f36a-53aa-9217-6386d15dddcd&l=2570
On my local Linux machine doing the following as a test with git bisect --first-parent:
$ mprof run pytest -m "not ultraslowtest" mne/tests/ --tb=short --cov=mne --cov-report=xml --cov-report=html -vv && mprof plot
takes ~12GB of memory usage on main and also 45357de:
but only a much more reasonable ~2.5GB of max memory usage on db550d5:
So I think this is due to #21995.
MNE does use SciPy, Numba, scikit-learn, dipy, etc. that could be interacting with NumPy in some bad way, too. I could at least rule out numba, dipy, and scikit-learn by uninstalling them and seeing the same overconsumption, though (some tests will just be skipped or our library with just use slower code paths this way).
I'm happy to work on isolating this further if it's not clear why this might be happening just from having it pinned down to the PR/commit.
Reproduce the code example:
<not easily reproduced / not yet isolated>Error message:
No response
NumPy/Python version information:
See commit numbers above
Context for the issue:
No response

