Skip to content

BUG: Drastic memory usage increase in recent builds #22233

@larsoner

Description

@larsoner

Describe the issue:

Recently in MNE-Python our CIs started dying due to memory overconsumption on both

On my local Linux machine doing the following as a test with git bisect --first-parent:

$ mprof run pytest -m "not ultraslowtest" mne/tests/ --tb=short --cov=mne --cov-report=xml --cov-report=html -vv && mprof plot

takes ~12GB of memory usage on main and also 45357de:

bad_45357de4cf83340bd05d279673f68a7fd42b2308

but only a much more reasonable ~2.5GB of max memory usage on db550d5:

good_db550d57ee72e097b50d776cce108c64c80c181b

So I think this is due to #21995.

MNE does use SciPy, Numba, scikit-learn, dipy, etc. that could be interacting with NumPy in some bad way, too. I could at least rule out numba, dipy, and scikit-learn by uninstalling them and seeing the same overconsumption, though (some tests will just be skipped or our library with just use slower code paths this way).

I'm happy to work on isolating this further if it's not clear why this might be happening just from having it pinned down to the PR/commit.

Reproduce the code example:

<not easily reproduced / not yet isolated>

Error message:

No response

NumPy/Python version information:

See commit numbers above

Context for the issue:

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions