Skip to content

Minor bug fixes to mpi support #390

Merged
quaquel merged 31 commits intomasterfrom
mpi_testing
Aug 27, 2025
Merged

Minor bug fixes to mpi support #390
quaquel merged 31 commits intomasterfrom
mpi_testing

Conversation

@quaquel
Copy link
Copy Markdown
Owner

@quaquel quaquel commented Aug 14, 2025

This PR contains various small changes to the MPI related code. It makes sure the tutorial and example are up to date with DelftBlue, and it fixes an incompatibility with mpi4py 4.x. It also changes around some of the log levels in the mpi evaluator to make it easier to diagnose problems.

EwoutH and others added 23 commits December 20, 2023 13:53
This is a release branch
* bugfix to finalizer

fixes a bug in the finalizer in case of working directory models

* Update futures_util.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fixes a mistake in the MPI evaluator where the pool is created twice (probably some code merging issue in the past)
* cleans up the mpi example and associated slurm file.
* additional logging messages
updates:
- [github.com/psf/black: 24.3.0 → 24.4.0](psf/black@24.3.0...24.4.0)
- [github.com/pre-commit/pre-commit-hooks: v4.5.0 → v4.6.0](pre-commit/pre-commit-hooks@v4.5.0...v4.6.0)

[pre-commit.ci] pre-commit autoupdate (#345)

* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/psf/black: 23.12.0 → 24.2.0](psf/black@23.12.0...24.2.0)
- [github.com/asottile/pyupgrade: v3.15.0 → v3.15.1](asottile/pyupgrade@v3.15.0...v3.15.1)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

[pre-commit.ci] pre-commit autoupdate (#347)

updates:
- [github.com/psf/black: 24.2.0 → 24.3.0](psf/black@24.2.0...24.3.0)

[pre-commit.ci] pre-commit autoupdate (#352)

updates:
- [github.com/asottile/pyupgrade: v3.15.1 → v3.15.2](asottile/pyupgrade@v3.15.1...v3.15.2)

Co-Authored-By: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
np.NaN will be removed in NumPy 2.0.
* Clear up DelftBlue tutorial
A few minor clarifications to the DelftBlue part of the MPI tutorial.

* MPI tutorial: pip upgrade existing EMAworkbench installation

This makes sure users don't accidently keep using an older version if they installed it previously.
* fixes a platypus related warning

* pandas related plotting fix

* Update optimization.py

* pandas related fix to ensure index col is handled properly

* platypus fix for warning

* Update optimization.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix for linter error

* Update b_and_w_plotting.py

matplotlib related

* matplotlib related fix

* work arround for python3.9 vs 3.10 and knock on for matplotlib




---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ewout ter Hoeven <[email protected]>
* version bump and backport of pre-commit

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@review-notebook-app
Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

Copy link
Copy Markdown
Collaborator

@EwoutH EwoutH left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don’t have (access to) a system to test this on currently, but aside from two small remarks it looks good.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is loading / installing Python modules now handles? I don’t find that totally clear yet.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't find a clear explanation in the docs. I just installed stuff via the command line while being logged in and in the scratch directory.

@quaquel
Copy link
Copy Markdown
Owner Author

quaquel commented Aug 26, 2025

I have confirmed that the latest commit works as intended on Delft Blue. I am still figuring out the tutorial side of things and, in particular, how to correctly install the workbench and other dependencies.

For testing, I just scp'ed the workbench to scratch and modified the example to add this folder to the top of sys.path.

@quaquel quaquel merged commit d2fdab5 into master Aug 27, 2025
14 of 18 checks passed
@quaquel quaquel deleted the mpi_testing branch August 27, 2025 11:15
@quaquel quaquel restored the mpi_testing branch August 27, 2025 11:16
@quaquel quaquel deleted the mpi_testing branch November 20, 2025 10:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants