Output space exploration by quaquel · Pull Request #170 · quaquel/EMAworkbench

quaquel · 2022-09-14T11:10:24Z

adds output space exploration to the workbench.

unit tests and documentation are in place.
I might still want to update the example notebook
unit tests adds pytest-mock as a dependency in ci, and thus also for Replace setup.py with pyproject.toml and implement optional dependencies #166

Closes #154.

initial commit for output space exploration * new module for output space exploration build on platypus-opt * new example with the lake problem * modifications to optimization.py to make new module work with existing optimization functionality * updates to TODO outlining further possible cleanup of optimization.py

seems unnecessary because archive is already non dominated for e-NSGAII and is problematic for OutputSpaceExploration

basic version now works properly. HitBox as per Plos paper, but also behaves as epsilon archive. per epsilon box, the most central solution is kept. Tournament selection is used based on novelty. Novelty is 1/number of seen solutions in hitbox

added link to Cherel paper and outlined the rest of the algorithm.

coveralls · 2022-09-14T11:14:05Z

Coverage increased (+0.7%) to 80.731% when pulling 986cdec on output_space_exploration into 0c61dcc on master.

EwoutH · 2022-09-14T12:38:36Z

Looks awesome! I will review it from a code-quality and user perspective tomorrow.

EwoutH · 2022-09-19T11:10:08Z

I started on a review, in general, it looks quite good!

Reviewing the docs on the other hand is quite a challenge without them begin built. Could you check if it's easy to enable doc building (see #171) before I review them?

quaquel · 2022-09-19T12:52:59Z

I have enabled the building of docs for pull requests. I guess we need one more commit here to trigger it.

EwoutH · 2022-09-19T13:25:45Z

If you can find the time, cleaning up you commits would be a great way to also trigger the CI! :)

Otherwise I will do some ci-trigger-magic when I review it (probably tomorrow).

* new module for output space exploration build on platypus-opt * new example with the lake problem * modifications to optimization.py to make new module work with existing optimization functionality * updates to TODO outlining further possible cleanup of optimization.py * unit tests for all new code

quaquel · 2022-09-21T09:39:17Z

That review app looks nice and the suggestions will indeed make the notebook a lot better. I'll try to make the suggested changes over the coming days.

quaquel · 2022-09-22T07:33:02Z

I have updated the notebook broadly in line with your suggestions. Note that I don't intend for these notebooks to be a tutorial on MOEAs so I have kept the additional text short but with links to relevant literature in case you want to know more.

I also added a small check to ensure the number of items in grid_spec matches the number of objectives (analogous to the problem with epsilon values in e-NSGA2). This in turn necessitated a small update to the test code.

initial commit for output space exploration * new module for output space exploration build on platypus-opt * new example with the lake problem * modifications to optimization.py to make new module work with existing optimization functionality * updates to TODO outlining further possible cleanup of optimization.py

…/EMAworkbench into output_space_exploration

quaquel · 2022-09-22T11:52:44Z

I cleaned the commit history again, so if you agree with the notebook update, this is ready to be merged.

EwoutH

Code quality and structure looks good, I didn't encounter strange things.

Docs are also coming along nicely, I added a few comments on things I found unclear or ambiguous.

If you take a look at those, could you let me merge afterwards? I will squash it into a nice tidy commit.

ema_workbench/em_framework/optimization.py

ema_workbench/examples/lake_model_outputspace_exploration.py

ema_workbench/em_framework/evaluators.py

ema_workbench/em_framework/outputspace_exploration.py

quaquel · 2022-09-23T14:09:05Z

I think this is now ready to be merged

EwoutH · 2022-09-23T14:24:06Z

Awesome, I agree!

Can I merge #178? Then I will throw in some CI, rebase and check if #170 passes, and then merge it.

quaquel · 2022-09-23T16:16:38Z

go ahead

for more information, see https://pre-commit.ci

EwoutH · 2022-09-23T21:32:09Z

@quaquel It's in, congratulations on an amazing new feature!

quaquel added 20 commits July 8, 2022 08:46

Merge branch 'master' into output_space_exploration

4e3b4c4

Create outputspace_exploration_lakeproblem.ipynb

c33383b

remove nondominated check in to_dataframe

4899034

seems unnecessary because archive is already non dominated for e-NSGAII and is problematic for OutputSpaceExploration

Update output_space_exploration.py

d36a25e

basic version now works properly. HitBox as per Plos paper, but also behaves as epsilon archive. per epsilon box, the most central solution is kept. Tournament selection is used based on novelty. Novelty is 1/number of seen solutions in hitbox

code cleaning and basic documentation

ec92c81

comparison of DE and SBX added in notebook

85a68ea

adds autoadaptive operator selection

4ac8aad

rename output_space_exploration

52c8022

Update lake_model_outputspace_exploration.py

a3351c9

add documentation file for outputspace exploration

4e104cf

part of rename

a1de8b8

placeholder file for unit tests

18727f4

more detailed documentation of algorithm

907fa8a

added link to Cherel paper and outlined the rest of the algorithm.

Merge branch 'master' into output_space_exploration

8e18c80

unit test added for Novelty and HitBox

b7a5f2d

adds pytest-mock as dependency for testing

1c83c35

typo in ci.yml

69dca27

unititest for OutputSpaceExplorationAlgorithm

9875bc5

final test for inits of user facing algorithms

55158d1

quaquel requested a review from EwoutH September 14, 2022 11:10

quaquel added 2 commits September 14, 2022 14:47

code cleaning

26dbdea

examples cleaned and notebook annotated

8ff2247

EwoutH mentioned this pull request Sep 19, 2022

Replace setup.py with pyproject.toml and implement optional dependencies #166

Merged

9 tasks

quaquel added 2 commits September 22, 2022 09:29

updated notebook, algorithm and test code

f76981f

black related changes

26732f0

quaquel linked an issue Sep 22, 2022 that may be closed by this pull request

Output Space Exploration using novelty search like optimization #154

Closed

quaquel added the enhancement label Sep 22, 2022

quaquel marked this pull request as draft September 22, 2022 10:14

quaquel marked this pull request as ready for review September 22, 2022 10:15

quaquel added 2 commits September 22, 2022 13:51

Merge branch 'output_space_exploration' of https://github.com/quaquel…

0421791

…/EMAworkbench into output_space_exploration

EwoutH requested changes Sep 23, 2022

View reviewed changes

EwoutH mentioned this pull request Sep 23, 2022

use masked arrays for storing outcomes #176

Merged

resolving code review comments

4e8f497

EwoutH added this to the 2.3.0 milestone Sep 23, 2022

quaquel mentioned this pull request Sep 23, 2022

Formatting: Format with Black, increase max line length to 100, combine multi-line blocks #178

Merged

quaquel added 2 commits September 23, 2022 16:08

additional documentation

5e94d17

layouting of gridspec

b9aea84

EwoutH approved these changes Sep 23, 2022

View reviewed changes

EwoutH and others added 2 commits September 23, 2022 23:13

Merge branch 'master' into output_space_exploration

4632aab

[pre-commit.ci] auto fixes from pre-commit.com hooks

52e5039

for more information, see https://pre-commit.ci

EwoutH added feature and removed enhancement labels Sep 23, 2022

Delete .coverage.TUD501192.8913.995292

986cdec

EwoutH merged commit 6c1b55c into master Sep 23, 2022

quaquel deleted the output_space_exploration branch October 28, 2022 13:37

Conversation

quaquel commented Sep 14, 2022 • edited by EwoutH Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coveralls commented Sep 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

EwoutH commented Sep 14, 2022

Uh oh!

EwoutH commented Sep 19, 2022

Uh oh!

quaquel commented Sep 19, 2022

Uh oh!

EwoutH commented Sep 19, 2022

Uh oh!

quaquel commented Sep 21, 2022

Uh oh!

quaquel commented Sep 22, 2022

Uh oh!

quaquel commented Sep 22, 2022

Uh oh!

EwoutH left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

quaquel commented Sep 23, 2022

Uh oh!

EwoutH commented Sep 23, 2022

Uh oh!

quaquel commented Sep 23, 2022

Uh oh!

EwoutH commented Sep 23, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

quaquel commented Sep 14, 2022 •

edited by EwoutH

Loading

coveralls commented Sep 14, 2022 •

edited

Loading