Reorganization of evaluator code and renaming of modules#320
Reorganization of evaluator code and renaming of modules#320
Conversation
in preparation for future updates to the mpi code, all evaluator code has been reorganized. All evaluators go into their respective module, and code sthared among 2 or more goes into a new util package.
All relevant modules also have been renamed to future_{style of parallelization}
|
I will try to give it a look monday! |
|
I have a few questions:
|
|
I am, in parallel, working on expanding on the MPI code. I noticed that I could reuse parts of the multiprocessing code in this. However, it also gave rise to circular imports. To resolve these, at a minimum, I needed to move all reused parts into a separate util module. Moreover, for convenience, I prefer to have related functionality in the same module and keep the modules rather short. Therefore, I moved all evaluators to their respective modules (i.e., multiprocessing, mpi, ipyparallel). The net result is a code base that is more cleanly separated, easier to test in isolation, and prepares for further updates to mpi and multiprocressing. The prefix was ema, but that is not informative. I went with futures to indicate it has to do with parallization, but I am open to other suggestions. The reorganization resulted in import problems that caused the docs to fail. I fixed it quickly by making ipyparallel required. I looked at other solutions, but because I am subclassing stuff from ipyparallel, I could not use the method level imports as used with mpi4py. |
|
Thanks for the explanation. Why do we need a prefix at all? Edit: These are breaking changes right? As in it break current imports? |
|
If you don't, you can get namespace classes (e.g., No these are not breaking changes, because all relevant imports were available at the ema_workbench level already. It is only an internal reorganization. |
In preparation for future updates to the mpi code, all evaluator code has been reorganized. All evaluators go into their respective modules, and code shared among 2 or more goes into a new util package.
tests and docs have been updated as well to reflect these changes.