Skip to content

Custom combinations of sampling schemes #411

@quaquel

Description

@quaquel

At the moment, the workbench by default samples levers and uncertainties, resulting in policies and scenarios. Next, these are combined in a full factorial manner so that each policy is evaluated over all scenarios. A single sampling scheme is used for levers, and a separate one for the uncertainties.

However, sometimes users want to create more complicated experimental designs. For example, one might want to sample several uncertainties in a full factorial manner, while sampling the remainder using a Latin hypercube. At the moment, this is not well supported by the workbench directly.

One possible direction is to draw a sharper distinction between BaseEvaluator.perform_experiments and BaseEvaluator.evaluate_experiments. At the moment, the former handles experiment generation, while the latter implements the details of running experiments depending on the kind of evaluator (e.g., ipyparallel, mpi, multiprocessing).

It might be an idea to turn evaluate_experiments into something that just does that: given some collection of experiments, it just runs them and that's it. If we then expand the sampling side with some helper functions for combining samples, it becomes possible to generate any complicated experimental design that you like.

builds on #47

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions