Skip to content

Copied config file may diverge from results #953

@lindnemi

Description

@lindnemi

Checklist

  • I am using the current master branch or the latest release. Please indicate.
  • I am running on an up-to-date pypsa-eur environment. Update via conda env update -f envs/environment.yaml.

Describe the Bug

I solved a model, changed the config, resolved the model. Then I look at results/.../config.yaml and find that it's the file from the first run, not from the second. Thus, the stored results have diverged from the stored config. The reason is that snakemake does not execute this rule again:

rule copy_config:
    params:
        RDIR=RDIR,
    output:
        RESULTS + "config.yaml",
    threads: 1
    resources:
        mem_mb=1000,
    conda:
        "../envs/environment.yaml"
    script:
        "../scripts/copy_config.py"

Together with @p-glaum we tried a bunch of ways to enforce re-executing this rule whenever the config.yaml changes, e.g., by adding config=config to the params section to check for changes in the snakemake.config. However, that did not work. Another idea would be to use one of the prenetworks or postnetworks as dummy input, but nothing was quite satisfactory. Scenario management and the dependence of the different solve_* rules on the copied config.yaml complicate the matter further. Any ideas?

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions