Skip to content

Adds network energy balance plots#1285

Merged
fneum merged 23 commits intomasterfrom
add_modular_map_plots
Mar 19, 2025
Merged

Adds network energy balance plots#1285
fneum merged 23 commits intomasterfrom
add_modular_map_plots

Conversation

@p-glaum
Copy link
Copy Markdown
Contributor

@p-glaum p-glaum commented Sep 12, 2024 •
edited by fneum
Loading

Closes # (if applicable).

Changes proposed in this Pull Request

This PR is still a draft and needs improvement. Only uses statistics functions to create network plots by including scripts from @bobbyxng, @FabianHofmann and me.

Feel free to make improvements/changes to make it even more modular.

Checklist

  • I tested my contribution locally and it works as intended.
  • Code and workflow changes are sufficiently documented.
  • Changed dependencies are added to envs/environment.yaml.
  • Changes in configuration options are added in config/config.default.yaml.
  • Changes in configuration options are documented in doc/configtables/*.csv.
  • Sources of newly added data are documented in doc/data_sources.rst.
  • A release note doc/release_notes.rst is added.

TODOS

  • Add all plotting config settings to the plotting.yaml
  • [ ] improve scaling parameters
  • automatically take all carriers from balance_map for plotting of maps
  • write collecting rule to plot all maps
  • [ ] try to calculate flow width from branch width. not sure if this is possible and how exactly flow and branch_width are related. But would be nice to only have branch_factor

@p-glaum p-glaum marked this pull request as draft September 12, 2024 12:52
FabianHofmann
FabianHofmann reviewed Sep 19, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@FabianHofmann FabianHofmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks nice @p-glaum, what is the status? can I support with something?

@p-glaum
Copy link
Copy Markdown
Contributor Author

p-glaum commented Sep 19, 2024 •
edited
Loading

this looks nice @p-glaum, what is the status? can I support with something?

status hasn't changed so far 😅 . It was just an initial draft that the code is not lost/forgotten. I think it still needs some cleaning and updating because not all graphs look nice at the moment. I collected some TODOs above.
Currently, I don't have the time to continue on this. @bobbyxng was also working on this, but I think he is currently also busy. So if you want, you can go ahead with the implementation :)

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Oct 9, 2024 •
edited
Loading

Validator Report

I am the Validator. Download all artifacts here.
I'll be back and edit this comment for each new commit.

General

Plots comparison
Main branch Feature branch
Image not available Image not available
Image not available Image not available
Image not available Image not available
Image not available Image not available
Image not available Image not available
Image not available Image not available
Files comparison
Status NRMSE MAE (norm)
market_values.csv ⚠️Changed 0.000 0.12
weighted_prices.csv ⚠️Changed 0.652 0.59
metrics.csv ⚠️Changed 0.004 0.15
price_statistics.csv ⚠️Changed 0.000 0.17
nodal_supply_energy.csv ⚠️ NaN mismatch
prices.csv ⚠️ NaN mismatch
nodal_cfs.csv ⚠️ NaN mismatch
supply.csv ⚠️ NaN mismatch
nodal_costs.csv ⚠️ NaN mismatch
nodal_capacities.csv ⚠️ NaN mismatch
capacities.csv ⚠️ NaN mismatch
curtailment.csv ⚠️ NaN mismatch
costs.csv ⚠️ NaN mismatch
cfs.csv ⚠️ NaN mismatch
supply_energy.csv ⚠️ NaN mismatch
energy.csv ⚠️ NaN mismatch

NRMSE: Normalized (combined-min-max) Root Mean Square Error
MAE (norm): Mean Absolute Error on normalized data (min-max)
Status Threshold: MAE (norm) > 0.05 and NRMSE > 0.3

Model Metrics

Benchmarks Image not available Image not available Image not available

Comparing add_modular_map_plots (5294621) with master (52b8179).
Branch is 8 commits ahead and 4 commits behind.
Last updated on 2024-11-05 15:02:53 CET.

@p-glaum
Copy link
Copy Markdown
Contributor Author

p-glaum commented Nov 1, 2024

I fixed some bugs and made it compatible with the newest master version. For me, it is working and almost all plots look nice. Maybe we can fine tune the config a bit further. This PR is currently not interfering with default snakemake workflow, as you need to call the plot function separately with plot_balance_maps.

@p-glaum p-glaum marked this pull request as ready for review November 1, 2024 11:11
bobbyxng added a commit to resilient-project/pypsa-eur-resilient that referenced this pull request Nov 4, 2024
@bobbyxng
Added balance plotting PyPSA#1285
f326c01
bobbyxng added a commit to bobbyxng/pci-pmi-policy-targets that referenced this pull request Nov 4, 2024
@bobbyxng
Added balance plotting PyPSA/pypsa-eur#1285 and initial notebook.
2434af5
@fneum fneum requested a review from bobbyxng November 4, 2024 20:54
@fneum
Copy link
Copy Markdown
Member

fneum commented Nov 4, 2024 •
edited
Loading

I will have a quick pass over this once @bobbyxng has given his feedback he announced bilaterally.

Don't forget to reference the rules in the documentation and add a release note :)

@p-glaum p-glaum added the DNMY Do not merge yet label Nov 5, 2024
bobbyxng
bobbyxng reviewed Nov 6, 2024
View reviewed changes
bobbyxng
bobbyxng approved these changes Nov 6, 2024
View reviewed changes
Copy link
Copy Markdown
Collaborator

@bobbyxng bobbyxng left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my tests, the plots of all carriers ran through @fneum, so the implementation generally looks good to me, only two smaller remarks:

  • All plotting-related configs will be moved from config.default.yaml to plotting.default.yaml, as @p-glaum suggested. I support the idea.
  • vmin, vmax, see comment here: #1285 (comment)

@p-glaum
Copy link
Copy Markdown
Contributor Author

p-glaum commented Nov 6, 2024

I will have a quick pass over this once @bobbyxng has given his feedback he announced bilaterally.

Don't forget to reference the rules in the documentation and add a release note :)

added release note and documentation. There are just two minor open TODOs but you can already take look if you want.

@coroa
Copy link
Copy Markdown
Member

coroa commented Mar 3, 2025

Just scrolling through old PRs. Any status update here?

@fneum
Copy link
Copy Markdown
Member

fneum commented Mar 3, 2025

I think the idea is to get PyPSA/PyPSA#1156 in first and then use these functions directly.

@fneum fneum removed the DNMY Do not merge yet label Mar 17, 2025
@fneum fneum added this to the v2025.03.0 milestone Mar 18, 2025
@fneum
Copy link
Copy Markdown
Member

fneum commented Mar 18, 2025 •
edited
Loading

This is amazing! Sorry for not pulling it in earlier @p-glaum!

The maps have a legend sizing problem, which is however a more general problem that needs to be tackled in PyPSA: PyPSA/PyPSA#1031

I have made a few changes:

  • The plotting configuration is still out-sourced to plotting.default.yaml but overwrites are still intended to be made in config.yaml.

  • Assumed that all configuration settings are set (i.e. no config.get(..., default))

@fneum
Copy link
Copy Markdown
Member

fneum commented Mar 19, 2025

@bobbyxng had a look at my changes and had no further comments.

There is some follow-on work to improve the default scaling, but since this is very scenario dependent it is difficult to give all-encompassing settings here. I think this can be merged now.

@fneum fneum merged commit 8ac0520 into master Mar 19, 2025
12 checks passed
@fneum fneum deleted the add_modular_map_plots branch March 19, 2025 18:21
@daniel-rdt daniel-rdt mentioned this pull request Aug 21, 2025
Merged
25 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@FabianHofmann FabianHofmann FabianHofmann left review comments

@bobbyxng bobbyxng bobbyxng approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

v2025.04.0

Development

Successfully merging this pull request may close these issues.

5 participants

@p-glaum @fneum @coroa @FabianHofmann @bobbyxng