* docs: add documentation page for describing cba workflow

* docs: add description of saved cba networks and calculating cba indicators

* docs: add document describing calculation of cba indicators

* chore: fix formatting for rst

* chore: fix formatting

* docs: add information of other indicators

* docs: add workflow diagram to documentation

* docs: upload diagram of workflow

* chore: format bullet points

* docs: add PR#392 to release notes

* typo in docs/cba-indicators.rst.rst

* Fix raw custom costs (PyPSA#2023)

* fix custom_costs scaling

* add release note

* avoid code duplication

* Update doc/release_notes.rst

Co-authored-by: Thomas Gilon <[email protected]>

* minor fix

---------

Co-authored-by: Thomas Gilon <[email protected]>

* fix: add schema related files to the REUSE.toml (PyPSA#2029)

* Update cba-indicators.rst

added doc for B4 and small layout cahnges to other KPI's

* Update index.rst

* adjust menu to show CBA and CBA indicators
* move PyPSA-Eur related pages a little down the menu and bring the Open-TYNDP references upwards

* Update index.rst

add PR's for B2, B3, and B4

* update GEM steel and use GEM cement plant tracker (PyPSA#1830)

* update GEM steel and use GEM cement plant tracker

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update doc/data-retrieval.rst

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Copilot <[email protected]>

* follow up on PyPSA#1830 (PyPSA#2031)

* update data sources for country-level electricity demand (PyPSA#1828)

* update data sources for country-level electricity demand

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update electricity demand data cleaning

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* extend interpolation limit

* Apply suggestions from code review

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Copilot <[email protected]>

* docs: list all available archive cutouts in own weather data section (PyPSA#2033)

* docs: list all available archive cutouts in docs

* Revert "docs: list all available archive cutouts in docs"

This reverts commit 7c7e78d.

* move to own section

* fix

* remove zenodo

* add new pypsa explorer rule to workflow

* [github-actions.ci] Update locked envs

* doc: add new rules to docs

* doc: add release note

* doc: add feature to dev table in README.md and index.rst

* Add lfp, vanadium, lair, pair battery technologies (Form-Energy Storage) (PyPSA#1961)

* Add lfp, vanadium, lair, pair battery technologies (Form-Energy Storage)

Co-authored-by: Gianvito Colucci <[email protected]>

* add release notes

Co-authored-by: Gianvito Colucci <[email protected]>

* add Copilot suggestion manually

* fix documentation as suggested by Copilot

* improve documentation with current method

* make roundtrip_correction linked to battery tech

* minor phrasing

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Gianvito Colucci <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Make corine optional (PyPSA#2013)

* Update renewable.py

* Update schema.json

---------

Co-authored-by: Fabian Neumann <[email protected]>

* feat: allow for bidirectional links between offshore hubs (#445)

* feat: allow for bidirectional links between offshore hubs

* doc: add release note

* Add script path provider (take 2) (PyPSA#1997)

* Add script path getter; update all `script` directives

* force integration test on windows runner

* Revert CI change

---------

Co-authored-by: Lukas Trippe <[email protected]>

* fix: use Windows specific parameter on Windows

* feat: add availability profiles to nuclear power plants (#447)

* feat: add rule retrieve_nuclear_profiles

* feat: add nuclear profiles as input to prepare_sector_network

* feat: attach profiles to network components

* doc: add release note

* chore: update schema

* fix: fix linter for missing variable

* doc: improve type hints

* feat: account for outages in must-runs

* Apply suggestions from code review

* chore: revivew docker image with pixi (PyPSA#2041)

* chore: image follow up (PyPSA#2042)

* chore: image follow up

* update version in pixi.toml

* Feat/355 clean market model output files (#372)

* feat: add cleaning plexos output data script and rule

* feat: year, scenario, source, table column to output

* fix: add scenario name to output

* feat: add warning that processing of the data is only implemented for NT currently

* feat: remove unncessary logger.info

* docs: improve file docstring

* docs: add types for lookup_dict

* [github-actions.ci] Update locked envs

* docs: clarify docstring

Co-authored-by: Daniel Rüdt <[email protected]>

* docs: clarify logger warning

Co-authored-by: Daniel Rüdt <[email protected]>

* fix: update input in rule to new folder structure

Co-authored-by: Daniel Rüdt <[email protected]>

* fix: remove redundant source

Co-authored-by: Daniel Rüdt <[email protected]>

* fix: remove word repetition in comment

Co-authored-by: Daniel Rüdt <[email protected]>

* fix: improve logger info

Co-authored-by: Daniel Rüdt <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: rename to TYNDP market model output

* feat: differentiate between charge (load)

* feat: add function which sets negative sign if load

* docs: add release note

* feat: add data retrieve

* feat: add MM output data to config schema, rule, validation

* fix: restore sb.smk and properly merge master changes

* fix: data ordering

* fix: another fix after merging concerning missing nuclear data

* docs: update release note

Co-authored-by: Daniel Rüdt <[email protected]>

* fix: set scenario only to NT

Co-authored-by: Daniel Rüdt <[email protected]>

* fix: remove unneeded helper function

Co-authored-by: Daniel Rüdt <[email protected]>

* fix: capitalize scenario name

Co-authored-by: Daniel Rüdt <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: remove need for helper function

Co-authored-by: Daniel Rüdt <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: constraint clean_tyndp_output_benchmark and plot_benchmark to specific planning horizons

---------

Co-authored-by: Daniel Rüdt <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: daniel-rdt <[email protected]>

* Update cba.rst

typo (tranmission > transmission)

* feat: move find_free_port to _helpers.py and add helpful logger message for Windows users

* feat: implement B4 indicator and automated benchmarking to CBA  (#398)

* feat: add `cba.co2_societal_cost` config options

* feat: add functions to calculate b2 indicator

* docs: add `co2_societal_cost` to `doc/configtables/cba`

* docs: add docstrings and type hints

* test: add `co2_societal_cost` to `config/test/config.tyndp.yaml`

* docs: add PR#348 to release notes

* docs: update `doc/index` with B2 indicator information

* feat: add calculation for B3 indicators

* docs: update `doc/index` with B3 indicator information

* docs: add PR#350 to `doc/release_notes`

* feat: add CBA RES carriers in `config/config.tyndp.yaml`

* refac: switch to using carriers specified in config

* docs: edit description of PR#350

* refac: use `get()` function

* refac: move check of method name into separate function

* feat: change to use net emissions (co2 atmosphere store)

* [github-actions.ci] Update locked envs

* [github-actions.ci] Update locked envs

* feat: add `nice_names=False` for `n.statistics()` and use `electricity.tyndp_renewable_carriers` from config

* feat: remove `cba.res_carriers` from config

* feat: rename `co2_diff` column to `co2_variation`

Co-authored-by: lisazeyen <[email protected]>

* feat: rename `B2_social_cost_*` columns to `B2_societal_cost_variation_*`

Co-authored-by: lisazeyen <[email protected]>

* feat: rename `B3_res_dump_change_mwh` column to `B3_annual_avoided_curtailment_mwh`

Co-authored-by: lisazeyen <[email protected]>

* feat: add processing of emission factors and calculation of b4 indicator

* feat: add non-co2 emissions csv into rule

* refac: use databundle and new data layer

* docs: update doc/data_inventory for tyndp_cba_non_co2_emissions

* docs: add non-co2 emission factors to doc/tyndp-2024-bundle

* docs: add PR#398 to release notes

* docs: reorganize PR#398 in doc/release_notes

* feat: add CARRIER_TO_FUEL_MAP and use tyndp_conventional_carriers in b4 calculation

* docs: update doc/index with b4 indicator

* refac: remove unneeded `aggregate_time="sum"`

Co-authored-by: lisazeyen <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* feat: reformat indicators csv to long format and add units (#401)

* feat: reformat indicators csv to long format and add units

* refac: change units mapping

* docs: add PR#401 to release notes

* fix: add custom mappings and use `bus_carrier` from `energy_balance`

* fix: use `Link` component for uranium

* fix: correct to use long format indicators data

* fix: remove doc/configtables/data.csv

no longer existing as part of recent changes to `master`. during the merge, i did not properly remove it, and the local version on my machine got committed

* fix: re-add `tyndp_cba_non_co2_emissions` to data/versions using new schema

* fix: add tyndp_cba_non_co2_emissions to scripts/lib/validation/config/data

* fix: update using `pixi run generate-config`

* fix: update using `pixi run python test/test_data_versions_layer.py`

* feat: update b4 calculation methodology

* feat: add automated benchmarking for cba (#405)

* feat: reformat indicators csv to long format and add units

* feat: add script to clean and process 2024 tyndp cba indicators

* feat: add tyndp2024 results to results indicators csv

* feat: add script to create benchmark plots of cba indicators

* feat: add rules for benchmarking cba indicators

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: update indicator names

* fix: change b4 indicator name and units

* refac: change units assignment

* fix: change mapping of indicators between models and 2024 tyndp

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refac: modify units

* feat: change mapping of indicators

* feat: change save location and name of benchmarking plot

* refac: change units mapping

* docs: add PR#401 to release notes

* feat: add rule to save the table of indicator names and units as a resources file

* fix: change units mapping between model results and report results

* fix: edit axis of benchmark plots

* feat: add plot title and legend

* feat: add horizontal dashed line at y=0

* refac: remove unneeded ordering

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refac: remove unnecessary mappings

* refac: remove units mapping

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refac: remove some code

* refac: cleanup parsing

* fix: correct mapping of B2a indicator

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* feat: edit arrangement of plots and legends

* docs: edit docstrings

* refac: remove MODEL_SUBINDEX_PREFERENCE

* fix: rearrange legend order

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* feat: rename indicators

* feat: rename indicators

* refac/docs: combine functions and modify docstrings

* refac: remove variable creation

* docs: updaet doc/index and doc/release_notes

* fix: add `planning_horizons` wildcard

Co-authored-by: lisazeyen <[email protected]>

* feat: change output folder from `benchmark` to `validation`

Co-authored-by: lisazeyen <[email protected]>

* fix: change naming convention to match SB benchmarking

* fix: change model name in plot

* refac: use planning horizon provided from config or wildcard

* refac: simplify filtering

* fix: correct filtering for scenario name

* refac: move scenario name checking to load_benchmark_rows() function

* fix: apply units for indicators

* fix: remove overwriting of numeric values with units

* feat: add `cba.area` config parameter

* feat: pass `feat.cba` through plots to add subtitle

* fix: add fallback checks for is_beneficial column

* feat: change save location for validation plots

* feat: add `cba.area` config parameter to scripts/lib/validation/config/cba

* refac: move `cba.area` lower down

* feat: update after running `pixi run generate-config`

* fix: correct `cba.area` type

* refac: update data paths

* fix: update b2a indicator name

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: lisazeyen <[email protected]>

* docs: fix PR number doc/index.rst

Co-authored-by: Carlos Gaete <[email protected]>

* fix: change to `not-supported` in data/versions.csv

Co-authored-by: lisazeyen <[email protected]>

* fix: change data source year in data/versions.csv

Co-authored-by: lisazeyen <[email protected]>

---------

Co-authored-by: measrainsey <[email protected]>
Co-authored-by: lisazeyen <[email protected]>
Co-authored-by: Thomas Gilon <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Carlos Gaete <[email protected]>

* docs: add B4 to doc/cba-indicators.rst

* docs: reorganize outputs for B1 indicator

* docs: edit wording for outputs

* docs: reorganize outputs for B2 indicators

* docs: edit wording for outputs

* docs: change wording for outputs

* docs: reformat outputs for B4 indicator

* docs: fix B4 indicator name

* Documentation: explain the Open TYNDP implementation of the scenario approach (#449)

* Add a scenarios page to the documentation. Move benchmarking page under header.

* Correct spacing

* Add introduction for the scenarios page

* Add information about main scenario assumptions

* Add breakdown of config keys and summary of the storylines

* Add link to stoyline report as epigraph author

* Responded to review comments

* Avoid merge conflict

* Final responses to review comments

* refactor: restructure TYNDP 2024 data retrievals to align with that of upstream (#451)

* refactor: remove individual retrieve rules

* doc: update documentation

* chore: update configuration validation schema

* doc: update data_inventory.csv

* doc: remove obsolete task from doc and minor doc fixes

* fix: remove pecd reference in retrieve_tyndp

* fix: make retrieve_tyndp lighter for non-tyndp runs

* doc: add release note

* doc: minor url fix in pr template

* doc: fix links in tyndp 2024 doc

Co-authored-by: Daniel Rüdt <[email protected]>

* doc: rename tyndp 2024 doc file

* Apply suggestions from code review

Co-authored-by: Daniel Rüdt <[email protected]>

* refactor: include market outputs in retrieve_tyndp

* refactor: move non_co2_emissions to data

* chore: update configs

* Apply suggestions from code review

---------

Co-authored-by: Daniel Rüdt <[email protected]>

* feat: assign cba method (TOOT/PINT) to project and run only assigned method (#426)

* feat: add table_B1_CBA_Implementations_Guidelines_TYNDP2024.csv data

* feat: remove `in_reference2030` and `in_reference2035` from clean_projects

* feat: add script to assign method to each project

* feat: add rules to read in table B3 and to assign CBA project method

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: add `cba_guidelines_reference_projects` to data schema

* fix: add missing projects as PINT for 2030 and 2040

* feat: read in output from assign_cba_project_method to prepare_toot_project and prepare_pint_project

* feat: remove filtering of projects based on method for now

* docs: add PR#426 to release notes

* fix: correct year in data/versions.csv

Co-authored-by: lisazeyen <[email protected]>

* feat: extend checkpoint and clean_projects to only run TOOT or PINT for a project; remove assign_cba_project_method rule

* fix: add check for if indicators file is empty

* fix: correct sorting after running `pixi run python test/test_data_versions_layer.py`

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* chore: delete scripts/cba/assign_cba_project_method.py

* refac: rename symbol from `_` to `method`

* chore: add TODO comment about storage

* refac: add ValueError for if method is not pint or toot

* chore: add comment for methods dependency

* refac: remove unused methods input

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: add check for if indicators file is empty

* chore: change valuerror to logger for missing benchmark rows

happens for DE scenario but not for NT

* feat: change output directory for cba projects

* feat: replace `prepare_pint_project.py` and `prepare_toot_project.py` with single `prepare_project.py`

* refac: remove method from snakemake call

* feat: update plotting scripts to detect method

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* feat: correct capacities from SB to CBA reference network (#450)

* feat: add rule to create table of corrections from SB to CBA network

* refac: change rule output name

* feat: update reference network using corrections

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* docs: add PR#450 to release notes

* fix: change bus name of UK00/UKNI to GB00/GBNI

* refac: remove default values

Co-authored-by: lisazeyen <[email protected]>

* fix: keep links uni-directional

Co-authored-by: lisazeyen <[email protected]>

* refac: remove default efficiency

Co-authored-by: lisazeyen <[email protected]>

* feat: add 2035 in reference grid preparation

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: lisazeyen <[email protected]>

* refac: remove unnecessary outputs

Co-authored-by: lisazeyen <[email protected]>

* refac: remove unneeded check

* fix: remove toot_projects and pint_projects from outputs

* refac: use `dataset["folder"]` convention for cba_guidelines_reference_projects

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: lisazeyen <[email protected]>

* feat: improve power generation benchmarking of renewables with curtailment (#457)

* feat: replace generation by curtailment for renewables generation in benchmarks

* doc: improve wording

* fix: filter for eu27 countries

* fix: account for h2 generating wind farms in

* doc: add release note

* feat: add market outputs to the benchmarking framework (#458)

* fix: fix unit in market outputs

* feat: add market outputs to the

* fix: remove loads from the data

* feat: group smr and smr ccs together and remove h2 store discharge

* feat: clean market output data for benchmarking

* feat: remove e-fuels from h2 demand

* feat: improve color management for bench

* refactor: rename TYNDP 2024 scenario source to be more explicit

* chore: change color of market outputs

* doc: add release note

* doc: update benchmarking tables and figures

* Prepare v0.5 (#456)

* doc: bump version

* doc: update release note

* doc: update features list

* doc: update benchmarking figures

* doc: update the README

* doc: add missing entries to the release note

* doc: update release date

Co-authored-by: Daniel Rüdt <[email protected]>

---------

Co-authored-by: Daniel Rüdt <[email protected]>

* doc: update tag in benchmarking doc (#465)

* doc: update tag in benchmarking doc

* doc: add release note

* fix: fix merge conflicts

* fix: expand the new build_electricity_demand features to open-tyndp

* [github-actions.ci] Update locked envs

* doc: update citation file to include Zenodo doi and improved references (#459)

* update citation file with improved doi and primary references

* add citation hint to the README.md and index.rst.

* fix: fix indentation error

* doc: update banner in README.md and index.rst

* doc: update version

* doc: update readme and index title to align with citation and Zenodo records

* doc: add release note

* Revert "doc: update readme and index title to align with citation and Zenodo records"

This reverts commit b5b61c5.

* doc: align citation title with README and index.rst

* Apply suggestions from code review

Co-authored-by: Thomas Gilon <[email protected]>

* fix: fix links in citation documentation

* doc: improve wording

* fix: use single quotes

* feat: keep TYNDP bundle zenodo badge

* feat: keep PyPSA-Eur-sec badge

* fix: use figure to improve rendering of single badge in documentation

* fix: left align badge

* fix: add line break

* Revert "fix: use figure to improve rendering of single badge in documentation"

This reverts commit a41a9c2.

---------

Co-authored-by: Thomas Gilon <[email protected]>

* fix: fix typo from TYDNP to TYNDP

* refactor: generalise condition in build_electricity_demand

* doc: add missing SPDX header

* fix: fix typo from TYDNP to TYNDP

* fix: use return code from windows uninstaller (#472)

* fix: use return code from uninstaller

when launched from the installer the uninstaller is unable to completely remove
uninstall.exe, but since that is overwritten anyway, we trust the return code.

* Update release_notes.rst

* refactor: add script path getter helper to sb and cba rule files

* fix: reindex load time series to snapshots year

* feat: refactors temporal aggregation as a separate rule (#469)

* refactor: rename time_aggregation as build_snapshot_weightings

 Conflicts:
	rules/build_sector.smk

* refactor: move set_temporal_aggregation to a new rule

* doc: improve doc

* refactor: update mock_snakemakes

* doc: add release note

 Conflicts:
	doc/release_notes.rst

* doc: add snakemake messages

* doc: update release note

* doc: update SPDX headers and wording of doc

* chore: improve type hint

* Apply suggestions from code review

Co-authored-by: Daniel Rüdt <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* doc: add logger for else case

* Apply suggestions from code review

---------

Co-authored-by: Daniel Rüdt <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* fix: h2 grid 2040 (#475)

* fix: remove fixed h2 pipelines assets from previous planning horizon

* fix: keep CH - IT H2 pipeline capacity

* doc: add release note

* doc: update release note

* feat: add uniform renewable profiles for solar and wind assets (#473)

* feat: add uniform_renewable_profiles feature

* doc: add release note

* chore: remove lost print

* fix: account for snapshot weigthings in generation

* fix: address non-unique columns

* fix: add missing aggregation for snapshots

* doc: fix type hint

* feat: sign windows binaries (#471)

* feat: add ossign for signing with the azure artifact signatures

* fix: also build and sign installer in PR

* fix: set azure credentials

* fix action name

* fix: ossign config should be json

* Add installer to PR as comment.

* fix: add permissions for commenting

* feat: update comment always for building failures

* fix: Link to artifact directly

* test failing

* Revert "test failing"

This reverts commit 88ae3fe.

* fix: update gh workflow name

* fix: update job names

* fix name references

* Update release_notes.rst

* feat: Only build windows installer on `/build-installer` comment.

* fix: only react to comments from people with write access

* fix: don't add comments for unrelated pr comments

---------

Co-authored-by: Markus Groissböck <[email protected]>

* chore: verify checksums for GCP (#486)

* Update pixi.toml

* [github-actions.ci] Update locked envs

* Update release_notes.rst

---------

Co-authored-by: coroa <[email protected]>

* fix: rename UK to GB (#489)

* fix: rename UK to GB

* doc: add release note

* feat: set build_years for new reference grid links via config file and include 2030 projects (#496)

* feat: set commissioning years for new links for reference grid via config file

* doc: add release note

* chore: use conda-forge for pypsa-explorer dependency (#491)

* feat: use conda-forge channel for pypsa-explorer dependency

* [github-actions.ci] Update locked envs

* doc: add release note

* manually update pixi.lock file again

* [github-actions.ci] Update locked envs

* chore: pin grpcio hotfix to silence warnings (#498)

* feat: use conda-forge channel for pypsa-explorer dependency

* [github-actions.ci] Update locked envs

* doc: add release note

* manually update pixi.lock file again

* [github-actions.ci] Update locked envs

* chore: pin grpcio<1.78

* [github-actions.ci] Update locked envs

* doc: add release note

* [github-actions.ci] Update locked envs

* doc: include comment for hotfix

Co-authored-by: Jonas Hörsch <[email protected]>

* [github-actions.ci] Update locked envs

---------

Co-authored-by: Jonas Hörsch <[email protected]>

* Prepare v0.5.1 (#495)

* doc: update version and Zenodo references

* doc: update release notes for v0.5.1

* doc: apply suggestions about capitalisations

Co-authored-by: meas <[email protected]>

* doc: add #471 to Windows installer in list of features

* fix: update version in pixi.toml

* [github-actions.ci] Update locked envs

---------

Co-authored-by: meas <[email protected]>

* fix: fix GH workflow for attaching windows installer to release (#501)

* fix: add explicit repo for windows installer upload

* Update release_notes.rst

* feat: add carrier specific load shedding and shedding costs (#494)

* feat: improve load shedding implementation to allow carrier specifc load shedding and shedding costs

* doc: add release note

* fix: adjust remaining config entries for load shedding

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: pass `msv_resample_method` and `hurdle_costs` as params

* fix: update cba config validation

* fix: change import path for `set_temporal_aggregation`

* refac: set `apply_to_all_carriers: true`

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refac: add `apply_to_all_carriers: true` to config/test/config.tyndp.yaml

* fix: remove setting `p_set` for generators and links due to infeasibilities

* chore: run pre-commit

* chore: remove comment (no longer needed)

* fix: set `e_sum_max` and `p_set` of biogas and biomass generators

---------

Co-authored-by: Markus Groissboeck <[email protected]>
Co-authored-by: Michael Lindner <[email protected]>
Co-authored-by: Thomas Gilon <[email protected]>
Co-authored-by: Markus Groissböck <[email protected]>
Co-authored-by: Fabian Neumann <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Lukas Trippe <[email protected]>
Co-authored-by: daniel-rdt <[email protected]>
Co-authored-by: daniel-rdt <[email protected]>
Co-authored-by: Virio Andreyana <[email protected]>
Co-authored-by: Gianvito Colucci <[email protected]>
Co-authored-by: Cristobal Gallego-Castillo <[email protected]>
Co-authored-by: Bryn Pickering <[email protected]>
Co-authored-by: lisazeyen <[email protected]>
Co-authored-by: measrainsey <[email protected]>
Co-authored-by: Carlos Gaete <[email protected]>
Co-authored-by: Will Usher <[email protected]>
Co-authored-by: Carlos Gaete <[email protected]>
Co-authored-by: tgilon <[email protected]>
Co-authored-by: Jonas Hörsch <[email protected]>
Co-authored-by: coroa <[email protected]>

* feat(cba): add MSV extraction for seasonal storage dispatch

Implement Marginal Storage Value (MSV) extraction to guide seasonal
storage dispatch in rolling horizon CBA optimization.

Changes:
- Add prepare_cba_base rule to fix capacities and remove co2_sequestration_limit
- Add solve_cba_msv_extraction rule for perfect foresight MSV extraction
- Apply MSV as marginal_cost to seasonal stores (H2, gas, co2 sequestered)
- Support temporal resampling with configurable resolution and method (ffill/interpolate)
- Refactor pipeline: base → reference → MSV extraction → simplify → dispatch

Config options:
- cba.storage.seasonal_carriers: carriers receiving MSV
- cba.msv_extraction.resolution: extraction resolution (false or 24H, etc.)
- cba.msv_extraction.resample_method: ffill or interpolate

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* doc: add release note for MSV extraction PR #441

* Update doc/release_notes.rst

Co-authored-by: lisazeyen <[email protected]>

* Update rules/cba.smk

Co-authored-by: lisazeyen <[email protected]>

* Update scripts/cba/prepare_cba_base.py

Co-authored-by: lisazeyen <[email protected]>

* Update scripts/cba/prepare_cba_base.py

Co-authored-by: lisazeyen <[email protected]>

* Update scripts/cba/solve_cba_msv_extraction.py

Co-authored-by: lisazeyen <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refactor(cba): restructure workflow per review, add MSV temporal aggregation

Address review feedback on workflow structure and naming:

- Rename prepare_cba_base -> simplify_sb_network (fixes capacities, hurdle costs,
  extends fuel sources, disables volume limits)
- Create prepare_rolling_horizon rule (disables cyclicity, applies MSV)
- Move MSV application from solve-time to network preparation phase
- Add build_msv_snapshot_weightings rule for temporal aggregation support

The set_temporal_aggregation function requires a CSV file with pre-computed
snapshot weightings, so a new rule generates this file when hourly resolution
is configured (e.g., "24H", "48H"). This follows existing snakemake patterns.

Additional changes:
- Add seasonal_carriers and msv_extraction to config schema
- Add _CbaMsvExtractionConfig to validation config
- Improve docstrings and comments for clarity
- Fix typo: "unustainable" -> "unsustainable"

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: move disabling of volume limits to prepare_rolling_horizon

* refactor: remove some of the logging

* feat: apply msv to storageunits as well

* feat: as intermediate solution, fix dispatch of bio generators to SB solution

* feat(cba): add leap day handling to MSV snapshot weightings

* feat(cba): three-tier storage classification with PF initial state

Split storage carriers into three categories for rolling horizon dispatch:
- seasonal_carriers: MSV + initial state from perfect foresight (hydro, H2, gas)
- accumulator_carriers: MSV only, start empty (co2 sequestered)
- cyclic_carriers: unchanged (batteries)

Add set_initial_state_from_pf() to initialize seasonal stores from PF e(t=-1),
addressing hydro reservoir under-dispatch in rolling horizon.

Extend MSV extraction and application to StorageUnit components (needed for
hydro modeled as StorageUnit in PyPSA).

Separate MSV extraction solving config (load_shedding disabled to preserve
duals) from rolling horizon solving config.

Add fallback solver mechanism for rolling horizon window failures.

* feat(cba): fix reservoir SOC from PF to eliminate load shedding

Fix state_of_charge_set on hydro-reservoir StorageUnits to the PF
trajectory, giving the RH optimizer SOC guidance while retaining
dispatch freedom for all other components.

Results (2030 reference network):
- Load shedding: 12.2 TWh → 0.0001 TWh (eliminated)
- Bus prices match PF: mean 71.9 vs 71.4 EUR/MWh, no VOLL hours
- Reservoir spill: 140 TWh → 0.25 TWh (water no longer wasted)
- Generation mix within 3% of PF across all carriers

Also documents failed Approach 1 (p_dispatch_set) which caused
infeasibility because spill remained free, draining reservoirs
between RH windows.

* WIP: add experimental SOC boundary fix (Approach 3) for RH windows

Approach 2 (fix full SOC trajectory) caused infeasibility in 2040 where
hourly SOC changes exceeded p_nom. Approach 3 fixes SOC only at the first
and last snapshot of each RH window, giving the optimizer dispatch freedom
within windows.

Results:
- 2030: identical to Approach 2 (0.1 GWh LS, 1 hour at year end)
- 2040: now feasible (2.8 GWh LS, 6 hours at year end)

Note: this commit retains all three approaches for reference. Next commit
will clean up and keep only the working implementation.

* refactor: keep only boundary SOC fix, remove Approaches 1 and 2

Remove fix_reservoir_dispatch_from_pf (Approach 1, caused infeasibility
via spill drain) and fix_reservoir_soc_from_pf (Approach 2, over-constrained
hourly dispatch beyond p_nom in 2040).

Keep only fix_reservoir_soc_at_boundaries which pins SOC at window
start/end from PF trajectory, giving the optimizer full dispatch freedom
within each RH window.

* refactor: add soc_boundary_carriers config, clean up prepare_rolling_horizon

- Add soc_boundary_carriers to config (tyndp, default, test), schema (all
  3 locations), pydantic validation, and snakemake rule params
- Also add missing accumulator_carriers to schema
- Move numpy import to top-level, remove stale EXPERIMENTAL label
- Use config-driven carriers instead of hardcoded list in script

* chore: merge `master` onto `feat/376-cba-msv-seasonal-stores` (#499)

* docs: add documentation page for describing cba workflow

* docs: add description of saved cba networks and calculating cba indicators

* docs: add document describing calculation of cba indicators

* chore: fix formatting for rst

* chore: fix formatting

* docs: add information of other indicators

* docs: add workflow diagram to documentation

* docs: upload diagram of workflow

* chore: format bullet points

* docs: add PR#392 to release notes

* typo in docs/cba-indicators.rst.rst

* Fix raw custom costs (PyPSA#2023)

* fix custom_costs scaling

* add release note

* avoid code duplication

* Update doc/release_notes.rst

Co-authored-by: Thomas Gilon <[email protected]>

* minor fix

---------

Co-authored-by: Thomas Gilon <[email protected]>

* fix: add schema related files to the REUSE.toml (PyPSA#2029)

* Update cba-indicators.rst

added doc for B4 and small layout cahnges to other KPI's

* Update index.rst

* adjust menu to show CBA and CBA indicators
* move PyPSA-Eur related pages a little down the menu and bring the Open-TYNDP references upwards

* Update index.rst

add PR's for B2, B3, and B4

* update GEM steel and use GEM cement plant tracker (PyPSA#1830)

* update GEM steel and use GEM cement plant tracker

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update doc/data-retrieval.rst

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Copilot <[email protected]>

* follow up on PyPSA#1830 (PyPSA#2031)

* update data sources for country-level electricity demand (PyPSA#1828)

* update data sources for country-level electricity demand

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update electricity demand data cleaning

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* extend interpolation limit

* Apply suggestions from code review

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Copilot <[email protected]>

* docs: list all available archive cutouts in own weather data section (PyPSA#2033)

* docs: list all available archive cutouts in docs

* Revert "docs: list all available archive cutouts in docs"

This reverts commit 7c7e78d.

* move to own section

* fix

* remove zenodo

* add new pypsa explorer rule to workflow

* [github-actions.ci] Update locked envs

* doc: add new rules to docs

* doc: add release note

* doc: add feature to dev table in README.md and index.rst

* Add lfp, vanadium, lair, pair battery technologies (Form-Energy Storage) (PyPSA#1961)

* Add lfp, vanadium, lair, pair battery technologies (Form-Energy Storage)

Co-authored-by: Gianvito Colucci <[email protected]>

* add release notes

Co-authored-by: Gianvito Colucci <[email protected]>

* add Copilot suggestion manually

* fix documentation as suggested by Copilot

* improve documentation with current method

* make roundtrip_correction linked to battery tech

* minor phrasing

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Gianvito Colucci <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Make corine optional (PyPSA#2013)

* Update renewable.py

* Update schema.json

---------

Co-authored-by: Fabian Neumann <[email protected]>

* feat: allow for bidirectional links between offshore hubs (#445)

* feat: allow for bidirectional links between offshore hubs

* doc: add release note

* Add script path provider (take 2) (PyPSA#1997)

* Add script path getter; update all `script` directives

* force integration test on windows runner

* Revert CI change

---------

Co-authored-by: Lukas Trippe <[email protected]>

* fix: use Windows specific parameter on Windows

* feat: add availability profiles to nuclear power plants (#447)

* feat: add rule retrieve_nuclear_profiles

* feat: add nuclear profiles as input to prepare_sector_network

* feat: attach profiles to network components

* doc: add release note

* chore: update schema

* fix: fix linter for missing variable

* doc: improve type hints

* feat: account for outages in must-runs

* Apply suggestions from code review

* chore: revivew docker image with pixi (PyPSA#2041)

* chore: image follow up (PyPSA#2042)

* chore: image follow up

* update version in pixi.toml

* Feat/355 clean market model output files (#372)

* feat: add cleaning plexos output data script and rule

* feat: year, scenario, source, table column to output

* fix: add scenario name to output

* feat: add warning that processing of the data is only implemented for NT currently

* feat: remove unncessary logger.info

* docs: improve file docstring

* docs: add types for lookup_dict

* [github-actions.ci] Update locked envs

* docs: clarify docstring

Co-authored-by: Daniel Rüdt <[email protected]>

* docs: clarify logger warning

Co-authored-by: Daniel Rüdt <[email protected]>

* fix: update input in rule to new folder structure

Co-authored-by: Daniel Rüdt <[email protected]>

* fix: remove redundant source

Co-authored-by: Daniel Rüdt <[email protected]>

* fix: remove word repetition in comment

Co-authored-by: Daniel Rüdt <[email protected]>

* fix: improve logger info

Co-authored-by: Daniel Rüdt <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: rename to TYNDP market model output

* feat: differentiate between charge (load)

* feat: add function which sets negative sign if load

* docs: add release note

* feat: add data retrieve

* feat: add MM output data to config schema, rule, validation

* fix: restore sb.smk and properly merge master changes

* fix: data ordering

* fix: another fix after merging concerning missing nuclear data

* docs: update release note

Co-authored-by: Daniel Rüdt <[email protected]>

* fix: set scenario only to NT

Co-authored-by: Daniel Rüdt <[email protected]>

* fix: remove unneeded helper function

Co-authored-by: Daniel Rüdt <[email protected]>

* fix: capitalize scenario name

Co-authored-by: Daniel Rüdt <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: remove need for helper function

Co-authored-by: Daniel Rüdt <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: constraint clean_tyndp_output_benchmark and plot_benchmark to specific planning horizons

---------

Co-authored-by: Daniel Rüdt <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: daniel-rdt <[email protected]>

* Update cba.rst

typo (tranmission > transmission)

* feat: move find_free_port to _helpers.py and add helpful logger message for Windows users

* feat: implement B4 indicator and automated benchmarking to CBA  (#398)

* feat: add `cba.co2_societal_cost` config options

* feat: add functions to calculate b2 indicator

* docs: add `co2_societal_cost` to `doc/configtables/cba`

* docs: add docstrings and type hints

* test: add `co2_societal_cost` to `config/test/config.tyndp.yaml`

* docs: add PR#348 to release notes

* docs: update `doc/index` with B2 indicator information

* feat: add calculation for B3 indicators

* docs: update `doc/index` with B3 indicator information

* docs: add PR#350 to `doc/release_notes`

* feat: add CBA RES carriers in `config/config.tyndp.yaml`

* refac: switch to using carriers specified in config

* docs: edit description of PR#350

* refac: use `get()` function

* refac: move check of method name into separate function

* feat: change to use net emissions (co2 atmosphere store)

* [github-actions.ci] Update locked envs

* [github-actions.ci] Update locked envs

* feat: add `nice_names=False` for `n.statistics()` and use `electricity.tyndp_renewable_carriers` from config

* feat: remove `cba.res_carriers` from config

* feat: rename `co2_diff` column to `co2_variation`

Co-authored-by: lisazeyen <[email protected]>

* feat: rename `B2_social_cost_*` columns to `B2_societal_cost_variation_*`

Co-authored-by: lisazeyen <[email protected]>

* feat: rename `B3_res_dump_change_mwh` column to `B3_annual_avoided_curtailment_mwh`

Co-authored-by: lisazeyen <[email protected]>

* feat: add processing of emission factors and calculation of b4 indicator

* feat: add non-co2 emissions csv into rule

* refac: use databundle and new data layer

* docs: update doc/data_inventory for tyndp_cba_non_co2_emissions

* docs: add non-co2 emission factors to doc/tyndp-2024-bundle

* docs: add PR#398 to release notes

* docs: reorganize PR#398 in doc/release_notes

* feat: add CARRIER_TO_FUEL_MAP and use tyndp_conventional_carriers in b4 calculation

* docs: update doc/index with b4 indicator

* refac: remove unneeded `aggregate_time="sum"`

Co-authored-by: lisazeyen <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* feat: reformat indicators csv to long format and add units (#401)

* feat: reformat indicators csv to long format and add units

* refac: change units mapping

* docs: add PR#401 to release notes

* fix: add custom mappings and use `bus_carrier` from `energy_balance`

* fix: use `Link` component for uranium

* fix: correct to use long format indicators data

* fix: remove doc/configtables/data.csv

no longer existing as part of recent changes to `master`. during the merge, i did not properly remove it, and the local version on my machine got committed

* fix: re-add `tyndp_cba_non_co2_emissions` to data/versions using new schema

* fix: add tyndp_cba_non_co2_emissions to scripts/lib/validation/config/data

* fix: update using `pixi run generate-config`

* fix: update using `pixi run python test/test_data_versions_layer.py`

* feat: update b4 calculation methodology

* feat: add automated benchmarking for cba (#405)

* feat: reformat indicators csv to long format and add units

* feat: add script to clean and process 2024 tyndp cba indicators

* feat: add tyndp2024 results to results indicators csv

* feat: add script to create benchmark plots of cba indicators

* feat: add rules for benchmarking cba indicators

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: update indicator names

* fix: change b4 indicator name and units

* refac: change units assignment

* fix: change mapping of indicators between models and 2024 tyndp

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refac: modify units

* feat: change mapping of indicators

* feat: change save location and name of benchmarking plot

* refac: change units mapping

* docs: add PR#401 to release notes

* feat: add rule to save the table of indicator names and units as a resources file

* fix: change units mapping between model results and report results

* fix: edit axis of benchmark plots

* feat: add plot title and legend

* feat: add horizontal dashed line at y=0

* refac: remove unneeded ordering

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refac: remove unnecessary mappings

* refac: remove units mapping

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refac: remove some code

* refac: cleanup parsing

* fix: correct mapping of B2a indicator

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* feat: edit arrangement of plots and legends

* docs: edit docstrings

* refac: remove MODEL_SUBINDEX_PREFERENCE

* fix: rearrange legend order

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* feat: rename indicators

* feat: rename indicators

* refac/docs: combine functions and modify docstrings

* refac: remove variable creation

* docs: updaet doc/index and doc/release_notes

* fix: add `planning_horizons` wildcard

Co-authored-by: lisazeyen <[email protected]>

* feat: change output folder from `benchmark` to `validation`

Co-authored-by: lisazeyen <[email protected]>

* fix: change naming convention to match SB benchmarking

* fix: change model name in plot

* refac: use planning horizon provided from config or wildcard

* refac: simplify filtering

* fix: correct filtering for scenario name

* refac: move scenario name checking to load_benchmark_rows() function

* fix: apply units for indicators

* fix: remove overwriting of numeric values with units

* feat: add `cba.area` config parameter

* feat: pass `feat.cba` through plots to add subtitle

* fix: add fallback checks for is_beneficial column

* feat: change save location for validation plots

* feat: add `cba.area` config parameter to scripts/lib/validation/config/cba

* refac: move `cba.area` lower down

* feat: update after running `pixi run generate-config`

* fix: correct `cba.area` type

* refac: update data paths

* fix: update b2a indicator name

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: lisazeyen <[email protected]>

* docs: fix PR number doc/index.rst

Co-authored-by: Carlos Gaete <[email protected]>

* fix: change to `not-supported` in data/versions.csv

Co-authored-by: lisazeyen <[email protected]>

* fix: change data source year in data/versions.csv

Co-authored-by: lisazeyen <[email protected]>

---------

Co-authored-by: measrainsey <[email protected]>
Co-authored-by: lisazeyen <[email protected]>
Co-authored-by: Thomas Gilon <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Carlos Gaete <[email protected]>

* docs: add B4 to doc/cba-indicators.rst

* docs: reorganize outputs for B1 indicator

* docs: edit wording for outputs

* docs: reorganize outputs for B2 indicators

* docs: edit wording for outputs

* docs: change wording for outputs

* docs: reformat outputs for B4 indicator

* docs: fix B4 indicator name

* Documentation: explain the Open TYNDP implementation of the scenario approach (#449)

* Add a scenarios page to the documentation. Move benchmarking page under header.

* Correct spacing

* Add introduction for the scenarios page

* Add information about main scenario assumptions

* Add breakdown of config keys and summary of the storylines

* Add link to stoyline report as epigraph author

* Responded to review comments

* Avoid merge conflict

* Final responses to review comments

* refactor: restructure TYNDP 2024 data retrievals to align with that of upstream (#451)

* refactor: remove individual retrieve rules

* doc: update documentation

* chore: update configuration validation schema

* doc: update data_inventory.csv

* doc: remove obsolete task from doc and minor doc fixes

* fix: remove pecd reference in retrieve_tyndp

* fix: make retrieve_tyndp lighter for non-tyndp runs

* doc: add release note

* doc: minor url fix in pr template

* doc: fix links in tyndp 2024 doc

Co-authored-by: Daniel Rüdt <[email protected]>

* doc: rename tyndp 2024 doc file

* Apply suggestions from code review

Co-authored-by: Daniel Rüdt <[email protected]>

* refactor: include market outputs in retrieve_tyndp

* refactor: move non_co2_emissions to data

* chore: update configs

* Apply suggestions from code review

---------

Co-authored-by: Daniel Rüdt <[email protected]>

* feat: assign cba method (TOOT/PINT) to project and run only assigned method (#426)

* feat: add table_B1_CBA_Implementations_Guidelines_TYNDP2024.csv data

* feat: remove `in_reference2030` and `in_reference2035` from clean_projects

* feat: add script to assign method to each project

* feat: add rules to read in table B3 and to assign CBA project method

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: add `cba_guidelines_reference_projects` to data schema

* fix: add missing projects as PINT for 2030 and 2040

* feat: read in output from assign_cba_project_method to prepare_toot_project and prepare_pint_project

* feat: remove filtering of projects based on method for now

* docs: add PR#426 to release notes

* fix: correct year in data/versions.csv

Co-authored-by: lisazeyen <[email protected]>

* feat: extend checkpoint and clean_projects to only run TOOT or PINT for a project; remove assign_cba_project_method rule

* fix: add check for if indicators file is empty

* fix: correct sorting after running `pixi run python test/test_data_versions_layer.py`

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* chore: delete scripts/cba/assign_cba_project_method.py

* refac: rename symbol from `_` to `method`

* chore: add TODO comment about storage

* refac: add ValueError for if method is not pint or toot

* chore: add comment for methods dependency

* refac: remove unused methods input

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: add check for if indicators file is empty

* chore: change valuerror to logger for missing benchmark rows

happens for DE scenario but not for NT

* feat: change output directory for cba projects

* feat: replace `prepare_pint_project.py` and `prepare_toot_project.py` with single `prepare_project.py`

* refac: remove method from snakemake call

* feat: update plotting scripts to detect method

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* feat: correct capacities from SB to CBA reference network (#450)

* feat: add rule to create table of corrections from SB to CBA network

* refac: change rule output name

* feat: update reference network using corrections

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* docs: add PR#450 to release notes

* fix: change bus name of UK00/UKNI to GB00/GBNI

* refac: remove default values

Co-authored-by: lisazeyen <[email protected]>

* fix: keep links uni-directional

Co-authored-by: lisazeyen <[email protected]>

* refac: remove default efficiency

Co-authored-by: lisazeyen <[email protected]>

* feat: add 2035 in reference grid preparation

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: lisazeyen <[email protected]>

* refac: remove unnecessary outputs

Co-authored-by: lisazeyen <[email protected]>

* refac: remove unneeded check

* fix: remove toot_projects and pint_projects from outputs

* refac: use `dataset["folder"]` convention for cba_guidelines_reference_projects

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: lisazeyen <[email protected]>

* feat: improve power generation benchmarking of renewables with curtailment (#457)

* feat: replace generation by curtailment for renewables generation in benchmarks

* doc: improve wording

* fix: filter for eu27 countries

* fix: account for h2 generating wind farms in

* doc: add release note

* feat: add market outputs to the benchmarking framework (#458)

* fix: fix unit in market outputs

* feat: add market outputs to the

* fix: remove loads from the data

* feat: group smr and smr ccs together and remove h2 store discharge

* feat: clean market output data for benchmarking

* feat: remove e-fuels from h2 demand

* feat: improve color management for bench

* refactor: rename TYNDP 2024 scenario source to be more explicit

* chore: change color of market outputs

* doc: add release note

* doc: update benchmarking tables and figures

* Prepare v0.5 (#456)

* doc: bump version

* doc: update release note

* doc: update features list

* doc: update benchmarking figures

* doc: update the README

* doc: add missing entries to the release note

* doc: update release date

Co-authored-by: Daniel Rüdt <[email protected]>

---------

Co-authored-by: Daniel Rüdt <[email protected]>

* doc: update tag in benchmarking doc (#465)

* doc: update tag in benchmarking doc

* doc: add release note

* fix: fix merge conflicts

* fix: expand the new build_electricity_demand features to open-tyndp

* [github-actions.ci] Update locked envs

* doc: update citation file to include Zenodo doi and improved references (#459)

* update citation file with improved doi and primary references

* add citation hint to the README.md and index.rst.

* fix: fix indentation error

* doc: update banner in README.md and index.rst

* doc: update version

* doc: update readme and index title to align with citation and Zenodo records

* doc: add release note

* Revert "doc: update readme and index title to align with citation and Zenodo records"

This reverts commit b5b61c5.

* doc: align citation title with README and index.rst

* Apply suggestions from code review

Co-authored-by: Thomas Gilon <[email protected]>

* fix: fix links in citation documentation

* doc: improve wording

* fix: use single quotes

* feat: keep TYNDP bundle zenodo badge

* feat: keep PyPSA-Eur-sec badge

* fix: use figure to improve rendering of single badge in documentation

* fix: left align badge

* fix: add line break

* Revert "fix: use figure to improve rendering of single badge in documentation"

This reverts commit a41a9c2.

---------

Co-authored-by: Thomas Gilon <[email protected]>

* fix: fix typo from TYDNP to TYNDP

* refactor: generalise condition in build_electricity_demand

* doc: add missing SPDX header

* fix: fix typo from TYDNP to TYNDP

* fix: use return code from windows uninstaller (#472)

* fix: use return code from uninstaller

when launched from the installer the uninstaller is unable to completely remove
uninstall.exe, but since that is overwritten anyway, we trust the return code.

* Update release_notes.rst

* refactor: add script path getter helper to sb and cba rule files

* fix: reindex load time series to snapshots year

* feat: refactors temporal aggregation as a separate rule (#469)

* refactor: rename time_aggregation as build_snapshot_weightings

 Conflicts:
	rules/build_sector.smk

* refactor: move set_temporal_aggregation to a new rule

* doc: improve doc

* refactor: update mock_snakemakes

* doc: add release note

 Conflicts:
	doc/release_notes.rst

* doc: add snakemake messages

* doc: update release note

* doc: update SPDX headers and wording of doc

* chore: improve type hint

* Apply suggestions from code review

Co-authored-by: Daniel Rüdt <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* doc: add logger for else case

* Apply suggestions from code review

---------

Co-authored-by: Daniel Rüdt <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* fix: h2 grid 2040 (#475)

* fix: remove fixed h2 pipelines assets from previous planning horizon

* fix: keep CH - IT H2 pipeline capacity

* doc: add release note

* doc: update release note

* feat: add uniform renewable profiles for solar and wind assets (#473)

* feat: add uniform_renewable_profiles feature

* doc: add release note

* chore: remove lost print

* fix: account for snapshot weigthings in generation

* fix: address non-unique columns

* fix: add missing aggregation for snapshots

* doc: fix type hint

* feat: sign windows binaries (#471)

* feat: add ossign for signing with the azure artifact signatures

* fix: also build and sign installer in PR

* fix: set azure credentials

* fix action name

* fix: ossign config should be json

* Add installer to PR as comment.

* fix: add permissions for commenting

* feat: update comment always for building failures

* fix: Link to artifact directly

* test failing

* Revert "test failing"

This reverts commit 88ae3fe.

* fix: update gh workflow name

* fix: update job names

* fix name references

* Update release_notes.rst

* feat: Only build windows installer on `/build-installer` comment.

* fix: only react to comments from people with write access

* fix: don't add comments for unrelated pr comments

---------

Co-authored-by: Markus Groissböck <[email protected]>

* chore: verify checksums for GCP (#486)

* Update pixi.toml

* [github-actions.ci] Update locked envs

* Update release_notes.rst

---------

Co-authored-by: coroa <[email protected]>

* fix: rename UK to GB (#489)

* fix: rename UK to GB

* doc: add release note

* feat: set build_years for new reference grid links via config file and include 2030 projects (#496)

* feat: set commissioning years for new links for reference grid via config file

* doc: add release note

* chore: use conda-forge for pypsa-explorer dependency (#491)

* feat: use conda-forge channel for pypsa-explorer dependency

* [github-actions.ci] Update locked envs

* doc: add release note

* manually update pixi.lock file again

* [github-actions.ci] Update locked envs

* chore: pin grpcio hotfix to silence warnings (#498)

* feat: use conda-forge channel for pypsa-explorer dependency

* [github-actions.ci] Update locked envs

* doc: add release note

* manually update pixi.lock file again

* [github-actions.ci] Update locked envs

* chore: pin grpcio<1.78

* [github-actions.ci] Update locked envs

* doc: add release note

* [github-actions.ci] Update locked envs

* doc: include comment for hotfix

Co-authored-by: Jonas Hörsch <[email protected]>

* [github-actions.ci] Update locked envs

---------

Co-authored-by: Jonas Hörsch <[email protected]>

* Prepare v0.5.1 (#495)

* doc: update version and Zenodo references

* doc: update release notes for v0.5.1

* doc: apply suggestions about capitalisations

Co-authored-by: meas <[email protected]>

* doc: add #471 to Windows installer in list of features

* fix: update version in pixi.toml

* [github-actions.ci] Update locked envs

---------

Co-authored-by: meas <[email protected]>

* fix: fix GH workflow for attaching windows installer to release (#501)

* fix: add explicit repo for windows installer upload

* Update release_notes.rst

* feat: add carrier specific load shedding and shedding costs (#494)

* feat: improve load shedding implementation to allow carrier specifc load shedding and shedding costs

* doc: add release note

* fix: adjust remaining config entries for load shedding

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: pass `msv_resample_method` and `hurdle_costs` as params

* fix: update cba config validation

* fix: change import path for `set_temporal_aggregation`

* refac: set `apply_to_all_carriers: true`

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refac: add `apply_to_all_carriers: true` to config/test/config.tyndp.yaml

* fix: remove setting `p_set` for generators and links due to infeasibilities

* chore: run pre-commit

* chore: remove comment (no longer needed)

* fix: set `e_sum_max` and `p_set` of biogas and biomass generators

---------

Co-authored-by: Markus Groissboeck <[email protected]>
Co-authored-by: Michael Lindner <[email protected]>
Co-authored-by: Thomas Gilon <[email protected]>
Co-authored-by: Markus Groissböck <[email protected]>
Co-authored-by: Fabian Neumann <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Lukas Trippe <[email protected]>
Co-authored-by: daniel-rdt <[email protected]>
Co-authored-by: daniel-rdt <[email protected]>
Co-authored-by: Virio Andreyana <[email protected]>
Co-authored-by: Gianvito Colucci <[email protected]>
Co-authored-by: Cristobal Gallego-Castillo <[email protected]>
Co-authored-by: Bryn Pickering <[email protected]>
Co-authored-by: lisazeyen <[email protected]>
Co-authored-by: measrainsey <[email protected]>
Co-authored-by: Carlos Gaete <[email protected]>
Co-authored-by: Will Usher <[email protected]>
Co-authored-by: Carlos Gaete <[email protected]>
Co-authored-by: tgilon <[email protected]>
Co-authored-by: Jonas Hörsch <[email protected]>
Co-authored-by: coroa <[email protected]>

* [github-actions.ci] Update locked envs

* fix: remove repeated `launch_explorer` option

* build: check pixi.lock and env files from `origin/master`

* Update config/schema.json

Co-authored-by: lisazeyen <[email protected]>

* Update config/schema.json

Co-authored-by: lisazeyen <[email protected]>

* Update scripts/cba/prepare_rolling_horizon.py

Co-authored-by: lisazeyen <[email protected]>

* Update scripts/cba/prepare_rolling_horizon.py

Co-authored-by: lisazeyen <[email protected]>

* Update scripts/cba/prepare_rolling_horizon.py

Co-authored-by: lisazeyen <[email protected]>

* Update scripts/cba/prepare_rolling_horizon.py

Co-authored-by: lisazeyen <[email protected]>

* Update scripts/cba/prepare_rolling_horizon.py

Co-authored-by: lisazeyen <[email protected]>

* Update scripts/cba/prepare_rolling_horizon.py

Co-authored-by: lisazeyen <[email protected]>

* Update scripts/cba/prepare_rolling_horizon.py

Co-authored-by: lisazeyen <[email protected]>

* Update config/config.tyndp.yaml

Co-authored-by: lisazeyen <[email protected]>

* Update scripts/cba/simplify_sb_network.py

Co-authored-by: lisazeyen <[email protected]>

* Update scripts/cba/prepare_rolling_horizon.py

Co-authored-by: lisazeyen <[email protected]>

* Update scripts/cba/prepare_rolling_horizon.py

Co-authored-by: lisazeyen <[email protected]>

* Update scripts/cba/prepare_rolling_horizon.py

Co-authored-by: lisazeyen <[email protected]>

* refactor(cba): replace seasonal/accumulator carrier lists with automatic classification

Eliminate seasonal_carriers and accumulator_carriers config lists. Now only
cyclic_carriers exists; all other stores automatically receive MSV and initial
state from PF (if they were cyclic in the full-year optimisation).

Note: this introduces a known infeasibility in window 53 of the rolling
horizon solve due to forced biomass dispatch (p_set) saturating the CO2
sequestration system. Addressed in the next commit.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix(cba): replace fixed biomass dispatch with per-window energy budgets

Instead of copying PF dispatch into p_set (which forces every timestep), tag volume-limited generators with has_volume_limit and set per-window e_sum_min = e_sum_max from PF dispatch totals in the rolling horizon loop.

This gives the optimizer hourly flexibility while preserving annual energy budgets, resolving the window 53 CO2 system infeasibility where co2 sequestered reaches capacity.

* chore: regenerate config/schema.json from Pydantic models

* fix: update using `pixi run generate-config`

---------

Co-authored-by: Carlos Gaete <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: lisazeyen <[email protected]>
Co-authored-by: lisa <[email protected]>
Co-authored-by: cdgaete <[email protected]>
Co-authored-by: meas <[email protected]>
Co-authored-by: Markus Groissboeck <[email protected]>
Co-authored-by: Michael Lindner <[email protected]>
Co-authored-by: Thomas Gilon <[email protected]>
Co-authored-by: Markus Groissböck <[email protected]>
Co-authored-by: Fabian Neumann <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Lukas Trippe <[email protected]>
Co-authored-by: daniel-rdt <[email protected]>
Co-authored-by: daniel-rdt <[email protected]>
Co-authored-by: Virio Andreyana <[email protected]>
Co-authored-by: Gianvito Colucci <[email protected]>
Co-authored-by: Cristobal Gallego-Castillo <[email protected]>
Co-authored-by: Bryn Pickering <[email protected]>
Co-authored-by: measrainsey <[email protected]>
Co-authored-by: Will Usher <[email protected]>
Co-authored-by: tgilon <[email protected]>
Co-authored-by: Jonas Hörsch <[email protected]>
Co-authored-by: coroa <[email protected]>