Heat dsm 2025 by lprieto1409 · Pull Request #1857 · PyPSA/pypsa-eur

lprieto1409 · 2025-10-14T03:30:01Z

Closes # (if applicable).

Changes proposed in this Pull Request

This PR introduces residential heat demand-side management (DSM) for electric heat pumps based on the smartEn / DNV methodology.

It enables flexible heat demand that shifts within configurable intraday periods, improving renewable integration and reducing system costs.

The new formulation models building thermal mass as an energy-storage resource, allowing load shifting within user-defined windows (default = 12 h) via the restriction_time parameter, while maintaining thermal comfort.

Implementation

Adds residential_heat section in config/config.default.yaml with dsm, restriction_value, and restriction_time parameters (restriction_value reflects projected EC heat-pump rollout, ≈ 27% by 2030)
Applies DSM to rural, urban decentral, and urban central (district-heating) systems
Introduces heat_dsm_profile() in scripts/build_hourly_heat_demand.py and DSM stores in scripts/prepare_sector_network.py
Updates rules/build_sector.smk, documentation (doc/configtables/sector.csv, doc/supply_demand.rst),
and release notes

Validation & Impact Analysis

The following analysis demonstrates the flexibility feature is working correctly using a test scenario (EU27, TY: 2030, 38 clusters). Comparison: Baseline (No DSM) vs. With DSM.

1. DSM Dispatch & Renewable Response

The plot validates dynamic optimization: DSM shifts from wind-following in winter to solar-driven midday charging in summer, confirming responsiveness to renewable availability.

2. Heat Pump Efficiency Improvement

DSM improves heat pump efficiency by shifting operation to higher COP periods. The time series shows consistently higher COP for DSM, particularly in summer, with the distribution revealing more frequent operation at higher efficiency levels.

3. Generation Mix Changes

The flexibility provided by DSM dynamically shifts the optimal generation mix, leading to the largest proportional increases in solar thermal and biogas dispatch:

4. Generation Dispatch Patterns

The bottom panel shows DSM's net impact on generation. During summer, DSM reduces generation through better COP utilization and solar alignment. In winter, it increases generation due to thermal storage losses and pre-heating during high-wind periods.

5. Renewable Curtailment

DSM reduces offshore wind curtailment by absorbing surplus generation, but increases onshore wind curtailment due to higher capacity deployment driven by storage losses.

6. System Cost Impact

Heat DSM reduces total system cost by 1.3 billion EUR (0.22%). Generator capacity investment remains unchanged at 156 B€, while storage and link investments decrease slightly due to improved system flexibility.

Definition of done

Update the code to current master and resolve the merge conflict
Check the code is still working as intended
Check the feature implementation, if the implementation is reasonable and confirmed by literature/assumptions supported by the referenced document
Check if the code matches the style and requirements of the remaining PyPSA-Eur code
Add some documentation (PyPSA-Eur style, i.e. the configuration options, but also to the body of the PyPSA-Eur docs where the logic of implemented features is explained)
Update the assessment of the impact this PR has on the model figure/qualitatively

Checklist

I tested my contribution locally and it works as intended.
Code and workflow changes are sufficiently documented.
Changed dependencies are added to envs/environment.yaml.
Changes in configuration options are added in config/config.default.yaml.
Changes in configuration options are documented in doc/configtables/*.csv.
Sources of newly added data are documented in doc/data_sources.rst.
A release note doc/release_notes.rst is added.

for more information, see https://pre-commit.ci

* fix for heat_dsm_profile for leap year * add fneum code suggestion * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

euronion · 2025-10-17T11:51:48Z

Supersedes #1856 and #1371

euronion

Thanks @lprieto1409 . I have a few questions and a few improvement suggestions, see the comments below.

config/test/config.overnight.yaml

config/config.default.yaml

config/plotting.default.yaml

doc/release_notes.rst

scripts/build_hourly_heat_demand.py

scripts/plot_balance_map.py

scripts/make_summary.py

scripts/plot_gas_network.py

IMPACT_ASSESSMENT_HEAT_DSM.md

euronion · 2025-10-30T17:15:02Z

@lprieto1409 The oet/master got merged into this branch instead of the pypsa-eur/master.
Can you please fix it? It makes it difficult to review and we don't want the OET changes in here.

fneum · 2025-11-01T13:12:56Z

Thanks for the PR. For getting it reviewed here, could you prepare:

An overview of what new functionality was added and how it can be used (configuration settings, new data dependencies, etc.). Please, not just an auto-generated AI summary.
Could you add a before (master) and after (this PR) comparison of system costs and energy balances, as well as any other aspects relevant to the feature? This allows us to assess whether the proposed changes are plausible quickly.

Thank you!

lprieto1409 · 2025-11-03T04:19:40Z

@lprieto1409 The oet/master got merged into this branch instead of the pypsa-eur/master. Can you please fix it? It makes it difficult to review and we don't want the OET changes in here.

Thanks @euronion, I fixed it and also address the conflicts, let me know if you have any additional comments

euronion · 2025-11-20T09:13:56Z

Thanks a lot for the updated figures @lprieto1409 , this helps discussion the PR impacts!

I'm not fully happy with the PR results.

Looking at the figures, it seems that most of the DSF serves the integration of solar thermal outside of winter/the main heating season
The impact of DSF during winter/heating season on the electricity generation profile seems small.
The DSF only works in pulling demand forward in time, not by delaying demand, i.e. it allows to heat earlier, but not to defer the heating decision. This is a modelling decision, by assuming e_min_pu = 0 for the DSM store.

Related to 1. and 2.:
This could either be an artifact of the model results you show only being for 6h resolution, or they could be and unexpected interaction with other aspects. If I look at the earlier results from @martacki 's implementation at #1371 , I would have exepcted a more pronounced impact during winter/heating season.
Can you please check with 1h results again. If then it doesn't change have a look and see if there is indeed not the behaviour that @martacki saw and what could be causing it. If it does change with different time resolution, then best to add a note to the documentation and hint at this PR with different results for different time resolutions.

Related to 3. I would argue that we should have e_min_pu = (-1) * e_max_pu, representing the thermal inertia of buildings (e_max_pu > 0 -> building get's heated above regular temperature and cools down slowly, e_min_pu < 0 -> building is allowed to cool down below regular temperature and then is heated up again). I suggest to make this configurable with a a dedicated config key, e.g. shift: {back-and-forth,back,forth}.

lprieto1409 · 2025-11-26T08:03:54Z

Thanks a lot for the updated figures @lprieto1409 , this helps discussion the PR impacts!

I'm not fully happy with the PR results.
1. Looking at the figures, it seems that most of the DSF serves the integration of `solar thermal` outside of winter/the main heating season

2. The impact of DSF during winter/heating season on the electricity generation profile seems small.

3. The DSF only works in pulling demand forward in time, not by delaying demand, i.e. it allows to heat earlier, but not to defer the heating decision. This is a modelling decision, by assuming `e_min_pu = 0` for the DSM store.
Related to 1. and 2.: This could either be an artifact of the model results you show only being for 6h resolution, or they could be and unexpected interaction with other aspects. If I look at the earlier results from @martacki 's implementation at #1371 , I would have exepcted a more pronounced impact during winter/heating season. Can you please check with 1h results again. If then it doesn't change have a look and see if there is indeed not the behaviour that @martacki saw and what could be causing it. If it does change with different time resolution, then best to add a note to the documentation and hint at this PR with different results for different time resolutions.

Related to 3. I would argue that we should have e_min_pu = (-1) * e_max_pu, representing the thermal inertia of buildings (e_max_pu > 0 -> building get's heated above regular temperature and cools down slowly, e_min_pu < 0 -> building is allowed to cool down below regular temperature and then is heated up again). I suggest to make this configurable with a a dedicated config key, e.g. shift: {back-and-forth,back,forth}.

Thanks for your comments @euronion, to look even further at points 1 and 2, I run the same test on hourly resolution from January to March (3 months) here are some of the results comparing both:

1 hour resolution

6 hour resolution

1 hour resolution

6 hour resolution

1 hour resolution

6 hour resolution

1 hour resolution

6 hour resolution

1 hour resolution

6 hour resolution

Rename config and restructure config. Add directionality option.

euronion · 2025-11-28T16:08:47Z

Thanks for the updated plots.
I was still puzzled by the little effects, but the 6h does smooth over some of the benefits of the 1h.
I had a look myself and made a few adjustments:

Implemented directionality: Structures can be cooled or heated below/above requirements (overheat and undercool or overheat-undercool). The previous implementation was only overheat.
Reduced the test scope to 5 countries, 2 months, hourly resolution for faster solving

Here are the results I'm getting:

The effect on the heat pump dispatch pattern is very noticeable:

Although the annual COP is not much affected:

The electricity generation / dispatch pattern is also affected, in peak by up to 27%:

Interestingly enough, the undercool-ing seems to be more prominent than the overheating. I think that change makes most of the difference. You can see this in the SoC of the heat dsm / virtual stores, as they are more often negative than positive. This corresponds to letting the housing structure cool below target temperature during the morning/midday and to re-heat it to target temperature in the early afternoon/evening.

euronion · 2025-11-28T16:10:49Z

I'm happy with the results now.

@martacki and @lprieto1409 feel free to have another look. I'd consider it RTM then

lprieto1409 · 2025-12-01T03:00:29Z

I'm happy with the results now.

@martacki and @lprieto1409 feel free to have another look. I'd consider it RTM then

Thanks @euronion it does look good to me, I only saw a minor typo in sector.csv but other than that looks great :)

martacki

thanks @euronion; the under-cool addition seems like a great addition! I like the upgrade 👍

@fneum I guess the latest analysis also covers your request(s) ?

fneum

Approved in principle! Nice feature, just a few editorial things to clean up. Takes 10 minutes. Then, RTM!

doc/configtables/sector.csv

config/test/config.heat_dsm_test.yaml

config/test/config.no_heat_dsm_test.yaml

config/config.default.yaml

doc/release_notes.rst

scripts/prepare_sector_network.py

config/config.default.yaml

scripts/prepare_sector_network.py

fneum · 2025-12-01T21:08:35Z

doc/supply_demand.rst

+
+*Thermal characteristics*
+
+Thermal losses in the building thermal mass are modeled using the same standing loss rates as decentralized water tank storage, which provides a reasonable proxy for the thermal inertia of building envelopes. The storage is cyclic, meaning the state of charge at the end of the optimization period must equal that at the beginning.


I think the standing losses in a hot water storage are likely much higher than in a building.

euronion · 2025-12-02T10:06:40Z

Tests are failing because Zenodo retrieval for seawater_temperature is not consistent with the data retrieval from #1675 . will be fixed in open-energy-transition#67

fneum · 2025-12-02T10:44:14Z

Tests are failing because Zenodo retrieval for seawater_temperature is not consistent with the data retrieval from #1675 . will be fixed in open-energy-transition#67

Good but then it's unrelated to this PR.

martacki and others added 11 commits October 18, 2024 11:10

initial dsm implementation for the heating sector

b9945b9

[pre-commit.ci] auto fixes from pre-commit.com hooks

5dc8eab

for more information, see https://pre-commit.ci

assume thermal losses in buildings and remove heating dsm from default

c483d3e

[pre-commit.ci] auto fixes from pre-commit.com hooks

1c6ba98

for more information, see https://pre-commit.ci

improve heat dsr documentation

0d1d1bb

Merge branch 'master' into heat_dsm

ca630ab

Merge branch 'master' into heat_dsm

8acb453

Merge branch 'master' into heat_dsm

bdf0a5c

addressing comments

a307d17

CI related

2866ac7

lprieto1409 marked this pull request as ready for review October 14, 2025 03:38

Merge remote-tracking branch 'upstream/master' into heat_dsm_2025

4248d09

lprieto1409 and others added 5 commits October 22, 2025 17:28

working workflow

765a8ba

working workflow

51d7189

adding documentation

5f6560e

Fixing CI and adding impact assessment

a36d123

Merge branch 'master' into heat_dsm_2025

dfedbbf

euronion requested changes Oct 23, 2025

View reviewed changes

lprieto1409 added 2 commits October 29, 2025 07:35

addressing comments

362bb86

deleting report

53b7a6a

reverting and addressing conflicts

6cdbfc4

lprieto1409 force-pushed the heat_dsm_2025 branch from c2d1a3f to 6cdbfc4 Compare October 31, 2025 16:27

fneum mentioned this pull request Nov 1, 2025

initial dsm implementation for the heating sector #1371

Closed

7 tasks

lprieto1409 added 2 commits November 3, 2025 05:12

Fixing merge and addressing comments

815fae7

addressing conflicts with master

79ac1be

lprieto1409 added 4 commits November 14, 2025 06:30

Larger example

3ffec69

higher temporal resolution

765495b

comparing heat dsm vs no heat dsm

d067395

Fix docs: heat DSM restriction_time is timezone-adjusted

f3dee63

code: Restructure feature

8ee450b

Rename config and restructure config. Add directionality option.

euronion and others added 4 commits November 28, 2025 17:09

Fix colours for plotting

b8f8c5d

code: Consistent labelling of heat dsm

f0b9741

Update tests

ae45f45

Merge branch 'master' into heat_dsm_2025

201aed1

minor typo

b380d8b

martacki approved these changes Dec 1, 2025

View reviewed changes

Merge branch 'master' into heat_dsm_2025

343b872

fneum reviewed Dec 1, 2025

View reviewed changes

addressing comments

9c0957c

fneum approved these changes Dec 2, 2025

View reviewed changes

fneum enabled auto-merge (squash) December 2, 2025 07:26

euronion added 5 commits December 2, 2025 08:57

doc: Fix time description

496fb71

doc: Update UTC/local time confusion

9bca1ac

code: Fix linting complaints

faba235

pre-commit

30e5d70

Fix test config

d67eaae

fneum disabled auto-merge December 2, 2025 11:12

fneum merged commit 2739717 into PyPSA:master Dec 2, 2025
5 of 6 checks passed


		Thermal characteristics

		Thermal losses in the building thermal mass are modeled using the same standing loss rates as decentralized water tank storage, which provides a reasonable proxy for the thermal inertia of building envelopes. The storage is cyclic, meaning the state of charge at the end of the optimization period must equal that at the beginning.

Conversation

lprieto1409 commented Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes proposed in this Pull Request

Implementation

Validation & Impact Analysis

1. DSM Dispatch & Renewable Response

2. Heat Pump Efficiency Improvement

3. Generation Mix Changes

4. Generation Dispatch Patterns

5. Renewable Curtailment

6. System Cost Impact

Checklist

Uh oh!

euronion commented Oct 17, 2025

Uh oh!

euronion left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

euronion commented Oct 30, 2025

Uh oh!

fneum commented Nov 1, 2025

Uh oh!

lprieto1409 commented Nov 3, 2025

Uh oh!

euronion commented Nov 20, 2025

Uh oh!

lprieto1409 commented Nov 26, 2025

Uh oh!

euronion commented Nov 28, 2025

Uh oh!

euronion commented Nov 28, 2025

Uh oh!

lprieto1409 commented Dec 1, 2025

Uh oh!

martacki left a comment

Choose a reason for hiding this comment

Uh oh!

fneum left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fneum Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

euronion commented Dec 2, 2025

Uh oh!

fneum commented Dec 2, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

lprieto1409 commented Oct 14, 2025 •

edited

Loading