Skip to content

Update calculations of retrofit costs#759

Merged
fneum merged 16 commits intoPyPSA:masterfrom
ekatef:fix_retrofit
Oct 18, 2023
Merged

Update calculations of retrofit costs#759
fneum merged 16 commits intoPyPSA:masterfrom
ekatef:fix_retrofit

Conversation

@ekatef
Copy link
Copy Markdown
Contributor

@ekatef ekatef commented Oct 13, 2023

The PR is aimed to update retrofit calculations to make the model work with retro_endogen: false. The main reasons why updates are needed is that data structure may have been altered and pandas version has changed.

Changes proposed in this Pull Request

  • replace pandas.append() was deprecated for a while and has been removed since pandas 2.0;
  • fix structure of area_tot (= total area) grouped dataframe to avoid messed concatenated columns;
  • fix a column name which contains subsector names for a building dataset: building_data.detail currently contains only [nan 'insulation' 'low-e'], while the script looks for "Totals" value which is currently contained in building_data.subsector;
  • deal with AssertionError: Length of new_levels (2) must be same as self.nlevels (3) which is currently arisen by reindex() function for prepare_building_topology( ) and map_tabula_to_hotmaps( ). That seems to be caused BUG: Regression in 2.1.0 when inserting usign MultiIndex frames pandas-dev/pandas#54909 for operation on insertion of smaller MultiIndex into larger MultiIndex and has been fixed recently with REGR: setitem with part of a MultiIndex raises pandas-dev/pandas#54885, which is one of milestones of pandas 2.1.1;
  • fix country codes in building_data to avoid messing-up "GB" and "UK";
  • remove duplicated country column in area_reordered;
  • fix call of difference( ) function;
  • fix computation of grouped means and total averages;
  • replace a level name with a number of the level.

Apart of the code fixes, the datakit structure should be ajusted a bit: tabula-calculator-calcsetbuilding.csv should be moved to /pypsa-eur/data/retro, where the workflow expects to find it.

Checklist

  • I tested my contribution locally and it seems to work fine.
  • Code and workflow changes are sufficiently documented.
  • Changed dependencies are added to envs/environment.yaml.
  • Changes in configuration options are added in all of config.default.yaml.
  • Changes in configuration options are also documented in doc/configtables/*.csv.
  • A release note doc/release_notes.rst is added.

ekatef and others added 14 commits September 27, 2023 12:47
@FabianHofmann
Copy link
Copy Markdown
Contributor

FabianHofmann commented Oct 13, 2023

Thanks, @ekatef for the PR. Looks good! CI is failing due to some internet connection problems, but let's wait till it's through. @lisazeyen, you approve?

martacki
martacki reviewed Oct 13, 2023
View reviewed changes
Copy link
Copy Markdown
Member

@martacki martacki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can confirm that the workflow runs with suggested data structure changes. But instead of moving the tabula data around, we could also change the respective path. Then the sector-bundle doesn't need updating (might be confusing if the files appears twice), i.e.
in rules/build_sector.smk, change

data_tabula="data/retro/tabula-calculator-calcsetbuilding,csv",

to

data_tabula="data/bundle-sector/retro/tabula-calculator-calcsetbuilding.csv",

Thoughts ?

martacki
martacki reviewed Oct 13, 2023
View reviewed changes
@ekatef
Copy link
Copy Markdown
Contributor Author

ekatef commented Oct 13, 2023

Can confirm that the workflow runs with suggested data structure changes. But instead of moving the tabula data around, we could also change the respective path. Then the sector-bundle doesn't need updating (might be confusing if the files appears twice), i.e. in rules/build_sector.smk, change

data_tabula="data/retro/tabula-calculator-calcsetbuilding,csv",

to

data_tabula="data/bundle-sector/retro/tabula-calculator-calcsetbuilding.csv",

Thoughts ?

Agree, fixed.

@lisazeyen
Copy link
Copy Markdown
Contributor

lisazeyen commented Oct 17, 2023

Thanks, @ekatef for the PR. Looks good! CI is failing due to some internet connection problems, but let's wait till it's through. @lisazeyen, you approve?

Looks good to me, only the CI needs to run through

@fneum
Copy link
Copy Markdown
Member

fneum commented Oct 18, 2023

None of the changes affect the CI, since retrofitting path is not covered by tests. Therefore, I am merging without passing tests (URL connection error).

@fneum fneum merged commit df5b9e9 into PyPSA:master Oct 18, 2023
martacki
martacki reviewed Oct 25, 2023
View reviewed changes
Copy link
Copy Markdown
Member

@martacki martacki left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@martacki martacki martacki left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@ekatef @FabianHofmann @lisazeyen @fneum @martacki