Skip to content

Try to remove a few deepcopy#45135

Merged
haampie merged 3 commits intospack:developfrom
alalazo:maintainers/remove-redundant-deepcopy
Jul 10, 2024
Merged

Try to remove a few deepcopy#45135
haampie merged 3 commits intospack:developfrom
alalazo:maintainers/remove-redundant-deepcopy

Conversation

@alalazo
Copy link
Copy Markdown
Member

@alalazo alalazo commented Jul 9, 2024

This just tries to remove a few deepcopy calls.

Double deepcopy in _write_section

This call:

validate_data = copy.deepcopy(data)
validate(validate_data, SECTION_SCHEMAS[section])

is redundant, since the first thing done within the validate function is to take a deepcopy of the input data.

Deepcopy when processing definitions in YAML files

This other call:

def _process_definition(self, item):
"""Process a single spec definition item."""
entry = copy.deepcopy(item)
when = _eval_conditional(entry.pop("when", "True"))
assert len(entry) == 1

seems to be needed to pop from the input. The function is reworked to avoid the need for popping.

Deepcopy on YAML validation

Before #20526 the validation of a YAML file was modifying the data (!), so we were using a defensive deepcopy to be able to round-trip data:

test_data = syaml.deepcopy(data)

We don't need that since #20526 so here I removed the deepcopy and simplified EnvironmentManifestFile, which was still keeping track of a "pristine" YAML file and a YAML file with defaults attached.


Measured on the same benchmark as in #43745 on:

  • Spack: 0.23.0.dev0 (15efcbe)
  • Python: 3.11.5
  • Platform: linux-ubuntu20.04-icelake
  • Concretizer: clingo
$ spack env create -d .
$ spack -e . add $(spack list)
$ time spack -e . find
% THE PR
==> 0 installed packages

real	0m3,125s
user	0m3,011s
sys	0m0,070s

% develop
==> 0 installed packages

real	0m3,934s
user	0m3,815s
sys	0m0,073s

@spackbot-app spackbot-app bot added core PR affects Spack core functionality environments labels Jul 9, 2024
Also, simplify EnvironmentManifestFile, which doesn't
need to keep track of both a pristine YAML and a YAML
with defaults anymore.
@spackbot-app spackbot-app bot added commands tests General test capability(ies) labels Jul 9, 2024
@alalazo alalazo marked this pull request as ready for review July 9, 2024 18:18
@alalazo
Copy link
Copy Markdown
Member Author

alalazo commented Jul 9, 2024

I'll leave

dest[sk] = copy.deepcopy(sv)

for another PR. Not clear to me if we can get rid of that easily.

@haampie haampie merged commit 9001e93 into spack:develop Jul 10, 2024
@alalazo alalazo deleted the maintainers/remove-redundant-deepcopy branch July 10, 2024 08:29
hariharan-devarajan pushed a commit to hariharan-devarajan/spack that referenced this pull request Jul 10, 2024
psakievich added a commit to sandialabs/spack-manager that referenced this pull request Jul 16, 2024
Address changes to spack environments and scopes
See:
- spack/spack#45044
- spack/spack#45135
psakievich added a commit to sandialabs/spack-manager that referenced this pull request Jul 16, 2024
Address changes to spack environments and scopes
See:
- spack/spack#45044
- spack/spack#45135
psakievich added a commit to sandialabs/spack-manager that referenced this pull request Jul 16, 2024
* Fixes for changes in core spack

Address changes to spack environments and scopes
See:
- spack/spack#45044
- spack/spack#45135

* Fixes

* Style ci

* Style
diehlpk pushed a commit to diehlpk/spack that referenced this pull request Aug 14, 2024
FrederickDeny pushed a commit to FrederickDeny/spack that referenced this pull request Aug 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

commands core PR affects Spack core functionality environments tests General test capability(ies)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants