new_installer.py: implement overwrite, keep_prefix#51622
Merged
Conversation
Add PrefixPivoter context manager to handle installation prefix management during overwrite installations. When overwrite=True, existing installations are backed up before the new install begins, and restored on failure. The keep_prefix option controls whether failed installations are preserved for debugging (when not overwriting). Priority order for failure handling: 1. overwrite=True always restores original prefix 2. keep_prefix=False removes failed installation 3. keep_prefix=True preserves failed installation Also update BuildGraph to skip pruning specs in the overwrite set, ensuring they are reinstalled even if already in the database. Recovery of the install dir on install failure during overwrite installs is prioritized by running - `rename(prefix, garbage)` - `rename(old_prefix, prefix) - remove garbage, allowing failure Failure to clean the garbage dir is not an error, it should be a unique dir name and easy to clean manually (if ever needed) Signed-off-by: Harmen Stoppels <[email protected]>
haampie
force-pushed
the
hs/fix/overwrite-install
branch
from
71792d3 to
a5b7758
Compare
when something is installed but marked for overwrite, we should not prune it from the build graph Signed-off-by: Harmen Stoppels <[email protected]>
Signed-off-by: Harmen Stoppels <[email protected]>
There was a problem hiding this comment.
Pull Request Overview
This PR implements overwrite and keep_prefix functionality for the new installer, adding robust prefix management during package installations. The implementation introduces a PrefixPivoter context manager that handles backing up existing installations during overwrites and cleaning up failed installations based on the specified options.
Key changes:
- Added
PrefixPivotercontext manager for installation prefix lifecycle management - Modified
BuildGraphto skip pruning specs in the overwrite set - Refactored installation logic by extracting
_install()function fromworker_function()
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| lib/spack/spack/new_installer.py | Implements PrefixPivoter class for prefix management, adds overwrite/keep_prefix parameters throughout the installation pipeline, and refactors worker_function to delegate installation to _install() |
| lib/spack/spack/test/test_new_installer.py | Adds comprehensive unit tests for PrefixPivoter covering success/failure scenarios, overwrite behavior, keep_prefix options, and failure recovery edge cases |
| lib/spack/spack/test/installer_build_graph.py | Adds test to verify that specs in overwrite_set are not pruned from the build graph even when already installed |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: Harmen Stoppels <[email protected]>
This was referenced Nov 21, 2025
alalazo
approved these changes
Nov 25, 2025
psakievich
pushed a commit
to psakievich/spack
that referenced
this pull request
Dec 1, 2025
Implement `--overwrite` and `--keep-prefix` for the new installer. This also adds support for installing "develop" specs with `dev_path=` variant, as they rely on overwrite installs. Add `PrefixPivoter` context manager to handle installation prefix management during overwrite installations. When `overwrite=True`, existing installations are backed up before the new install begins, and restored on failure. The `keep_prefix` option controls whether failed installations are preserved for debugging (when not overwriting). Priority order for failure handling: 1. `overwrite=True` always restores original prefix 2. `keep_prefix=False` removes failed installation 3. `keep_prefix=True` preserves failed installation Also update `BuildGraph` to skip pruning specs in the overwrite set, ensuring they are reinstalled even if already in the database. Recovery of the old install dir on failure during overwrite installs is prioritized by running 1. `rename(prefix, garbage)` 2. `rename(old_prefix, prefix)` 3. remove garbage dir recursively, allowing failure Failure to clean the garbage dir is not an error, it should be a unique dir name and easy to clean manually (if ever needed). Cleanup of install prefixes happens in the child process, in contrast to the old `installer.py`. This avoids a blocking operation in the UI process. Adds unit tests which run in 0.1s.
hippo91
pushed a commit
to hippo91/spack
that referenced
this pull request
Dec 1, 2025
Implement `--overwrite` and `--keep-prefix` for the new installer. This also adds support for installing "develop" specs with `dev_path=` variant, as they rely on overwrite installs. Add `PrefixPivoter` context manager to handle installation prefix management during overwrite installations. When `overwrite=True`, existing installations are backed up before the new install begins, and restored on failure. The `keep_prefix` option controls whether failed installations are preserved for debugging (when not overwriting). Priority order for failure handling: 1. `overwrite=True` always restores original prefix 2. `keep_prefix=False` removes failed installation 3. `keep_prefix=True` preserves failed installation Also update `BuildGraph` to skip pruning specs in the overwrite set, ensuring they are reinstalled even if already in the database. Recovery of the old install dir on failure during overwrite installs is prioritized by running 1. `rename(prefix, garbage)` 2. `rename(old_prefix, prefix)` 3. remove garbage dir recursively, allowing failure Failure to clean the garbage dir is not an error, it should be a unique dir name and easy to clean manually (if ever needed). Cleanup of install prefixes happens in the child process, in contrast to the old `installer.py`. This avoids a blocking operation in the UI process. Adds unit tests which run in 0.1s.
becker33
pushed a commit
that referenced
this pull request
Jan 11, 2026
Implement `--overwrite` and `--keep-prefix` for the new installer. This also adds support for installing "develop" specs with `dev_path=` variant, as they rely on overwrite installs. Add `PrefixPivoter` context manager to handle installation prefix management during overwrite installations. When `overwrite=True`, existing installations are backed up before the new install begins, and restored on failure. The `keep_prefix` option controls whether failed installations are preserved for debugging (when not overwriting). Priority order for failure handling: 1. `overwrite=True` always restores original prefix 2. `keep_prefix=False` removes failed installation 3. `keep_prefix=True` preserves failed installation Also update `BuildGraph` to skip pruning specs in the overwrite set, ensuring they are reinstalled even if already in the database. Recovery of the old install dir on failure during overwrite installs is prioritized by running 1. `rename(prefix, garbage)` 2. `rename(old_prefix, prefix)` 3. remove garbage dir recursively, allowing failure Failure to clean the garbage dir is not an error, it should be a unique dir name and easy to clean manually (if ever needed). Cleanup of install prefixes happens in the child process, in contrast to the old `installer.py`. This avoids a blocking operation in the UI process. Adds unit tests which run in 0.1s.
becker33
pushed a commit
that referenced
this pull request
Jan 11, 2026
Implement `--overwrite` and `--keep-prefix` for the new installer. This also adds support for installing "develop" specs with `dev_path=` variant, as they rely on overwrite installs. Add `PrefixPivoter` context manager to handle installation prefix management during overwrite installations. When `overwrite=True`, existing installations are backed up before the new install begins, and restored on failure. The `keep_prefix` option controls whether failed installations are preserved for debugging (when not overwriting). Priority order for failure handling: 1. `overwrite=True` always restores original prefix 2. `keep_prefix=False` removes failed installation 3. `keep_prefix=True` preserves failed installation Also update `BuildGraph` to skip pruning specs in the overwrite set, ensuring they are reinstalled even if already in the database. Recovery of the old install dir on failure during overwrite installs is prioritized by running 1. `rename(prefix, garbage)` 2. `rename(old_prefix, prefix)` 3. remove garbage dir recursively, allowing failure Failure to clean the garbage dir is not an error, it should be a unique dir name and easy to clean manually (if ever needed). Cleanup of install prefixes happens in the child process, in contrast to the old `installer.py`. This avoids a blocking operation in the UI process. Adds unit tests which run in 0.1s. Signed-off-by: Gregory Becker <[email protected]>
becker33
pushed a commit
that referenced
this pull request
Jan 12, 2026
Implement `--overwrite` and `--keep-prefix` for the new installer. This also adds support for installing "develop" specs with `dev_path=` variant, as they rely on overwrite installs. Add `PrefixPivoter` context manager to handle installation prefix management during overwrite installations. When `overwrite=True`, existing installations are backed up before the new install begins, and restored on failure. The `keep_prefix` option controls whether failed installations are preserved for debugging (when not overwriting). Priority order for failure handling: 1. `overwrite=True` always restores original prefix 2. `keep_prefix=False` removes failed installation 3. `keep_prefix=True` preserves failed installation Also update `BuildGraph` to skip pruning specs in the overwrite set, ensuring they are reinstalled even if already in the database. Recovery of the old install dir on failure during overwrite installs is prioritized by running 1. `rename(prefix, garbage)` 2. `rename(old_prefix, prefix)` 3. remove garbage dir recursively, allowing failure Failure to clean the garbage dir is not an error, it should be a unique dir name and easy to clean manually (if ever needed). Cleanup of install prefixes happens in the child process, in contrast to the old `installer.py`. This avoids a blocking operation in the UI process. Adds unit tests which run in 0.1s. Signed-off-by: Gregory Becker <[email protected]>
becker33
pushed a commit
that referenced
this pull request
Jan 15, 2026
Implement `--overwrite` and `--keep-prefix` for the new installer. This also adds support for installing "develop" specs with `dev_path=` variant, as they rely on overwrite installs. Add `PrefixPivoter` context manager to handle installation prefix management during overwrite installations. When `overwrite=True`, existing installations are backed up before the new install begins, and restored on failure. The `keep_prefix` option controls whether failed installations are preserved for debugging (when not overwriting). Priority order for failure handling: 1. `overwrite=True` always restores original prefix 2. `keep_prefix=False` removes failed installation 3. `keep_prefix=True` preserves failed installation Also update `BuildGraph` to skip pruning specs in the overwrite set, ensuring they are reinstalled even if already in the database. Recovery of the old install dir on failure during overwrite installs is prioritized by running 1. `rename(prefix, garbage)` 2. `rename(old_prefix, prefix)` 3. remove garbage dir recursively, allowing failure Failure to clean the garbage dir is not an error, it should be a unique dir name and easy to clean manually (if ever needed). Cleanup of install prefixes happens in the child process, in contrast to the old `installer.py`. This avoids a blocking operation in the UI process. Adds unit tests which run in 0.1s. Signed-off-by: Gregory Becker <[email protected]>
vjranagit
pushed a commit
to vjranagit/spack
that referenced
this pull request
Jan 18, 2026
Implement `--overwrite` and `--keep-prefix` for the new installer. This also adds support for installing "develop" specs with `dev_path=` variant, as they rely on overwrite installs. Add `PrefixPivoter` context manager to handle installation prefix management during overwrite installations. When `overwrite=True`, existing installations are backed up before the new install begins, and restored on failure. The `keep_prefix` option controls whether failed installations are preserved for debugging (when not overwriting). Priority order for failure handling: 1. `overwrite=True` always restores original prefix 2. `keep_prefix=False` removes failed installation 3. `keep_prefix=True` preserves failed installation Also update `BuildGraph` to skip pruning specs in the overwrite set, ensuring they are reinstalled even if already in the database. Recovery of the old install dir on failure during overwrite installs is prioritized by running 1. `rename(prefix, garbage)` 2. `rename(old_prefix, prefix)` 3. remove garbage dir recursively, allowing failure Failure to clean the garbage dir is not an error, it should be a unique dir name and easy to clean manually (if ever needed). Cleanup of install prefixes happens in the child process, in contrast to the old `installer.py`. This avoids a blocking operation in the UI process. Adds unit tests which run in 0.1s. Signed-off-by: Gregory Becker <[email protected]>
becker33
pushed a commit
that referenced
this pull request
Jan 22, 2026
Implement `--overwrite` and `--keep-prefix` for the new installer. This also adds support for installing "develop" specs with `dev_path=` variant, as they rely on overwrite installs. Add `PrefixPivoter` context manager to handle installation prefix management during overwrite installations. When `overwrite=True`, existing installations are backed up before the new install begins, and restored on failure. The `keep_prefix` option controls whether failed installations are preserved for debugging (when not overwriting). Priority order for failure handling: 1. `overwrite=True` always restores original prefix 2. `keep_prefix=False` removes failed installation 3. `keep_prefix=True` preserves failed installation Also update `BuildGraph` to skip pruning specs in the overwrite set, ensuring they are reinstalled even if already in the database. Recovery of the old install dir on failure during overwrite installs is prioritized by running 1. `rename(prefix, garbage)` 2. `rename(old_prefix, prefix)` 3. remove garbage dir recursively, allowing failure Failure to clean the garbage dir is not an error, it should be a unique dir name and easy to clean manually (if ever needed). Cleanup of install prefixes happens in the child process, in contrast to the old `installer.py`. This avoids a blocking operation in the UI process. Adds unit tests which run in 0.1s. Signed-off-by: Gregory Becker <[email protected]>
becker33
pushed a commit
that referenced
this pull request
Jan 26, 2026
Implement `--overwrite` and `--keep-prefix` for the new installer. This also adds support for installing "develop" specs with `dev_path=` variant, as they rely on overwrite installs. Add `PrefixPivoter` context manager to handle installation prefix management during overwrite installations. When `overwrite=True`, existing installations are backed up before the new install begins, and restored on failure. The `keep_prefix` option controls whether failed installations are preserved for debugging (when not overwriting). Priority order for failure handling: 1. `overwrite=True` always restores original prefix 2. `keep_prefix=False` removes failed installation 3. `keep_prefix=True` preserves failed installation Also update `BuildGraph` to skip pruning specs in the overwrite set, ensuring they are reinstalled even if already in the database. Recovery of the old install dir on failure during overwrite installs is prioritized by running 1. `rename(prefix, garbage)` 2. `rename(old_prefix, prefix)` 3. remove garbage dir recursively, allowing failure Failure to clean the garbage dir is not an error, it should be a unique dir name and easy to clean manually (if ever needed). Cleanup of install prefixes happens in the child process, in contrast to the old `installer.py`. This avoids a blocking operation in the UI process. Adds unit tests which run in 0.1s. Signed-off-by: Gregory Becker <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implement
--overwriteand--keep-prefixfor the new installer.This also adds support for installing "develop" specs with
dev_path=variant, as they rely on overwrite installs.
Add
PrefixPivotercontext manager to handle installation prefix managementduring overwrite installations. When
overwrite=True, existing installationsare backed up before the new install begins, and restored on failure. The
keep_prefixoption controls whether failed installations are preserved fordebugging (when not overwriting).
Priority order for failure handling:
overwrite=Truealways restores original prefixkeep_prefix=Falseremoves failed installationkeep_prefix=Truepreserves failed installationAlso update
BuildGraphto skip pruning specs in the overwrite set, ensuringthey are reinstalled even if already in the database.
Recovery of the old install dir on failure during overwrite installs is
prioritized by running
rename(prefix, garbage)rename(old_prefix, prefix)Failure to clean the garbage dir is not an error, it should be a unique dir
name and easy to clean manually (if ever needed).
Cleanup of install prefixes happens in the child process, in contrast to the
old
installer.py. This avoids a blocking operation in the UI process.Adds unit tests which run in 0.1s.