spack develop: stage build artifacts in same root as non-dev builds#41373
spack develop: stage build artifacts in same root as non-dev builds#41373tgamblin merged 31 commits intospack:developfrom
spack develop: stage build artifacts in same root as non-dev builds#41373Conversation
…urce_path) points to the dev_path
…lure) can now generate artifacts in stage (they definitely will if e.g. including the build phase of a cmake package, but could in other cases as well)
…context manager semantics
|
So to confirm, the source will still default to cloning into the environment, but the |
spack develop: stage build artifacts in same root as non-dev builds
|
Does this PR fix the cloning and building of resources with develop specs? #25370 It seems like it has the potential too, but it is not clear to me if it does in its current form. |
…amespace syntax in test
… can be thrown depending on what OS we're on
|
@psakievich FYI there has been a significant change since your last review: there is no longer a symlink in the stage directory to the source directory (i.e. stages for |
|
Sorry just seeing this. I will try to review in the next day or two. |
…spack#41373) Currently (outside of this PR) when you `spack develop` a path, this path is treated as the staging directory (this means that for example all build artifacts are placed in the develop path). This PR creates a separate staging directory for all `spack develop`ed builds. It looks like ``` # the stage root /the-stage-root-for-all-spack-builds/ spack-stage-<hash> # Spack packages inheriting CMakePackage put their build artifacts here spack-build-<hash>/ ``` Unlike non-develop builds, there is no `spack-src` directory, `source_path` is the provided `dev_path`. Instead, separately, in the `dev_path`, we have: ``` /dev/path/for/foo/ build-{arch}-<hash> -> /the-stage-root-for-all-spack-builds/spack-stage-<hash>/ ``` The main benefit of this is that build artifacts for out-of-source builds that are relative to `Stage.path` are easily identified (and you can delete them with `spack clean`). Other behavior added here: - [x] A symlink is made from the `dev_path` to the stage directory. This symlink name incorporates spec details, so that multiple Spack environments that develop the same path will not conflict with one another - [x] `spack cd` and `spack location` have added a `-c` shorthand for `--source-dir` Spack builds can still change the develop path (in particular to keep track of applied patches), and for in-source builds, this doesn't change much (although logs would not be written into the develop path). Packages inheriting from `CMakePackage` should get this benefit automatically though.
After #41373, where we stopped considering the source directory to be the stage for develop builds, we resumed *deleting* the stage even after a successful build. We don't want this for develop builds because developers need to iterate; we should keep the artifacts unless they explicitly run `spack clean`. Now: - [x] Build artifacts for develop packages are not removed after a successful install - [x] They are also not removed before an install starts, i.e. develop packages always reuse prior artifacts, if available. - [x] They can be deleted in any other context, e.g. by running `spack clean --stage`
After spack#41373, where we stopped considering the source directory to be the stage for develop builds, we resumed *deleting* the stage even after a successful build. We don't want this for develop builds because developers need to iterate; we should keep the artifacts unless they explicitly run `spack clean`. Now: - [x] Build artifacts for develop packages are not removed after a successful install - [x] They are also not removed before an install starts, i.e. develop packages always reuse prior artifacts, if available. - [x] They can be deleted in any other context, e.g. by running `spack clean --stage`
After spack#41373, where we stopped considering the source directory to be the stage for develop builds, we resumed *deleting* the stage even after a successful build. We don't want this for develop builds because developers need to iterate; we should keep the artifacts unless they explicitly run `spack clean`. Now: - [x] Build artifacts for develop packages are not removed after a successful install - [x] They are also not removed before an install starts, i.e. develop packages always reuse prior artifacts, if available. - [x] They can be deleted in any other context, e.g. by running `spack clean --stage`
…spack#41373) Currently (outside of this PR) when you `spack develop` a path, this path is treated as the staging directory (this means that for example all build artifacts are placed in the develop path). This PR creates a separate staging directory for all `spack develop`ed builds. It looks like ``` # the stage root /the-stage-root-for-all-spack-builds/ spack-stage-<hash> # Spack packages inheriting CMakePackage put their build artifacts here spack-build-<hash>/ ``` Unlike non-develop builds, there is no `spack-src` directory, `source_path` is the provided `dev_path`. Instead, separately, in the `dev_path`, we have: ``` /dev/path/for/foo/ build-{arch}-<hash> -> /the-stage-root-for-all-spack-builds/spack-stage-<hash>/ ``` The main benefit of this is that build artifacts for out-of-source builds that are relative to `Stage.path` are easily identified (and you can delete them with `spack clean`). Other behavior added here: - [x] A symlink is made from the `dev_path` to the stage directory. This symlink name incorporates spec details, so that multiple Spack environments that develop the same path will not conflict with one another - [x] `spack cd` and `spack location` have added a `-c` shorthand for `--source-dir` Spack builds can still change the develop path (in particular to keep track of applied patches), and for in-source builds, this doesn't change much (although logs would not be written into the develop path). Packages inheriting from `CMakePackage` should get this benefit automatically though.
After spack#41373, where we stopped considering the source directory to be the stage for develop builds, we resumed *deleting* the stage even after a successful build. We don't want this for develop builds because developers need to iterate; we should keep the artifacts unless they explicitly run `spack clean`. Now: - [x] Build artifacts for develop packages are not removed after a successful install - [x] They are also not removed before an install starts, i.e. develop packages always reuse prior artifacts, if available. - [x] They can be deleted in any other context, e.g. by running `spack clean --stage`
Currently (outside of this PR) when you
spack developa path, this path is treated as the staging directory (this means that for example all build artifacts are placed in the develop path).This PR creates a separate staging directory for all
spack developed builds. It looks like:Unlike non-develop builds, there is no
spack-srcdirectory,source_pathis the provideddev_path. Instead, separately, in thedev_path, we have:The main benefit of this is that build artifacts for out-of-source builds that are relative to
Stage.pathare easily identified (and you can delete them withspack clean).Other behavior added here:
A symlink is made from the
dev_pathto the stage directory. This symlink name incorporates spec details, so that multiple Spack environments that develop the same path will not conflict with one anotherspack cdandspack locationhave added a-cshorthand for--source-dirSpack builds can still change the develop path (in particular to keep track of applied patches), and for in-source builds, this doesn't change much (although logs would not be written into the develop path). Packages inheriting from
CMakePackageshould get this benefit automatically though.