stage: fix resources being deleted from local cache#10152
Merged
scheibelp merged 2 commits intospack:developfrom Jan 5, 2019
Merged
stage: fix resources being deleted from local cache#10152scheibelp merged 2 commits intospack:developfrom
scheibelp merged 2 commits intospack:developfrom
Conversation
adamjstewart
approved these changes
Dec 19, 2018
healther
approved these changes
Dec 28, 2018
| src = os.path.realpath(source_path) | ||
|
|
||
| if os.path.isdir(src): | ||
| copy_tree(src, destination_path) |
Contributor
There was a problem hiding this comment.
out of curiosity why doesn't copy handle this distinction?
Member
There was a problem hiding this comment.
I guess I can't think of a reason why we couldn't redesign copy and install to support both files and directories. It's just a design choice. For comparison, Python's shutil library has separate copy and copytree methods. Python's distutils library also has separate dir_util.copy_tree and file_util.copy_file methods.
scheibelp
force-pushed
the
resource-cache
branch
from
f2050d6 to
f1a269d
Compare
Non-expanded resources were being deleted from the cache on account of two behaviors: * ResourceStage was moving files rather than copying them, and uses "os.path.realpath" to resolve symlinks * CacheFetchStrategy creates a symlink to a cached resource rather than copying it This alters the first behavior: ResourceStage now copies the file rather than moving it.
scheibelp
force-pushed
the
resource-cache
branch
from
f1a269d to
08d0734
Compare
Member
|
Thanks! |
scheibelp
pushed a commit
that referenced
this pull request
Jan 5, 2019
Non-expanded resources were being deleted from the cache on account of two behaviors: * ResourceStage was moving files rather than copying them, and uses "os.path.realpath" to resolve symlinks * CacheFetchStrategy creates a symlink to a cached resource rather than copying it This alters the first behavior: ResourceStage now copies the file rather than moving it.
This was referenced Jan 8, 2019
gonsie
pushed a commit
that referenced
this pull request
Jan 15, 2019
Non-expanded resources were being deleted from the cache on account of two behaviors: * ResourceStage was moving files rather than copying them, and uses "os.path.realpath" to resolve symlinks * CacheFetchStrategy creates a symlink to a cached resource rather than copying it This alters the first behavior: ResourceStage now copies the file rather than moving it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resources can be deleted from the local cache because
ResourceStagemoves its resources to the root stage instead of copying them. If a locally cached resource exists, it is symlinked into the resource stage and then moved out of the local cache. This can be reproduced with the following commands:spack stage sqlite+functions(fetches archive and resource)spack cleanspack stage sqlite+functions(uses locally cached versions)spack cleanspack stage sqlite+functions(fetches resource again)