Add GCS Bucket Mirrors#24422
Closed
fluidnumerics-joe wants to merge 24 commits intospack:developfrom
FluidNumerics:develop
Closed
Add GCS Bucket Mirrors#24422fluidnumerics-joe wants to merge 24 commits intospack:developfrom FluidNumerics:develop
fluidnumerics-joe wants to merge 24 commits intospack:developfrom
FluidNumerics:develop
Conversation
* Needed to implement workaround on compute engine to create credentials necessary for creating signed url. See https://gist.github.com/jezhumble/91051485db4462add82045ef9ac2a0ec * Corrected issue with blob list - prefix argument was needed to properly list storage blobs
* The signed_url generation is replaced with using a blob.path() (for testing). Some organizations do not allow for service accounts or users to created signed urls for temporary resource accces.
This commit updates the GCS support to be more inline with how the S3 implementation works.
* vermin validation was failing complaining that the target version 2.6 was not met
Member
|
Superseded by #26382. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request contains changes to support Google Cloud Storage buckets as mirrors, meant for hosting Spack build-caches. This feature is beneficial for folks that are running infrastructure on Google Cloud Platform. On public cloud systems, resources are ephemeral and in many cases, installing compilers, MPI flavors, and user packages from scratch takes up considerable time.
Giving users the ability to host a Spack mirror that can store build caches in GCS buckets offers a clean solution for reducing application rebuilds for Google Cloud infrastructure.