etags for index.json invalidation, test coverage#34641
Merged
alalazo merged 8 commits intospack:developfrom Dec 21, 2022
Merged
etags for index.json invalidation, test coverage#34641alalazo merged 8 commits intospack:developfrom
alalazo merged 8 commits intospack:developfrom
Conversation
Implement an alternative strategy to do index.json invalidation. The current approach of pairs of index.json / index.json.hash is problematic because it leads to races. The standard solution for cache invalidation is etags, which are supported by both http and s3 protocols, which allows one to do conditional fetches. This PR implements that for the http/https schemes. It should also work for s3 schemes, but that requires other prs to be merged. Also it improves unit tests for index.json fetches.
spackbot-app
bot
added
binary-packages
core
Member
Author
|
@blue42u the stuff you're overhauling has really bad test coverage, so I'm inclined to get this in first. |
blue42u
suggested changes
Dec 21, 2022
blue42u
approved these changes
Dec 21, 2022
Member
Author
|
It would be nice if Gitlab CI would work here (and everywhere else too) |
Member
|
@spackbot run pipeline |
|
I've started that pipeline for you! |
Member
Author
|
@spackbot rebuild everything |
|
I've started that pipeline for you! |
Member
Author
|
@spackbot run pipeline |
|
I've started that pipeline for you! |
stephenmsachs
pushed a commit
to stephenmsachs/spack
that referenced
this pull request
Jan 3, 2023
Implement an alternative strategy to do index.json invalidation. The current approach of pairs of index.json / index.json.hash is problematic because it leads to races. The standard solution for cache invalidation is etags, which are supported by both http and s3 protocols, which allows one to do conditional fetches. This PR implements that for the http/https schemes. It should also work for s3 schemes, but that requires other prs to be merged. Also it improves unit tests for index.json fetches.
RikkiButler20
pushed a commit
to RikkiButler20/spack
that referenced
this pull request
Jan 24, 2023
Implement an alternative strategy to do index.json invalidation. The current approach of pairs of index.json / index.json.hash is problematic because it leads to races. The standard solution for cache invalidation is etags, which are supported by both http and s3 protocols, which allows one to do conditional fetches. This PR implements that for the http/https schemes. It should also work for s3 schemes, but that requires other prs to be merged. Also it improves unit tests for index.json fetches.
amd-toolchain-support
pushed a commit
to amd-toolchain-support/spack
that referenced
this pull request
Feb 16, 2023
Implement an alternative strategy to do index.json invalidation. The current approach of pairs of index.json / index.json.hash is problematic because it leads to races. The standard solution for cache invalidation is etags, which are supported by both http and s3 protocols, which allows one to do conditional fetches. This PR implements that for the http/https schemes. It should also work for s3 schemes, but that requires other prs to be merged. Also it improves unit tests for index.json fetches.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implement an alternative strategy to do index.json invalidation.
The current approach of pairs of index.json / index.json.hash is
problematic because it leads to races.
The standard solution for cache invalidation is etags, which are
supported by both http and s3 protocols, which allows one to do
conditional fetches.
This PR implements that for the http/https schemes. It should also work
for s3 schemes, but that requires other prs to be merged.
Also it improves unit tests for index.json fetches.