tests: more targeted caching for devtools build #13540
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
brendankenny
reviewed
Jan 5, 2022
.github/workflows/devtools.yml
Outdated
|
|
||
| - name: Generate cache hash | ||
| run: bash $GITHUB_WORKSPACE/lighthouse/.github/scripts/generate-devtools-hash.sh > cdt-test-hash.txt | ||
| # Caches are invalidated by GH after 7 days, so that's the max time between testing |
Contributor
There was a problem hiding this comment.
FWIW, this page says "GitHub will remove any cache entries that have not been accessed in over 7 days", which is slightly different.
Collaborator
Author
There was a problem hiding this comment.
good catch, let's just do this bit ourselves.
brendankenny
reviewed
Jan 5, 2022
.github/workflows/devtools.yml
Outdated
| # 3) every change to file in Lighthouse repo important to running these tests. | ||
| # | ||
| # The number is how many times this hash key was manually updated to break the cache. | ||
| key: ${{ runner.os }}-2-${{ hashFiles('cdt-test-hash.txt') }} |
Contributor
There was a problem hiding this comment.
to avoid all the sh business, could run
- name: Set week of the year
run: echo "WEEK_OF_THE_YEAR=$(date +%U)" >> $GITHUB_ENVand this could become
Suggested change
| key: ${{ runner.os }}-2-${{ hashFiles('cdt-test-hash.txt') }} | |
| key: ${{ env.WEEK_OF_THE_YEAR }}-2-${{ hashFiles('cdt-test-hash.txt') }} |
(maybe date could be inlined in the key here but github actions syntax 🤷 )
Collaborator
Author
There was a problem hiding this comment.
good call on the +%U
Collaborator
Author
There was a problem hiding this comment.
for some reason I prefer mondays tho so +%V :)
adamraine
approved these changes
Jan 5, 2022
Co-authored-by: Adam Raine <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I added some comments explaining the caching more explicitly, then realized that the cache-key generator script was considering files that were irrelevant.
the purpose of this cache is to hold on to content_shell, blink_tools, and devtools frontend repo ... but most importantly, all the stuff in
out/that gets generated duringrun_web_tests.sh. This make subsequent runs build faster, as it only needs to rebuild rules for which Lighthouse-rolled files are a dependency.The files in the cache-key generator script that didn't belong were added for the mistaken reason that "if this file changed, we need to re-run the tests" ... but we want to always run the tests, just throttle how often the DevTools repo updates.
The simplest implementation would be using no cache key, and just have DevTools update every 7 days (GHA cache ttl). Next up is checking for important LH-specific files changing in CDT source. Next up (this PR) includes considering the scripts that setup the test environment / the test files themselves as cache-breaking: most likely, you'd want to see the tests using ToT if you are actively changing them.