fix(github-artifacts): Better pagination and retry on missing artifacts#190
Conversation
GitHub sometimes takes a few minutes until it lists freshly uploaded artifacts on its API. This PR adds a retry mechanism when we cannot find any artifacts for the given revision. It also improves the pagination mechanism to no scan artifacts beyond the revision's commit date to shorten lookup times. Fixes #182.
|
This is a draft because I don't know how to test this without screwing something up. |
|
Tested this locally and it works (at least the happy path). It is hard to test the flaky GitHub scenario so we'll have to wait for that one. |
jan-auer
left a comment
jan-auer
left a comment
There was a problem hiding this comment.
An overall different approach could be to query workflow runs for the release branch and then using the run id query the artifacts for this single run.
Ah, I actually checked that API but didn't look at the parameters to see it allowed filtering based on branch name and trigger. That said I think it would still be a challenge to scan all the runs and get relevant artifacts so our "happy path" would end up making more requests and more complex. However, if we consider tying status checks to artifacts, we can tell Craft to get an artifact from a specific workflow run. This may cause some brittle config as a downside but might be worth exploring. |
This PR reverts #177 as not only it did not address the issue it was intending to fix, it introduced delays in where we actually expect a 404 response. It also caused a weird failure where we threw an error due to a 404 response which should have been ignored: https://github.com/getsentry/publish/runs/2281793805?check_suite_focus=true#step:8:59 The retry issue was resolved via #190.
This PR reverts #177 as not only it did not address the issue it was intending to fix, it introduced delays in where we actually expect a 404 response. It also caused a weird failure where we threw an error due to a 404 response which should have been ignored: https://github.com/getsentry/publish/runs/2281793805?check_suite_focus=true#step:8:59 The retry issue was resolved via #190.
3 participants
GitHub sometimes takes a few minutes until it lists freshly uploaded artifacts on its API. This PR adds a retry mechanism when we cannot find any artifacts for the given revision. It also improves the pagination mechanism to no scan artifacts beyond the revision's commit date to shorten lookup times.
Fixes #182.