Repository navigation
Closed
Labels
bugSomething isn't workingSomething isn't working
Description
What happened?
Artifacts sometimes fail to upload, due to github API failures.
This is similar to the issue reported in #2506. AFAICT, it's an issue on github's end. Here's a larger log of output from my build running in a github action:
• uploading to release file=dist/checksums.txt name=checksums.txt
• failed to upload artifact, will retry artifact=checksums.txt error=POST https://uploads.github.com/repos/jittering/traefik-kop/releases/54810593/assets?name=checksums.txt: 502 Error uploading to https://objects-origin.githubusercontent.com/github-production-release-asset-2e65be: 500 [] try=1
• uploading to release file=dist/checksums.txt name=checksums.txt
• failed to upload artifact, will retry artifact=checksums.txt error=POST https://uploads.github.com/repos/jittering/traefik-kop/releases/54810593/assets?name=checksums.txt: 422 Validation Failed [{Resource:ReleaseAsset Field:name Code:already_exists Message:}] try=2
• uploading to release file=dist/traefik-kop_0.8.1_Linux_arm64.tar.gz name=traefik-kop_0.8.1_Linux_arm64.tar.gz
• failed to upload artifact, will retry artifact=traefik-kop_0.8.1_Linux_arm64.tar.gz error=POST https://uploads.github.com/repos/jittering/traefik-kop/releases/54810593/assets?name=traefik-kop_0.8.1_Linux_arm64.tar.gz: 502 Error uploading to https://objects-origin.githubusercontent.com/github-production-release-asset-2e65be: 500 [] try=1
• uploading to release file=dist/traefik-kop_0.8.1_Linux_arm64.tar.gz name=traefik-kop_0.8.1_Linux_arm64.tar.gz
• failed to upload artifact, will retry artifact=traefik-kop_0.8.1_Linux_arm64.tar.gz error=POST https://uploads.github.com/repos/jittering/traefik-kop/releases/54810593/assets?name=traefik-kop_0.8.1_Linux_arm64.tar.gz: 422 Validation Failed [{Resource:ReleaseAsset Field:name Code:already_exists Message:}] try=2
• uploading to release file=dist/traefik-kop_0.8.1_Linux_x86_64.tar.gz name=traefik-kop_0.8.1_Linux_x86_64.tar.gz
• uploading to release file=dist/traefik-kop_0.8.1_Linux_armv6.tar.gz name=traefik-kop_0.8.1_Linux_armv6.tar.gz
• failed to upload artifact, will retry artifact=traefik-kop_0.8.1_Linux_armv6.tar.gz error=POST https://uploads.github.com/repos/jittering/traefik-kop/releases/54810593/assets?name=traefik-kop_0.8.1_Linux_armv6.tar.gz: 502 Error uploading to https://objects-origin.githubusercontent.com/github-production-release-asset-2e65be: 500 [] try=1
• uploading to release file=dist/traefik-kop_0.8.1_Linux_armv6.tar.gz name=traefik-kop_0.8.1_Linux_armv6.tar.gz
• failed to upload artifact, will retry artifact=traefik-kop_0.8.1_Linux_armv6.tar.gz error=Post "https://uploads.github.com/repos/jittering/traefik-kop/releases/54810593/assets?name=traefik-kop_0.8.1_Linux_armv6.tar.gz": http2: Transport: cannot retry err [stream error: stream ID 11; REFUSED_STREAM; received from peer] after Request.Body was written; define Request.GetBody to avoid this error try=2
• uploading to release file=dist/traefik-kop_0.8.1_Linux_armv6.tar.gz name=traefik-kop_0.8.1_Linux_armv6.tar.gz
• failed to upload artifact, will retry artifact=traefik-kop_0.8.1_Linux_armv6.tar.gz error=Post "https://uploads.github.com/repos/jittering/traefik-kop/releases/54810593/assets?name=traefik-kop_0.8.1_Linux_armv6.tar.gz": http2: Transport: cannot retry err [stream error: stream ID 13; REFUSED_STREAM; received from peer] after Request.Body was written; define Request.GetBody to avoid this error try=3
• uploading to release file=dist/traefik-kop_0.8.1_Linux_armv6.tar.gz name=traefik-kop_0.8.1_Linux_armv6.tar.gz
• failed to upload artifact, will retry artifact=traefik-kop_0.8.1_Linux_armv6.tar.gz error=POST https://uploads.github.com/repos/jittering/traefik-kop/releases/54810593/assets?name=traefik-kop_0.8.1_Linux_armv6.tar.gz: 422 Validation Failed [{Resource:ReleaseAsset Field:name Code:already_exists Message:}] try=4
• uploading to release file=dist/traefik-kop_0.8.1_Darwin_arm64.tar.gz name=traefik-kop_0.8.1_Darwin_arm64.tar.gz
• failed to upload artifact, will retry artifact=traefik-kop_0.8.1_Darwin_arm64.tar.gz error=POST https://uploads.github.com/repos/jittering/traefik-kop/releases/54810593/assets?name=traefik-kop_0.8.1_Darwin_arm64.tar.gz: 502 Error uploading to https://objects-origin.githubusercontent.com/github-production-release-asset-2e65be: 500 [] try=1
• uploading to release file=dist/traefik-kop_0.8.1_Darwin_arm64.tar.gz name=traefik-kop_0.8.1_Darwin_arm64.tar.gz
• failed to upload artifact, will retry artifact=traefik-kop_0.8.1_Darwin_arm64.tar.gz error=Post "https://uploads.github.com/repos/jittering/traefik-kop/releases/54810593/assets?name=traefik-kop_0.8.1_Darwin_arm64.tar.gz": http2: Transport: cannot retry err [stream error: stream ID 25; REFUSED_STREAM; received from peer] after Request.Body was written; define Request.GetBody to avoid this error try=2
• uploading to release file=dist/traefik-kop_0.8.1_Darwin_arm64.tar.gz name=traefik-kop_0.8.1_Darwin_arm64.tar.gz
• failed to upload artifact, will retry artifact=traefik-kop_0.8.1_Darwin_arm64.tar.gz error=Post "https://uploads.github.com/repos/jittering/traefik-kop/releases/54810593/assets?name=traefik-kop_0.8.1_Darwin_arm64.tar.gz": http2: Transport: cannot retry err [stream error: stream ID 29; REFUSED_STREAM; received from peer] after Request.Body was written; define Request.GetBody to avoid this error try=3
• uploading to release file=dist/traefik-kop_0.8.1_Darwin_arm64.tar.gz name=traefik-kop_0.8.1_Darwin_arm64.tar.gz
• failed to upload artifact, will retry artifact=traefik-kop_0.8.1_Darwin_arm64.tar.gz error=Post "https://uploads.github.com/repos/jittering/traefik-kop/releases/54810593/assets?name=traefik-kop_0.8.1_Darwin_arm64.tar.gz": http2: Transport: cannot retry err [stream error: stream ID 33; REFUSED_STREAM; received from peer] after Request.Body was written; define Request.GetBody to avoid this error try=4
• uploading to release file=dist/traefik-kop_0.8.1_Darwin_arm64.tar.gz name=traefik-kop_0.8.1_Darwin_arm64.tar.gz
• failed to upload artifact, will retry artifact=traefik-kop_0.8.1_Darwin_arm64.tar.gz error=POST https://uploads.github.com/repos/jittering/traefik-kop/releases/54810593/assets?name=traefik-kop_0.8.1_Darwin_arm64.tar.gz: 422 Validation Failed [{Resource:ReleaseAsset Field:name Code:already_exists Message:}] try=5
• uploading to release file=dist/traefik-kop_0.8.1_Linux_armv7.tar.gz name=traefik-kop_0.8.1_Linux_armv7.tar.gz
• failed to upload artifact, will retry artifact=traefik-kop_0.8.1_Linux_armv7.tar.gz error=POST https://uploads.github.com/repos/jittering/traefik-kop/releases/54810593/assets?name=traefik-kop_0.8.1_Linux_armv7.tar.gz: 502 Error uploading to https://objects-origin.githubusercontent.com/github-production-release-asset-2e65be: 500 [] try=1
• uploading to release file=dist/traefik-kop_0.8.1_Linux_armv7.tar.gz name=traefik-kop_0.8.1_Linux_armv7.tar.gz
• failed to upload artifact, will retry artifact=traefik-kop_0.8.1_Linux_armv7.tar.gz error=Post "https://uploads.github.com/repos/jittering/traefik-kop/releases/54810593/assets?name=traefik-kop_0.8.1_Linux_armv7.tar.gz": http2: Transport: cannot retry err [stream error: stream ID 23; REFUSED_STREAM; received from peer] after Request.Body was written; define Request.GetBody to avoid this error try=2
• uploading to release file=dist/traefik-kop_0.8.1_Linux_armv7.tar.gz name=traefik-kop_0.8.1_Linux_armv7.tar.gz
• failed to upload artifact, will retry artifact=traefik-kop_0.8.1_Linux_armv7.tar.gz error=Post "https://uploads.github.com/repos/jittering/traefik-kop/releases/54810593/assets?name=traefik-kop_0.8.1_Linux_armv7.tar.gz": http2: Transport: cannot retry err [stream error: stream ID 27; REFUSED_STREAM; received from peer] after Request.Body was written; define Request.GetBody to avoid this error try=3
• uploading to release file=dist/traefik-kop_0.8.1_Linux_armv7.tar.gz name=traefik-kop_0.8.1_Linux_armv7.tar.gz
• failed to upload artifact, will retry artifact=traefik-kop_0.8.1_Linux_armv7.tar.gz error=POST https://uploads.github.com/repos/jittering/traefik-kop/releases/54810593/assets?name=traefik-kop_0.8.1_Linux_armv7.tar.gz: 422 Validation Failed [{Resource:ReleaseAsset Field:name Code:already_exists Message:}] try=4
• uploading to release file=dist/traefik-kop_0.8.1_Darwin_x86_64.tar.gz name=traefik-kop_0.8.1_Darwin_x86_64.tar.gz
• failed to upload artifact, will retry artifact=traefik-kop_0.8.1_Darwin_x86_64.tar.gz error=POST https://uploads.github.com/repos/jittering/traefik-kop/releases/54810593/assets?name=traefik-kop_0.8.1_Darwin_x86_64.tar.gz: 502 Error uploading to https://objects-origin.githubusercontent.com/github-production-release-asset-2e65be: 500 [] try=1
• uploading to release file=dist/traefik-kop_0.8.1_Darwin_x86_64.tar.gz name=traefik-kop_0.8.1_Darwin_x86_64.tar.gz
• failed to upload artifact, will retry artifact=traefik-kop_0.8.1_Darwin_x86_64.tar.gz error=Post "https://uploads.github.com/repos/jittering/traefik-kop/releases/54810593/assets?name=traefik-kop_0.8.1_Darwin_x86_64.tar.gz": http2: Transport: cannot retry err [stream error: stream ID 39; REFUSED_STREAM; received from peer] after Request.Body was written; define Request.GetBody to avoid this error try=2
• uploading to release file=dist/traefik-kop_0.8.1_Darwin_x86_64.tar.gz name=traefik-kop_0.8.1_Darwin_x86_64.tar.gz
• failed to upload artifact, will retry artifact=traefik-kop_0.8.1_Darwin_x86_64.tar.gz error=POST https://uploads.github.com/repos/jittering/traefik-kop/releases/54810593/assets?name=traefik-kop_0.8.1_Darwin_x86_64.tar.gz: 422 Validation Failed [{Resource:ReleaseAsset Field:name Code:already_exists Message:}] try=3
I cleaned up and sorted the output so its easier to read. It seems that in each case, a 5xx error is followed by 422 on each successive retry. In this scenario, it might be necessary to check the list of release assets and/or delete it before retrying.
In my case, it took my several rebuilds (5 tries) with manually deleting the release in between until I was able to get a clean build where every artifact uploaded properly.
How can we reproduce this?
Seems to be a github issue, not sure there will be an easy way to reproduce
goreleaser version
goreleaser version 1.1.0
commit: af4a8642bddb171914ee932bfb5768f102c3516fGoReleaser Check
- goreleaser check shows no errors
Search
- I did search for other open and closed issues before opening this.
Code of Conduct
- I agree to follow this project's Code of Conduct
Additional context
No response
Reacted by Yngvar Kristiansen, Ben Moss and Leo Robinovitch
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working
Activity
[-][Bug]: [/-][+][Bug]: error uploading artifacts during release[/+][-][Bug]: error uploading artifacts during release[/-][+][Bug]: error uploading artifacts to github releases[/+]maxmcd commentedon Dec 9, 2021
I've been getting this error today as well: https://github.com/maxmcd/bramble/runs/4465299050?check_suite_focus=true
jlucktay commentedon Dec 11, 2021
Same here: https://github.com/jlucktay/stack/runs/4491068251?check_suite_focus=true
avelino commentedon Dec 28, 2021
I am having the same problem at prestd
https://github.com/prest/prest/runs/4649770692?check_suite_focus=true#step:15:426
caarlos0 commentedon Dec 28, 2021
when this happens, it usually means that goreleaser ran multiple times against the same tag, as the files are already there...
we should probably drop the retries in this case though
caarlos0 commentedon Dec 28, 2021
hmm, seems to be this google/go-github#2113
avelino commentedon Dec 28, 2021
If the file already exists and you are generating a "new" release, shouldn't you overwrite it?
Thinking that you are consuming an upload api you should delete the existing file and upload it again
caarlos0 commentedon Dec 28, 2021
too destructive, rather not... who knows why the file already exists... it may not even have been created by goreleaser
avelino commentedon Dec 29, 2021
@caarlos0 if we put a parameter to force (aka
--force) the replacement of the file?in my case that solves it, all builds are (and will be) generated "automatically" via pipeline (github actions), if there is one, it was because of some error in the middle of the process, it's ok to overwrite.
caarlos0 commentedon Jan 7, 2022
I don't like the idea...
to me it seems the right thing is to fix this on go-github
chetan commentedon Jan 9, 2022
@caarlos0 Is there a way to re-release a specific tag? In my case, when I ran into this issue, I had to manually delete the release from github and then run goreleaser again.
goreleaser already fails-fast when a release already exists, so if it's gotten to the point of uploading artifacts, it means that the release was created during the current run. If that's the case, I don't see any reason why the file can't simply be deleted and retried automatically.
As you can see from my original problem report, the
already_existserror only returns after the initial522error. My guess is that despite github throwing 522 on the initial upload, the asset object is actually getting created and stored, but an error was thrown at some other point in the process. Thus, any subsequent retries hit thealready_existserror. Obviously the core issue is at github's end, but it can be worked around by simply cleaning up and retrying, possibly with some appropriate exponential backoff between each try.caarlos0 commentedon Jan 10, 2022
it does not...
chetan commentedon Jan 10, 2022
Sorry, you're right, I'm misremembering the sequence from when I was troubleshooting and don't have those logs any more. I just re-ran a release for an existing tag manually and I see this message:
All of the artifact uploads indeed fail immediately (only 422 errors, no 5xx) as they already exist. I think in this case, I agree with what @avelino said above -- if it's a new release, overwrite the file; if updating, do not retry a
422 already_exists, just skip or fail the process.Unless there is some race condition with multiple releases happening simultaneously, I don't see why this wouldn't be a safe option. For extra caution, a flag which users can decide to always include in their build should be a good compromise.
I don't think the linked go-github error linked above is relevant here as that appears to be some internal problem while the
502here is come backing from github's API. This google/go-github#304 issue does mention 502s on upload and the response was (rightly) that it should be handled by the client.caarlos0 commentedon Jan 10, 2022
this happens more often than not, people setup goreleaser on commit to main AND tag, and get concurrent releases... if we allow to override, who knows what the end release will look like
the issue seems to be within go-github google/go-github#2113
caarlos0 commentedon Feb 5, 2022
is this still happening?
avelino commentedon Feb 5, 2022
No reply, I had no more problem (after I reported, ANY more releases were generated using goreleaser)
caarlos0 commentedon Feb 5, 2022
ok, will close this then
thanks all!
vandot commentedon Apr 19, 2022
@caarlos0 it started happening to me from yesterday here is this fixed in some newer versions of
goreleaser?I am using
troian/golang-crossdidn't had time to migrate it to new setup.Edit: switching to latest version of
goreleaser/goreleaser-crossfixed it, false alarmcaarlos0 commentedon Apr 19, 2022
This is an issue on github api, happens from time time to time to some people in some places in some repos, nothing I can do about it.
Best thing is open a support ticket for them to make some noise.
I also opened this discussion a while back: community/community#14341
rfay commentedon Jun 11, 2022
Shoot, I was hoping that goreleaser would have a workaround for this problem at github. I've seen it so very many times- * https://github.com/skx/github-action-publish-binaries/issues/33
The problem is that when you're doing a release and it fails the recovery is tremendously awkward, can result in deleting the release or at least the stuff already uploaded to the release so you can start over again. But from what I hear there's no workaround in goreleaser, right?
caarlos0 commentedon Jun 12, 2022
correct - see #2746 (comment)
github-actions commentedon Dec 5, 2022
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.