verify_upstream_unchanged fails on github actions when git repo is unauthenticated #1373

Activity

1. estyrke

   added

   bugSomething isn't working properlySomething isn't working properly

   triagewaiting for initial maintainer reviewwaiting for initial maintainer review

   on Nov 11, 2025
2. codejedi365

   removed

   triagewaiting for initial maintainer reviewwaiting for initial maintainer review

   on Nov 11, 2025
3. codejedi365 commented on Nov 11, 2025

   

   codejedi365

   on Nov 11, 2025

   Last edited by codejedi365

   Contributor

   More actions

   Thank you for reporting your issue as contributing to PSR only helps make PSR better.

   My analysis (request response to my assumptions & diagnosis if I am incorrect):

   • The final error is the most telling as it identifies that you have branch restrictions on main (Repository rule violations found for refs/heads/main...Changes must be made through a pull request.). Branch restrictions are very difficult to work through with PSR's release commit mechanism without some additional configuration. Until recently, I did not have a solution for branch protections until Bypass protected branches with deploy keys #1343 besides what you are doing with a custom PAT and I assume you have bypass restrictions enabled for that token?

   • Estyrke's workflow does not limit or define GITHUB_TOKEN permissions which means it has all write permissions to authorized CI API endpoints. Required permissions for release committing PSR are contents: write. Estyrke sets persist-credentials: false to remove GITHUB_TOKEN from git configuration. This causes git fetch to fail as the git repository does not have credentials to the repository any more and PSR's git fetch does not use a credentialed url as part of fetch, it expects origin the git repository to be authenticated (unlike PSR's git push).

     • [Possible PSR feature patch] If the git fetch used the same mechanism as PSR's git push it would use the custom secrets.SEMANTIC_RELEASE_GITHUB_TOKEN as part of the fetch url and it would have read access to the repository to validate the upstream has not changed prior to the release commit git push. Requires the setting tool.semantic_release.remote.ignore_token_for_push = false for success.

     • [Possible PSR bug] When actions/checkout is provided persist-credentials: true (default), the git fetch used to verify the upstream has not changed succeeds because the git repo has contents: write via the fully-authorized GITHUB_TOKEN. This allows PSR to continue to the git push https://$GITHUB_ACTOR:$TOKEN@$ORIGIN_URL main step and this fails supposedly because of the branch protections on main (Changes must be made through a pull request.). Based on github.py#L477:L487, my expectation is that $TOKEN is SEMANTIC_RELEASE_GITHUB_TOKEN because token is passed in from the configuration and not interpreted from git config --get remote.origin.url which currently has the $GITHUB_TOKEN value. This expectation is either wrong and GITHUB_TOKEN trips the branch protections, or SEMANTIC_RELEASE_GITHUB_TOKEN does not have bypass permissions to push to main. This is also dependent on the value of tool.semantic_release.remote.ignore_token_for_push which should not be an issue as the default is false.

   • Your workflow might not of had a verify_upstream script (as recommended in example workflows) prior to executing PSR in v10.4.1. If so, it was possible that if the Actions workflow prior to the release job took too long, there could of been another push to main. This situation would of caused an upstream collision in the git push of the release commit. PSR's v10.5.2 intent is to eliminate this situation without consumer need for a separate verify_upstream step and minimize the possible collision time window to milliseconds before push, but requires an authenticated git fetch mechanism.

   • Your workflow is not specific on where the repo is checked out. It is unclear if actions/checkout is going to check out the ref or the sha directly. Given there is no branch checkout step included in your workflow, I assume it is checking out main blindly. This is problematic as if there was another commit to main in the duration of the workflow, it is likely a untested change will be released accidentally. PSR recommends setting ref: ${{ github.sha }} on the actions/checkout step which sets the repo in a detached head state but deliberately on the commit that was validated through prior actions test jobs. Then add a step to deliberately check out the correct release branch at that commit sha (ex. git checkout -B ${{ github.ref_name }}).

   • The fetch-depth: 0 parameter is no longer required as of [email protected] since functionality has been added to automatically un-shallow repositories as long as the git repository is credentialed. This will also fail in your workflow as the authorized git access to the remote has been severed.

   • Your semantic release configuration still has some old configuration options that no longer exist. Specifically, commit_version_number, upload_to_pypi, upload_to_repository, and version_source do not exist. The setting upload_to_release has been moved to tool.semantic_release.publish.upload_to_vcs_release. This likely occurred in the transition from v7 to v8 and unfortunately PSR's config parser does not alert you of these mismatches (an update is under development). Recommend running the following command to cross-check your desired results with the new defaults:

     semantic-release generate-config --pyproject

   ---

   After this analysis, I think we can make your situation work if you wanted to change the CI implementation significantly (and maybe try deploy keys as that seems the most ideal for security GitHub solution) but I likely will also need to implement the patch to use an authenticated git fetch deliberately. I just didn't have this in mind and honestly have been confused at the use case for the current git push with an authenticated url rather than just origin. After seeing your workflow, now I see the situation.
4. codejedi365

   added

   awaiting-replyWaiting for responseWaiting for response

   on Nov 11, 2025
5. estyrke commented on Nov 12, 2025

   

   estyrke

   on Nov 12, 2025

   Author

   More actions

   • I assume you have bypass restrictions enabled for that token?

   Yes, correct.

   • [Possible PSR bug] When actions/checkout is provided persist-credentials: true (default), the git fetch used to verify the upstream has not changed succeeds because the git repo has contents: write via the fully-authorized GITHUB_TOKEN. This allows PSR to continue to the git push https://$GITHUB_ACTOR:$TOKEN@$ORIGIN_URL main step and this fails supposedly because of the branch protections on main (Changes must be made through a pull request.). Based on github.py#L477:L487, my expectation is that $TOKEN is SEMANTIC_RELEASE_GITHUB_TOKEN because token is passed in from the configuration and not interpreted from git config --get remote.origin.url which currently has the $GITHUB_TOKEN value. This expectation is either wrong and GITHUB_TOKEN trips the branch protections, or SEMANTIC_RELEASE_GITHUB_TOKEN does not have bypass permissions to push to main. This is also dependent on the value of tool.semantic_release.remote.ignore_token_for_push which should not be an issue as the default is false.

   I am a bit confused by this as well. Even if "persist-credentials" is true, PSR should reasonably use the provided GH_TOKEN for the push, but apparently it gets overridden somehow (due to .insteadOf magic added to the git config by actions/checkout perhaps?)

   • Your workflow is not specific on where the repo is checked out. It is unclear if actions/checkout is going to check out the ref or the sha directly. Given there is no branch checkout step included in your workflow, I assume it is checking out main blindly. This is problematic as if there was another commit to main in the duration of the workflow, it is likely a untested change will be released accidentally. PSR recommends setting ref: ${{ github.sha }} on the actions/checkout step which sets the repo in a detached head state but deliberately on the commit that was validated through prior actions test jobs. Then add a step to deliberately check out the correct release branch at that commit sha (ex. git checkout -B ${{ github.ref_name }}).

   Ouch! I have always assumed that actions/checkout checks out the ref that triggered the workflow...

   Actually, this seems to be what it does according to the readme (emphasis mine):

   Only a single commit is fetched by default, for the ref/SHA that triggered the workflow. Set fetch-depth: 0 to fetch all history for all branches and tags. Refer here to learn which commit $GITHUB_SHA points to for different events.

   Specifically, we use the 'workflow_call' trigger, which is supposed to use the same GITHUB_SHA as the caller, which in turn uses the push trigger.

   After this analysis, I think we can make your situation work if you wanted to change the CI implementation significantly (and maybe try deploy keys as that seems the most ideal for security GitHub solution) but I likely will also need to implement the patch to use an authenticated git fetch deliberately. I just didn't have this in mind and honestly have been confused at the use case for the current git push with an authenticated url rather than just origin. After seeing your workflow, now I see the situation.

   Ideally both our current workflow (with non-persisted credentials and provided GH_TOKEN) and the one with persisted credentials, correctly applying the provided GH_TOKEN to the push command should work, but I'll look into updating our workflow according to best practices.
6. estyrke commented on Nov 12, 2025

   

   estyrke

   on Nov 12, 2025

   Author

   More actions

   As an aside, I don't really see what the verify_upstream_unchanged functionality is supposed to accomplish. Wouldn't the push itself fail if there are new commits on the branch?
7. codejedi365 commented on Nov 13, 2025

   

   codejedi365

   on Nov 13, 2025

   Contributor

   More actions

   As an aside, I don't really see what the verify_upstream_unchanged functionality is supposed to accomplish. Wouldn't the push itself fail if there are new commits on the branch?

   Yes, there should be a failure, but I'm not going to be parsing the error messages from github to help people determine why there was a failure. A few people have also asked for a force push option to get around the branch protections. In that case, there would be no error only destructive actions. I felt it was best to add a script beforehand as a recommended course of action and this release I integrated it into PSR so users didn't have to worry about it. This mechanism is a proactive way to detect when an error will occur and provide a descriptive user message of why no release occurred.
8. codejedi365 commented on Nov 13, 2025

   

   codejedi365

   on Nov 13, 2025

   Contributor

   More actions

   • Your workflow is not specific on where the repo is checked out. It is unclear if actions/checkout is going to check out the ref or the sha directly. Given there is no branch checkout step included in your workflow, I assume it is checking out main blindly. This is problematic as if there was another commit to main in the duration of the workflow, it is likely a untested change will be released accidentally. PSR recommends setting ref: ${{ github.sha }} on the actions/checkout step which sets the repo in a detached head state but deliberately on the commit that was validated through prior actions test jobs. Then add a step to deliberately check out the correct release branch at that commit sha (ex. git checkout -B ${{ github.ref_name }}).

   Ouch! I have always assumed that actions/checkout checks out the ref that triggered the workflow...

   Actually, this seems to be what it does according to the readme (emphasis mine):

   Only a single commit is fetched by default, for the ref/SHA that triggered the workflow. Set fetch-depth: 0 to fetch all history for all branches and tags. Refer here to learn which commit $GITHUB_SHA points to for different events.

   Although it says this, it is an incomplete statement. It says ref/SHA. The branch name is a ref and we know it is the branch because otherwise you would have a step that checks out the branch because PSR does not work on a detached head state. Detached head states occur when you git checkout SHA. I also warn you of the exact situation above, because it happened with a release of PSR. I was lucky it did not break anything, but it released a commit that was added to the default branch after the releasing workflow was executing (during its testing phase). So when the two simultaneously running workflows were executing the first arrived at the release job and released the current state of master even though I had a concurrency flag set. They may have updated it in v5 but I'm pretty sure it still exists and it happened to me in v4, in the last 1.5 yrs.

   Specifically, we use the 'workflow_call' trigger, which is supposed to use the same GITHUB_SHA as the caller, which in turn uses the push trigger.

   I don't disagree that the GITHUB_SHA is defined, I'm telling you that checkout does not use it if you leave it undefined.
9. estyrke commented on Nov 13, 2025

   

   estyrke

   on Nov 13, 2025

   Author

   More actions

   It sounded like such an unreasonable behavior that I had to test it: https://github.com/estyrke/test-actions-checkout/actions

   This shows that both for push and workflow_dispatch triggers, the actions/checkout@v4 action checks out the original commit that triggered the workflow even if there are commits added during the workflow run. That is if no ref input is provided, so that the default is used. If you specify ref: main the result may be different.
10. rdctmeconomou commented on Nov 15, 2025

    

    rdctmeconomou

    on Nov 15, 2025

    More actions

    Maybe it's unrelated, but I'm getting a similar error (fatal: could not read Username for 'https://github.com/': No such device or address) for both versions 8.3.0 and 10.5.2 of the python-semantic-release/python-semantic-release action, which I upgraded as part of my troubleshooting:

    The next version is: 2.0.0! 🚀
               INFO     No contents found in                 changelog_writer.py:210
                        PosixPath('/github/workspace/templat                        
                        es'), using default changelog                               
                        template                                                    
               INFO     found 'project.version' in source file contents,  toml.py:69
                        replacing with 2.0.0                                        
    No build command specified, skipping
               INFO     Upstream branch name: origin/main          gitproject.py:379
               INFO     Fetching latest changes from remote        gitproject.py:410
                        'origin'                                                    
               ERROR    Cmd('git') failed due to: exit code(128)   gitproject.py:414
                          cmdline: git fetch -v -- origin                           
                          stderr: 'fatal: could not read Username                   
                        for 'https://github.com/': No such device                    
                        or address'
    

    This worked on September 15 with version 8.3.0:

    The next version is: 1.1.0! 🚀
               INFO     No contents found in                 changelog_writer.py:210
                        PosixPath('/home/runner/work/rdctdev                        
                        -meta/rdctdev-meta/templates'),                             
                        using default changelog template                            
               INFO     found 'project.version' in source file contents,  toml.py:69
                        replacing with 1.1.0                                        
    No build command specified, skipping
    [22:43:19] INFO     Creating release for tag v1.1.0                github.py:252
               INFO     Successfully created release with ID:          github.py:272
                        247457275
    

    I double-checked the commit history and organization audit logs. Aside from me updating actions' version pins (in troubleshooting this issue) and editing a comment, my CI workflow hasn't changed. At the repository level, I've changed several GitHub Actions secrets and variables, but they only affect my deployment workflow. We recently subscribed to the GitHub Team plan, but that was three hours before the last successful execution of the action. I literally have the receipts. We aren't using branch protection rules or anything like that. I'm at a loss as to what might have gone wrong.
11. codejedi365 commented on Nov 15, 2025

    

    codejedi365

    on Nov 15, 2025

    Contributor

    More actions

    @rdctmeconomou, Is your git repository authenticated? Because a git fetch origin was added in v10.5.

    Please provide your release job steps to show how you are setting up the repo prior to the release step.
12. rdctmeconomou commented on Nov 15, 2025

    

    rdctmeconomou

    on Nov 15, 2025

    More actions

    @codejedi365 It doesn't contain anything proprietary, so here's the current workflow definition, which includes the changes I made while troubleshooting this issue, and here's the original version, which worked several months ago but is broken now.

    The project lives in a private repository hosted by GitHub.com, so yes, access to it would require authentication. Note that the lint job uses a GitHub App to access private repositories containing project dependencies, but those credentials get discarded at the end of the job.
13. codejedi365 commented on Nov 15, 2025

    

    codejedi365

    on Nov 15, 2025

    Last edited by codejedi365

    Contributor

    More actions

    @rdctmeconomou, reviewed your ci workflow. Your issue is similar because your git repository is unauthenticated prior to python-semantic-release execution. The fact you checkout the repo in a prior job, the post-checkout step removes the authentication from git at the end of the setup job and separately the cache does not return it when entering the release job. You would also need the patch I discussed above to use the passed github token to perform git fetch.

    To make it work today, you should revert to v10.4.1.

    Other recommendations:

    • You should override your concurrency.cancel-in-progress setting in the release job, you don't want to cancel mid release.
    • Your non-standard use of cache from the main job is interesting and likely for good reason but it is important to consider that cache can be deleted at any point, including between jobs in the workflow, which can cause your workflow to fail. Most workflows use checkout in each job and then transfer generated artifacts via actions/upload-artifact and actions/download-artifact.
    • I have deprecated and archived python-semantic-release/upload-to-gh-release@main quite a while ago. Recommend switching to the new repository to maintain to compatibility of your PSR configuration. The new repo is python-semantic-release/publish-action.
    • When you jump 2 full major versions of PSR, you should review the migration instructions and changelog release notes on our website to prevent from unexpected issues.
    • You should also cross reference your repo actions with the GitHub Actions usage example on our documentation website to make sure you are using the best practices to have consistent and predicable releases.

    ❤️React with ❤️1Reacted by Matthew X. Economou
14. codejedi365 commented on Nov 15, 2025

    

    codejedi365

    on Nov 15, 2025

    Last edited by codejedi365

    Contributor

    More actions

    It sounded like such an unreasonable behavior that I had to test it: https://github.com/estyrke/test-actions-checkout/actions

    This shows that both for push and workflow_dispatch triggers, the actions/checkout@v4 action checks out the original commit that triggered the workflow even if there are commits added during the workflow run. That is if no ref input is provided, so that the default is used. If you specify ref: main the result may be different.

    I see what you did, but I don't think your workflow test is comprehensive enough to test the scenario that could occur. The fact you are in the same job, the provided GITHUB_SHA has no chance in changing. I have tested this when the example I discussed above occurred, and I used a sleep action in a test job and then manually pushed a change from my local machine before it moved on to a separate release job and at that point it did fail in a reproducible pattern. But, continue at your own risk.
15. 2 remaining items

    Load more
16. github-actions

    added

    unresponsive

    on Dec 3, 2025
17. codejedi365

    added

    confirmedPrevent from becoming stalePrevent from becoming stale

    and removed

    awaiting-replyWaiting for responseWaiting for response

    unresponsive

    on Dec 8, 2025
18. python-semantic-release

    deleted a comment from github-actions on Dec 8, 2025
19. codejedi365

    added 4 commits that reference this issue on Dec 13, 2025

    fix(cmd-version): resolve unauthenticated git repo issues for upstrea…

    ...

    This issue will close once commit 36c2199 is merged into the 'master' branch.

    36c2199

    fix(cmd-version): resolve unauthenticated git repo issues for upstrea…

    ...

    This issue will close once commit 676ab78 is merged into the 'master' branch.

    676ab78

    fix(cmd-version): resolve unauthenticated git repo issues for upstrea…

    ...

    This issue will close once commit c316fa9 is merged into the 'master' branch.

    c316fa9

    fix(cmd-version): resolve unauthenticated git repo issues for upstrea…

    ...

    This issue will close once commit 93e168a is merged into the 'master' branch.

    93e168a
20. codejedi365

    closed this as completedin #1388on Dec 14, 2025
21. codejedi365

    added a commit that references this issue on Dec 14, 2025

    fix(cmd-version): resolve unauthenticated git repo issues for upstrea…

    ...

    This issue will close once commit e164f68 is merged into the 'master' branch.

    e164f68
22. codejedi365

    added

    released

    and removed

    confirmedPrevent from becoming stalePrevent from becoming stale

    on Dec 14, 2025
23. codejedi365

    changed the title [-]verify_upstream_unchanged fails on github actions[/-] [+]`verify_upstream_unchanged` fails on github actions when git repo is unauthenticated[/+] on Dec 14, 2025
24. codejedi365 commented on Dec 14, 2025

    

    codejedi365

    on Dec 14, 2025

    Contributor

    More actions

    🎉 This issue has been resolved in v10.5.3 🎉

    You can find more information about this release on the GitHub Releases page.

    If your issue persists, please reply here and we will re-open the issue. If you receive a new error, please open a new issue and reference this issue there.

    ❤️React with ❤️1Reacted by Emil Styrke