Skip to content

Deploy to PyPI using trusted publishing#603

Merged
hugovk merged 1 commit intoultrajson:mainfrom
hugovk:trusted-publishers
Aug 17, 2023
Merged

Deploy to PyPI using trusted publishing#603
hugovk merged 1 commit intoultrajson:mainfrom
hugovk:trusted-publishers

Conversation

@hugovk
Copy link
Copy Markdown
Member

@hugovk hugovk commented Aug 2, 2023

Follow on from #594:

We're using https://github.com/pypa/gh-action-pypi-publish directly to upload sdists, but twine for the wheels. I've switched the wheels to https://github.com/pypa/gh-action-pypi-publish as well, because I'd like to improve security by switching over to using a "trusted publisher" that makes use of short-lived tokens instead of a persistent one (which lives on my personal account) and is saved in repo secrets.

More info:

But to configure this on PyPI, I'd need "owner" access rather than "maintainer".

I emailed Jonas Tarnstrom, one of the former maintainers, and he's given me access so I've now set up trusted publishing for this project at https://pypi.org/manage/project/ujson/settings/publishing/:

image

We should have the same thing on TestPyPI too: @segfault, I think you initially set up https://test.pypi.org/project/ujson/? Please could you give me owner access there too? Thanks!

TODO

@codecov-commenter
Copy link
Copy Markdown

Codecov Report

Merging #603 (b5621e8) into main (a179fb5) will not change coverage.
The diff coverage is n/a.

❗ Current head b5621e8 differs from pull request most recent head 2080d73. Consider uploading reports for the commit 2080d73 to get more accurate results

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

@@           Coverage Diff           @@
##             main     #603   +/-   ##
=======================================
  Coverage   91.66%   91.66%           
=======================================
  Files           6        6           
  Lines        1944     1944           
=======================================
  Hits         1782     1782           
  Misses        162      162           

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@hugovk hugovk added the changelog: Added For new features label Aug 2, 2023
@hugovk
Copy link
Copy Markdown
Member Author

hugovk commented Aug 17, 2023

Update: I've sent an email to @segfault.

@segfault
Copy link
Copy Markdown
Collaborator

Can do

@segfault
Copy link
Copy Markdown
Collaborator

Done. Sorry for the lag. Missed the initial email notification.

@hugovk
Copy link
Copy Markdown
Member Author

hugovk commented Aug 17, 2023

No problem, and thank you!

@hugovk
Copy link
Copy Markdown
Member Author

hugovk commented Aug 17, 2023

@hugovk hugovk marked this pull request as ready for review August 17, 2023 14:50
@hugovk hugovk merged commit e9e1e6c into ultrajson:main Aug 17, 2023
@hugovk hugovk deleted the trusted-publishers branch August 17, 2023 14:51
@hugovk
Copy link
Copy Markdown
Member Author

hugovk commented Aug 17, 2023

  • Merge, check deploys to TestPyPI

After updating cibiuldwheel (#604), it successfully uploaded to TestPyPI via trusted publishing 🚀

So I've also:

So we should all be set for whenever we next want a prod release.

@webknjaz
Copy link
Copy Markdown

@hugovk FYI building in the same job as uploading opens up a possibility for privilege escalation through build dependency poisoning, that's why it's recommended to build in a separate job. Besides, it would be useful to build the platform-specific wheels from sdist and not from a Git checkout.

@bwoodsend
Copy link
Copy Markdown
Collaborator

What privilege would be higher than poisoning the wheels we build?

it would be useful to build the platform-specific wheels from sdist and not from a Git checkout

What makes you say that?

@webknjaz
Copy link
Copy Markdown

What privilege would be higher than poisoning the wheels we build?

GitHub's OIDC token can eventually let someone access services other, than PyPI. That's why, while working on the secretless publishing in my pypi-publish action during the private beta, we wanted explicitly ask that people separate the jobs. My PyPUG guide caught up with that recommendation a bit later: https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/.

it would be useful to build the platform-specific wheels from sdist and not from a Git checkout

What makes you say that?

That's kinda tribal knowledge, partially. But essentially, that's what pip install does when installing from sdist.

If you don't build from sdist, you may end up shipping a tarball that doesn't contain all the necessary bits for building+install wheels from it. What's present in Git isn't necessarily a part of sdist.

Also, this helps make sdists usable by the downstreams. Distro packagers mostly use sdists from PyPI to repackage RPMs and similar, treating them as the buildable source.

cibuildwheel supports building from sdist natively, by the way, to support this specific use-case.

Additionally, I have a GitHub Action to support using sdists in CI instead of a Git checkout, to improve the downstream compatibility even more — https://github.com/marketplace/actions/checkout-python-sdist.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog: Added For new features

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants