Feature Request / Improvement

Starting an issue to gather feedback on providing nightly builds for pyiceberg. Resolves #734.
Thanks @syun64 for the pointers and feedback.

PyIceberg Release Process

The current release process as documented in How to release.
To publish a release candidate (RC) to the public,

• Tag and sign Major/Minor release via git, push to the apache branch
• Build artifacts with Github action
  • Python Release Github action
  • Outputs release-main artifact, available for download
• Upload to Apache SVN
  • Both the source distribution (sdist) and the binary distributions (wheels) need to be published for the RC
  • Generate SVN hash for artifacts, sign with gpg
  • https://dist.apache.org/repos/dist/dev/iceberg/
• Upload to PyPi
  • Download artifact from Python release Github action
  • twine upload

Proposed Nightly Build Process

Goals

• Only PyPi is needed, can skip SVN
• Automate nightly build. Using cron-based Github Action
• Automate upload to PyPi. Using Github Action to push directly to PyPi
• Make sure the PyPi package is uploaded as pre-release/development versions
• Make nightly build installable via pip install pyiceberg --pre, preferred.
  • Alternatively, install via a new nightly package, i.e. pyiceberg-nightly

Reference

• https://medium.com/@blackary/publishing-a-python-package-from-github-to-pypi-in-2024-a6fb8635d45d
• https://github.com/marketplace/actions/pypi-publish