Repository navigation
Expose package version as bytewax.__version__ - #637
Conversation
Signed-off-by: Shahzeb Ali <[email protected]>
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
|
Thank you for your PR. It would make sense to me if we could add The values could come from:
These could be stored in a generated file included in wheels and sdists, so users wouldn't need Git installed. @Shahzeb-DataEngineer, could you make these changes, or would it be better to split them into multiple PRs? |
A new build.rs runs as part of the Maturin build and generates pysrc/bytewax/_build_info.py with the version from Cargo.toml, the git revision from `git rev-parse HEAD`, and the dirty state from `git status --porcelain`, all captured at build time. The generated file is packaged into wheels and sdists, so the exact build can be identified from Python without Git installed. bytewax/__init__.py falls back to importlib.metadata and "unknown" values when the generated file is absent. Signed-off-by: Shahzeb Ali <[email protected]>
|
Thank you for the review, and for sketching out the design. I have added How it works:
One consequence worth naming: a build made from an sdist has no |
|
This is starting to look more complicated than I expected :) Thanks for exploring this. Let's track the commit SHA and dirty-tree metadata in separate issues. For this PR, I'd suggest returning to the original Longer term, collecting build information through Cargo and exposing it through Rust/PyO3 could be useful, but getting the build and packaging behaviour right requires more work than I initially anticipated. Let's defer that and keep this PR focused on Thank you! |
The `repo-checks` lint job runs `mypy -p bytewax` on a fresh checkout before the Rust build ever runs, so `bytewax._build_info` (generated by `build.rs` during the Maturin build) does not exist yet and mypy reports `import-not-found`. Tell mypy to ignore the missing module, the same way the config already handles `confluent_kafka` and `jsonpickle`. Also silence a `clippy::map_unwrap_or` warning in `build.rs` so the new build script stays lint-clean. Signed-off-by: Shahzeb Ali <[email protected]>
|
Fixed the lint failure from the last run: |
The build.rs approach turned out more complex than expected. This PR goes back to the original approach: read the installed version from the package metadata with importlib.metadata, without loading the native extension. Tracking the commit SHA and dirty-tree metadata in separate issues. Signed-off-by: Shahzeb Ali <[email protected]>
Use byte-exact upstream content for pyproject.toml, .gitignore and CHANGELOG.md so the PR diff only shows the intended changes. Signed-off-by: Shahzeb Ali <[email protected]>
|
Done, this PR is back to the original importlib.metadata approach: |
Add-on packages such as bytewax-duckdb install modules into bytewax without an __init__.py of their own, which worked while bytewax was a namespace package. With the new __init__.py it is a regular package, and an add-on in another sys.path entry than bytewax, as with an editable install or PYTHONPATH, was no longer found. Extend __path__ with pkgutil.extend_path, as namespace packages did before PEP 420. Of the 17 add-ons on PyPI with code, all import in the same environment, from a --target directory on PYTHONPATH and with an editable bytewax, as on main; without this, 11 of them were not found in the last two cases. The one difference from main: a path added to sys.path after import bytewax is not searched. The new test fails without the change and passes with it and on main. The CHANGELOG entry says that bytewax is now a regular package.
With pysrc/bytewax/__init__.py, ruff treats bytewax as first-party and wants third-party imports in their own group (I001). Re-sort the ten files it reports with ruff check --fix; nothing else changes.
|
Thank you, @Shahzeb-DataEngineer, for this PR, and for following through on the back and forth, which was on me. I decided to finish it with two small commits on top of yours:
You remain the author of the change. Thanks again! |
Resolve the CHANGELOG conflict with bytewax#639 by keeping both entries. Signed-off-by: Mykyta Bozhenko <[email protected]>
Description
Adds a top-level
__init__.pyfor thebytewaxpackage that exposes the installed version asbytewax.__version__, read from the package metadata viaimportlib.metadata:This makes it easy to sanity-check which Bytewax build is actually installed and to report the version when filing issues. If the package metadata is unavailable (for example, running from an unbuilt source checkout),
__version__falls back to"unknown"instead of raising.The
__init__is intentionally lightweight and does not import any submodules, so importing the package stays cheap and does not pull in the compiled extension.Also adds a test asserting
bytewax.__version__matches the installed distribution version, and a changelog entry.Closes #326