Initial support for Poetry was added in #7, and included support for bootstrapping Poetry and then using it to install app dependencies.
This issue is for adding support for controlling the Python version via the tool.poetry.dependencies.python TOML field in pyproject.toml when using Poetry. This will be in addition to the existing ability to use runtime.txt, and the planned ability to use .python-version (see #6).
An example Poetry config created using poetry init with when using Poetry 1.8.3 and Python 3.11 contains:
[tool.poetry.dependencies]
python = "^3.11"
However, the next release of Poetry is due to change the default constraint marker from ^ to >= (see python-poetry/poetry#9558), which will give:
[tool.poetry.dependencies]
python = ">=3.11"
It's worth noting that:
- Poetry's dependency syntax doesn't use PEP-440 style versioning, but instead something closer to the Node.js style semver syntax. This means it differs from the syntax used in requirements files and in other non-Poetry parts of
pyproject.toml (including the project.requires-python field, which is what uv uses), which makes things more complicated both for the buildpack and for user UX (for example both syntaxes have a specifier that includes the tilde character, but they do drastically different things for some edge cases). See: https://python-poetry.org/docs/dependency-specification/
- The Python version used in the output of
poetry init uses the version of the local Python version to set the minimum Python version.
- If the
python field isn't specified at all, Poetry defaults to a wide range (currently >=2.7,<2.8 || >=3.4), which then breaks installing any packages that have a smaller compatibility range than that (ie: any package that only supports Python 3) - since Poetry treats its python field as a "this project must be compatible with all of these Python versions" value. As such, omitting python really isn't viable for users - so it's going to be set most of the time.
- Both the
"^3.11" and ">=3.11" forms allow higher major versions of Python such as 3.12 or 3.13, which can include breaking changes. Whilst supporting a range of versions makes sense for a library, for applications these unbounded ranges can cause issues (as we've seen with the Node.js buildpacks over the years), and so ideally we wouldn't want apps to use these forms.
All of the above means that adding support for tool.poetry.dependencies.python is going to mean making compromises in one area or another sadly, since we either have to:
- Explicitly support "unsafe" version ranges such as those above. (Pros: Compatibility with the default output of
poetry init. Cons: Breaking changes when new major Python versions are released + encourages environment drift between CNB and local development environments - and even from one developer's machine to another.)
- Ignore the
tool.poetry.dependencies.python field (either completely, or perhaps only if it uses an unsafe range) and instead install the buildpack's curated default Python version. (Pros/cons: Pretty much the same as (1), apart from the curated Python version perhaps being marginally more compatible with packages in the wild, since we wait a couple of months before making new Python versions the default.)
- Partially support "unsafe" ranges, by using the lower bound of the range as the Python version to be installed, rather than the upper bound. (Pros: Compatibility with the default output of
poetry init and avoids breakage when new Python versions are released. Cons: Doesn't prevent environment drift - developers can still be using a different Python version locally - particularly as new versions are released, unless they remember to bump the project's minimum Python version.)
- Error on "unsafe" version ranges telling users to change to a stricter range instead (eg
3.11.*). (Pros: Prevents breaking changes when new versions of Python released + prevents environment drift. Cons: First build on any app using Poetry will likely fail, unless they are using a template that already has a stricter range set.)
At the moment I'm leaning towards (4), since:
- Our priority should be encouraging users towards safe patterns, rather than those that can cause hard to debug issues later (we already get support tickets where users using pip+runtime.txt were using different version of Python locally vs on Heroku and blame Heroku saying "it works on my machine")
- Users using a Python package/project manager that uses a lockfile have choosing to use a more advanced tool that offers determinism over short-term convenience. Enforcing that a safe range is used (that forces use of a specific major Python version) would be in line with that.
- We can always make
.python-version take priority over tool.poetry.dependencies.python, and so the error message can say to either adjust the tool.poetry.dependencies.python range or create a .python-version file as an alternative (if they want to keep the wide range in the Poetry config).
- Poetry's
init command supports a --python option, so we could always encourage users to use poetry init --python '3.12.*' in our docs to save them having to fix the default version afterwards.
- We could always advocate for Poetry supporting an "--type app" (or similar) for
poetry init which picks defaults more appropriate for an app vs a library (in addition to it setting a Python version like 3.11.* instead of an unbounded range, it could also set package-mode = false which would avoid all of the other boilerplate)
We'll also want to factor in the pyproject.toml project.requires-python field since that is what uv uses and so we may need to support that too in the future.
See also:
GUS-W-9608268.
Initial support for Poetry was added in #7, and included support for bootstrapping Poetry and then using it to install app dependencies.
This issue is for adding support for controlling the Python version via the
tool.poetry.dependencies.pythonTOML field inpyproject.tomlwhen using Poetry. This will be in addition to the existing ability to useruntime.txt, and the planned ability to use.python-version(see #6).An example Poetry config created using
poetry initwith when using Poetry 1.8.3 and Python 3.11 contains:However, the next release of Poetry is due to change the default constraint marker from
^to>=(see python-poetry/poetry#9558), which will give:It's worth noting that:
pyproject.toml(including theproject.requires-pythonfield, which is whatuvuses), which makes things more complicated both for the buildpack and for user UX (for example both syntaxes have a specifier that includes the tilde character, but they do drastically different things for some edge cases). See: https://python-poetry.org/docs/dependency-specification/poetry inituses the version of the local Python version to set the minimum Python version.pythonfield isn't specified at all, Poetry defaults to a wide range (currently>=2.7,<2.8 || >=3.4), which then breaks installing any packages that have a smaller compatibility range than that (ie: any package that only supports Python 3) - since Poetry treats itspythonfield as a "this project must be compatible with all of these Python versions" value. As such, omittingpythonreally isn't viable for users - so it's going to be set most of the time."^3.11"and">=3.11"forms allow higher major versions of Python such as 3.12 or 3.13, which can include breaking changes. Whilst supporting a range of versions makes sense for a library, for applications these unbounded ranges can cause issues (as we've seen with the Node.js buildpacks over the years), and so ideally we wouldn't want apps to use these forms.All of the above means that adding support for
tool.poetry.dependencies.pythonis going to mean making compromises in one area or another sadly, since we either have to:poetry init. Cons: Breaking changes when new major Python versions are released + encourages environment drift between CNB and local development environments - and even from one developer's machine to another.)tool.poetry.dependencies.pythonfield (either completely, or perhaps only if it uses an unsafe range) and instead install the buildpack's curated default Python version. (Pros/cons: Pretty much the same as (1), apart from the curated Python version perhaps being marginally more compatible with packages in the wild, since we wait a couple of months before making new Python versions the default.)poetry initand avoids breakage when new Python versions are released. Cons: Doesn't prevent environment drift - developers can still be using a different Python version locally - particularly as new versions are released, unless they remember to bump the project's minimum Python version.)3.11.*). (Pros: Prevents breaking changes when new versions of Python released + prevents environment drift. Cons: First build on any app using Poetry will likely fail, unless they are using a template that already has a stricter range set.)At the moment I'm leaning towards (4), since:
.python-versiontake priority overtool.poetry.dependencies.python, and so the error message can say to either adjust thetool.poetry.dependencies.pythonrange or create a.python-versionfile as an alternative (if they want to keep the wide range in the Poetry config).initcommand supports a--pythonoption, so we could always encourage users to usepoetry init --python '3.12.*'in our docs to save them having to fix the default version afterwards.poetry initwhich picks defaults more appropriate for an app vs a library (in addition to it setting a Python version like3.11.*instead of an unbounded range, it could also setpackage-mode = falsewhich would avoid all of the other boilerplate)We'll also want to factor in the
pyproject.tomlproject.requires-python field since that is what uv uses and so we may need to support that too in the future.See also:
.python-versionfile #6requires-pythonastral-sh/uv#7429GUS-W-9608268.