CLI tools defined in 'build_requires' not available during dependency install #8628
Description
- Poetry version: 1.7.0
- Python version: 3.10
- OS version and name: Debian GNU/Linux 12 (bookworm)
- pyproject.toml: https://github.com/greschd/poetry_bug_repro/blob/main/poetry_project/pyproject.toml
Issue
When installing a poetry project with dependencies which require a CLI tool (defined in their pyproject.toml [build-system]requires section), the build fails since the CLI tool is not available when the dependency is built. Building the dependency directly using python -m build works.
The following repository contains a minimal reproduction: https://github.com/greschd/poetry_bug_repro The CLI tool chosen is pre-commit==2.3.0. [1]
Steps to reproduce:
Option 1 (using Docker):
git clone https://github.com/greschd/poetry_bug_repro.gitcd poetry_bug_reprodocker build .
This builds the failing_build target, which errors out with the traceback below (pre-commit does not exist). To see that python -m build works (in the same directory):
docker build --target=working_build .
Option 2 (direct):
git clone https://github.com/greschd/poetry_bug_repro.gitcd poetry_bug_repro/poetry_projectpoetry install
This will either produce the same traceback as below, or complain that the pre-commit version is wrong (if pre-commit is available in the wider environment).
Full log:
> [failing_build 6/6] RUN poetry install -vvv:
1.220 Creating virtualenv poetry-project-2nQEdy2R-py3.10 in /root/.cache/pypoetry/virtualenvs
1.552 Using virtualenv: /root/.cache/pypoetry/virtualenvs/poetry-project-2nQEdy2R-py3.10
1.689 Installing dependencies from lock file
1.692
1.692 Finding the necessary packages for the current system
1.697
1.697 Package operations: 1 install, 0 updates, 0 removals
1.697
1.698 • Installing dependency-project (0.2.0 /dependency_project)
6.914
6.914 Stack trace:
6.914
6.914 6 /usr/local/lib/python3.10/site-packages/poetry/installation/executor.py:269 in _execute_operation
6.958 267│
6.958 268│ try:
6.958 → 269│ result = self._do_execute_operation(operation)
6.958 270│ except EnvCommandError as e:
6.958 271│ if e.e.returncode == -2:
6.958
6.959 5 /usr/local/lib/python3.10/site-packages/poetry/installation/executor.py:379 in _do_execute_operation
6.992 377│ return 0
6.992 378│
6.992 → 379│ result: int = getattr(self, f"_execute_{method}")(operation)
6.992 380│
6.992 381│ if result != 0:
6.992
6.992 4 /usr/local/lib/python3.10/site-packages/poetry/installation/executor.py:504 in _execute_install
7.028 502│
7.028 503│ def _execute_install(self, operation: Install | Update) -> int:
7.028 → 504│ status_code = self._install(operation)
7.028 505│
7.028 506│ self._save_url_reference(operation)
7.028
7.029 3 /usr/local/lib/python3.10/site-packages/poetry/installation/executor.py:536 in _install
7.063 534│ archive = self._prepare_archive(operation)
7.063 535│ elif package.source_type == "directory":
7.063 → 536│ archive = self._prepare_archive(operation)
7.063 537│ cleanup_archive = True
7.063 538│ elif package.source_type == "url":
7.063
7.063 2 /usr/local/lib/python3.10/site-packages/poetry/installation/executor.py:608 in _prepare_archive
7.097 606│ self._populate_hashes_dict(archive, package)
7.097 607│
7.097 → 608│ return self._chef.prepare(
7.097 609│ archive, editable=package.develop, output_dir=output_dir
7.097 610│ )
7.097
7.098 1 /usr/local/lib/python3.10/site-packages/poetry/installation/chef.py:121 in prepare
7.109 119│ if archive.is_dir():
7.109 120│ destination = output_dir or Path(tempfile.mkdtemp(prefix="poetry-chef-"))
7.110 → 121│ return self._prepare(archive, destination=destination, editable=editable)
7.110 122│
7.110 123│ return self._prepare_sdist(archive, destination=output_dir)
7.110
7.110 ChefBuildError
7.110
7.110 Backend subprocess exited when trying to invoke get_requires_for_build_wheel
7.110
7.110 Traceback (most recent call last):
7.110 File "/usr/local/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
7.110 main()
7.110 File "/usr/local/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 335, in main
7.110 json_out['return_val'] = hook(**hook_input['kwargs'])
7.110 File "/usr/local/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
7.110 return hook(config_settings)
7.110 File "/tmp/tmps0obq4kj/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 355, in get_requires_for_build_wheel
7.110 return self._get_build_requires(config_settings, requirements=['wheel'])
7.110 File "/tmp/tmps0obq4kj/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 325, in _get_build_requires
7.110 self.run_setup()
7.110 File "/tmp/tmps0obq4kj/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 507, in run_setup
7.110 super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
7.110 File "/tmp/tmps0obq4kj/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 341, in run_setup
7.110 exec(code, locals())
7.110 File "<string>", line 5, in <module>
7.110 File "/usr/local/lib/python3.10/subprocess.py", line 421, in check_output
7.110 return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
7.110 File "/usr/local/lib/python3.10/subprocess.py", line 503, in run
7.110 with Popen(*popenargs, **kwargs) as process:
7.110 File "/usr/local/lib/python3.10/subprocess.py", line 971, in __init__
7.110 self._execute_child(args, executable, preexec_fn, close_fds,
7.110 File "/usr/local/lib/python3.10/subprocess.py", line 1863, in _execute_child
7.110 raise child_exception_type(errno_num, err_msg, err_filename)
7.110 FileNotFoundError: [Errno 2] No such file or directory: 'pre-commit'
7.110
7.110
7.110 at /usr/local/lib/python3.10/site-packages/poetry/installation/chef.py:166 in _prepare
7.122 162│
7.122 163│ error = ChefBuildError("\n\n".join(message_parts))
7.122 164│
7.122 165│ if error is not None:
7.122 → 166│ raise error from None
7.122 167│
7.122 168│ return path
7.122 169│
7.122 170│ def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:
7.123
7.123 Note: This error originates from the build backend, and is likely not a problem with poetry but with dependency-project (0.2.0 /dependency_project) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "dependency-project @ file:///dependency_project"'.
7.123
[1] This is only for demonstration purposes, it could be any CLI tool. A somewhat old version of pre-commit is used to make it less likely to already exist in the environment.