Poetry doesn't install build-system.requires from dependencies not using Poetry. #7702
Description
- Poetry version: 1.4.1 and current master 71b3d6e
- Python version: 3.10.6
- OS version and name: Ubuntu 22.04
- pyproject.toml: https://github.com/adamgreig/poetry_build_requires_bug/blob/master/poetry_bug_2/pyproject.toml
- I am on the latest stable Poetry version, installed using a recommended method.
- I have searched the issues of this repo and believe that this is not a duplicate.
- I have consulted the FAQ and blog for any relevant entries or release notes.
- If an exception occurs when executing a command, I executed it again in debug mode (
-vvvoption) and have included the output below.
Issue
Poetry doesn't install entries in [build-system.requires] from dependencies not using Poetry. In my case a dependency has a build system requirement on setuptools_scm, which must be installed for setup.py to run successfully. setuptools_scm is listed in the dependency's [build-system.requires] but isn't installed by Poetry, so the overall installation fails.
Adding setuptools_scm to the dependency's actual dependencies list causes it to be installed first, and then the build succeeds.
I ran into this issue trying to install amaranth but have reproduced the error in a minimal test case here. Run poetry install in poetry_bug_2/ to see the error.
It seems like the issue (thanks to @whitequark for investigating) is here:
poetry/src/poetry/inspection/info.py
Line 599 in 71b3d6e
which should possibly include the extra requirements:
- *PEP517_META_BUILD_DEPS,
+ *(PEP517_META_BUILD_DEPS + PyProjectTOML(path.joinpath("pyproject.toml")).build_system.requires),I'm happy to prepare a PR with that change if you would like.
Full error output is below but the pertinent error is:
Unable to determine package info for path: /home/adam/Projects/tmp/poetry_bug/poetry_bug_1
Fallback egg_info generation failed.
Command ['/tmp/tmpq237jcb0/.venv/bin/python', 'setup.py', 'egg_info'] errored with the following return code 1
Output:
Traceback (most recent call last):
File "/home/adam/Projects/tmp/poetry_bug/poetry_bug_1/setup.py", line 1, in <module>
import setuptools_scm
ModuleNotFoundError: No module named 'setuptools_scm'
Output of `poetry install -vvv`
Using virtualenv: /home/adam/.cache/pypoetry/virtualenvs/poetry-bug-2-MKW0P_o3-py3.10 Updating dependencies Resolving dependencies... 1: fact: poetry-bug-2 is 0.1.0 1: derived: poetry-bug-2 1: Version solving took 7.074 seconds. 1: Tried 1 solutions.Stack trace:
1 ~/.local/lib/python3.10/site-packages/poetry/utils/env.py:1530 in run
1528│ try:
1529│ if input:
→ 1530│ output = subprocess.run(
1531│ cmd,
1532│ stdout=subprocess.PIPE,CalledProcessError
Command '['/tmp/tmpfrovmlq9/.venv/bin/python', '-']' returned non-zero exit status 1.
at /usr/lib/python3.10/subprocess.py:524 in run
520│ # We don't call process.wait() as .exit does that for us.
521│ raise
522│ retcode = process.poll()
523│ if check and retcode:
→ 524│ raise CalledProcessError(retcode, process.args,
525│ output=stdout, stderr=stderr)
526│ return CompletedProcess(process.args, retcode, stdout, stderr)
527│
528│The following error occurred when trying to handle this error:
Stack trace:
3 ~/.local/lib/python3.10/site-packages/poetry/inspection/info.py:601 in get_pep517_metadata
599│ *PEP517_META_BUILD_DEPS,
600│ )
→ 601│ venv.run(
602│ "python",
603│ "-",2 ~/.local/lib/python3.10/site-packages/poetry/utils/env.py:1500 in run
1498│ def run(self, bin: str, *args: str, **kwargs: Any) -> str | int:
1499│ cmd = self.get_command_from_bin(bin) + list(args)
→ 1500│ return self._run(cmd, **kwargs)
1501│
1502│ def run_pip(self, *args: str, **kwargs: Any) -> int | str:1 ~/.local/lib/python3.10/site-packages/poetry/utils/env.py:1782 in _run
1780│ def _run(self, cmd: list[str], **kwargs: Any) -> int | str:
1781│ kwargs["env"] = self.get_temp_environ(environ=kwargs.get("env"))
→ 1782│ return super()._run(cmd, **kwargs)
1783│
1784│ def get_temp_environ(EnvCommandError
Command ['/tmp/tmpfrovmlq9/.venv/bin/python', '-'] errored with the following return code 1
Output:
Traceback (most recent call last):
File "/tmp/tmpfrovmlq9/.venv/lib/python3.10/site-packages/build/init.py", line 494, in _handle_backend
yield
File "/tmp/tmpfrovmlq9/.venv/lib/python3.10/site-packages/build/init.py", line 486, in _call_backend
basename: str = callback(outdir, config_settings, **kwargs)
File "/tmp/tmpfrovmlq9/.venv/lib/python3.10/site-packages/pep517/wrappers.py", line 189, in prepare_metadata_for_build_wheel
return self._call_hook('prepare_metadata_for_build_wheel', {
File "/tmp/tmpfrovmlq9/.venv/lib/python3.10/site-packages/pep517/wrappers.py", line 309, in _call_hook
self._subprocess_runner(
File "/tmp/tmpfrovmlq9/.venv/lib/python3.10/site-packages/build/init.py", line 327, in _runner
self._hook_runner(cmd, cwd, extra_environ)
File "/tmp/tmpfrovmlq9/.venv/lib/python3.10/site-packages/pyproject_hooks/_impl.py", line 71, in quiet_subprocess_runner
check_output(cmd, cwd=cwd, env=env, stderr=STDOUT)
File "/usr/lib/python3.10/subprocess.py", line 420, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 524, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/tmp/build-env-hbzwyj_3/bin/python', '/tmp/tmpfrovmlq9/.venv/lib/python3.10/site-packages/pep517/in_process/_in_process.py', 'prepare_metadata_for_build_wheel', '/tmp/tmpslsegi50']' returned non-zero exit status 1.During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "", line 17, in
File "/tmp/tmpfrovmlq9/.venv/lib/python3.10/site-packages/build/init.py", line 454, in metadata_path
metadata = self.prepare('wheel', output_directory)
File "/tmp/tmpfrovmlq9/.venv/lib/python3.10/site-packages/build/init.py", line 411, in prepare
return self._call_backend(
File "/tmp/tmpfrovmlq9/.venv/lib/python3.10/site-packages/build/init.py", line 485, in _call_backend
with self._handle_backend(hook_name):
File "/usr/lib/python3.10/contextlib.py", line 153, in exit
self.gen.throw(typ, value, traceback)
File "/tmp/tmpfrovmlq9/.venv/lib/python3.10/site-packages/build/init.py", line 502, in _handle_backend
raise BuildBackendException( # noqa: B904 # use raise from
build.BuildBackendException: Backend subprocess exited when trying to invoke prepare_metadata_for_build_wheelInput:
import build
import build.env
import pyproject_hookssource = '/home/adam/Projects/tmp/poetry_bug/poetry_bug_1'
dest = '/tmp/tmpfrovmlq9/dist'with build.env.IsolatedEnvBuilder() as env:
builder = build.ProjectBuilder(
srcdir=source,
scripts_dir=env.scripts_dir,
python_executable=env.executable,
runner=pyproject_hooks.quiet_subprocess_runner,
)
env.install(builder.build_system_requires)
env.install(builder.get_requires_for_build('wheel'))
builder.metadata_path(dest)at ~/.local/lib/python3.10/site-packages/poetry/utils/env.py:1546 in run
1542│ )
1543│ else:
1544│ output = subprocess.check_output(cmd, stderr=stderr, env=env, **kwargs)
1545│ except CalledProcessError as e:
→ 1546│ raise EnvCommandError(e, input=input)
1547│
1548│ return decode(output)
1549│
1550│ def execute(self, bin: str, *args: str, **kwargs: Any) -> int:The following error occurred when trying to handle this error:
Stack trace:
2 ~/.local/lib/python3.10/site-packages/poetry/utils/env.py:1544 in run
1542│ )
1543│ else:
→ 1544│ output = subprocess.check_output(cmd, stderr=stderr, env=env, **kwargs)
1545│ except CalledProcessError as e:
1546│ raise EnvCommandError(e, input=input)1 /usr/lib/python3.10/subprocess.py:420 in check_output
418│ kwargs['input'] = empty
419│
→ 420│ return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
421│ **kwargs).stdout
422│CalledProcessError
Command '['/tmp/tmpfrovmlq9/.venv/bin/python', 'setup.py', 'egg_info']' returned non-zero exit status 1.
at /usr/lib/python3.10/subprocess.py:524 in run
520│ # We don't call process.wait() as .exit does that for us.
521│ raise
522│ retcode = process.poll()
523│ if check and retcode:
→ 524│ raise CalledProcessError(retcode, process.args,
525│ output=stdout, stderr=stderr)
526│ return CompletedProcess(process.args, retcode, stdout, stderr)
527│
528│The following error occurred when trying to handle this error:
Stack trace:
3 ~/.local/lib/python3.10/site-packages/poetry/inspection/info.py:622 in get_pep517_metadata
620│ os.chdir(path.as_posix())
621│ try:
→ 622│ venv.run("python", "setup.py", "egg_info")
623│ info = PackageInfo.from_metadata(path)
624│ except EnvCommandError as fbe:2 ~/.local/lib/python3.10/site-packages/poetry/utils/env.py:1500 in run
1498│ def run(self, bin: str, *args: str, **kwargs: Any) -> str | int:
1499│ cmd = self.get_command_from_bin(bin) + list(args)
→ 1500│ return self._run(cmd, **kwargs)
1501│
1502│ def run_pip(self, *args: str, **kwargs: Any) -> int | str:1 ~/.local/lib/python3.10/site-packages/poetry/utils/env.py:1782 in _run
1780│ def _run(self, cmd: list[str], **kwargs: Any) -> int | str:
1781│ kwargs["env"] = self.get_temp_environ(environ=kwargs.get("env"))
→ 1782│ return super()._run(cmd, **kwargs)
1783│
1784│ def get_temp_environ(EnvCommandError
Command ['/tmp/tmpfrovmlq9/.venv/bin/python', 'setup.py', 'egg_info'] errored with the following return code 1
Output:
Traceback (most recent call last):
File "/home/adam/Projects/tmp/poetry_bug/poetry_bug_1/setup.py", line 1, in
import setuptools_scm
ModuleNotFoundError: No module named 'setuptools_scm'at ~/.local/lib/python3.10/site-packages/poetry/utils/env.py:1546 in run
1542│ )
1543│ else:
1544│ output = subprocess.check_output(cmd, stderr=stderr, env=env, **kwargs)
1545│ except CalledProcessError as e:
→ 1546│ raise EnvCommandError(e, input=input)
1547│
1548│ return decode(output)
1549│
1550│ def execute(self, bin: str, *args: str, **kwargs: Any) -> int:The following error occurred when trying to handle this error:
Stack trace:
21 ~/.local/lib/python3.10/site-packages/cleo/application.py:327 in run
325│
326│ try:
→ 327│ exit_code = self._run(io)
328│ except BrokenPipeError:
329│ # If we are piped to another process, it may close early and send a20 ~/.local/lib/python3.10/site-packages/poetry/console/application.py:190 in _run
188│ self._load_plugins(io)
189│
→ 190│ exit_code: int = super()._run(io)
191│ return exit_code
192│19 ~/.local/lib/python3.10/site-packages/poethepoet/plugin.py:249 in _run
247│ tokens.insert(task_name_index, "--")
248│
→ 249│ continue_run(self, io)
250│
251│ # Apply the patch18 ~/.local/lib/python3.10/site-packages/cleo/application.py:431 in _run
429│ io.input.interactive(interactive)
430│
→ 431│ exit_code = self._run_command(command, io)
432│ self._running_command = None
433│17 ~/.local/lib/python3.10/site-packages/cleo/application.py:473 in _run_command
471│
472│ if error is not None:
→ 473│ raise error
474│
475│ return terminate_event.exit_code16 ~/.local/lib/python3.10/site-packages/cleo/application.py:457 in _run_command
455│
456│ if command_event.command_should_run():
→ 457│ exit_code = command.run(io)
458│ else:
459│ exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED15 ~/.local/lib/python3.10/site-packages/cleo/commands/base_command.py:119 in run
117│ io.input.validate()
118│
→ 119│ status_code = self.execute(io)
120│
121│ if status_code is None:14 ~/.local/lib/python3.10/site-packages/cleo/commands/command.py:62 in execute
60│
61│ try:
→ 62│ return self.handle()
63│ except KeyboardInterrupt:
64│ return 113 ~/.local/lib/python3.10/site-packages/poetry/console/commands/install.py:156 in handle
154│ self.installer.verbose(self.io.is_verbose())
155│
→ 156│ return_code = self.installer.run()
157│
158│ if return_code != 0:12 ~/.local/lib/python3.10/site-packages/poetry/installation/installer.py:116 in run
114│ self._execute_operations = False
115│
→ 116│ return self._do_install()
117│
118│ def dry_run(self, dry_run: bool = True) -> Installer:11 ~/.local/lib/python3.10/site-packages/poetry/installation/installer.py:263 in _do_install
261│ source_root=self._env.path.joinpath("src")
262│ ):
→ 263│ ops = solver.solve(use_latest=self._whitelist).calculate_operations()
264│ else:
265│ self._io.write_line("Installing dependencies from lock file")10 ~/.local/lib/python3.10/site-packages/poetry/puzzle/solver.py:74 in solve
72│ with self._progress(), self._provider.use_latest_for(use_latest or []):
73│ start = time.time()
→ 74│ packages, depths = self._solve()
75│ end = time.time()
76│9 ~/.local/lib/python3.10/site-packages/poetry/puzzle/solver.py:157 in _solve
155│
156│ try:
→ 157│ result = resolve_version(self._package, self._provider)
158│
159│ packages = result.packages8 ~/.local/lib/python3.10/site-packages/poetry/mixology/init.py:18 in resolve_version
16│ solver = VersionSolver(root, provider)
17│
→ 18│ return solver.solve()
19│7 ~/.local/lib/python3.10/site-packages/poetry/mixology/version_solver.py:112 in solve
110│ while next is not None:
111│ self._propagate(next)
→ 112│ next = self._choose_package_version()
113│
114│ return self._result()6 ~/.local/lib/python3.10/site-packages/poetry/mixology/version_solver.py:428 in _choose_package_version
426│ package = locked
427│
→ 428│ package = self._provider.complete_package(package)
429│
430│ conflict = False5 ~/.local/lib/python3.10/site-packages/poetry/puzzle/provider.py:657 in complete_package
655│ if locked is not None and locked.package.is_same_package_as(dep):
656│ continue
→ 657│ self.search_for_direct_origin_dependency(dep)
658│ else:
659│ for dep in _dependencies:4 ~/.local/lib/python3.10/site-packages/poetry/puzzle/provider.py:280 in search_for_direct_origin_dependency
278│ elif dependency.is_directory():
279│ dependency = cast("DirectoryDependency", dependency)
→ 280│ package = self._search_for_directory(dependency)
281│
282│ elif dependency.is_url():3 ~/.local/lib/python3.10/site-packages/poetry/puzzle/provider.py:426 in _search_for_directory
424│ def _search_for_directory(self, dependency: DirectoryDependency) -> Package:
425│ dependency.validate(raise_error=True)
→ 426│ package = self.get_package_from_directory(dependency.full_path)
427│
428│ self.validate_package_for_dependency(dependency=dependency, package=package)2 ~/.local/lib/python3.10/site-packages/poetry/puzzle/provider.py:439 in get_package_from_directory
437│ @classmethod
438│ def get_package_from_directory(cls, directory: Path) -> Package:
→ 439│ return PackageInfo.from_directory(path=directory).to_package(root_dir=directory)
440│
441│ def _search_for_url(self, dependency: URLDependency) -> Package:1 ~/.local/lib/python3.10/site-packages/poetry/inspection/info.py:499 in from_directory
497│ info = cls.from_setup_files(path)
498│ else:
→ 499│ info = get_pep517_metadata(path)
500│ except PackageInfoError:
501│ if not info:PackageInfoError
Unable to determine package info for path: /home/adam/Projects/tmp/poetry_bug/poetry_bug_1
Fallback egg_info generation failed.
Command ['/tmp/tmpfrovmlq9/.venv/bin/python', 'setup.py', 'egg_info'] errored with the following return code 1
Output:
Traceback (most recent call last):
File "/home/adam/Projects/tmp/poetry_bug/poetry_bug_1/setup.py", line 1, in
import setuptools_scm
ModuleNotFoundError: No module named 'setuptools_scm'at ~/.local/lib/python3.10/site-packages/poetry/inspection/info.py:625 in get_pep517_metadata
621│ try:
622│ venv.run("python", "setup.py", "egg_info")
623│ info = PackageInfo.from_metadata(path)
624│ except EnvCommandError as fbe:
→ 625│ raise PackageInfoError(
626│ path, "Fallback egg_info generation failed.", fbe
627│ )
628│ finally:
629│ os.chdir(cwd.as_posix())