poetry 1.2.0b3 lock fails with no output #6054
Description
- I am on the latest Poetry version.
- I have searched the issues of this repo and believe that this is not a duplicate.
- If an exception occurs when executing a command, I executed it again in debug mode (
-vvvoption).
- OS version and name: macOS Monterey 12.4
- Poetry version: 1.2.0b3
- Link of a Gist with the contents of your pyproject.toml file: click
Issue
I've discovered a case where the latest pre-release version of Poetry (1.2.0b3) fails to lock dependencies in some cases. Unfortunately, I haven't been able to track down why exactly this happens, but I do have a concrete example (linked above).
Steps to reproduce:
- Install Poetry 1.2.0b3 (I installed it with
pipx install --suffix @1.2.0b3 'poetry==1.2.0b3'to distinguish from my other installations) - Create a
pyproject.tomlwithkfp = "1.8.12"andgsutil = "^5.6.0" - Run
[email protected] lock
The output I get is:
$ [email protected] lock
Updating dependencies
Resolving dependencies... (0.1s)
$
Running the same command with -vvv:
$ [email protected] lock -vvv
Loading configuration file /Users/vstrimaitis/Library/Preferences/pypoetry/config.toml
Using virtualenv: /Users/vstrimaitis/dev/random/reproduce-poetry-lock-bug/.venv
Project environment contains an empty path in sys_path, ignoring.
Updating dependencies
Resolving dependencies...
1: fact: reproduce-poetry-lock-bug is 0.1.0
1: derived: reproduce-poetry-lock-bug
1: fact: reproduce-poetry-lock-bug depends on kfp (1.8.12)
1: fact: reproduce-poetry-lock-bug depends on gsutil (^5.6.0)
1: selecting reproduce-poetry-lock-bug (0.1.0)
1: derived: gsutil (>=5.6.0,<6.0.0)
1: derived: kfp (==1.8.12)
[keyring.backend] Loading KWallet
[keyring.backend] Loading SecretService
[keyring.backend] Loading Windows
[keyring.backend] Loading chainer
[keyring.backend] Loading libsecret
[keyring.backend] Loading macOS
Creating new session for pypi.org
Source (PyPI): No release information found for gsutil-3.33, skipping
Source (PyPI): 6 packages found for gsutil >=5.6.0,<6.0.0
Source (PyPI): 1 packages found for kfp 1.8.12
1: fact: kfp (1.8.12) depends on absl-py (>=0.9,<2)
1: fact: kfp (1.8.12) depends on PyYAML (>=5.3,<6)
1: fact: kfp (1.8.12) depends on google-api-core (>=1.31.5,<2.0.0 || >2.3.0,<3.0.0dev)
1: fact: kfp (1.8.12) depends on google-cloud-storage (>=1.20.0,<2)
1: fact: kfp (1.8.12) depends on kubernetes (>=8.0.0,<19)
1: fact: kfp (1.8.12) depends on google-api-python-client (>=1.7.8,<2)
1: fact: kfp (1.8.12) depends on google-auth (>=1.6.1,<2)
1: fact: kfp (1.8.12) depends on requests-toolbelt (>=0.8.0,<1)
1: fact: kfp (1.8.12) depends on cloudpickle (>=2.0.0,<3)
1: fact: kfp (1.8.12) depends on kfp-server-api (>=1.1.2,<2.0.0)
1: fact: kfp (1.8.12) depends on jsonschema (>=3.0.1,<4)
1: fact: kfp (1.8.12) depends on tabulate (>=0.8.6,<1)
1: fact: kfp (1.8.12) depends on click (>=7.1.2,<9)
1: fact: kfp (1.8.12) depends on Deprecated (>=1.2.7,<2)
1: fact: kfp (1.8.12) depends on strip-hints (>=0.1.8,<1)
1: fact: kfp (1.8.12) depends on docstring-parser (>=0.7.3,<1)
1: fact: kfp (1.8.12) depends on kfp-pipeline-spec (>=0.1.14,<0.2.0)
1: fact: kfp (1.8.12) depends on fire (>=0.3.1,<1)
1: fact: kfp (1.8.12) depends on protobuf (>=3.13.0,<4)
1: fact: kfp (1.8.12) depends on uritemplate (>=3.0.1,<4)
1: fact: kfp (1.8.12) depends on pydantic (>=1.8.2,<2)
1: fact: kfp (1.8.12) depends on typer (>=0.3.2,<1.0)
1: selecting kfp (1.8.12)
1: derived: typer (>=0.3.2,<1.0)
1: derived: pydantic (>=1.8.2,<2)
1: derived: uritemplate (>=3.0.1,<4)
1: derived: protobuf (>=3.13.0,<4)
1: derived: fire (>=0.3.1,<1)
1: derived: kfp-pipeline-spec (>=0.1.14,<0.2.0)
1: derived: docstring-parser (>=0.7.3,<1)
1: derived: strip-hints (>=0.1.8,<1)
1: derived: Deprecated (>=1.2.7,<2)
1: derived: click (>=7.1.2,<9)
1: derived: tabulate (>=0.8.6,<1)
1: derived: jsonschema (>=3.0.1,<4)
1: derived: kfp-server-api (>=1.1.2,<2.0.0)
1: derived: cloudpickle (>=2.0.0,<3)
1: derived: requests-toolbelt (>=0.8.0,<1)
1: derived: google-auth (>=1.6.1,<2)
1: derived: google-api-python-client (>=1.7.8,<2)
1: derived: kubernetes (>=8.0.0,<19)
1: derived: google-cloud-storage (>=1.20.0,<2)
1: derived: google-api-core (>=1.31.5,<2.0.0 || >2.3.0,<3.0.0dev)
1: derived: PyYAML (>=5.3,<6)
1: derived: absl-py (>=0.9,<2)
Source (PyPI): 7 packages found for typer >=0.3.2,<1.0
Source (PyPI): 3 packages found for pydantic >=1.8.2,<2
Source (PyPI): 1 packages found for uritemplate >=3.0.1,<4
Source (PyPI): 25 packages found for protobuf >=3.13.0,<4
Source (PyPI): 2 packages found for fire >=0.3.1,<1
Source (PyPI): 3 packages found for kfp-pipeline-spec >=0.1.14,<0.2.0
Source (PyPI): 11 packages found for docstring-parser >=0.7.3,<1
Source (PyPI): 3 packages found for strip-hints >=0.1.8,<1
Source (PyPI): 7 packages found for deprecated >=1.2.7,<2
Source (PyPI): 10 packages found for click >=7.1.2,<9
Source (PyPI): 5 packages found for tabulate >=0.8.6,<1
Source (PyPI): 5 packages found for jsonschema >=3.0.1,<4
Source (PyPI): 12 packages found for kfp-server-api >=1.1.2,<2.0.0
Source (PyPI): No release information found for cloudpickle-0.2.0, skipping
Source (PyPI): 2 packages found for cloudpickle >=2.0.0,<3
Source (PyPI): No release information found for requests-toolbelt-0.0.0, skipping
Source (PyPI): 3 packages found for requests-toolbelt >=0.8.0,<1
Source (PyPI): 62 packages found for google-auth >=1.6.1,<2
Source (PyPI): Unable to parse version "1.0beta5prerelease" for the google-api-python-client package, skipping
Source (PyPI): Unable to parse version "1.0beta5prerelease2" for the google-api-python-client package, skipping
Source (PyPI): 28 packages found for google-api-python-client >=1.7.8,<2
Source (PyPI): 13 packages found for kubernetes >=8.0.0,<19
Source (PyPI): No release information found for google-cloud-storage-0.0.1, skipping
Source (PyPI): 37 packages found for google-cloud-storage >=1.20.0,<2
Source (PyPI): 16 packages found for google-api-core >=1.31.5,<2.0.0 || >2.3.0,<3.0.0dev
Source (PyPI): No release information found for pyyaml-3.01, skipping
Source (PyPI): No release information found for pyyaml-3.02, skipping
Source (PyPI): No release information found for pyyaml-3.03, skipping
Source (PyPI): No release information found for pyyaml-3.04, skipping
Source (PyPI): No release information found for pyyaml-3.05, skipping
Source (PyPI): No release information found for pyyaml-3.06, skipping
Source (PyPI): No release information found for pyyaml-3.07, skipping
Source (PyPI): No release information found for pyyaml-3.08, skipping
Source (PyPI): No release information found for pyyaml-3.09, skipping
Source (PyPI): 4 packages found for pyyaml >=5.3,<6
Source (PyPI): No release information found for absl-py-0.0, skipping
Source (PyPI): 11 packages found for absl-py >=0.9,<2
1: selecting uritemplate (3.0.1)
1: fact: fire (0.4.0) depends on six (*)
1: fact: fire (0.4.0) depends on termcolor (*)
1: selecting fire (0.4.0)
1: derived: termcolor
1: derived: six
Source (PyPI): 7 packages found for termcolor *
Source (PyPI): 27 packages found for six *
1: selecting cloudpickle (2.1.0)
1: fact: pydantic (1.9.1) depends on typing-extensions (>=3.7.4.3)
1: selecting pydantic (1.9.1)
1: derived: typing-extensions (>=3.7.4.3)
Source (PyPI): 10 packages found for typing-extensions >=3.7.4.3
1: fact: kfp-pipeline-spec (0.1.16) depends on protobuf (>=3.13.0,<4)
1: selecting kfp-pipeline-spec (0.1.16)
1: fact: strip-hints (0.1.10) depends on wheel (*)
1: selecting strip-hints (0.1.10)
1: derived: wheel
Source (PyPI): 62 packages found for wheel *
1: fact: requests-toolbelt (0.9.1) depends on requests (>=2.0.1,<3.0.0)
1: selecting requests-toolbelt (0.9.1)
1: derived: requests (>=2.0.1,<3.0.0)
Source (PyPI): No release information found for requests-0.0.1, skipping
Source (PyPI): No release information found for requests-0.12.01, skipping
Source (PyPI): No release information found for requests-2.15.0, skipping
Source (PyPI): 66 packages found for requests >=2.0.1,<3.0.0
1: selecting pyyaml (5.4.1)
1: selecting tabulate (0.8.10)
1: fact: jsonschema (3.2.0) depends on attrs (>=17.4.0)
1: fact: jsonschema (3.2.0) depends on pyrsistent (>=0.14.0)
1: fact: jsonschema (3.2.0) depends on setuptools (*)
1: fact: jsonschema (3.2.0) depends on six (>=1.11.0)
1: selecting jsonschema (3.2.0)
1: derived: six (>=1.11.0)
1: derived: setuptools
1: derived: pyrsistent (>=0.14.0)
1: derived: attrs (>=17.4.0)
Source (PyPI): No release information found for setuptools-13.0, skipping
Source (PyPI): 478 packages found for setuptools *
Source (PyPI): 28 packages found for pyrsistent >=0.14.0
Source (PyPI): 13 packages found for attrs >=17.4.0
1: fact: gsutil (5.11) depends on argcomplete (>=1.9.4)
1: fact: gsutil (5.11) depends on crcmod (>=1.7)
1: fact: gsutil (5.11) depends on fasteners (>=0.14.1)
1: fact: gsutil (5.11) depends on gcs-oauth2-boto-plugin (>=3.0)
1: fact: gsutil (5.11) depends on google-apitools (>=0.5.32)
1: fact: gsutil (5.11) depends on httplib2 (>=0.20.4)
1: fact: gsutil (5.11) depends on google-reauth (>=0.1.0)
1: fact: gsutil (5.11) depends on monotonic (>=1.4)
1: fact: gsutil (5.11) depends on pyOpenSSL (>=0.13)
1: fact: gsutil (5.11) depends on retry_decorator (>=1.0.0)
1: fact: gsutil (5.11) depends on six (>=1.12.0)
1: fact: gsutil (5.11) depends on google-auth (>=2.5.0)
1: selecting gsutil (5.11)
1: derived: google-auth[aiohttp] (>=2.5.0)
1: Version solving took 0.153 seconds.
1: Tried 1 solutions.
Stack trace:
18 ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/cleo/application.py:329 in run
327│
328│ try:
→ 329│ exit_code = self._run(io)
330│ except Exception as e:
331│ if not self._catch_exceptions:
17 ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/console/application.py:188 in _run
186│ self._load_plugins(io)
187│
→ 188│ exit_code: int = super()._run(io)
189│ return exit_code
190│
16 ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/cleo/application.py:423 in _run
421│ io.input.set_stream(stream)
422│
→ 423│ exit_code = self._run_command(command, io)
424│ self._running_command = None
425│
15 ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/cleo/application.py:465 in _run_command
463│
464│ if error is not None:
→ 465│ raise error
466│
467│ return event.exit_code
14 ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/cleo/application.py:449 in _run_command
447│
448│ if event.command_should_run():
→ 449│ exit_code = command.run(io)
450│ else:
451│ exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED
13 ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/cleo/commands/base_command.py:119 in run
117│ io.input.validate()
118│
→ 119│ status_code = self.execute(io)
120│
121│ if status_code is None:
12 ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/cleo/commands/command.py:83 in execute
81│
82│ try:
→ 83│ return self.handle()
84│ except KeyboardInterrupt:
85│ return 1
11 ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/console/commands/lock.py:54 in handle
52│ self._installer.lock(update=not self.option("no-update"))
53│
→ 54│ return self._installer.run()
55│
10 ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/installation/installer.py:111 in run
109│ self._execute_operations = False
110│
→ 111│ return self._do_install()
112│
113│ def dry_run(self, dry_run: bool = True) -> Installer:
9 ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/installation/installer.py:244 in _do_install
242│ source_root=self._env.path.joinpath("src")
243│ ):
→ 244│ ops = solver.solve(use_latest=self._whitelist).calculate_operations()
245│ else:
246│ self._io.write_line("Installing dependencies from lock file")
8 ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/puzzle/solver.py:73 in solve
71│ with self._provider.progress():
72│ start = time.time()
→ 73│ packages, depths = self._solve(use_latest=use_latest)
74│ end = time.time()
75│
7 ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/puzzle/solver.py:141 in _solve
139│
140│ try:
→ 141│ result = resolve_version(
142│ self._package, self._provider, locked=locked, use_latest=use_latest
143│ )
6 ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/mixology/__init__.py:24 in resolve_version
22│ solver = VersionSolver(root, provider, locked=locked, use_latest=use_latest)
23│
→ 24│ return solver.solve()
25│
5 ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/mixology/version_solver.py:124 in solve
122│ next: str | None = self._root.name
123│ while next is not None:
→ 124│ self._propagate(next)
125│ next = self._choose_package_version()
126│
4 ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/mixology/version_solver.py:153 in _propagate
151│ continue
152│
→ 153│ result = self._propagate_incompatibility(incompatibility)
154│
155│ if result is _conflict:
3 ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/mixology/version_solver.py:222 in _propagate_incompatibility
220│ self._log(f"derived: {adverb}{unsatisfied.dependency}")
221│
→ 222│ self._solution.derive(
223│ unsatisfied.dependency, not unsatisfied.is_positive(), incompatibility
224│ )
2 ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/mixology/partial_solution.py:100 in derive
98│ Adds an assignment of package as a derivation.
99│ """
→ 100│ self._assign(
101│ Assignment.derivation(
102│ dependency,
1 ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/mixology/partial_solution.py:115 in _assign
113│ """
114│ self._assignments.append(assignment)
→ 115│ self._register(assignment)
116│
117│ def backtrack(self, decision_level: int) -> None:
AssertionError
at ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/mixology/partial_solution.py:160 in _register
156│ )
157│ old_positive = Term(dep, is_positive=True)
158│ if old_positive is not None:
159│ value = old_positive.intersect(assignment)
→ 160│ assert value is not None
161│ self._positive[name] = value
162│
163│ return
164│
The line 1: selecting gsutil (5.11) seems very weird to me because gsutil==5.11 is incompatible with kfp==1.8.12 because the first one depends on google-auth >=2.5.0 and the second one - on google-auth >=1.6.1,<2
With some other combinations of dependencies I also ran into a similar issue where there is no output from running [email protected] lock, but [email protected] lock -vvv reveals an OverrideNeeded exception, which sounds similar to #5791 (and also prints the same line with pandas and numpy). If needed, I can try to find a combination of dependencies that reveals this exact exception, but I don't have it right now.
If I try to do the same with Poetry version 1.2.0b2, everything works fine and resolves gsutil to version 5.6 because that's the latest version compatible with the other library (kfp).