-
Notifications
You must be signed in to change notification settings - Fork 2.4k
Description
- Poetry version:
1.2.1and1.3.0.dev01 - Python version:
3.9.13 - OS version and name:
Windows 11 Enterprise (build 22000.856) - pyproject.toml:
[tool.poetry]
name = "test-poetry-master"
version = "0.1.0"
description = ""
authors = ["kristang"]
readme = "README.md"
[[tool.poetry.source]]
name = "private-pypi"
url = "https://private/pypi/private-pypi/simple/"
secondary = true
[tool.poetry.dependencies]
python = "^3.9"
[tool.poetry.dev-dependencies]
pytest = "^7.1.1"
mypy = "^0.982"
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"- I am on the latest stable Poetry version, installed using a recommended method.
- I have searched the issues of this repo and believe that this is not a duplicate.
- I have consulted the FAQ and blog for any relevant entries or release notes.
- If an exception occurs when executing a command, I executed it again in debug mode (
-vvvoption) and have included the output below.
Additionally
- Discussed the issue with @neersighted as well as @finswimmer and @radoering on the Poetry Discord.
Issue
When adding a secondary, private source and attempting to authenticate via basic-http, poetry silently ignores a UnicodeEncodingError originating from a failed latin1 encoding of a str.
Instead, poetry moves on to the public pypi.org feed and informs the user that the searched package is not available - even if it is available on pypi.org
The error stems from loading in a user/password from Windows Certificate Manager which appears to be corrupted and cannot be encoded with 'latin1'
The source of the encoding error is poetry invoking _basic_auth_str from auth.py in the requests package:
https://github.com/psf/requests/blob/7104ad4b135daab0ed19d8e41bd469874702342b/requests/auth.py#L56-L60
The originating call in poetry is request in utils.authenticator.
poetry/src/poetry/utils/authenticator.py
Lines 213 to 216 in 0ffe91c
| if credential.username is not None or credential.password is not None: | |
| request = requests.auth.HTTPBasicAuth( | |
| credential.username or "", credential.password or "" | |
| )(request) |
solve() has a fairly wide try-catch with a base Exception
poetry/src/poetry/mixology/version_solver.py
Lines 110 to 123 in 0ffe91c
| try: | |
| next: str | None = self._root.name | |
| while next is not None: | |
| self._propagate(next) | |
| next = self._choose_package_version() | |
| return self._result() | |
| except Exception: | |
| raise | |
| finally: | |
| self._log( | |
| f"Version solving took {time.time() - start:.3f} seconds.\n" | |
| f"Tried {self._solution.attempted_solutions} solutions." | |
| ) |
Reproducing
As this involves a broken str that fails to encode reproducibility is a bit hard. I will try and create tests and create a PR if my skills are sufficient enough to create something useful.
Running pipenv lock -vvv yields:
Stack trace:
4 ~\Documents\Projects\GITHUB\poetry\src\poetry\puzzle\solver.py:157 in _solve
155│
156│ try:
→ 157│ result = resolve_version(self._package, self._provider)
158│
159│ packages = result.packages
3 ~\Documents\Projects\GITHUB\poetry\src\poetry\mixology\__init__.py:18 in resolve_version
16│ solver = VersionSolver(root, provider)
17│
→ 18│ return solver.solve()
19│
2 ~\Documents\Projects\GITHUB\poetry\src\poetry\mixology\version_solver.py:113 in solve
111│ next: str | None = self._root.name
112│ while next is not None:
→ 113│ self._propagate(next)
114│ next = self._choose_package_version()
115│
1 ~\Documents\Projects\GITHUB\poetry\src\poetry\mixology\version_solver.py:152 in _propagate
150│ # where that incompatibility will allow us to derive new assignments
151│ # that avoid the conflict.
→ 152│ root_cause = self._resolve_conflict(incompatibility)
153│
154│ # Back jumping erases all the assignments we did at the previous
SolveFailure
Because test-poetry-master depends on mypy (^0.982) which doesn't exist, version solving failed.
at ~\Documents\Projects\GITHUB\poetry\src\poetry\mixology\version_solver.py:351 in _resolve_conflict
347│ )
348│ self._log(f'! which is caused by "{most_recent_satisfier.cause}"')
349│ self._log(f"! thus: {incompatibility}")
350│
→ 351│ raise SolveFailure(incompatibility)
352│
353│ def _choose_package_version(self) -> str | None:
354│ """
355│ Tries to select a version of a required package.
The following error occurred when trying to handle this error:
Stack trace:
11 ~\AppData\Local\pypoetry\Cache\virtualenvs\poetry-4e7WOrbt-py3.9\lib\site-packages\cleo\application.py:329 in run
327│
328│ try:
→ 329│ exit_code = self._run(io)
330│ except Exception as e:
331│ if not self._catch_exceptions:
10 ~\Documents\Projects\GITHUB\poetry\src\poetry\console\application.py:185 in _run
183│ self._load_plugins(io)
184│
→ 185│ exit_code: int = super()._run(io)
186│ return exit_code
187│
9 ~\AppData\Local\pypoetry\Cache\virtualenvs\poetry-4e7WOrbt-py3.9\lib\site-packages\cleo\application.py:423 in _run
421│ io.input.set_stream(stream)
422│
→ 423│ exit_code = self._run_command(command, io)
424│ self._running_command = None
425│
8 ~\AppData\Local\pypoetry\Cache\virtualenvs\poetry-4e7WOrbt-py3.9\lib\site-packages\cleo\application.py:465 in _run_command
463│
464│ if error is not None:
→ 465│ raise error
466│
467│ return event.exit_code
7 ~\AppData\Local\pypoetry\Cache\virtualenvs\poetry-4e7WOrbt-py3.9\lib\site-packages\cleo\application.py:449 in _run_command
447│
448│ if event.command_should_run():
→ 449│ exit_code = command.run(io)
450│ else:
451│ exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED
6 ~\AppData\Local\pypoetry\Cache\virtualenvs\poetry-4e7WOrbt-py3.9\lib\site-packages\cleo\commands\base_command.py:119 in run
117│ io.input.validate()
118│
→ 119│ status_code = self.execute(io)
120│
121│ if status_code is None:
5 ~\AppData\Local\pypoetry\Cache\virtualenvs\poetry-4e7WOrbt-py3.9\lib\site-packages\cleo\commands\command.py:83 in execute
81│
82│ try:
→ 83│ return self.handle()
84│ except KeyboardInterrupt:
85│ return 1
4 ~\Documents\Projects\GITHUB\poetry\src\poetry\console\commands\lock.py:54 in handle
52│ self.installer.lock(update=not self.option("no-update"))
53│
→ 54│ return self.installer.run()
55│
3 ~\Documents\Projects\GITHUB\poetry\src\poetry\installation\installer.py:114 in run
112│ self._execute_operations = False
113│
→ 114│ return self._do_install()
115│
116│ def dry_run(self, dry_run: bool = True) -> Installer:
2 ~\Documents\Projects\GITHUB\poetry\src\poetry\installation\installer.py:247 in _do_install
245│ source_root=self._env.path.joinpath("src")
246│ ):
→ 247│ ops = solver.solve(use_latest=self._whitelist).calculate_operations()
248│ else:
249│ self._io.write_line("Installing dependencies from lock file")
1 ~\Documents\Projects\GITHUB\poetry\src\poetry\puzzle\solver.py:74 in solve
72│ with self._progress(), self._provider.use_latest_for(use_latest or []):
73│ start = time.time()
→ 74│ packages, depths = self._solve()
75│ end = time.time()
76│
SolverProblemError
Because test-poetry-master depends on mypy (^0.982) which doesn't exist, version solving failed.
at ~\Documents\Projects\GITHUB\poetry\src\poetry\puzzle\solver.py:163 in _solve
159│ packages = result.packages
160│ except OverrideNeeded as e:
161│ return self._solve_in_compatibility_mode(e.overrides)
162│ except SolveFailure as e:
→ 163│ raise SolverProblemError(e)
164│
165│ combined_nodes = depth_first_search(PackageNode(self._package, packages))
166│ results = dict(aggregate_package_nodes(nodes) for nodes in combined_nodes)
167│No-where in the stacktrace is a user informed that the underlying issue is an encoding failure of credentials.
Pseduo fix
Wrapping requests.auth.HTTPBasicAuth in a try-except can catch the UnicodeEncodeError and propagate it to the user:
if credential.username is not None or credential.password is not None:
try:
request = requests.auth.HTTPBasicAuth(
credential.username or "", credential.password or ""
)(request)
except UnicodeEncodeError as e:
raise PoetryException(e)The new stacktrace now contains the correct UnicodeEncodeError:
Stack trace:
2 ~\Documents\Projects\GITHUB\poetry\src\poetry\utils\authenticator.py:216 in request
214│ if credential.username is not None or credential.password is not None:
215│ try:
→ 216│ request = requests.auth.HTTPBasicAuth(
217│ credential.username or "", credential.password or ""
218│ )(request)
1 ~\AppData\Local\pypoetry\Cache\virtualenvs\poetry-4e7WOrbt-py3.9\lib\site-packages\requests\auth.py:95 in __call__
93│
94│ def __call__(self, r):
→ 95│ r.headers["Authorization"] = _basic_auth_str(self.username, self.password)
96│ return r
97│
UnicodeEncodeError
'latin-1' codec can't encode characters in position 0-5: ordinal not in range(256)
at ~\AppData\Local\pypoetry\Cache\virtualenvs\poetry-4e7WOrbt-py3.9\lib\site-packages\requests\auth.py:60 in _basic_auth_str
56│ if isinstance(username, str):
57│ username = username.encode("latin1")
58│
59│ if isinstance(password, str):
→ 60│ password = password.encode("latin1")
61│
62│ authstr = "Basic " + to_native_string(
63│ b64encode(b":".join((username, password))).strip()
64│ )
The following error occurred when trying to handle this error:
Stack trace:
24 ~\AppData\Local\pypoetry\Cache\virtualenvs\poetry-4e7WOrbt-py3.9\lib\site-packages\cleo\application.py:329 in run
327│
328│ try:
→ 329│ exit_code = self._run(io)
330│ except Exception as e:
331│ if not self._catch_exceptions:
23 ~\Documents\Projects\GITHUB\poetry\src\poetry\console\application.py:185 in _run
183│ self._load_plugins(io)
184│
→ 185│ exit_code: int = super()._run(io)
186│ return exit_code
187│
22 ~\AppData\Local\pypoetry\Cache\virtualenvs\poetry-4e7WOrbt-py3.9\lib\site-packages\cleo\application.py:423 in _run
421│ io.input.set_stream(stream)
422│
→ 423│ exit_code = self._run_command(command, io)
424│ self._running_command = None
425│
21 ~\AppData\Local\pypoetry\Cache\virtualenvs\poetry-4e7WOrbt-py3.9\lib\site-packages\cleo\application.py:465 in _run_command
463│
464│ if error is not None:
→ 465│ raise error
466│
467│ return event.exit_code
20 ~\AppData\Local\pypoetry\Cache\virtualenvs\poetry-4e7WOrbt-py3.9\lib\site-packages\cleo\application.py:449 in _run_command
447│
448│ if event.command_should_run():
→ 449│ exit_code = command.run(io)
450│ else:
451│ exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED
19 ~\AppData\Local\pypoetry\Cache\virtualenvs\poetry-4e7WOrbt-py3.9\lib\site-packages\cleo\commands\base_command.py:119 in run
117│ io.input.validate()
118│
→ 119│ status_code = self.execute(io)
120│
121│ if status_code is None:
18 ~\AppData\Local\pypoetry\Cache\virtualenvs\poetry-4e7WOrbt-py3.9\lib\site-packages\cleo\commands\command.py:83 in execute
81│
82│ try:
→ 83│ return self.handle()
84│ except KeyboardInterrupt:
85│ return 1
17 ~\Documents\Projects\GITHUB\poetry\src\poetry\console\commands\lock.py:54 in handle
52│ self.installer.lock(update=not self.option("no-update"))
53│
→ 54│ return self.installer.run()
55│
16 ~\Documents\Projects\GITHUB\poetry\src\poetry\installation\installer.py:114 in run
112│ self._execute_operations = False
113│
→ 114│ return self._do_install()
115│
116│ def dry_run(self, dry_run: bool = True) -> Installer:
15 ~\Documents\Projects\GITHUB\poetry\src\poetry\installation\installer.py:247 in _do_install
245│ source_root=self._env.path.joinpath("src")
246│ ):
→ 247│ ops = solver.solve(use_latest=self._whitelist).calculate_operations()
248│ else:
249│ self._io.write_line("Installing dependencies from lock file")
14 ~\Documents\Projects\GITHUB\poetry\src\poetry\puzzle\solver.py:74 in solve
72│ with self._progress(), self._provider.use_latest_for(use_latest or []):
73│ start = time.time()
→ 74│ packages, depths = self._solve()
75│ end = time.time()
76│
13 ~\Documents\Projects\GITHUB\poetry\src\poetry\puzzle\solver.py:157 in _solve
155│
156│ try:
→ 157│ result = resolve_version(self._package, self._provider)
158│
159│ packages = result.packages
12 ~\Documents\Projects\GITHUB\poetry\src\poetry\mixology\__init__.py:18 in resolve_version
16│ solver = VersionSolver(root, provider)
17│
→ 18│ return solver.solve()
19│
11 ~\Documents\Projects\GITHUB\poetry\src\poetry\mixology\version_solver.py:114 in solve
112│ while next is not None:
113│ self._propagate(next)
→ 114│ next = self._choose_package_version()
115│
116│ return self._result()
10 ~\Documents\Projects\GITHUB\poetry\src\poetry\mixology\version_solver.py:413 in _choose_package_version
411│ dependency = unsatisfied[0]
412│ else:
→ 413│ dependency = min(*unsatisfied, key=_get_min)
414│
415│ locked = self._provider.get_locked(dependency)
9 ~\Documents\Projects\GITHUB\poetry\src\poetry\mixology\version_solver.py:399 in _get_min
397│
398│ try:
→ 399│ num_packages = len(self._dependency_cache.search_for(dependency))
400│ except ValueError:
401│ num_packages = 0
8 ~\Documents\Projects\GITHUB\poetry\src\poetry\mixology\version_solver.py:62 in _search_for
60│ packages = self.cache.get(key)
61│ if packages is None:
→ 62│ packages = self.provider.search_for(dependency)
63│ else:
64│ packages = [
7 ~\Documents\Projects\GITHUB\poetry\src\poetry\puzzle\provider.py:316 in search_for
314│ return PackageCollection(dependency, packages)
315│
→ 316│ packages = self._pool.find_packages(dependency)
317│
318│ packages.sort(
6 ~\Documents\Projects\GITHUB\poetry\src\poetry\repositories\pool.py:136 in find_packages
134│ packages: list[Package] = []
135│ for repo in self.repositories:
→ 136│ packages += repo.find_packages(dependency)
137│ return packages
138│
5 ~\Documents\Projects\GITHUB\poetry\src\poetry\repositories\repository.py:42 in find_packages
40│ ignored_pre_release_packages = []
41│
→ 42│ for package in self._find_packages(dependency.name, constraint):
43│ if package.yanked and not isinstance(constraint, Version):
44│ # PEP 592: yanked files are always ignored, unless they are the only
4 ~\Documents\Projects\GITHUB\poetry\src\poetry\repositories\legacy_repository.py:96 in _find_packages
94│ versions = self._cache.store("matches").get(key)
95│ else:
→ 96│ page = self._get_page(f"/{name}/")
97│ if page is None:
98│ self._log(
3 ~\Documents\Projects\GITHUB\poetry\src\poetry\repositories\legacy_repository.py:150 in _get_page
148│ def _get_page(self, endpoint: str) -> SimpleRepositoryPage | None:
149│ try:
→ 150│ response = self._get_response(endpoint)
151│ except UnicodeEncodeError as e:
152│ raise e
2 ~\Documents\Projects\GITHUB\poetry\src\poetry\repositories\http.py:272 in _get_response
270│ url = self._url + endpoint
271│ try:
→ 272│ response: requests.Response = self.session.get(
273│ url, raise_for_status=False, timeout=REQUESTS_TIMEOUT
274│ )
1 ~\Documents\Projects\GITHUB\poetry\src\poetry\utils\authenticator.py:274 in get
272│
273│ def get(self, url: str, **kwargs: Any) -> requests.Response:
→ 274│ return self.request("get", url, **kwargs)
275│
276│ def post(self, url: str, **kwargs: Any) -> requests.Response:
PoetryException
'latin-1' codec can't encode characters in position 0-5: ordinal not in range(256)
at ~\Documents\Projects\GITHUB\poetry\src\poetry\utils\authenticator.py:220 in request
216│ request = requests.auth.HTTPBasicAuth(
217│ credential.username or "", credential.password or ""
218│ )(request)
219│ except UnicodeEncodeError as e:
→ 220│ raise PoetryException(e)
221│
222│ session = self.get_session(url=url)
223│ prepared_request = session.prepare_request(request)
224│