Skip to content

[poetry install] Error resolving packages from virtual repositories #9056

@tgarciaalv

Description

@tgarciaalv

Description

Hello. We have a internal pip mirror to download python packages, configured as a JFrog virtual repository. Since we updated to 1.8.0 or 1.8.1 we got a failure resolving the packages when executing poetry install command.
We get ` Package('xxx-lib', '0.17.0') is not in list `  when actually it is. 

Setting the real repository URL (instead of the virtual repo url) works, but we need to use the virtual repo url.

Workarounds

fix the version to Poetry (version 1.7.1)

Poetry Installation Method

pipx

Operating System

ubuntu

Poetry Version

Poetry (version 1.8.1)

Poetry Configuration

cache-dir = "/root/.cache/pypoetry"
experimental.system-git-client = false
installer.max-workers = null
installer.modern-installation = true
installer.no-binary = null
installer.parallel = true
keyring.enabled = true
repositories.matillion.url = "https://<my jfrog virtual repo>/simple"
solver.lazy-wheel = true
virtualenvs.create = true
virtualenvs.in-project = true
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.no-setuptools = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}/virtualenvs"  # /root/.cache/pypoetry/virtualenvs
virtualenvs.prefer-active-python = false
virtualenvs.prompt = "{project_name}-py{python_version}"
warnings.export = true

Python Sysconfig

No response

Example pyproject.toml

No response

Poetry Runtime Logs

pip install poetry --progress-bar off
poetry config virtualenvs.create true
poetry config virtualenvs.in-project true
poetry config http-basic.<my-organisation> ${JFROG_USER} ${JFROG_TOKEN}
poetry config --list
poetry -vvv install --no-ansi --no-interaction --no-root

[...]

[urllib3:urllib3.connectionpool] https://xxxxxxx.jfrog.io:443 "GET /artifactory/api/xxxxxxx/xxx-lib/0.17.0/xxx_lib-0.17.0-py3-none-any.whl HTTP/1.1" 206 10001
   1: Version solving took 14.901 seconds.
   1: Tried 1 solutions.

  ValueError

  Package('xxx-lib', '0.17.0') is not in list

  at /usr/local/lib/python3.11/site-packages/poetry/repositories/legacy_repository.py:66 in package
       62│         Note that this will be cached so the subsequent operations
       63│         should be much faster.
       64│         """
       65│         try:
    →  66│             index = self._packages.index(Package(name, version))
       67│ 
       68│             return self._packages[index]
       69│         except ValueError:
       70│             package = super().package(name, version, extras)

The following error occurred when trying to handle this error:


  Stack trace:

  29  /usr/local/lib/python3.11/site-packages/cleo/application.py:327 in run
       325│ 
       326│             try:
     → 327│                 exit_code = self._run(io)
       328│             except BrokenPipeError:
       329│                 # If we are piped to another process, it may close early and send a

  28  /usr/local/lib/python3.11/site-packages/poetry/console/application.py:190 in _run
       188│         self._load_plugins(io)
       189│ 
     → 190│         exit_code: int = super()._run(io)
       191│         return exit_code
       192│ 

  27  /usr/local/lib/python3.11/site-packages/cleo/application.py:431 in _run
       429│             io.input.interactive(interactive)
       430│ 
     → 431│         exit_code = self._run_command(command, io)
       432│         self._running_command = None
       433│ 

  26  /usr/local/lib/python3.11/site-packages/cleo/application.py:473 in _run_command
       471│ 
       472│         if error is not None:
     → 473│             raise error
       474│ 
       475│         return terminate_event.exit_code

  25  /usr/local/lib/python3.11/site-packages/cleo/application.py:457 in _run_command
       455│ 
       456│             if command_event.command_should_run():
     → 457│                 exit_code = command.run(io)
       458│             else:
       459│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

  24  /usr/local/lib/python3.11/site-packages/cleo/commands/base_command.py:117 in run
       115│         io.input.validate()
       116│ 
     → 117│         return self.execute(io) or 0
       118│ 
       119│     def merge_application_definition(self, merge_args: bool = True) -> None:

  23  /usr/local/lib/python3.11/site-packages/cleo/commands/command.py:61 in execute
        59│ 
        60│         try:
     →  61│             return self.handle()
        62│         except KeyboardInterrupt:
        63│             return 1

  22  /usr/local/lib/python3.11/site-packages/poetry/console/commands/install.py:153 in handle
       151│         self.installer.verbose(self.io.is_verbose())
       152│ 
     → 153│         return_code = self.installer.run()
       154│ 
       155│         if return_code != 0:

  21  /usr/local/lib/python3.11/site-packages/poetry/installation/installer.py:104 in run
       102│             self.verbose(True)
       103│ 
     → 104│         return self._do_install()
       105│ 
       106│     def dry_run(self, dry_run: bool = True) -> Installer:

  20  /usr/local/lib/python3.11/site-packages/poetry/installation/installer.py:241 in _do_install
       239│                 source_root=self._env.path.joinpath("src")
       240│             ):
     → 241│                 ops = solver.solve(use_latest=self._whitelist).calculate_operations()
       242│         else:
       243│             self._io.write_line("Installing dependencies from lock file")

  19  /usr/local/lib/python3.11/site-packages/poetry/puzzle/solver.py:71 in solve
        69│         with self._progress(), self._provider.use_latest_for(use_latest or []):
        70│             start = time.time()
     →  71│             packages, depths = self._solve()
        72│             end = time.time()
        73│ 

  18  /usr/local/lib/python3.11/site-packages/poetry/puzzle/solver.py:154 in _solve
       152│ 
       153│         try:
     → 154│             result = resolve_version(self._package, self._provider)
       155│ 
       156│             packages = result.packages

  17  /usr/local/lib/python3.11/site-packages/poetry/mixology/__init__.py:18 in resolve_version
        16│     solver = VersionSolver(root, provider)
        17│ 
     →  18│     return solver.solve()
        19│ 

  16  /usr/local/lib/python3.11/site-packages/poetry/mixology/version_solver.py:175 in solve
       173│             while next is not None:
       174│                 self._propagate(next)
     → 175│                 next = self._choose_package_version()
       176│ 
       177│             return self._result()

  15  /usr/local/lib/python3.11/site-packages/poetry/mixology/version_solver.py:514 in _choose_package_version
       512│             package = locked
       513│ 
     → 514│         package = self._provider.complete_package(package)
       515│ 
       516│         conflict = False

  14  /usr/local/lib/python3.11/site-packages/poetry/puzzle/provider.py:489 in complete_package
       487│                 dependency_package = DependencyPackage(
       488│                     dependency,
     → 489│                     self._pool.package(
       490│                         package.pretty_name,
       491│                         package.version,

  13  /usr/local/lib/python3.11/site-packages/poetry/repositories/repository_pool.py:204 in package
       202│         for repo in self.repositories:
       203│             try:
     → 204│                 return repo.package(name, version, extras=extras)
       205│             except PackageNotFound:
       206│                 continue

  12  /usr/local/lib/python3.11/site-packages/poetry/repositories/legacy_repository.py:70 in package
        68│             return self._packages[index]
        69│         except ValueError:
     →  70│             package = super().package(name, version, extras)
        71│             package._source_type = "legacy"
        72│             package._source_url = self._url

  11  /usr/local/lib/python3.11/site-packages/poetry/repositories/cached_repository.py:75 in package
        73│         extras: list[str] | None = None,
        74│     ) -> Package:
     →  75│         return self.get_release_info(canonicalize_name(name), version).to_package(
        76│             name=name, extras=extras
        77│         )

  10  /usr/local/lib/python3.11/site-packages/poetry/repositories/cached_repository.py:52 in get_release_info
        50│             return PackageInfo.load(self._get_release_info(name, version))
        51│ 
     →  52│         cached = self._release_cache.remember(
        53│             f"{name}:{version}", lambda: self._get_release_info(name, version)
        54│         )

   9  /usr/local/lib/python3.11/site-packages/poetry/utils/cache.py:147 in remember
       145│         value = self.get(key)
       146│         if value is None:
     → 147│             value = callback() if callable(callback) else callback
       148│             self.put(key, value, minutes)
       149│         return value

   8  /usr/local/lib/python3.11/site-packages/poetry/repositories/cached_repository.py:53 in <lambda>
        51│ 
        52│         cached = self._release_cache.remember(
     →  53│             f"{name}:{version}", lambda: self._get_release_info(name, version)
        54│         )
        55│ 

   7  /usr/local/lib/python3.11/site-packages/poetry/repositories/legacy_repository.py:123 in _get_release_info
       121│         yanked = page.yanked(name, version)
       122│ 
     → 123│         return self._links_to_data(
       124│             links,
       125│             PackageInfo(

   6  /usr/local/lib/python3.11/site-packages/poetry/repositories/http_repository.py:360 in _links_to_data
       358│ 
       359│         # drop yanked files unless the entire release is yanked
     → 360│         info = self._get_info_from_links(links, ignore_yanked=not data.yanked)
       361│ 
       362│         data.summary = info.summary

   5  /usr/local/lib/python3.11/site-packages/poetry/repositories/http_repository.py:310 in _get_info_from_links
       308│                 return self._get_info_from_metadata(
       309│                     universal_python3_wheel
     → 310│                 ) or self._get_info_from_wheel(universal_python3_wheel)
       311│ 
       312│             if universal_python2_wheel:

   4  /usr/local/lib/python3.11/site-packages/poetry/repositories/http_repository.py:123 in _get_info_from_wheel
       121│             try:
       122│                 package_info = PackageInfo.from_metadata(
     → 123│                     metadata_from_wheel_url(link.filename, link.url, self.session)
       124│                 )
       125│             except HTTPRangeRequestUnsupported:

   3  /usr/local/lib/python3.11/site-packages/poetry/inspection/lazy_wheel.py:77 in metadata_from_wheel_url
        75│         # After context manager exit, wheel.name will point to a deleted file path.
        76│         # Add `delete_backing_file=False` to disable this for debugging.
     →  77│         with LazyWheelOverHTTP(url, session) as lazy_file:
        78│             metadata_bytes = lazy_file.read_metadata(name)
        79│ 

   2  /usr/local/lib/python3.11/site-packages/poetry/inspection/lazy_wheel.py:289 in __enter__
       287│     def __enter__(self: U) -> U:
       288│         super().__enter__()
     → 289│         self._setup_content()
       290│         return self
       291│ 

   1  /usr/local/lib/python3.11/site-packages/poetry/inspection/lazy_wheel.py:357 in _setup_content
       355│         if self._length is None:
       356│             logger.debug("begin fetching content length")
     → 357│             self._length = self._fetch_content_length()
       358│             logger.debug("done fetching content length (is: %d)", self._length)
       359│             # Enable us to seek and write anywhere in the backing file up to this

  AssertionError

  

  at /usr/local/lib/python3.11/site-packages/poetry/inspection/lazy_wheel.py:526 in _fetch_content_length
      522│             # If we *could* download some file contents, then write them to the end of
      523│             # the file and set up our bisect boundaries by hand.
      524│             with self._stay():
      525│                 response_length = int(tail.headers["Content-Length"])
    → 526│                 assert response_length == min(initial_chunk_size, ret_length)
      527│                 self.seek(-response_length, io.SEEK_END)
      528│                 # Default initial chunk size is currently 1MB, but streaming content
      529│                 # here allows it to be set arbitrarily large.
      530│                 for chunk in tail.iter_content(CONTENT_CHUNK_SIZE):

Exited with code exit status 1

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugSomething isn't working as expected

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions