Skip to content

poetry install hugely slowed down by deepcopy operations #7459

@ralbertazzi

Description

@ralbertazzi
  • Poetry version: 1.3.2
  • Python version: 3.9.11
  • OS version and name: MacOS 12.5.1
  • pyproject.toml: not shared
  • I am on the latest stable Poetry version, installed using a recommended method.
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • I have consulted the FAQ and blog for any relevant entries or release notes.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

poetry install takes considerable time (70 seconds) to run what should be a no-op. In fact:

  • All packages to be installed are already cached
  • All packages are already installed in the virtual environment. The command terminates with No dependencies to install or update

Although I cannot share the pyproject.toml and poetry.lock files as they contain private dependencies (so you wouldn't be able to reproduce it), I can share a profiling of poetry that obtained running cProfile (link here). I'll share a few links that obtained plotting with snakeviz the result of

/path/to/pypoetry/venv/bin/python -m cProfile -o test.profile -m poetry install

As you can see lots of time seems to be spent in deepcopy operations in poetry-core.

I'll try to create a fully reproducible use case. In the meantime I can also share that:

  • The project contains a great number of dependencies (around 200), some of them related to ML (PyTorch, TensorFlow, ..).
  • The same issue happens in our CI systems running on Linux
  • I experienced the same issue on both Poetry 1.2.2 ad Poetry 1.3.2
  • While "stuck", the poetry process runs at full 100% CPU utilization
  • We fetch all packages (including public ones) through Artifactory, a private packages repo. AFAIK Artifactory does not offer any JSON API like PyPI, which I know Poetry uses extensively

Thanks a lot for your support and for maintaining this amazing tool ❤️

Screenshot 2023-02-02 at 15 40 25

Screenshot 2023-02-02 at 15 40 41

Screenshot 2023-02-02 at 15 41 00

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugSomething isn't working as expectedstatus/triageThis issue needs to be triaged

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions