fix: performance regression when parsing links from legacy repositories#6442
Conversation
neersighted
added
area/solver
|
This is like what I did but way better. Woohoo! I am so happy to have been even the slightest bit helpful in making Poetry a better tool for everyone. I have run this against other projects that I was seeing performance issues with internally and can see that performance in dependency resolution has dramatically improved. I can't stress enough the importance of having this performance issue fixed and in a release branch (whether it be this PR or some other one). The performance drop with the logic as it stands on v1.2.0 just makes using poetry in environments where a secondary repo is used practically unusable. I have had two backends show this same performance issue, one being pypicloud which seems to direct clients to pypi.org when it does not have a package, and AWS CodeArtifact, which can be configured to pull packages from pypi.org itself. I would just hate to have people advocate against tools like this or be hesitant to upgrade. |
neersighted
force-pushed
the
legacy-repo-performance
branch
from
3ba17af to
cf441e7
Compare
radoering
force-pushed
the
legacy-repo-performance
branch
2 times, most recently
from
f3065b9 to
738723c
Compare
|
As proposed by @neersighted on discord, I added |
neersighted
left a comment
neersighted
left a comment
There was a problem hiding this comment.
LGTM overall, I have high confidence in backporting this -- one minor nit that we can skip if you don't like it.
neersighted
changed the title
radoering
force-pushed
the
legacy-repo-performance
branch
from
738723c to
4fc67d6
Compare
…rmance for legacy repositories Co-authored-by: Jarrod Moore <[email protected]>
radoering
force-pushed
the
legacy-repo-performance
branch
from
4fc67d6 to
e521873
Compare
|
pre-commit.ci autofix |
|
The backport to To backport manually, run these commands in your terminal: # Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.2 1.2
# Navigate to the new working tree
cd .worktrees/backport-1.2
# Create a new branch
git switch --create backport-6442-to-1.2
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 c4b2253793cd6b41a99e25e479e40b776cca0a0e
# Push it to GitHub
git push --set-upstream origin backport-6442-to-1.2
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.2Then, create a pull request where the |
…es (python-poetry#6442) Resolves: python-poetry#6436 Measurements of `poetry lock` with warm cache with example pyproject.toml from python-poetry#6436: |test case|time in s|peak memory usage in MB| |---|---|---| |legacy repository (before)|422|113| |legacy repository (after)|3|118| |pypi repository|1|92| `backports.cached-property` is used in order to support cached_property on Python 3.7. Co-authored-by: Jarrod Moore <[email protected]> Co-authored-by: Bjorn Neergaard <[email protected]> (cherry picked from commit c4b2253)
…es (python-poetry#6442) Resolves: python-poetry#6436 Measurements of `poetry lock` with warm cache with example pyproject.toml from python-poetry#6436: |test case|time in s|peak memory usage in MB| |---|---|---| |legacy repository (before)|422|113| |legacy repository (after)|3|118| |pypi repository|1|92| `backports.cached-property` is used in order to support cached_property on Python 3.7. (cherry picked from commit c4b2253) Co-authored-by: Jarrod Moore <[email protected]> Co-authored-by: Bjorn Neergaard <[email protected]>
…es (python-poetry#6442) Resolves: python-poetry#6436 Measurements of `poetry lock` with warm cache with example pyproject.toml from python-poetry#6436: |test case|time in s|peak memory usage in MB| |---|---|---| |legacy repository (before)|422|113| |legacy repository (after)|3|118| |pypi repository|1|92| `backports.cached-property` is used in order to support cached_property on Python 3.7. (cherry picked from commit c4b2253) Co-authored-by: Jarrod Moore <[email protected]> Co-authored-by: Bjorn Neergaard <[email protected]>
…es (#6442) Resolves: #6436 Measurements of `poetry lock` with warm cache with example pyproject.toml from #6436: |test case|time in s|peak memory usage in MB| |---|---|---| |legacy repository (before)|422|113| |legacy repository (after)|3|118| |pypi repository|1|92| `backports.cached-property` is used in order to support cached_property on Python 3.7. (cherry picked from commit c4b2253) Co-authored-by: Jarrod Moore <[email protected]> Co-authored-by: Bjorn Neergaard <[email protected]>
|
This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
4 participants
Resolves: #6436
Measurements of
poetry lockwith warm cache with example pyproject.toml from #6436:Usage of
@cached_propertywould be nice, but unfortunately is not available in Python 3.7 and@propertycombined with@cachedas proposed in https://docs.python.org/3/library/functools.html#functools.cached_property results in memory leaks (flake8-bugbear B019).