Prefer compatible wheels over source distribution for legacy repository dependency resolution#5385
Closed
gpfv wants to merge 2 commits intopython-poetry:masterfrom
Closed
Prefer compatible wheels over source distribution for legacy repository dependency resolution#5385gpfv wants to merge 2 commits intopython-poetry:masterfrom
gpfv wants to merge 2 commits intopython-poetry:masterfrom
Conversation
Member
|
@gpfv this might need some TLC. :) |
… repository dependency resolution
Author
|
Testing Linux Capabilities added |
Closed
This was referenced Sep 15, 2022
Member
|
Superseded by #6547 |
|
This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request Check List
Resolves: #4439 and #3464
PyPi repositories use the pypi.org JSON api for fetching dependencies, where for an external repo (legacy repo) the dependencies are determined by going through the files available in the repo and inspecting the metadata (actually PyPi repositories also use this method as fallback when requires_dist is None). Universal wheels are prefered, then source distribution and only after that a desperate attempt (# Pick the first wheel available and hope for the best) is made to extract the dependencies from a platform specific wheel.
It is cumbersome to retrieve this from source, the file needs to be extracted, which takes a long time, and even after it is not always certain the correct ones can be determined, due to lack of standardization and arbitrary code execution in setup.py. So either this step has to be improved, implying all past and future build systems need to be supported, or another option is to use a well defined package distribution format, which already exist. The wheel does not need to be reinvented, it just needs to be used.
As dependencies could potentially differ (although could not find an example of this easily) depending on the platform and python version, only the appropriate wheels are considered by matching with compatibility tags.
-> Using compatible wheels over sdist will result in a more correct dependency resolution at a greater speed.
for example PyQt5 (5.15.4) requires_dist
wheel : ~3s not cached, ~0.2s cached
tar.gz: ~10s both not cached and cached (needs to be extracted everytime)