Skip to content

Pip packaging and publishing improvements in pytorch wheels for better integration with poetry #76557

@suneeta-mall

Description

@suneeta-mall

🐛 Describe the bug

Poerty is a great build toolchain for python applications that lean on pip packaging.

Please see a detailed thread here python-poetry/poetry#4231 (comment) on why the integration of applications written using PyTorch eco-system fails. As mentioned in the follow-up reply from @abn here python-poetry/poetry#4231 (comment) that some improvements in our PyTorch packages are published and indexed can greatly improve the experience and integration. To highlight the requests:

  1. The lack of file hashes being exposed in the repository means that Poetry has to download each artifact unnecessarily. As per PEP 503. this can be appended to the file URL as #=. This can be done easily by the index admins I believe.
  2. While fixing (1) will reduce the number of required downloads, files still will be downloaded again when there is a need to inspect metadata (ie. Requires-Dist, etc. from the wheel). This can be avoided too if PEP 658 gets implemented. I suspect this will need to also be supported by Poetry as well.
  3. Would be great if requesting a non-existant resource returns a 404 instead of 403. If I understand correctly, this is because they have not set s3:ListBucket permission for public users (assuming they use S3 for this).

I am not 100% sure this ticket is the best place on the main torch repo but I did not find any CI-specific repo either in pytorc. Please help direct if I got the main repo to report this wrong.

Versions

Has been tried with multiple versions including 1.11

cc @seemethere @malfet @osalpekar @atalman @ezyang

Metadata

Metadata

Assignees

No one assigned

    Labels

    module: binariesAnything related to official binaries that we release to userstriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions