Skip to content

Comments

Preserve original PKG-INFO contents when creating wheel (instead of calling wheel.metadata.pkginfo_to_metadata)#4701

Merged
abravalheri merged 7 commits intopypa:mainfrom
abravalheri:obviate-pkginfo-to-metadata
Nov 19, 2024
Merged

Preserve original PKG-INFO contents when creating wheel (instead of calling wheel.metadata.pkginfo_to_metadata)#4701
abravalheri merged 7 commits intopypa:mainfrom
abravalheri:obviate-pkginfo-to-metadata

Conversation

@abravalheri
Copy link
Contributor

@abravalheri abravalheri commented Oct 23, 2024 •
edited
Loading

This seems to be a necessary step towards implementing new versions of metadata (e.g. PEP 685).

As pointed out in #4698, even if we modify setuptools to emit new versions of the metadata, pypa/wheel still hardcodes version 2.1, which can be problematic.

The good news is that recently setuptools has started emitting valid and complete 2.1 metadata on its own (see setuptools/_core_metadata.py). This PR assumes that it is OK just to reuse the file generated by setuptools without any need to post-process it in bdist_wheel.

Summary of changes

  • Strengthening of test_core_metadata to ensure the generated PKG-INFO is compatible with standards for METADATA.
    • Add tests for the generated PKG-INFO of real-world configuration scenarios, makimg sure they are compatible with wheel.metadata.pkginfo_to_metadata.
    • Add tests for the generated PKG-INFO to make sure they "round-trip" when subject to the parsing and serialisation methods used by pypa/wheel.
  • Modify setuptools.command.bdist_wheel to simply re-use the already generated PKG-INFO.

Closes

Limitations

There are "minor" differences between METADATA generated by pypa/wheel and PKG-INFO generated by setuptools:

  • The order of some headers (trivial)
  • The normalisation of requirements

I believe that an acceptable level of normalisation, fully compliant with the spec, will be "automatically" achieved once pypa/packaging#845 and pypa/packaging#644 (comment) are handled1.

Pull Request Checklist

Footnotes

  1. Right now, the specs mandate the extras to be normalised, but not the name. Setuptools relies on packaging for the normalisation, so once the potential bug in https://github.com/pypa/packaging/issues/845 is solved, it will be compliant to the spec. If packaging introduces names normalisation we will also adopt it. ↩

@abravalheri abravalheri mentioned this pull request Oct 23, 2024
Merged
2 tasks
@abravalheri abravalheri changed the title Preserver original PKG-INFO contents when creating wheel (instead of calling wheel.metadata.pkginfo_to_metadata) Preserve original PKG-INFO contents when creating wheel (instead of calling wheel.metadata.pkginfo_to_metadata) Oct 23, 2024
@abravalheri abravalheri force-pushed the obviate-pkginfo-to-metadata branch from 2924da8 to 9841171 Compare October 23, 2024 12:47
@abravalheri abravalheri marked this pull request as ready for review October 23, 2024 19:22
@abravalheri
Copy link
Contributor Author

abravalheri commented Oct 24, 2024

@agronholm do you have any reservations regarding this approach?

@agronholm
Copy link
Contributor

agronholm commented Oct 24, 2024

If I understood this correctly, then it's fine by me. It's about time we start ditching the awkward conversion code from egg-info metadata. The only thing preventing me from advancing this myself is mental exhaustion.

@abravalheri abravalheri mentioned this pull request Oct 25, 2024
Merged
@abravalheri
Copy link
Contributor Author

abravalheri commented Oct 25, 2024

I will wait to see the assessment on #4698 before merging the PR.

@agronholm
Copy link
Contributor

agronholm commented Nov 9, 2024

Given that the bdist_wheel implementation on pypa/wheel is no longer actually being used (unless you have an old version of setuptools but upgrade only wheel), wouldn't it make sense to just skip the PKG-INFO creation altogether and just emit the proper 2.4 metadata directly?

@abravalheri abravalheri force-pushed the obviate-pkginfo-to-metadata branch from a8e0012 to a465e30 Compare November 11, 2024 18:20
@abravalheri
Copy link
Contributor Author

abravalheri commented Nov 11, 2024 •
edited
Loading

Given that the bdist_wheel implementation on pypa/wheel is no longer actually being used (unless you have an old version of setuptools but upgrade only wheel), wouldn't it make sense to just skip the PKG-INFO creation altogether and just emit the proper 2.4 metadata directly?

There are 2 aspects to this question. The first one is the existence of .egg-info and all files that exist within it... At this point in time .egg-info is very intertwined with the rest of the setuptools implementation, and although there is ambition to remove it eventually, for the time being it seems to difficult/risky.

The second aspect is that this PR in some way is a step towards generating proper 2.4 metadata directly. What I plan in #4698 is to change setuptools._core_metadata.write_pkg_file so that it is produces proper 2.4 metadata and can be used to generate both PKG-INFO and METADATA. We need to change bdist_wheel to avoid the conversion and the hardcoded parts, so that we can use the proper metadata that is generated directly.

@abravalheri abravalheri force-pushed the obviate-pkginfo-to-metadata branch from a465e30 to 089aca9 Compare November 18, 2024 15:05
@abravalheri
Copy link
Contributor Author

abravalheri commented Nov 19, 2024

I will go ahead and merge this so that the follow up PRs can be rebased.

@abravalheri abravalheri merged commit e622859 into pypa:main Nov 19, 2024
@abravalheri abravalheri deleted the obviate-pkginfo-to-metadata branch November 19, 2024 19:36
@abravalheri
Copy link
Contributor Author

abravalheri commented Nov 20, 2024

I plan to create a release that consists mostly of this change so that we can receive feedback and see if there is an unintended consequence.

@lkubb lkubb mentioned this pull request Mar 28, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@abravalheri @agronholm