Skip to content

cpython: fix extensions when using a musl toolchain#110293

Merged
flokli merged 1 commit intoNixOS:masterfrom
pjjw:python-extension-musl-fix
Jan 22, 2021
Merged

cpython: fix extensions when using a musl toolchain#110293
flokli merged 1 commit intoNixOS:masterfrom
pjjw:python-extension-musl-fix

Conversation

@pjjw
Copy link
Contributor

@pjjw pjjw commented Jan 21, 2021

Motivation for this change

it looks like while attempting to fix cross-compilation in #98915, python extensions under musl-libc were entirely broken due to python's build process not differentiating between musl and gnu abi. this fixes this.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@pjjw pjjw requested a review from FRidh as a code owner January 21, 2021 09:07
@ofborg ofborg bot added 6.topic: python Python is a high-level, general-purpose programming language. 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Jan 21, 2021
@FRidh FRidh self-assigned this Jan 21, 2021
@FRidh FRidh added this to the 21.05 milestone Jan 21, 2021
@flokli
Copy link
Member

flokli commented Jan 21, 2021

@pjjw do you have an easy before/after repro?

@pjjw
Copy link
Contributor Author

pjjw commented Jan 21, 2021

@flokli yea- try nix-build -A pkgsMusl.python39Packages.bootstrapped-pip - before and after should give you a clear look at what's going on.

It's kind of explained in the file itself, but above the section where this change is made there's a link to cpython's configure scripts, and you can see there that it will never detect musl because it's choosing from enumerated options of which musl is not one. musl showing up in the first place as part of the triple in sysconfigdata is purely an artifact of how we're dealing with cross-compilation- python itself will never use that string.

@pjjw
Copy link
Contributor Author

pjjw commented Jan 21, 2021

@GrahamcOfBorg build pkgsMusl.python39Packages.bootstrapped-pip

and then compare to hydra for the broken one.

@pjjw
Copy link
Contributor Author

pjjw commented Jan 21, 2021

oops, i fed something rotten to the bot.

Copy link
Contributor

@lopsided98 lopsided98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't tested, but looks reasonable to me. One minor suggestion would be to change the variable from fakeAbiName to pythonAbiName.

@pjjw pjjw force-pushed the python-extension-musl-fix branch from 4732cc9 to b8f4a97 Compare January 22, 2021 01:09
@pjjw
Copy link
Contributor Author

pjjw commented Jan 22, 2021

feel free to squash that cleanup commit out, only didn't force-push because github doesn't handle comments well in that case.

@flokli flokli force-pushed the python-extension-musl-fix branch from b8f4a97 to 858eebe Compare January 22, 2021 23:23
@flokli flokli merged commit e1c86b9 into NixOS:master Jan 22, 2021
@flokli
Copy link
Member

flokli commented Jan 22, 2021

squashed and merged, thanks!

@ofborg ofborg bot removed the 6.topic: python Python is a high-level, general-purpose programming language. label Jan 22, 2021
@pjjw pjjw deleted the python-extension-musl-fix branch January 25, 2021 01:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants