Fix finding headers when cross compiling#2569
Closed
lopsided98 wants to merge 1 commit intopypa:mainfrom
lopsided98:python-inc-cross
Closed
Fix finding headers when cross compiling#2569lopsided98 wants to merge 1 commit intopypa:mainfrom lopsided98:python-inc-cross
lopsided98 wants to merge 1 commit intopypa:mainfrom
lopsided98:python-inc-cross
Conversation
When cross-compiling third-party extensions, get_python_inc() may be called to return the path to Python's headers. However, it uses the sys.prefix or sys.exec_prefix of the build Python, which returns incorrect paths when cross-compiling (paths pointing to build system headers). To fix this, we use the INCLUDEPY and CONFINCLUDEPY conf variables, which can be configured to point at host Python by setting _PYTHON_SYSCONFIGDATA_NAME. The existing behavior is maintained on non-POSIX platforms or if a prefix is manually specified.
Member
|
Thanks for the report. All changes to distutils should be submitted to pypa/distutils. Please re-submit the PR there and it will be subsequently merged into Setuptools. Sorry for any inconvenience. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of changes
When cross-compiling extension modules using
build_ext,get_python_inc()may be called to return the path to Python's headers. However, it uses thesys.prefixorsys.exec_prefixof the build Python, which returns incorrect paths when cross-compiling (paths pointing to build system headers).To fix this, we use the
INCLUDEPYandCONFINCLUDEPYconf variables, which can be configured to point at host Python by setting_PYTHON_SYSCONFIGDATA_NAME. The existing behavior is maintained on non-POSIX platforms or if a prefix is manually specified.This was originally submitted to stdlib distutils as python/cpython#22440, but unfortunately they are no longer accepting patches to it even though setuptools still uses stdlib distutils by default.
I tested this with the netifaces package, which compiles an extension module and fails with the following error if it is cross-compiled from x86_64 to armv7l without this patch, due to the word size difference:
After applying this patch and setting
_PYTHON_SYSCONFIGDATA_NAMEandSETUPTOOLS_USE_DISTUTILSappropriately, the build succeeds.The stdlib distutils version of this patch is currently applied in nixpkgs (NixOS/nixpkgs#98915) and we have not observed any breakage.
Pull Request Checklist
changelog.d/.(See documentation for details)