Skip to content

Comments

pyodide_build.out_of_tree.venv: Monkey-patch platform.system for pip#4400

Merged
hoodmane merged 1 commit intopyodide:mainfrom
passagemath:venv-pip-macos
Jan 22, 2024
Merged

pyodide_build.out_of_tree.venv: Monkey-patch platform.system for pip#4400
hoodmane merged 1 commit intopyodide:mainfrom
passagemath:venv-pip-macos

Conversation

@mkoeppe
Copy link
Contributor

@mkoeppe mkoeppe commented Jan 21, 2024

Description

On macOS, in out-of-tree venvs, pip cannot install platform wheels.

(venv-pyodide) worktree-pristine ➤ pip install -v -v -v numpy@https://cdn.jsdelivr.net/pyodide/v0.25.0/full/numpy-1.26.1-cp311-cp311-emscripten_3_1_46_wasm32.whl
Using pip 23.3.2 from /Users/mkoeppe/s/sage/sage-rebasing/worktree-pristine/venv-pyodide/lib/python3.11/site-packages/pip (python 3.11)
Non-user install because user site-packages disabled
Created temporary directory: /private/var/folders/38/wnh4gf1552g_crsjnv2vmmww0000gp/T/pip-build-tracker-0zbfwyi8
Initialized build tracking at /private/var/folders/38/wnh4gf1552g_crsjnv2vmmww0000gp/T/pip-build-tracker-0zbfwyi8
Created build tracker: /private/var/folders/38/wnh4gf1552g_crsjnv2vmmww0000gp/T/pip-build-tracker-0zbfwyi8
Entered build tracker: /private/var/folders/38/wnh4gf1552g_crsjnv2vmmww0000gp/T/pip-build-tracker-0zbfwyi8
Created temporary directory: /private/var/folders/38/wnh4gf1552g_crsjnv2vmmww0000gp/T/pip-install-fldpigj1
Created temporary directory: /private/var/folders/38/wnh4gf1552g_crsjnv2vmmww0000gp/T/pip-ephem-wheel-cache-2_ot62c6
Looking in indexes: https://pypi.org/simple, file:///Users/mkoeppe/s/sage/sage-rebasing/worktree-pristine/.pyodide-xbuildenv/xbuildenv/pyodide-root/pypa_index
ERROR: numpy-1.26.1-cp311-cp311-emscripten_3_1_46_wasm32.whl is not a supported wheel on this platform.

The computation of the supported wheel tags goes through https://github.com/pypa/packaging/blame/main/src/packaging/tags.py#L516; on macOS, it gives:

>>> list(pip._vendor.packaging.tags.platform_tags())
['macosx_14_0_x86_64', 'macosx_14_0_intel', 'macosx_14_0_fat64', 'macosx_14_0_fat32', 'macosx_14_0_universal2', 'macosx_14_0_universal', 'macosx_13_0_x86_64', 'macosx_13_0_intel', 'macosx_13_0_fat64', 'macosx_13_0_fat32', 'macosx_13_0_universal2', 'macosx_13_0_universal', 'macosx_12_0_x86_64', 'macosx_12_0_intel', 'macosx_12_0_fat64', 'macosx_12_0_fat32', 'macosx_12_0_universal2', 'macosx_12_0_universal', 'macosx_11_0_x86_64', 'macosx_11_0_intel', 'macosx_11_0_fat64', 'macosx_11_0_fat32', 'macosx_11_0_universal2', 'macosx_11_0_universal', 'macosx_10_16_x86_64', 'macosx_10_16_intel', 'macosx_10_16_fat64', 'macosx_10_16_fat32', 'macosx_10_16_universal2', 'macosx_10_16_universal', 'macosx_10_15_x86_64', 'macosx_10_15_intel', 'macosx_10_15_fat64', 'macosx_10_15_fat32', 'macosx_10_15_universal2', 'macosx_10_15_universal', 'macosx_10_14_x86_64', 'macosx_10_14_intel', 'macosx_10_14_fat64', 'macosx_10_14_fat32', 'macosx_10_14_universal2', 'macosx_10_14_universal', 'macosx_10_13_x86_64', 'macosx_10_13_intel', 'macosx_10_13_fat64', 'macosx_10_13_fat32', 'macosx_10_13_universal2', 'macosx_10_13_universal', 'macosx_10_12_x86_64', 'macosx_10_12_intel', 'macosx_10_12_fat64', 'macosx_10_12_fat32', 'macosx_10_12_universal2', 'macosx_10_12_universal', 'macosx_10_11_x86_64', 'macosx_10_11_intel', 'macosx_10_11_fat64', 'macosx_10_11_fat32', 'macosx_10_11_universal2', 'macosx_10_11_universal', 'macosx_10_10_x86_64', 'macosx_10_10_intel', 'macosx_10_10_fat64', 'macosx_10_10_fat32', 'macosx_10_10_universal2', 'macosx_10_10_universal', 'macosx_10_9_x86_64', 'macosx_10_9_intel', 'macosx_10_9_fat64', 'macosx_10_9_fat32', 'macosx_10_9_universal2', 'macosx_10_9_universal', 'macosx_10_8_x86_64', 'macosx_10_8_intel', 'macosx_10_8_fat64', 'macosx_10_8_fat32', 'macosx_10_8_universal2', 'macosx_10_8_universal', 'macosx_10_7_x86_64', 'macosx_10_7_intel', 'macosx_10_7_fat64', 'macosx_10_7_fat32', 'macosx_10_7_universal2', 'macosx_10_7_universal', 'macosx_10_6_x86_64', 'macosx_10_6_intel', 'macosx_10_6_fat64', 'macosx_10_6_fat32', 'macosx_10_6_universal2', 'macosx_10_6_universal', 'macosx_10_5_x86_64', 'macosx_10_5_intel', 'macosx_10_5_fat64', 'macosx_10_5_fat32', 'macosx_10_5_universal2', 'macosx_10_5_universal', 'macosx_10_4_x86_64', 'macosx_10_4_intel', 'macosx_10_4_fat64', 'macosx_10_4_fat32', 'macosx_10_4_universal2', 'macosx_10_4_universal']

By monkey-patching platform.system to return something other than "Darwin", we get a suitable list of tags.

>>> platform.system = lambda: "dksdks"
>>> list(pip._vendor.packaging.tags._generic_platforms())
['emscripten_3_1_46_wasm32']
>>> list(pip._vendor.packaging.tags.platform_tags())
['emscripten_3_1_46_wasm32']
>>> list(pip._vendor.packaging.tags.sys_tags())
[<cp311-cp311-emscripten_3_1_46_wasm32 @ 4585950912>, <cp311-abi3-emscripten_3_1_46_wasm32 @ 4585951232>, <cp311-none-emscripten_3_1_46_wasm32 @ 4585951424>, <cp310-abi3-emscripten_3_1_46_wasm32 @ 4585950720>, <cp39-abi3-emscripten_3_1_46_wasm32 @ 4585951616>, <cp38-abi3-emscripten_3_1_46_wasm32 @ 4585951744>, <cp37-abi3-emscripten_3_1_46_wasm32 @ 4585951936>, <cp36-abi3-emscripten_3_1_46_wasm32 @ 4585952128>, <cp35-abi3-emscripten_3_1_46_wasm32 @ 4585952320>, <cp34-abi3-emscripten_3_1_46_wasm32 @ 4585952512>, <cp33-abi3-emscripten_3_1_46_wasm32 @ 4585950784>, <cp32-abi3-emscripten_3_1_46_wasm32 @ 4585952832>, <py311-none-emscripten_3_1_46_wasm32 @ 4585953408>, <py3-none-emscripten_3_1_46_wasm32 @ 4585950976>, <py310-none-emscripten_3_1_46_wasm32 @ 4585953536>, <py39-none-emscripten_3_1_46_wasm32 @ 4585953664>, <py38-none-emscripten_3_1_46_wasm32 @ 4585953792>, <py37-none-emscripten_3_1_46_wasm32 @ 4585953984>, <py36-none-emscripten_3_1_46_wasm32 @ 4585954176>, <py35-none-emscripten_3_1_46_wasm32 @ 4585954368>, <py34-none-emscripten_3_1_46_wasm32 @ 4585954560>, <py33-none-emscripten_3_1_46_wasm32 @ 4585954752>, <py32-none-emscripten_3_1_46_wasm32 @ 4585954944>, <py31-none-emscripten_3_1_46_wasm32 @ 4585955136>, <py30-none-emscripten_3_1_46_wasm32 @ 4585955328>, <py311-none-any @ 4585955520>, <py3-none-any @ 4585955712>, <py310-none-any @ 4585956096>, <py39-none-any @ 4585956288>, <py38-none-any @ 4585956544>, <py37-none-any @ 4585956800>, <py36-none-any @ 4585957056>, <py35-none-any @ 4585957312>, <py34-none-any @ 4585957568>, <py33-none-any @ 4585957824>, <py32-none-any @ 4585958080>, <py31-none-any @ 4585958336>, <py30-none-any @ 4585958592>]

Checklists

Copy link
Member

@hoodmane hoodmane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @mkoeppe!

@hoodmane hoodmane merged commit 056db73 into pyodide:main Jan 22, 2024
@mkoeppe mkoeppe deleted the venv-pip-macos branch January 22, 2024 02:07
@hoodmane
Copy link
Member

As a side effect, we'll be getting _generic_platforms instead of _linux_platforms on linux. I hope that's a good thing...

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Jan 22, 2024

Yes, I think that's an improvement for Linux too.

@hoodmane
Copy link
Member

It sure seems like it should be, but one never knows what sort of horrible crazy behavior we might be accidentally relying on.

@hoodmane
Copy link
Member

Now I'm running into trouble because we don't monkeypatch platform.machine()...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants