-
-
Notifications
You must be signed in to change notification settings - Fork 993
Description
There was earlier discussion about this in #38 and #344 but I think it could be useful to have a separate issue to discuss what binary wheels for pyodide could look like.
Python wheels are basically just zip files with the package contents, see PEP 427. We are currently already supporting pure python wheels.
We could also consider trying to build wheels for packages that have C extensions. I'm not exactly sure what would be necessary on the technical side to make it work, and run the wasm binary through the compiler when the modules are loaded. That would require more investigation, and it could be a blocker, but I imagine it could be made to work.
For instance a numpy wheel could be,
numpy-1.4.0-py37-pyodide015-wasm.whl
where the latter two parts are the binary ABI and the platform.
Advantages
- wheels have already a clear spec, deal with ABI compatibility and are natural for python users.
- I think they could be distributed through PyPi (or any other
warehouse / PyPi index). If the platform tag is not supported now, I wouldn't likely be too difficult to support it. - Because it's just a zip file, it should be relatively easy to repackage a set of wheels into one or strip unused parts for a given project (without dealing with emscripten side of things at all).
- They might make package building a bit more sustainable, because we would have to deal less with it in pyodide, and users can do it and distribute results independently.
- One file per package to download instead of two (
.js+.data).
Disadvantages
- it puts stronger constraints on ABI compatibility, meaning that we cannot make releases every other month that breaks compatibility (or in particular update emscripten). That's certainly not ideal, and we considered it a blocker in the past, Package from PyPI #344 but we do have to start to deal with it at some point in any case probably.
- in particular, it likely means we cannot make releases too often. For the situation with PyPy wheels https://github.com/antocuni/pypy-wheels is a relevant reference.
- related to the previous point, it makes re-generation of a full distribution more difficult, as one has lots of distinct packages.
- I'm not sure that a zip format is ideal for the web, but at the same time we do need some container to distributes multiple files in a python package.
Again I'm not saying that we should migrate to using wheels, but it could be interesting to consider what would be necessary to make it work.