Add Pyodide support and CI jobs for statsmodels#9270
Add Pyodide support and CI jobs for statsmodels#9270bashtage merged 78 commits intostatsmodels:mainfrom
statsmodels#9270Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
changes in unit test looks good to me Question (I never looked at this) |
|
Hi, @bashtage and @josef-pkt! Is it okay to tag you both to seek another review on this PR, since it has been a month since my last review request? It is complete and ready to go from my end, and I've merged the latest changes from |
|
Thanks for the work. I'll get around to it next week. |
|
Sure, thanks, @bashtage! Could you trigger the workflows in the meantime here, please? |
|
Hi again, @bashtage – here is a gentle reminder since you mentioned you would get around to it the week after. There is no hurry here whatsoever – whenever you get the time. Thanks! |
|
I am away this week but will absolutely take a look when I'm back. |
bashtage
left a comment
bashtage
left a comment
There was a problem hiding this comment.
One small change and one quesation. Overall LGTM. I can't really comment n the wasm/js stuff.
| @pytest.mark.matplotlib | ||
| @pytest.mark.skipif( | ||
| PYTHON_IMPL_WASM, | ||
| reason="Matplotlib uses different backend in WASM/Pyodide" |
There was a problem hiding this comment.
Does the backend matter for these and the others that are marked to skip?
There was a problem hiding this comment.
If you mean the functionality of the plots, I don't think so. Normally, one will not need to adapt the code to use matplotlib-pyodide to use the HTML5/WASM canvases, since it gets installed when one installs matplotlib using micropip. The details are a little murky since it has been some time for me, but I received the error: 'Line2D' object has no attribute "other", so it is probably a missing feature in matplotlib-pyodide at the time of writing.
For example, https://github.com/statsmodels/statsmodels/blob/main/examples/python/categorical_interaction_plot.py (I can't find any other failing example right now) can be tried out in the Pyodide console (https://pyodide.org/en/stable/console.html), though it won't display any of the matplotlib.figure.Figure objects, of course, lacking a canvas for the renderer – they can be viewed in a JupyterLite notebook (https://jupyter.org/try-jupyter/lab/) and they should all work without any code changes. Not all of the tests marked with @pytest.mark.matplotlib have been skipped, and the ones skipped here are those that were failing because of the error noted above. I could investigate further as a follow-up if you would want me to do so.
|
Going to merge this and we'll see if other changes are needed. |
|
I'll let it run one more time to make sure there is nothing stange |
|
Thank you for the merge! |
Backport of #9270: add Pyodide support and CI jobs for v0.14.x
4 participants
statsmodels#9166NumPy's guide.
This PR adds a GitHub Actions CI job in
emscripten.ymlthat builds and tests WASM wheels forstatsmodels(Pyodide version 0.26.0, Python 3.12), compiled using the Emscripten toolchain (version 3.1.58). Thepytesttest suite is then executed against the built wheel using the in-housestatsmodels.test()API – the tests have been fixed or skipped in order to adapt to the limitations of Pyodide in the browser, such as not being able to usethreading/multiprocessingfrom the standard library or the fact that the bitness for the Python interpreter that is powered by Pyodide is 32-bit.A Node.js-based test suite that calls Pyodide from JS is used instead of the Pyodide-managed special virtual environment due to differences in being able to load shared objects, where there are some problems for
statsmodelsregarding the visibility of symbols most likely coming from OpenBLAS, please see previous discussion(s) on #9166 for more information.Additional context
Here are a few xrefs to some other PRs that have been implemented for packages in the Scientific Python ecosystem:
scikit-image): Build and test PyWavelets Pyodide wheels in CI PyWavelets/pywt#701 and Upload dev wheels to Anaconda.org + revamp wheels publishing workflow PyWavelets/pywt#714pandasrepository: BLD, TST: Build and test Pyodide wheels forpandasin CI pandas-dev/pandas#57896scikit-imagescikit-image/scikit-image#7350The greater, long-term goal towards this change is to implement Sphinx-based interactive documentation via JupyterLite and WASM-powered, in-browser kernels, as referenced in Quansight-Labs/czi-scientific-python-mgmt#19, see also: Quansight-Labs/czi-scientific-python-mgmt#18. A pilot that can be readily tried out is available for the "API reference" pages under the PyWavelets documentation. This will be preceded by configuring a place to publish these WebAssembly/Emscripten wheels nightly or on a scheduled cadence for
statsmodels(which could be a third-party, PyPI-like index such as Anaconda.org) and then integrating Sphinx extensions such asjupyterlite-sphinxfor hosted documentation. Work towards these areas is going on in-line with the above tracking issue(s).Details
Notes:
needed for doc changes.
then show that it is fixed with the new code.
verify your changes are well formatted by running
flake8is installed. This command is also available on Windowsusing the Windows System for Linux once
flake8is installed in thelocal Linux environment. While passing this test is not required, it is good practice and it helps
improve code quality in
statsmodels.