Repository navigation
Installation problem in mipsel and mips64el (caused by cpuinfo) #257
Description
Minimal, reproducible code sample, a copy-pastable example if possible
$ python3.8 setup.py config
Traceback (most recent call last):
File "setup.py", line 4, in <module>
import cpuinfo
File "/<<PKGBUILDDIR>>/cpuinfo.py", line 2406, in <module>
_check_arch()
File "/<<PKGBUILDDIR>>/cpuinfo.py", line 231, in _check_arch
raise Exception("py-cpuinfo currently only works on X86 and some ARM/PPC/S390X CPUs.")
Exception: py-cpuinfo currently only works on X86 and some ARM/PPC/S390X CPUsProblem description
Some of the tests run on the Debian CI infrastructure are failing on mipsel and mips64el architectures [1, 2, 3].
The problem is related to the installation phase and clearly caused by cpuinfo the does not support those architectures.
Not sure if updating cpuinfo would help.
My plan is to prepare a small patch for the Debian packaging script to ignore the failure and set both have_sse2 and have_avx2 to False.
I would appreciate some comments/hints form developers.
Also if it is considered of interested for the project I can submit a PR (assuming that the fix works on Debian).
[1] https://buildd.debian.org/status/fetch.php?pkg=numcodecs&arch=mipsel&ver=0.7.2%2Bds-1&stamp=1601287342&raw=0
[2] https://buildd.debian.org/status/fetch.php?pkg=numcodecs&arch=mips64el&ver=0.7.2%2Bds-1&stamp=1601287778&raw=0
[3] https://buildd.debian.org/status/package.php?p=numcodecs
Version and installation information
Please provide the following:
- Value of
numcodecs.__version__: 0.7.2 - Version of Python interpreter: 3.8.6
- Operating system (Linux/Windows/Mac): GNU/Linix (Debian sid)
- How NumCodecs was installed (e.g., "using pip into virtual environment", or "using conda"): setup.py
Activity
jeromekelleher commentedon Nov 17, 2020
I can't comment on the technical details here @avalentino, but I just wanted to weigh in and say thanks. It's super exciting to see numcodecs going in to Debian!
joshmoore commentedon Nov 18, 2020
Last update seems to be:
commit c5becae
Author: John Kirkham
Date: Sun Nov 3 18:43:48 2019 -0500
for s390x support, but I don't see any further additions in https://github.com/workhorsy/py-cpuinfo/blob/master/ChangeLog.
@jakirkham, for my own benefit, could you explain the background of the vendoring itself?
jakirkham commentedon Nov 18, 2020
It’s used by the
setup.pyscript at build time. Historically Python hasn’t had a good mechanism for requesting build time dependencies be installed. Also the project that this file comes from has everything contained in this file. So it’s pretty easy to vendor.If there’s been an update upstream that fixes this issue, a PR upgrading this file would be welcome. If not, we might want to raise an upstream issue with them as well.
jakirkham commentedon Nov 18, 2020
Also this is the PR ( #202 ) where that commit came from for more context
jakirkham commentedon Nov 18, 2020
cc @QuLogic (in case you have any thoughts here 🙂)
Be robust against cpuinfo failures (Closes: zarr-developers#257)
Be robust against cpuinfo failures (Closes: zarr-developers#257)
vincentsarago commentedon Apr 7, 2021
👋 sorry, and feel free to ignore if you want. but using
pyproject.tomlyou can easily add build time dependenciesMore on
pyproject.tomlhttps://snarky.ca/clarifying-pep-518/e.g https://github.com/mapbox/rasterio/blob/master/pyproject.toml#L1-L2
jakirkham commentedon Apr 7, 2021
We vendored it to save people from installing it and to make sure we get a version (or commit) that works for us and avoid any that have issues. It's one file so this is pretty easy to update
Cython isn't required as we include the C files
Setuptools comes with pip and is in basically every Python environment these days
vincentsarago commentedon Apr 7, 2021
@jakirkham I understand, feel free to close the PR.
TBH, I'm hitting #268 and I was confuse where to fix. there is workhorsy/py-cpuinfo#161 which might fix the issue ... but then it will have to be updated also on this repo. To me because, it's only a one file python module, I think it's a good reason why not vendoring it ... but that's my personal opinion.
sure this is just an example
jakirkham commentedon Apr 7, 2021
Right so that is exactly the sort of situation where I would expect vendoring to help. Does using those changes fix the issue for you? If so, would you be able to open a PR including those changes in the vendored file?
jakirkham commentedon Apr 8, 2021
Should add Matthew just reviewed and merged that py-cpuinfo PR
jakirkham commentedon Oct 17, 2022
Think this should have been addressed by PR ( #280 ). Please feel free to open a new issue if other problems persist. Thanks!