Repository navigation
Can't import zarr<3 with latest numcodecs==0.16.0 #2963
Description
Zarr version
v2.18.5
Numcodecs version
v0.16.0
Python Version
3.11 (but probably all)
Operating System
Ubuntu and Mac (but probably all)
Installation
pip install 'zarr<3' numcodecs
Description
In zarr<3, zarr/util.py imports numcodecs.blosc.cbuffer_sizes (e.g. here in 2.18.5), which was removed in numcodecs 0.16.0 (released 40mins ago at time of writing)
This reference was removed in #2182, prior to 3.0.0, but causes zarr 2.x to fail to import when latest numcodecs is installed; GHA example:
File "/home/runner/work/TileDB-SOMA/TileDB-SOMA/venv-soma/lib/python3.11/site-packages/zarr/convenience.py", line 8, in <module>
from zarr._storage.store import data_root, meta_root, assert_zarr_v3_api_available
File "/home/runner/work/TileDB-SOMA/TileDB-SOMA/venv-soma/lib/python3.11/site-packages/zarr/_storage/store.py", line 10, in <module>
from zarr.meta import Metadata2, Metadata3
File "/home/runner/work/TileDB-SOMA/TileDB-SOMA/venv-soma/lib/python3.11/site-packages/zarr/meta.py", line 10, in <module>
from zarr.util import json_dumps, json_loads
File "/home/runner/work/TileDB-SOMA/TileDB-SOMA/venv-soma/lib/python3.11/site-packages/zarr/util.py", line 34, in <module>
from numcodecs.blosc import cbuffer_sizes, cbuffer_metainfo
ImportError: cannot import name 'cbuffer_sizes' from 'numcodecs.blosc' (/home/runner/work/TileDB-SOMA/TileDB-SOMA/venv-soma/lib/python3.11/site-packages/numcodecs/blosc.cpython-311-x86_64-linux-gnu.so)
Error: Process completed with exit code 1.
Not sure it's worth a 2.18.6 to pin numcodecs<0.16 (or remove the offending references), just flagging the issue.
Steps to reproduce
python -m venv .venv
. .venv/bin/activate
pip install 'zarr<3' numcodecs
python -c 'import zarr' # ❌ ImportError: cannot import name 'cbuffer_sizes' from 'numcodecs.blosc' (/Users/ryan/c/single-cell-data/tiledb-soma/tmp/.venv/lib/python3.11/site-packages/numcodecs/blosc.cpython-311-darwin.so)or:
docker run --rm --entrypoint bash python:3.11 -c 'pip install "zarr<3" numcodecs && python -c "import zarr"'
# ❌ ImportError: cannot import name 'cbuffer_sizes' from 'numcodecs.blosc' (/usr/local/lib/python3.11/site-packages/numcodecs/blosc.cpython-311-x86_64-linux-gnu.so)Additional output
No response
Reacted by CANEVETGASPARD
Activity
pin `numcodecs<0.16`
pin `numcodecs<0.16`
dstansby commentedon Apr 7, 2025
Thanks for the quick report, and sorry about this. I think the best thing to do is pin numcodecs here, I'll do that now.
cspades commentedon Apr 7, 2025
Also hit this error, which affects NeMo:
Following for the fix, thank you!
`setup.py`: remove deprecated `License ::` from `classifiers` (#3916)
`setup.py`: remove deprecated `License ::` from `classifiers` (#3916)
`setup.py`: remove deprecated `License ::` from `classifiers` (#3916) (…
dstansby commentedon Apr 8, 2025
This should now be fixed with a new 2.18.6 release - would be great if someone could confirm before we close this.
tcompa commentedon Apr 8, 2025
Hi there
Something is still not working for me:
At a first look, I think that this is because #2965 modified the docs entry for
numcodecs(https://github.com/zarr-developers/zarr-python/blob/support/v2/pyproject.toml#L45), rather than the actual line (https://github.com/zarr-developers/zarr-python/blob/support/v2/pyproject.toml#L16).dstansby commentedon Apr 8, 2025
😱 thanks for checking - working on a working fix...
Add numcodecs<0.16 to fix zarr-developers/zarr-python#2963
gastoneb commentedon Apr 8, 2025
Experiencing the same issue even with older zarr versions. Workaround until a fix is available is to explicitly require numcodecs==0.15.0 in your deps, since it will always try to install 0.16 otherwise.
dstansby commentedon Apr 9, 2025
Right, I'm pretty sure this is fixed now - just did a fresh install, and with zarr 2.18.7 it pulls numcodecs 0.15.1. Could someone else double check? Sorry again for the disruption.
schenker commentedon Apr 9, 2025
With zarr 2.18.7 it works as expected, thanks!
4 remaining items
rm numcodecs<0.16 pin (#3946)
rm numcodecs<0.16 pin (#3946) (#3950)
Squashed commit of the following: