Repository navigation
zarr 2.10.0 fails to load pre-existing nested files #840
Description
Minimal, reproducible code sample, a copy-pastable example if possible
import numpy as np
import zarr
a = np.ones((2,2))
z = zarr.open(zarr.NestedDirectoryStore("test.zarr"), 'w')
z['test'] = a
z = zarr.open(zarr.NestedDirectoryStore("test.zarr"), 'r')
print(np.array(z['test']))
arr = zarr.open(zarr.NestedDirectoryStore("path/to/existing/zarr/file.zarr"), 'r')
print(np.max(arr['volumes/raw']))output with zarr 2.10.0:
array([[1., 1.],
[1., 1.]])
0
output with zarr 2.3.0:
array([[1., 1.],
[1., 1.]])
6733
Problem description
With the current version of zarr (2.10.0) pre-existing zarr files using NestedDirectoryStore are all zero when read.
With an older version (tested with 2.3.0) the correct values are loaded.
Creating a new zarr file using NestedDirectoryStore on 2.10.0 and reading it works correctly.
Version and installation information
Please provide the following:
- Value of
zarr.__version__: '2.10.0' - Value of
numcodecs.__version__: '0.9.1' - Version of Python interpreter: 3.8.11
- Operating system (Linux/Windows/Mac): Linux
- How Zarr was installed (e.g., "using pip into virtual environment", or "using conda"): pip install
Activity
joshmoore commentedon Oct 6, 2021
Hi @abred. My apologies. I look at this almost every morning and think, "I need to write a test for this" but keep failing to do so. Is there any chance of your uploading something that will reproduce the error (e.g. by adding a very small dataset to the
fixtures/directory). I don't think the error will be hard to debug once we can see it happening.Fix zarr-developers#840
Add legacy tests (See zarr-developers#840)
joshmoore commentedon Oct 18, 2021
I'm managed to reproduce. Please see #850
Handle legacy nested datasets (with tests) (#850)
joshmoore commentedon Oct 19, 2021
@abred : please let us know if the issues persist.
abred commentedon Oct 21, 2021
Thank you for fixing this! And sorry for not getting back to you in time, I was on vacation. It's working now.
joshmoore commentedon Oct 21, 2021
👍 Thanks for letting us know!