You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Reading/writing F-order data w/o compression is broken #1279
I was trying to upgrade zarr-python in our webKnossos libs and noticed failing tests. The tests essentially write some data, reopen the array, read the data back, and compare before/after.
We use F-order to write our data. I tried to replace F with C order and the tests succeeded. Interestingly, the issue does not occur when writing with compression (tested with Blosc). I didn't check whether the writing or reading part is the cause for this problem.
I debugged a bit further. Previously, bytes were passed to the FSStore and now the arrays are passed directly. That causes issues with the order while serializing in fsspec. Not sure whether to fix on fsspec or zarr side (leaning towards zarr).
Activity
normanrz commentedon Dec 1, 2022
I just checked: the write path seems to be the culprit.
joshmoore commentedon Dec 6, 2022
Can confirm that using
FSStorefails whileDirectoryStorepasses:cc: @martindurant
normanrz commentedon Dec 6, 2022
I debugged a bit further. Previously, bytes were passed to the
FSStoreand now the arrays are passed directly. That causes issues with the order while serializing infsspec. Not sure whether to fix onfsspecorzarrside (leaning towardszarr).The change is that
ensure_bytesis not called anymore forFSStore:https://github.com/zarr-developers/zarr-python/blob/main/zarr/core.py#L2232-L2233
Change: 43266ee#diff-f640a0c570765203632323d71a9d1145ec6121c90db73f56e1271ab0efd5b8d8R2232
normanrz commentedon Dec 6, 2022
cc: @madsbk @jakirkham @d70-t