Skip to content

Commit 817d911

Browse files
committed
fix(codecs): reshape filter output to the resolved output shape
1 parent e3d6d20 commit 817d911

3 files changed

Lines changed: 13 additions & 13 deletions

File tree

‎changes/4560.bugfix.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
Encoding a chunk through a numcodecs filter no longer passes the
22
filter's flattened output to the next codec: `_encode_sync` reshapes the
3-
flattened array back to the chunk shape, so downstream codecs
4-
(e.g. `TransposeCodec` or an `imagecodecs` serializer chained after a
3+
flattened array to the filter's output shape (from
4+
`resolve_metadata`), so downstream codecs (e.g. `TransposeCodec` or an `imagecodecs` serializer chained after a
55
`Delta` or `FixedScaleOffset` filter) receive a chunk of the expected
66
shape instead of a flat 1-D array that could make writing fail or
7-
silently corrupt data. Shape-changing codecs such as `PackBits` are
8-
unaffected.
7+
silently corrupt data. Shape-changing codecs such as `PackBits` get
8+
their own resolved output shape.

‎src/zarr/codecs/numcodecs/_codecs.py‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -164,12 +164,12 @@ def _encode_sync(self, chunk_data: NDBuffer, chunk_spec: ArraySpec) -> NDBuffer:
164164
# would be encoded in transposed element order (gh-3558)
165165
chunk_ndarray = np.ascontiguousarray(chunk_data.as_ndarray_like())
166166
out = self._codec.encode(chunk_ndarray)
167-
if out.ndim == 1 and out.size == np.prod(chunk_spec.shape):
168-
# most numcodecs filters return a flattened array; reshape it so
169-
# downstream codecs receive a chunk of the shape the pipeline
170-
# promised them. Shape-changing codecs (e.g. PackBits) keep out.
171-
out = out.reshape(chunk_spec.shape)
172-
return chunk_spec.prototype.nd_buffer.from_ndarray_like(out)
167+
# numcodecs filters return a flattened array; reshape it to the
168+
# output shape so downstream codecs receive the chunk shape that
169+
# resolve_metadata promised them (gh-4553)
170+
return chunk_spec.prototype.nd_buffer.from_ndarray_like(
171+
out.reshape(self.resolve_metadata(chunk_spec).shape)
172+
)
173173

174174
async def _encode_single(self, chunk_data: NDBuffer, chunk_spec: ArraySpec) -> NDBuffer:
175175
return await asyncio.to_thread(self._encode_sync, chunk_data, chunk_spec)

‎tests/test_codecs/test_numcodecs.py‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -214,9 +214,9 @@ def test_generic_filter_flattened_output_reshaped() -> None:
214214
np.testing.assert_array_equal(data, b[:, :])
215215

216216

217-
def test_generic_filter_packedbits_output_not_reshaped() -> None:
218-
# PackBits drives the size of the chunk, so `_encode_sync` must not reshape
219-
# its output
217+
def test_generic_filter_packedbits_output_shape() -> None:
218+
# PackBits changes the size of the chunk, so `_encode_sync` must reshape its
219+
# output to the shape from `resolve_metadata`, not the input chunk shape
220220
data = np.tril(np.ones((16, 16), dtype=bool))
221221

222222
a = create_array(

0 commit comments

Comments
 (0)