Skip to content

feat: Support rectilinear chunk grids - #69

Closed
maxrjones wants to merge 3 commits into
mainfrom
rectilinear-chunks
Closed

maxrjones wants to merge 3 commits into
mainfrom
rectilinear-chunks

Conversation

@maxrjones

Copy link
Copy Markdown
Member

Summary

  • Support TIFFs where image height is not evenly divisible by rows-per-strip, using zarr's rectilinear chunk grid instead of erroring out
  • Add a TruncateCodec to handle edge strips that are padded to full size on disk
  • Remove ~30 previously-xfailed test files

Details

Previously, strip-based TIFFs with partial final strips (e.g., 100px height with 30px rows-per-strip) raised a ValueError because zarr's regular chunk grid requires uniform chunk sizes. This PR uses rectilinear chunk grids instead, allowing the final strip to have a different size.

Key changes:

  • parser.py: _get_chunks_from_strips now returns a nested sequence (e.g., [[30, 30, 30, 10], [width]]) for non-uniform strips instead of raising. _construct_manifest_array selects between regular and rectilinear chunk grids accordingly.
  • codecs.py: New TruncateCodec (bytes-to-bytes) trims oversized edge-chunk buffers to their expected logical size before downstream codecs process them.
  • utils.py: Added _is_nested_sequence helper (vendored from zarr-python#3534) to distinguish rectilinear chunk specs from regular ones.
  • test_gdal_examples.py: Removed the entire partial_chunks xfail list (~30 files). Moved a few files back into their correct category lists (jpeg_tables, YCbCr).

Dependencies

Requires upstream rectilinear chunk grid support in zarr-python, VirtualiZarr, and xarray (see pyproject.toml for feature branches).

@maxrjones

Copy link
Copy Markdown
Member Author

superseded by #87

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant