Repository navigation
Conversation
maxrjones
had a problem deploying
to
test-release
April 14, 2026 03:23 — with
GitHub Actions
Failure
8 tasks
maxrjones
had a problem deploying
to
test-release
April 20, 2026 15:50 — with
GitHub Actions
Failure
maxrjones
had a problem deploying
to
test-release
April 20, 2026 16:20 — with
GitHub Actions
Failure
Collaborator
|
Super excited for this! Just ran into a use-case today that it'll be great for. |
Member
Author
nice! I'll try to get back to this tomorrow. I got a bit behind on development due to travel. Are you able to share anything about the use-case? I'm supposed to demo this feature at our project wrap-up next week and if it's related to NASA data it could be a good collaboration opportunity. |
Appending now promotes the destination array's chunk grid to rectilinear when either the incoming or existing array is already rectilinear, or when both are regular but declare genuinely different chunk sizes along the append axis - mirroring concatenate(), gated behind the same array.rectilinear_chunks opt-in check. - num_chunks() counts rectilinear chunks (len(edges)) instead of assuming shape // chunk_size. - check_compatible_arrays() no longer crashes comparing a rectilinear array's declared chunk sizes, and excludes the append axis from the comparison (chunk sizes are allowed to differ there). - resize_array() only calls zarr's Array.resize() when both sides stay regular with the same chunk size. Otherwise it writes the correctly merged chunk edges directly via copy_and_replace_metadata + save_metadata, since zarr's own rectilinear resize logic only ever appends a single new edge covering the whole size increase - wrong whenever the appended array itself has more than one chunk along that axis. Also fixes a related bug the append test suite caught: comparing chunk shapes via full_chunk_edges (shape-dependent boundary-truncated edges) is wrong wherever the two arrays being compared aren't meant to share the same shape, as in a region write comparing a small tile against the full destination array. check_combinable_zarr_arrays and check_compatible_arrays now compare the declared, shape-independent chunk_grid_sizes instead; check_same_chunk_shapes normalizes a bare int against a matching 1-tuple so the comparison still treats a grid zarr auto-classified as regular the same as an equivalent rectilinear declaration. Region writes now raise a clear NotImplementedError for a rectilinear array up front, rather than crashing on chunk_size-based alignment math that has no equivalent for irregular chunk boundaries. Replaces the old blanket "appending is not yet supported for rectilinear arrays" test with tests covering the two promotion cases and the disabled-flag error, and adds a region-write guard test.
Removes stale "appending is not yet supported" wording from the data structures explanation, the usage guide's append section, and the unreleased release notes entry, now that appending can promote an array to rectilinear. Region writes remain unsupported.
TomNicholas
temporarily deployed
to
test-release
September 17, 2026 19:53 — with
GitHub Actions
Inactive
# Conflicts: # virtualizarr/manifests/utils.py # virtualizarr/tests/test_manifests/test_array.py
TomNicholas
temporarily deployed
to
test-release
September 18, 2026 14:52 — with
GitHub Actions
Inactive
Avoids depending on zarr.core.chunk_grids._is_rectilinear_chunks, a private internal that could change without notice, for the trivial nested-sequence check we actually need.
TomNicholas
temporarily deployed
to
test-release
September 18, 2026 19:27 — with
GitHub Actions
Inactive
TomNicholas
temporarily deployed
to
test-release
September 22, 2026 20:22 — with
GitHub Actions
Inactive
# Conflicts: # virtualizarr/writers/icechunk.py
TomNicholas
temporarily deployed
to
test-release
September 30, 2026 01:25 — with
GitHub Actions
Inactive
Reading rectilinear chunk grids is new in xarray 2026.9.0, so the round-trip test fails on the minimum supported xarray version.
TomNicholas
temporarily deployed
to
test-release
September 30, 2026 15:52 — with
GitHub Actions
Inactive
Member
|
Now that xarray v2026.09.0 has been released, this should pass CI here too... We also should not need to bump our minimum xarray version for non-rectilinear usage, which is nice. |
xarray 2026.9.0 and zarr 3.2.1 are released with rectilinear chunk grid support, so only the unreleased virtual-tiff branch is still needed.
TomNicholas
temporarily deployed
to
test-release
September 30, 2026 16:06 — with
GitHub Actions
Inactive
Rectilinear TIFF strips need virtual-tiff changes that aren't released yet (virtual-zarr/virtual-tiff#87), so drop the TIFF concat test, the tifffile dev dependency and the rectilinear pixi env from this PR.
TomNicholas
temporarily deployed
to
test-release
September 30, 2026 16:10 — with
GitHub Actions
Inactive
_warn_about_oversized_virtual_chunks runs on every opened dataset and compared chunk shape to array shape via manifest_chunk_shape, which raises on a rectilinear grid, so any parser returning one crashed open_virtual_dataset. Also reword it and the partial-chunk concat error, which blamed regular-only chunk grids: a partial chunk before the last input still can't be concatenated with rectilinear grids, because the stored chunk decodes to the full chunk length.
TomNicholas
temporarily deployed
to
test-release
September 30, 2026 21:02 — with
GitHub Actions
Inactive
Indexing, broadcast_to and np.where assume a regular chunk grid, so on a rectilinear array they failed with an AttributeError from deep inside manifest_chunk_shape. That included xarray's isel, expand_dims and concat along a new dimension, which index under the hood. Raise NotImplementedError naming the operation instead, and document the limitation; supporting it is left to a follow-up. with_fill_value_only only needed the chunk grid's shape, so compute that via ChunkGrid, which handles rectilinear grids, instead of raising.
TomNicholas
temporarily deployed
to
test-release
September 30, 2026 21:28 — with
GitHub Actions
Inactive
This was referenced Oct 15, 2026
TomNicholas
temporarily deployed
to
test-release
October 16, 2026 18:59 — with
GitHub Actions
Inactive
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a companion PR to zarr-developers/zarr-python#3802 that adds support for rectilinear chunk grids.
Acceptance criteria:
docs/releases.md*.mdfile underdocs/api