Repository navigation
Description
Found by the property-suite widening that came with W4 (#99). It predates that work — the rule crossed an "auto" spec before the chunk taxonomy existed too; the widening is simply the first thing to generate the chain.
The divergence
ds = xr.Dataset({"temperature": (("lat", "lon"), np.zeros((1, 1)))})
ds.chunk({"lon": "auto"}).isel(lat=[]) # fine -- lat: 0, lon: 1
ds.plan.chunk({"lon": "auto"}).isel(lat=[]).collect() # ZeroDivisionErrorpushdown_selects_past_rechunks hoists the select in front, so the rechunk is handed a zero-size array and asked to size lon's blocks against it. dask's auto_chunks divides by the largest block, which is 0 (dask/array/core.py:3414).
This is the bad direction of divergence: the optimised plan raises where the eager chain succeeds. (Contrast 07-small-wins.md §8, where the optimiser skipping an eager error was settled as desirable.)
Scope, measured rather than guessed
Against the pinned xarray 2026.7.0 / dask 2026.7.1, on an array with a zero-length dim:
| form | result |
|---|---|
chunk({dim: "auto"}), chunk({dim: "10MB"}) |
ZeroDivisionError |
chunk("auto"), chunk("10MB") (uniform) |
fine — a different path in dask |
chunk({dim: 2}), chunk({dim: -1}), chunk({dim: None}), chunk(2) |
fine |
So it is the mapping form carrying Auto or ByteSize only — in taxonomy terms, exactly two of the seven ChunkSpec variants.
Why it wasn't just fixed with #99
The guard has to answer "does this select leave a dim with no positions?", and only some of that is provable from the indexer alone:
- provable, no schema needed:
Positions(()), aMaskwith noTrue, aForwardSlicewithstop <= start; - not provable without the dim's length:
isel(dim=slice(10, 20))on a length-3 dim, and anyGeneralSlice.
A partial guard still leaves the divergence reachable, so it would not let the generator exclusion be lifted — which is the only thing that would prove the fix. Doing it properly means the rule reading sizes at its own position (it currently ignores the schema argument outright), and deciding what to do when a size is unknown: refusing then is correct but costs the rule on every chain downstream of an unmodelled op. That is a design decision, not a patch.
State today
- Pinned by example:
tests/test_accessor.py::test_an_emptying_select_may_not_cross_an_auto_rechunk, a strict xfail — it flips to passing the moment the rule learns to refuse. - Excluded from generation:
tests/test_properties.py'sEMPTIED_DIM_BUG, so the suite reports new breakage rather than this one every run. Lifting that exclusion is the acceptance test for a fix.
Done when
- The rule refuses to hoist a select that leaves any dim empty in front of a rechunk carrying
Auto/ByteSize(narrower than refusing outright, per the table above). - The unknown-size case is decided explicitly and stated in the docstring.
EMPTIED_DIM_BUG's exclusion is removed and the property suite stays green; thetest_accessor.pyxfail is flipped to a passing test.
Activity
optimize: guard #121 on the indexer, not on the extent (#121)
schema: mint weight dims per variable, not per dataset (#125) (#128)
optimize: barrier the extent-dependent chunk specs (#121)
docs: link the emptying-select divergence to its issue (#121)
optimize: barrier the extent-dependent chunk specs (#121)
optimize: wire Rechunk to the chunk taxonomy (W4, PR 2 of 2) (#123)
optimize: barrier the extent-dependent chunk specs (#121) (#129)