Skip to content

An emptying select crosses an auto-sizing rechunk and makes dask divide by zero #121

Closed

Description

@charles-turner-1

Found by the property-suite widening that came with W4 (#99). It predates that work — the rule crossed an "auto" spec before the chunk taxonomy existed too; the widening is simply the first thing to generate the chain.

The divergence

ds = xr.Dataset({"temperature": (("lat", "lon"), np.zeros((1, 1)))})

ds.chunk({"lon": "auto"}).isel(lat=[])            # fine -- lat: 0, lon: 1
ds.plan.chunk({"lon": "auto"}).isel(lat=[]).collect()   # ZeroDivisionError

pushdown_selects_past_rechunks hoists the select in front, so the rechunk is handed a zero-size array and asked to size lon's blocks against it. dask's auto_chunks divides by the largest block, which is 0 (dask/array/core.py:3414).

This is the bad direction of divergence: the optimised plan raises where the eager chain succeeds. (Contrast 07-small-wins.md §8, where the optimiser skipping an eager error was settled as desirable.)

Scope, measured rather than guessed

Against the pinned xarray 2026.7.0 / dask 2026.7.1, on an array with a zero-length dim:

form result
chunk({dim: "auto"}), chunk({dim: "10MB"}) ZeroDivisionError
chunk("auto"), chunk("10MB") (uniform) fine — a different path in dask
chunk({dim: 2}), chunk({dim: -1}), chunk({dim: None}), chunk(2) fine

So it is the mapping form carrying Auto or ByteSize only — in taxonomy terms, exactly two of the seven ChunkSpec variants.

Why it wasn't just fixed with #99

The guard has to answer "does this select leave a dim with no positions?", and only some of that is provable from the indexer alone:

  • provable, no schema needed: Positions(()), a Mask with no True, a ForwardSlice with stop <= start;
  • not provable without the dim's length: isel(dim=slice(10, 20)) on a length-3 dim, and any GeneralSlice.

A partial guard still leaves the divergence reachable, so it would not let the generator exclusion be lifted — which is the only thing that would prove the fix. Doing it properly means the rule reading sizes at its own position (it currently ignores the schema argument outright), and deciding what to do when a size is unknown: refusing then is correct but costs the rule on every chain downstream of an unmodelled op. That is a design decision, not a patch.

State today

  • Pinned by example: tests/test_accessor.py::test_an_emptying_select_may_not_cross_an_auto_rechunk, a strict xfail — it flips to passing the moment the rule learns to refuse.
  • Excluded from generation: tests/test_properties.py's EMPTIED_DIM_BUG, so the suite reports new breakage rather than this one every run. Lifting that exclusion is the acceptance test for a fix.

Done when

  • The rule refuses to hoist a select that leaves any dim empty in front of a rechunk carrying Auto/ByteSize (narrower than refusing outright, per the table above).
  • The unknown-size case is decided explicitly and stated in the docstring.
  • EMPTIED_DIM_BUG's exclusion is removed and the property suite stays green; the test_accessor.py xfail is flipped to a passing test.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions