Repository navigation
ODD PI 26.3 Objective 2: 🌳 Evolve the Zarr Ecosystem #347
Description
Motivation
We need to ensure a strong foundation for developers building tools on top of the Zarr format. Together, documenting the Zarr ecosystem and zarr-python roadmap will anchor zarr-python within the broader Zarr ecosystem.
Description
During this PI, we will evolve the Zarr ecosystem by:
- documenting the ecosystem
- developing + soliciting feedback on the zarr-python roadmap
- onboarding a new team member to zarr-python work
To develop the roadmap, we will have:
- A set of dedicated meetings where Davis can get feedback on his vision for the future of Zarr. Davis has already invested a lot of time in a zarr planning document to kick off these discussions. Request and address feedback from Joe H, Max, Ilan and Lachy.
- Present the “final” roadmap for feedback in the Pangeo showcase. Include stakeholders from Earthmover, NVIDIA and Josh Munroe.
We will also onboard a new team member to zarr-python work and mentor them towards becoming a zarr-python core developer. This will reduce the dependency on the 2 existing zarr-python developers for development and review to make progress on current features and roadmap featues.
Acceptance Criteria
Detailed in sub-issues:
- Document the Zarr ecosystem
- Finish roadmap
- Get feedback on roadmap
- Onboard Chuck to zarr-python development
LOE
1 FTE (for non-stretch tasks)
Dependencies
External dependency on zarr-python stakeholders
Related issues and PRs
zarr-developers/zarr-python#3925
developmentseed/datacube-guide#35
developmentseed/datacube-guide#30
pydata/xarray#11279
Activity
[-]ODD PI 26.3 Objective 2: Evolve the Zarr Ecosystem[/-][+]ODD PI 26.3 Objective 2: 🌳 Evolve the Zarr Ecosystem[/+]6 remaining items
memory://URL scheme no longer routes to fsspec, breaking interop withMemoryFileSystemzarr-developers/zarr-python#3939abarciauskas-bgse commentedon Jul 20, 2026
Closing as complete
maxrjones commentedon Jul 22, 2026
Close-Out Update
Acceptance criteria: Finish roadmap ✅ / Onboard Chuck to zarr-python ✅
Roadmap — d-v-b/zarr-python-planning: config design, proposed API stubs, authorship clarification, distributed-writes proposal, and re-sequenced roadmap. Discussion issues: 2→3 migration story, zarrista integration, non-coordinated writes. Davis reviewed roadmap PRs (#11, #10, #7, #3).
Onboarding Chuck — Chuck landed 8 PRs in zarr-developers/zarr-python: doctests, ChunksLike widening, CuPy darwin guard, V3 time types, mypy-via-hatch fix, S3 ambient credentials, custom-dtype example, and a beginner guide (open). Davis reviewed all but one.
Acceptance criteria: Document the Zarr ecosystem — substantial progress, not fully closed
d-v-b): new zarr-metadata model layer (#4119, #4129, d-v-b#210, #3982, #3986, #4037, #4083, #4084); pydantic-zarr overhaul (9 PRs); zarr-cm (6 PRs); new data type extensions in zarr-extensionsAcceptance criteria: Get feedback on roadmap — partial at cutoff
Conversation is ongoing in zarr-developers/zarr-python#4149
zarr-python work this quarter (Davis: 62 authored PRs, 80 reviews; Max: 12 authored PRs, 34 reviews; Chuck: 8 authored PRs, 9 reviews)
Releases (Davis): 3.2.0 (#3927, #3932), 3.2.1 (#3942), CD workflow (#3935).
Performance (Davis): Cache lexicographic chunk coords in sharding, store chunks_per_shard in _ShardIndex, remove random subchunk ordering, internal simplifications.
Bug fixes (Davis): FsspecStore leading slashes, CastValue validation, contextual arrayspec in codec validation, memory:// URL routing, metadata comparison by JSON form, storage/codec fixes, struct unstable warning. Also xarray fill_value roundtrip fix.
Ergonomics / cleanup (Davis): Deprecate blosc enums, deprecate more enums, prune old string dtype class, Py312 modernizations, consolidated JSON IO refactor, backported sentinel type, docs tested as code snippets, remove fixed-length utf32 warning.
Governance / process (Max): PR template reviewer guidance, self-merge policy, deprecation policy (open), AGENTS.md (open).
Testing (Davis + Max): Davis: indexing test cleanup, test correctness fixes, pytest modernization, strict mode + ephemeral moto port, comprehensive indexing oracle, vlen F-contiguous regression tests. Max: block/mask indexing property tests, rectilinear block indexing, LatencyStore init fix, docs linting CI, xarray downstream CI.
API design (Davis + Max): Davis: zarr-metadata model layer (#4119, #4129, d-v-b#210), one Array class refactor. Max: ChunkLayout POC (open), one Array class refactor (open), sharding/chunk-grid introspection (#4036, #4035), rectilinear stabilization.
Build / CI (Davis + Max): Davis: CD workflow, harden dev environment. Max: hatch→uv migration (open), xarray downstream CI.
numcodecs (Davis): 3 PRs (pytest-cov, pyzstd skip, pytest pin) and 17 reviews.
Open items at quarter's end
cc @d-v-b @chuckwondo — feel free to edit this comment directly or reply with corrections/additions.