Repository navigation
Release Zarr-Python v3.1.4 #3508
Description
Release: v3.1.4
Scheduled Date: 2025/10/12
Priority PRs/issues to complete prior to release
Before release:
- Make sure the release branch (e.g.,
3.1.x) is up to date with any backports. - Make sure that all pull requests which will be included in the release have been properly documented as changelog files in the
changes/directory. - Run
towncrier build --version x.y.zto create the changelog, and commit the result to the release branch. - Check SPEC 0 to see if the minimum supported version of Python or NumPy needs bumping.
- Check to ensure that:
- Deprecated workarounds/codes/tests are removed. Run
grep "# TODO" **/*.pyto find all potential TODOs. - All tests pass in the "Tests" workflow.
- All tests pass in the "GPU Tests" workflow.
- All tests pass in the "Hypothesis" workflow.
- Check that downstream libraries work well (maintainers can make executive decisions about whether all checks are required for this release).
- numcodecs
- Xarray (@jhamman @dcherian @TomNicholas)
- Zarr's upstream compatibility is tested via the Upstream Dev CI worklow.
- Click on the most recent workflow and check that the
upstream-devjob has run and passed.upstream-devis not run on all all workflow runs. - Check that the expected version of Zarr-Python was tested using the
Version Infostep of theupstream-devjob. - If testing on a branch other than
mainis needed, open a PR modifying https://github.com/pydata/xarray/blob/90ee30943aedba66a37856b2332a41264e288c20/ci/install-upstream-wheels.sh#L56 and add therun-upstreamlabel.
- Titiler.Xarray (@maxrjones)
- Modify dependencies for titiler.xarray.
- Modify triggers for running the test workflow.
- Push the branch to the repository and check for the actions for any failures.
- Deprecated workarounds/codes/tests are removed. Run
Release:
- Go to https://github.com/zarr-developers/zarr-python/releases.
- Click "Draft a new release".
- Choose a version number prefixed with a
v(e.g.v0.0.0). For pre-releases, include the appropriate suffix (e.g.v0.0.0a1orv0.0.0rc2). - Set the target branch to the release branch (e.g.,
3.1.x) - Set the description of the release to:
See release notes https://zarr.readthedocs.io/en/stable/release-notes.html#release-0-0-0, replacing the correct version numbers. For pre-release versions, the URL should omit the pre-release suffix, e.g. "a1" or "rc1". - Click on "Generate release notes" to auto-fill the description.
- Make a release by clicking the 'Publish Release' button, this will automatically create a tag too.
- Verify that release workflows succeeded.
- The latest version is correct on PyPI.
- The stable version is correct on ReadTheDocs.
After release:
- Review and merge the pull request on the conda-forge zarr-feedstock that will be automatically generated.
- Party 🎉
Reacted by Davis Bennett
Activity
ianhi commentedon Oct 6, 2025
All good there
d-v-b commentedon Oct 23, 2025
over in #3540 @maxrjones suggested marking this release 3.2.0:
The test of whether this release should be 3.2.0 or 3.1.4 is how much effort it requires for downstream projects to adapt to these changes. Personally I don't think switching our docs backend will require substantial effort from zarr users, but I don't have a clear feeling about the concurrency limits change. How can that change break existing workflows or generally require people to change their code?
cc @jhamman
d-v-b commentedon Oct 23, 2025
@dcherian over in #3526 you said "because it's going to cause errors with dask as we've learned with Icechunk.", could you elaborate on this? What's the likelihood that #3526 breaks people's workflows?
I don't want to release something that's problematic for people, even if it's in a 3.2.0 release.
ianhi commentedon Oct 23, 2025
I think that #3542 should probably block releae until fixed. It will break a lot of links to the docs from around the internet
ilan-gold commentedon Oct 23, 2025
That's also true, I hadn't thought of those of us not crazy enough to use intersphinx. https://zarr.readthedocs.io/en/latest/user-guide/config.html doesn't exist but https://zarr.readthedocs.io/en/stable/user-guide/config.html does and https://zarr.readthedocs.io/en/stable/user-guide/config/ doesn't exist but https://zarr.readthedocs.io/en/latest/user-guide/config/ does
d-v-b commentedon Oct 23, 2025
can we programmatically check that all endpoints of our old docs site resolve to endpoints in the new docs site?
ianhi commentedon Oct 23, 2025
It's also possible to make some redirect rules in readthedocs which might be the easiest solution for this
or to do something like this: https://github.com/earth-mover/icechunk/pull/1092/files
ilan-gold commentedon Oct 24, 2025
Was thinking about this a bit last night, and I think the check should be "is the old
objects.inva subset of the new one?"dcherian commentedon Oct 24, 2025
I say 3.2.0 since downstream users may need some changes to get things back to working. Not nice to wake up to broken automated pipelines...
We've seen that it's really easy to overload a machine with a combo of zarr concurrency & another source of parallelism (e.g. dask). It's hard to reason through because the zarr limit is NOT a per-process limit but a per-array-call limit (at best I think). So 10 threads running zarr reads in parallel would set up 640 connections (IIUC). This usually manifests as a "timeout of sorts". With icechunk we see "0B/s" errors.
d-v-b commentedon Oct 24, 2025
seems like we should implement a per-process limit instead of making the per-array-call limit higher.
maxrjones commentedon Oct 24, 2025
#3544 should be sufficient to unblock the release (at least from the docs perspective, maybe not from the concurrency perspective)Nevermind, I'll need to do a bit more work on sphinx inter-compatibility
ianhi commentedon Nov 7, 2025
What's teh conclusion here? Is this ok to go?
d-v-b commentedon Nov 7, 2025
we decided in the dev meeting to revert the concurrency changes, then I think we can release
maxrjones commentedon Nov 21, 2025
Done in https://github.com/zarr-developers/zarr-python/releases/tag/v3.1.4