Skip to content

Commit 98dcde7

Browse files
maxrjonesmaxrjones
committed
chore: use just
1 parent a78726c commit 98dcde7

11 files changed

Lines changed: 183 additions & 106 deletions

File tree

‎.github/workflows/docs.yml‎

Lines changed: 4 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -25,15 +25,9 @@ jobs:
2525
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
2626
with:
2727
python-version: '3.12'
28-
# Mirror the former hatch `docs` env exactly: remote extra + docs group,
29-
# from the universal lock. --strict turns warnings into errors, so a docs
30-
# code block that fails to execute at build time (e.g. a non-exec python
31-
# fence disrupting a later exec="true" block) fails CI instead of merging
32-
# as a silent warning.
33-
- run: uv sync --locked --no-default-groups --extra remote --group docs
34-
- run: uv run --no-sync mkdocs build --strict
35-
env:
36-
DISABLE_MKDOCS_2_WARNING: "true"
37-
NO_MKDOCS_2_WARNING: "true"
28+
# `just docs-build` runs `mkdocs build --strict` (warnings are errors, so a
29+
# docs code block that fails to execute at build time fails CI) against the
30+
# locked docs env. check_unlinked_types runs in that same synced env.
31+
- run: uvx --from rust-just just docs-build
3832
- run: uv run --no-sync python ci/check_unlinked_types.py
3933
continue-on-error: true

‎.github/workflows/gpu_test.yml‎

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -60,10 +60,7 @@ jobs:
6060
enable-cache: true
6161
python-version: ${{ matrix.python-version }}
6262
- name: Run Tests
63-
run: |
64-
uv sync --locked --no-default-groups --group test --extra gpu --extra optional
65-
uv pip install pytest-examples
66-
bash ci/run-coverage.sh -m gpu
63+
run: uvx --from rust-just just gpu
6764

6865
- name: Upload coverage
6966
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0

‎.github/workflows/hypothesis.yaml‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -69,8 +69,7 @@ jobs:
6969
id: status
7070
run: |
7171
echo "Using Hypothesis profile: $HYPOTHESIS_PROFILE"
72-
uv sync --locked --no-default-groups --group test --group remote-tests --extra remote --extra optional --extra cli --extra cast-value-rs
73-
bash ci/run-hypothesis.sh
72+
uvx --from rust-just just hypothesis
7473
7574
# explicitly save the cache so it gets updated, also do this even if it fails.
7675
- name: Save cached hypothesis directory

‎.github/workflows/test.yml‎

Lines changed: 13 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -60,21 +60,16 @@ jobs:
6060
with:
6161
enable-cache: true
6262
python-version: ${{ matrix.python-version }}
63-
# One universal uv.lock serves every os/python cell of this matrix;
64-
# `uv sync --locked` fails if it is stale vs pyproject.toml — a
65-
# cross-platform drift gate. setup-uv's python-version sets UV_PYTHON, so uv
66-
# selects the right interpreter. The shared coverage runner lives in
67-
# ci/run-coverage.sh so CI and local dev invoke an identical command.
63+
# The `just` recipe (see Justfile) runs `uv sync --locked`, so one universal
64+
# uv.lock serves every os/python cell — and it fails if the lock is stale vs
65+
# pyproject.toml (a cross-platform drift gate). setup-uv's python-version
66+
# sets UV_PYTHON so uv picks the right interpreter. CI and local dev invoke
67+
# the identical recipe; `uvx --from rust-just` provides `just` without a
68+
# separate install.
6869
- name: Run Tests
6970
env:
7071
HYPOTHESIS_PROFILE: ci
71-
run: |
72-
EXTRAS=""
73-
if [ "${{ matrix.dependency-set }}" = "optional" ]; then
74-
EXTRAS="--extra remote --extra optional --extra cli --extra cast-value-rs --group remote-tests"
75-
fi
76-
uv sync --locked --no-default-groups --group test $EXTRAS
77-
bash ci/run-coverage.sh
72+
run: uvx --from rust-just just test-${{ matrix.dependency-set }}
7873
- name: Upload coverage
7974
if: ${{ matrix.dependency-set == 'optional' && matrix.os == 'ubuntu-latest' }}
8075
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
@@ -108,17 +103,11 @@ jobs:
108103
with:
109104
enable-cache: true
110105
python-version: ${{ matrix.python-version }}
111-
# Both intentionally unlocked: min_deps derives floors via uv's --resolution
112-
# lowest-direct (replacing the hand-pinned tool.hatch.envs.min_deps);
113-
# upstream overlays nightly + git mains (ci/sync-upstream.sh).
106+
# min_deps and upstream are intentionally unlocked (floors via --resolution
107+
# lowest-direct; nightly + git mains overlay). The recipe names match the
108+
# matrix values — see the Justfile.
114109
- name: Run Tests
115-
run: |
116-
if [ "${{ matrix.dependency-set }}" = "min_deps" ]; then
117-
uv sync --resolution lowest-direct --no-default-groups --group test --group remote-tests --extra remote --extra optional
118-
else
119-
bash ci/sync-upstream.sh
120-
fi
121-
bash ci/run-coverage.sh
110+
run: uvx --from rust-just just ${{ matrix.dependency-set }}
122111
- name: Upload coverage
123112
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
124113
with:
@@ -140,10 +129,7 @@ jobs:
140129
enable-cache: true
141130
python-version: '3.13'
142131
- name: Run doctests
143-
run: |
144-
uv sync --locked --no-default-groups --extra remote --group remote-tests
145-
uv pip list
146-
uv run --no-sync --with pytest-examples pytest tests/test_docs.py -v
132+
run: uvx --from rust-just just doctest
147133

148134
benchmarks:
149135
name: Benchmark smoke test
@@ -159,9 +145,7 @@ jobs:
159145
enable-cache: true
160146
python-version: '3.13'
161147
- name: Run Benchmarks
162-
run: |
163-
uv sync --locked --no-default-groups --group test
164-
uv run --no-sync pytest --benchmark-enable tests/benchmarks
148+
run: uvx --from rust-just just benchmark
165149

166150
test-complete:
167151
name: Test complete

‎Justfile‎

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
# zarr-python developer task runner (https://github.com/casey/just).
2+
#
3+
# `just` is a thin verb-runner over uv; uv owns the environments. Each recipe is
4+
# the single source of truth for a dev/CI task — CI calls the same recipes (via
5+
# `uvx --from rust-just just <recipe>`), so local and CI behavior cannot drift.
6+
#
7+
# Install just: uv tool install rust-just (or `brew install just`, `cargo install just`)
8+
# List recipes: just (or `just --list`)
9+
#
10+
# The matrix lives in GitHub Actions; pass the Python version via UV_PYTHON
11+
# (setup-uv sets it from `python-version`). Locally, override per call, e.g.
12+
# UV_PYTHON=3.13 just test-optional
13+
14+
# Extras + groups that make up the "optional" (full integration) test environment.
15+
optional_deps := "--extra remote --extra optional --extra cli --extra cast-value-rs --group remote-tests"
16+
17+
[private]
18+
default:
19+
@just --list
20+
21+
[doc("Run the unit tests with the minimal dependency set")]
22+
test-minimal *args:
23+
#!/usr/bin/env bash
24+
set -euo pipefail
25+
uv sync --locked --no-default-groups --group test
26+
if [ -n "${CI:-}" ]; then uv pip list; fi
27+
uv run --no-sync coverage run --source=src -m pytest --ignore tests/benchmarks \
28+
--junitxml=junit.xml -o junit_family=legacy {{args}}
29+
uv run --no-sync coverage xml
30+
31+
[doc("Run the unit tests with the full (optional) integration dependency set")]
32+
test-optional *args:
33+
#!/usr/bin/env bash
34+
set -euo pipefail
35+
uv sync --locked --no-default-groups --group test {{optional_deps}}
36+
if [ -n "${CI:-}" ]; then uv pip list; fi
37+
uv run --no-sync coverage run --source=src -m pytest --ignore tests/benchmarks \
38+
--junitxml=junit.xml -o junit_family=legacy {{args}}
39+
uv run --no-sync coverage xml
40+
41+
[doc("Generate an HTML coverage report (optional deps); open htmlcov/index.html")]
42+
coverage-html *args:
43+
#!/usr/bin/env bash
44+
set -euo pipefail
45+
uv sync --locked --no-default-groups --group test {{optional_deps}}
46+
uv run --no-sync coverage run --source=src -m pytest --ignore tests/benchmarks {{args}}
47+
uv run --no-sync coverage html
48+
49+
[doc("Run the slow Hypothesis property tests")]
50+
hypothesis *args:
51+
#!/usr/bin/env bash
52+
set -euo pipefail
53+
uv sync --locked --no-default-groups --group test {{optional_deps}}
54+
if [ -n "${CI:-}" ]; then uv pip list; fi
55+
uv run --no-sync coverage run --source=src -m pytest -nauto --run-slow-hypothesis \
56+
tests/test_properties.py tests/test_store/test_stateful* {{args}}
57+
uv run --no-sync coverage xml
58+
59+
[doc("Validate executable code blocks in the docs (tests/test_docs.py)")]
60+
doctest *args:
61+
#!/usr/bin/env bash
62+
set -euo pipefail
63+
uv sync --locked --no-default-groups --extra remote --group remote-tests
64+
if [ -n "${CI:-}" ]; then uv pip list; fi
65+
uv run --no-sync --with pytest-examples pytest tests/test_docs.py -v {{args}}
66+
67+
[doc("Run the benchmark suite (minimal deps)")]
68+
benchmark *args:
69+
#!/usr/bin/env bash
70+
set -euo pipefail
71+
uv sync --locked --no-default-groups --group test
72+
uv run --no-sync pytest --benchmark-enable tests/benchmarks {{args}}
73+
74+
[doc("Run the tests against the lowest supported direct dependency versions")]
75+
min_deps *args:
76+
#!/usr/bin/env bash
77+
set -euo pipefail
78+
# uv derives the floors from the `>=` constraints in pyproject.toml.
79+
uv sync --resolution lowest-direct --no-default-groups \
80+
--group test --group remote-tests --extra remote --extra optional
81+
if [ -n "${CI:-}" ]; then uv pip list; fi
82+
uv run --no-sync coverage run --source=src -m pytest --ignore tests/benchmarks \
83+
--junitxml=junit.xml -o junit_family=legacy {{args}}
84+
uv run --no-sync coverage xml
85+
86+
[doc("Run the tests against bleeding-edge (nightly + git main) dependencies")]
87+
upstream *args:
88+
#!/usr/bin/env bash
89+
set -euo pipefail
90+
uv sync --no-default-groups --group test --group remote-tests --extra remote
91+
uv pip install --prerelease=allow \
92+
--index https://pypi.anaconda.org/scientific-python-nightly-wheels/simple/ \
93+
--extra-index-url https://pypi.org/simple/ \
94+
numpy \
95+
"packaging @ git+https://github.com/pypa/packaging" \
96+
"numcodecs @ git+https://github.com/zarr-developers/numcodecs" \
97+
"s3fs @ git+https://github.com/fsspec/s3fs" \
98+
"universal_pathlib @ git+https://github.com/fsspec/universal_pathlib" \
99+
"typing_extensions @ git+https://github.com/python/typing_extensions" \
100+
"donfig @ git+https://github.com/pytroll/donfig" \
101+
"obstore @ git+https://github.com/developmentseed/obstore@main#subdirectory=obstore"
102+
if [ -n "${CI:-}" ]; then uv pip list; fi
103+
uv run --no-sync coverage run --source=src -m pytest --ignore tests/benchmarks \
104+
--junitxml=junit.xml -o junit_family=legacy {{args}}
105+
uv run --no-sync coverage xml
106+
107+
[doc("Run the GPU tests (requires CUDA + a GPU); `pytest -m gpu`")]
108+
gpu *args:
109+
#!/usr/bin/env bash
110+
set -euo pipefail
111+
uv sync --locked --no-default-groups --group test --extra gpu --extra optional
112+
uv pip install pytest-examples
113+
if [ -n "${CI:-}" ]; then uv pip list; fi
114+
uv run --no-sync coverage run --source=src -m pytest -m gpu --ignore tests/benchmarks \
115+
--junitxml=junit.xml -o junit_family=legacy {{args}}
116+
uv run --no-sync coverage xml
117+
118+
[doc("Build the documentation (strict: warnings are errors)")]
119+
docs-build *args:
120+
#!/usr/bin/env bash
121+
set -euo pipefail
122+
uv sync --locked --no-default-groups --extra remote --group docs
123+
DISABLE_MKDOCS_2_WARNING=true NO_MKDOCS_2_WARNING=true \
124+
uv run --no-sync mkdocs build --strict {{args}}
125+
126+
[doc("Serve the documentation locally with live reload at http://0.0.0.0:8000/")]
127+
docs-serve *args:
128+
#!/usr/bin/env bash
129+
set -euo pipefail
130+
uv sync --locked --no-default-groups --extra remote --group docs
131+
DISABLE_MKDOCS_2_WARNING=true NO_MKDOCS_2_WARNING=true \
132+
uv run --no-sync mkdocs serve --watch src {{args}}
133+
134+
[doc("Run all pre-commit hooks (ruff, codespell, mypy, repo-review, ...)")]
135+
lint *args:
136+
prek run --all-files {{args}}
137+
138+
[doc("Check that uv.lock is in sync with pyproject.toml")]
139+
lock-check:
140+
uv lock --check

‎changes/4096.misc.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
Replaced hatch with uv for development environments and CI task management. Contributors now create the development environment with `uv sync --locked` and run tests with `uv run pytest` (or the shared runners in `ci/`, e.g. `bash ci/run-coverage.sh`); see the contributing guide for details. The `tool.hatch.envs.*` environments and their run scripts were removed from `pyproject.toml`, and a committed `uv.lock` now pins the full transitive dependency closure for reproducible, cross-platform CI. hatchling remains the build backend, so package builds and version derivation are unchanged.
1+
Replaced hatch with uv and [just](https://github.com/casey/just) for development environments and CI task management. Contributors create the development environment with `uv sync --locked` and run tasks via `just` recipes defined in the `Justfile` (e.g. `just test-optional`, `just docs-build`; run `just` to list them). The recipes are thin wrappers over uv and are shared by CI, so local and CI behavior cannot drift. The `tool.hatch.envs.*` environments were removed and a committed `uv.lock` now pins the full transitive dependency closure for reproducible, cross-platform CI. hatchling remains the build backend, so package builds and version derivation are unchanged.

‎ci/run-coverage.sh‎

Lines changed: 0 additions & 16 deletions
This file was deleted.

‎ci/run-hypothesis.sh‎

Lines changed: 0 additions & 12 deletions
This file was deleted.

‎ci/sync-upstream.sh‎

Lines changed: 0 additions & 18 deletions
This file was deleted.

‎docs/contributing.md‎

Lines changed: 18 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,13 @@ uv sync --locked # create .venv/ with Zarr and all development dependencies
8888

8989
This creates a virtual environment in `.venv/`. Prefix commands with `uv run` to run them inside it, or activate it directly with `source .venv/bin/activate`. To use a specific Python version, pass `-p`, e.g. `uv sync --locked -p 3.13`.
9090

91+
Common developer tasks (running the matrix test environments, building docs, etc.) are defined as [`just`](https://github.com/casey/just) recipes in the `Justfile`. Install `just` and list the available recipes with:
92+
93+
```bash
94+
uv tool install rust-just # or: brew install just
95+
just # list all recipes
96+
```
97+
9198
To verify that your development environment is working, you can run the unit tests:
9299

93100
```bash
@@ -132,13 +139,15 @@ Zarr includes a suite of unit tests. The simplest way to run them is within your
132139
uv run pytest
133140
```
134141

135-
To reproduce a CI test environment exactly — the optional integration dependencies, with coverage — sync that environment and run the shared coverage script:
142+
To reproduce a CI test environment exactly — the same dependency set CI uses, with coverage — run the matching `just` recipe. These sync the locked environment and run the test suite the way CI does:
136143

137144
```bash
138-
uv sync --locked --no-default-groups --group test --extra remote --extra optional --extra cli --extra cast-value-rs --group remote-tests
139-
bash ci/run-coverage.sh
145+
just test-minimal # the minimal dependency set
146+
just test-optional # the full optional/integration dependency set
140147
```
141148

149+
Pass extra pytest arguments through, e.g. `just test-optional -k test_attributes`, and select a Python version with `UV_PYTHON`, e.g. `UV_PYTHON=3.13 just test-optional`.
150+
142151
All tests are automatically run via GitHub Actions for every pull request and must pass before code can be accepted. Test coverage is also collected automatically via the Codecov service.
143152

144153
> **Note:** Previous versions of Zarr-Python made extensive use of doctests. These tests were not maintained during the 3.0 refactor but may be brought back in the future. See issue #2614 for more details.
@@ -195,18 +204,18 @@ If you would like to skip the failing checks and push the code for further discu
195204

196205
> **Note:** Test coverage for Zarr-Python 3 is currently not at 100%. This is a known issue and help is welcome to bring test coverage back to 100%. See issue #2613 for more details.
197206
198-
Zarr strives to maintain 100% test coverage under the latest Python stable release. Both unit tests and docstring doctests are included when computing coverage. From your development environment, running:
207+
Zarr strives to maintain 100% test coverage under the latest Python stable release. Both unit tests and docstring doctests are included when computing coverage. Running:
199208

200209
```bash
201-
bash ci/run-coverage.sh
210+
just test-optional
202211
```
203212

204213
will run the test suite with coverage and produce an XML coverage report. This should be 100% before code can be accepted into the main code base.
205214

206-
You can then generate an HTML coverage report from the collected data by running:
215+
You can also generate a browsable HTML coverage report by running:
207216

208217
```bash
209-
uv run --no-sync coverage html
218+
just coverage-html
210219
```
211220

212221
When submitting a pull request, coverage will also be collected across all supported Python versions via the Codecov service, and will be reported back within the pull request. Codecov coverage must also be 100% before code can be accepted.
@@ -220,15 +229,15 @@ Zarr uses mkdocs for documentation, hosted on readthedocs.org. Documentation is
220229
The documentation can be built locally by running:
221230

222231
```bash
223-
uv run --group docs --extra remote mkdocs build
232+
just docs-build
224233
```
225234

226235
The resulting built documentation will be available in the `site/` folder.
227236

228237
You can also serve a continuously updating version of the documentation during development at [http://0.0.0.0:8000/](http://0.0.0.0:8000/) by running:
229238

230239
```bash
231-
uv run --group docs --extra remote mkdocs serve --watch src
240+
just docs-serve
232241
```
233242

234243
#### Adding executable code blocks in the documentation

0 commit comments

Comments
 (0)