Skip to content

Commit 9801fef

Browse files
authored
Merge branch 'main' into claude/great-stonebraker-732ded
2 parents 1aafb6e + ee1b44c commit 9801fef

30 files changed

Lines changed: 883 additions & 217 deletions

‎changes/4458.feature.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Reading part of an uncompressed chunk now fetches only the bytes it needs. When `BytesCodec` is the only codec, a selection is read with a single byte range request covering the rows along the first axis that it touches, instead of the whole chunk. Chunks with a compressor or any other codec are read whole, as before.

‎changes/4521.removal.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
The following have been removed, having been deprecated since Zarr-Python 3.0.0:
2+
3+
- `zarr.tree`, `zarr.api.synchronous.tree` and `zarr.api.asynchronous.tree`. Use `Group.tree` / `AsyncGroup.tree` instead.
4+
- The `Array.compressor` and `AsyncArray.compressor` properties. Use `Array.compressors` instead (for Zarr format 2 arrays, `compressors[0]` is the old `compressor`, and an empty tuple means no compressor).

‎changes/4522.removal.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
The Zarr-Python 2 keyword arguments `synchronizer`, `chunk_store`, `cache_metadata`, `cache_attrs`, `object_codec`, `read_only` and `meta_array` accepted by `zarr.create`, `zarr.group` and `zarr.open_group` (and the functions that wrap them, such as `zarr.zeros`) have never had any effect in Zarr-Python 3. Passing one of them now emits a `ZarrDeprecationWarning` instead of a "not yet implemented" `ZarrRuntimeWarning`. They will be removed in a release after February 2027. Calls that do not pass these arguments are unaffected.

‎changes/4612.bugfix.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
The `ArrayV2Metadata.chunk_grid` and `zarr.storage.default_compressor` deprecation warnings now point at the calling code instead of zarr internals, so Python no longer hides them by default.

‎docs/contributing.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -479,7 +479,7 @@ If, after weighing these factors, a removal is not clearly worthwhile, prefer to
479479

480480
Once a deprecation is decided, it must follow these steps so that users get a consistent, actionable signal:
481481

482-
1. **Emit a warning at runtime.** Raise `zarr.errors.ZarrDeprecationWarning` for an API that will be *removed*. Use `zarr.errors.ZarrFutureWarning` instead when behavior will *change* rather than disappear (for example, a default value that will change). The distinction is about who needs to see the warning. Python hides `DeprecationWarning` by default except in `__main__` and under test runners, so a removal warning reaches the authors of the code that calls the deprecated API, whereas `FutureWarning` is shown to everyone, which a behavior change needs because it alters results without any change to the user's code. The [`typing_extensions.deprecated`](https://typing-extensions.readthedocs.io/en/latest/#deprecated) decorator is the preferred way to deprecate whole functions, methods, and classes. When calling `warnings.warn` directly, set `stacklevel` so the warning points at the caller's code rather than zarr internals.
482+
1. **Emit a warning at runtime.** Raise `zarr.errors.ZarrDeprecationWarning` for an API that will be *removed*. Use `zarr.errors.ZarrFutureWarning` instead when behavior will *change* rather than disappear (for example, a default value that will change). The distinction is about who needs to see the warning. Python hides `DeprecationWarning` by default except in `__main__` and under test runners, so a removal warning reaches the authors of the code that calls the deprecated API, whereas `FutureWarning` is shown to everyone, which a behavior change needs because it alters results without any change to the user's code. The [`typing_extensions.deprecated`](https://typing-extensions.readthedocs.io/en/latest/#deprecated) decorator is the preferred way to deprecate whole functions, methods, and classes. When calling `warnings.warn` directly, set `stacklevel` so the warning points at the caller's code rather than zarr internals. This matters because Python shows a `DeprecationWarning` by default only when it points at `__main__`; one that points at zarr internals is seen only under test runners. A warning raised inside a coroutine cannot point at the caller when the synchronous API runs it, because `sync()` runs coroutines on a separate IO thread, so if the deprecation must reach users running scripts, emit it from the public synchronous function instead. Test the location, for example with `assert caught[0].filename == __file__`.
483483
2. **Write an actionable message.** The warning message must state the planned removal or change (a specific version if known, otherwise "a future release") and the migration path (i.e., what the user should do instead). A deprecation warning with no replacement guidance is incomplete.
484484
3. **Document it.** Add a "Deprecated" admonition to the docstring, stating the same removal target and migration path as the warning, so the deprecation appears in the rendered API reference, and update any affected user-guide prose.
485485
4. **Add a changelog entry.** Add a `removal` changelog fragment in `changes/` (for example, `changes/1234.removal.md`) so the deprecation is announced under "Deprecations and Removals" in the release notes. When the API is finally removed, add a second `removal` fragment for that release.

‎docs/user-guide/v3_migration.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -231,9 +231,9 @@ The following functions / methods have not been ported to Zarr-Python 3:
231231
- `zarr.copy_all` ([issue #2407](https://github.com/zarr-developers/zarr-python/issues/2407))
232232
- `zarr.copy_store` ([issue #2407](https://github.com/zarr-developers/zarr-python/issues/2407))
233233

234-
The following features (corresponding to function arguments to functions in
235-
`zarr`) have not been ported to Zarr-Python 3. Using these features
236-
will raise a warning or a `NotImplementedError`:
234+
The following function arguments from Zarr-Python 2 are accepted but ignored by
235+
Zarr-Python 3. Passing them emits a `ZarrDeprecationWarning`, and they will be
236+
removed in a release after February 2027:
237237

238238
- `cache_attrs`
239239
- `cache_metadata`
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
`to_key_value` takes a writer instead of an `indent` keyword. The writer is a `JSONWriter`, a frozen dataclass exported from `zarr_metadata` and `zarr_metadata.model` that owns how a document becomes JSON bytes: its settings are `indent`, `sort_keys`, `ensure_ascii` and `trailing_newline`, and its `encode` method does the writing. `to_key_value` also accepts a mapping of the settings (`JSONWriterSchema`), so `to_key_value(indent=2)` becomes `to_key_value({"indent": 2})`, with no import. The defaults are those of `json.dumps`, which the models already used, so a bare `to_key_value()` writes the same bytes as before. A setting that `json.dumps` would accept but misread, such as `indent=True`, a negative indent or a non-whitespace string indent, raises, and so does a mapping key that is not a setting.

‎packages/zarr-metadata/docs/models.md‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,34 @@ To change a model, use one of these methods:
120120
must claim what this context left unclaimed and contradict nothing.
121121
Otherwise it raises `ScopeConflictError`.
122122

123+
## Writing store bytes
124+
125+
`to_key_value` writes a model's document as the JSON bytes a store holds.
126+
A `JSONWriter` decides how those bytes are spelled: `indent`, `sort_keys`,
127+
`ensure_ascii` and `trailing_newline`. Its defaults are those of
128+
`json.dumps`. `to_key_value` takes a `JSONWriter` or a mapping of its
129+
settings, so you do not need to import the class to change one:
130+
131+
```python test="true" session="models"
132+
from zarr_metadata.model import JSONWriter, ZarrV3GroupMetadata
133+
134+
group = ZarrV3GroupMetadata.create_default(attributes={"b": 1, "a": 2})
135+
print(group.to_key_value({"indent": 2, "sort_keys": True})["zarr.json"].decode())
136+
# {
137+
# "attributes": {
138+
# "a": 2,
139+
# "b": 1
140+
# },
141+
# "node_type": "group",
142+
# "zarr_format": 3
143+
# }
144+
assert group.to_key_value({"indent": 2}) == group.to_key_value(JSONWriter(indent=2))
145+
```
146+
147+
A mapping with a key that is not a setting raises `TypeError`, so a
148+
misspelled setting is not ignored. `JSONWriter.from_dict` turns a mapping
149+
into a writer, and `to_dict` turns a writer back into a mapping.
150+
123151
## Extra fields
124152

125153
An extra field is a top-level member that the spec does not define. The

‎packages/zarr-metadata/src/zarr_metadata/__init__.py‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,9 @@
1919

2020
from zarr_metadata.model import (
2121
UNSET,
22+
JSONWriter,
23+
JSONWriterLike,
24+
JSONWriterSchema,
2225
ZarrV2ArrayMetadata,
2326
ZarrV2ArrayMetadataUpdate,
2427
ZarrV2ConsolidatedMetadata,
@@ -363,6 +366,9 @@
363366
"Int64DataTypeName",
364367
"Int64FillValue",
365368
"JSONValue",
369+
"JSONWriter",
370+
"JSONWriterLike",
371+
"JSONWriterSchema",
366372
"MetadataValidationError",
367373
"NumpyDatetime64DataTypeName",
368374
"NumpyDatetime64FillValue",

‎packages/zarr-metadata/src/zarr_metadata/model/__init__.py‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -43,8 +43,11 @@
4343
invalid model is never built. `to_json` returns the document as written.
4444
`update` reads new members in the model's own scope. `with_context` and
4545
`refined_in` read the document in another scope. `to_key_value` writes
46-
the model as it is. A v3 group reads its `consolidated_metadata` as an
47-
envelope and does not read the documents it lists, because consolidated
46+
the model as it is, with a `JSONWriter`, or a mapping of its settings, deciding how
47+
the bytes are spelled (indentation, key order, ASCII escaping, trailing
48+
newline); the default is that of `json.dumps`. A v3 group reads its
49+
`consolidated_metadata` as an envelope and does not read the documents
50+
it lists, because consolidated
4851
metadata is a convention of the reference implementation, not core
4952
metadata. `ZarrV3ConsolidatedMetadata` and
5053
`read_consolidated_metadata_v3` read those documents. The model also
@@ -98,6 +101,7 @@
98101
read_node_metadata_v3,
99102
read_openable_node_metadata_v3,
100103
)
104+
from zarr_metadata.model._json_writer import JSONWriter, JSONWriterLike, JSONWriterSchema
101105
from zarr_metadata.model._repair import (
102106
Repair,
103107
RepairKind,
@@ -122,6 +126,9 @@
122126

123127
__all__ = [
124128
"UNSET",
129+
"JSONWriter",
130+
"JSONWriterLike",
131+
"JSONWriterSchema",
125132
"Repair",
126133
"RepairKind",
127134
"ZarrV2ArrayMetadata",

0 commit comments

Comments
 (0)