Skip to content

Commit bc177b3

Browse files
d-v-bclaude
andauthored
feat(zarr-metadata)!: read what zarr-python writes in a ZARR_PYTHON_CONTEXT scope (zarr-developers#4583)
* feat(zarr-metadata)!: read what zarr-python writes in a ZARR_PYTHON_CONTEXT scope Add definitions for the extensions zarr-python writes. zarr-extensions registers `fixed_length_utf32` and the `vlen-utf8` and `vlen-bytes` codecs, so they join CORE_AND_EXTENSIONS_CONTEXT. zarr-python's own `null_terminated_bytes` and `variable_length_bytes`, which no specification registers, are only in the new ZARR_PYTHON_CONTEXT, so the default scope stays spec-pure. Rename CORE to CORE_CONTEXT and CORE_AND_EXTENSIONS to CORE_AND_EXTENSIONS_CONTEXT, so that a scope's name says it is a context. Fixes #423 Assisted-by: ClaudeCode:claude-opus-5-5 * docs(zarr-metadata): number the changelog fragments for PR 426 Assisted-by: ClaudeCode:claude-opus-5-5 * docs(zarr-metadata): number the changelog fragments for PR 4583 Assisted-by: ClaudeCode:claude-opus-5-5 * fix(zarr-metadata): keep the old scope names and tighten the string rules Keep `CORE` and `CORE_AND_EXTENSIONS` importable from `zarr_metadata.v3` and `zarr_metadata.v3.definition` as the same scopes with a DeprecationWarning, so a consumer such as pydantic-zarr keeps working across the rename. Check that `vlen-utf8` encodes `string` and `vlen-bytes` encodes `bytes` (or zarr-python's `variable_length_bytes`), as zarr-extensions says, instead of accepting any data type whose values vary in size. Do not count trailing null bytes against the size of a `null_terminated_bytes` fill value, since they are padding. Spell a `fixed_length_utf32` fill value without its trailing U+0000 padding, so `"a"` and `"a\u0000"` have one canonical spelling. Assisted-by: ClaudeCode:claude-opus-5-5 * feat(zarr-metadata): read zarr-python's raw_bytes and rename CORE_V2 Add zarr-python's `raw_bytes` data type (NumPy's `V`) to ZARR_PYTHON_CONTEXT, so the scope reads every data type zarr-python writes. Its names start with `ZarrPython`, since `RawBytes` already names the spec's raw bits. A fill value longer than `length_bytes` is a problem, because zarr-python cuts it short, and the canonical spelling pads it with null bytes, as zarr-python writes it. Rename CORE_V2 to CORE_V2_CONTEXT, so every scope's name ends in `_CONTEXT`. The old name keeps working with a DeprecationWarning, through the same `renamed_scope` helper as the v3 names, which moves to `zarr_metadata._renamed`. Share the base64 decoding and spelling of fill values in `_byte.py`, and say that the vlen codecs' `configuration` key is optional, since zarr-python writes an empty object. Assisted-by: ClaudeCode:claude-opus-5-5 * refactor(zarr-metadata): declare the v2 scope's old name once Move CORE_V2_CONTEXT and its table of old names into `v2/_core.py`, as `v3/_core.py` holds the v3 scopes, so `zarr_metadata.v2` and `zarr_metadata.v2.definition` share one table. Say why a `fixed_length_utf32` fill value counts its trailing U+0000 code points, as the spec requires, while `null_terminated_bytes` follows zarr-python and does not, and test a value that only its padding makes too long. Assisted-by: ClaudeCode:claude-opus-5-5 * fix(zarr-metadata): give main's strategies the renamed scopes and new definitions The Hypothesis strategies merged from main (zarr-developers#4575) imported `CORE`, `CORE_AND_EXTENSIONS` and `CORE_V2` from modules that now hold only the `_CONTEXT` names, and their coverage test requires a strategy for every definition in `CORE_AND_EXTENSIONS_CONTEXT`. Rename the imports, and add strategies for `fixed_length_utf32`, `vlen-utf8` and `vlen-bytes`; each vlen codec draws nothing for a data type it does not read. Assisted-by: ClaudeCode:claude-opus-5-5 * fix(zarr-metadata): draw zarr-python's own data types in the testing strategies `DEFAULT_STRATEGIES` holds a strategy for each definition the package defines, but `null_terminated_bytes`, `variable_length_bytes` and `raw_bytes` had none, so `fields()` raised `LookupError` for them. Draw each, and check the strategies against `ZARR_PYTHON_CONTEXT`, which holds every definition. Say that the documents in test_zarr_python_scope.py leave out the compressors zarr-python adds. Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <[email protected]> * test(zarr-metadata): pin the registered extensions to CORE_AND_EXTENSIONS_CONTEXT The strategies are now checked against `ZARR_PYTHON_CONTEXT`, which no longer proves that `fixed_length_utf32`, `vlen-utf8` and `vlen-bytes` are in `CORE_AND_EXTENSIONS_CONTEXT`. Assert it directly. Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <[email protected]> * refactor(zarr-metadata): let the vlen strategies ask their codec's rules Each vlen codec strategy named its compatible data types a second time; draw nothing for exactly the chunks the codec's own chunk rules refuse, so the two cannot drift. Draw `fixed_length_utf32` fill values with trailing U+0000 padding too, which the canonical spelling removes. Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <[email protected]> --------- Co-authored-by: Claude Opus 5.5 <[email protected]>
1 parent be4d9d4 commit bc177b3

74 files changed

Lines changed: 1987 additions & 664 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎packages/zarr-metadata/README.md‎

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ closely model the content of the Zarr specifications, such as:
4141
`TypedDict` with its rules, and the scope that decides which definitions
4242
read a document. `zarr_metadata.v3` and `zarr_metadata.v2` export every
4343
definition and scope of their format, such as `BYTES_CODEC` and
44-
`CORE_AND_EXTENSIONS`.
44+
`CORE_AND_EXTENSIONS_CONTEXT`.
4545
- **JSON type checking** ([`zarr_metadata.typed_json`](https://zarr-metadata.readthedocs.io/en/latest/api/typed_json/)):
4646
`check` checks a JSON value against any `TypedDict` and reports every
4747
problem with its location.
@@ -104,7 +104,9 @@ of context-free consistency rules, e.g.:
104104
In a v3 document, validation also checks each extension point: the data
105105
type, the chunk grid, the chunk key encoding, each codec, and each storage
106106
transformer. Each one is checked by the definition that claims its name in
107-
a scope. The scope is `CORE_AND_EXTENSIONS` unless a `context` is passed.
107+
a scope. The scope is `CORE_AND_EXTENSIONS_CONTEXT` unless a `context` is passed.
108+
`ZARR_PYTHON_CONTEXT` adds the data types that zarr-python writes but no
109+
specification registers.
108110
If the definition rejects a configuration, the reader rejects it too,
109111
and a configuration key the definition does not declare is reported as
110112
`unknown_key`. If no definition in the scope claims a name, the reader
@@ -232,13 +234,13 @@ each field:
232234
```python test="true" session="readme"
233235
from zarr_metadata.model import read_array_metadata_v3
234236
from zarr_metadata.definition import canonical_of, with_problems
235-
from zarr_metadata.v3.definition import CORE
237+
from zarr_metadata.v3.definition import CORE_CONTEXT
236238

237239
reading = read_array_metadata_v3(raw)
238240
beyond_core = [
239241
loc
240242
for loc, field in reading.fields()
241-
if field.name is not None and CORE.claimant(field.read_as, field.name) is None
243+
if field.name is not None and CORE_CONTEXT.claimant(field.read_as, field.name) is None
242244
]
243245
simplest = {
244246
loc: canonical_of(field, problems)
@@ -283,15 +285,15 @@ scope claims is an `UnclaimedField` too, also when it sits inside another
283285
field, such as a codec inside a shard. A model's `unclaimed_fields` lists
284286
every field nothing in scope claims, at any depth, by its location, and
285287
leaves out the members that waive `must_understand`: a reader can open
286-
the node when it is empty. `CORE` holds
288+
the node when it is empty. `CORE_CONTEXT` holds
287289
`CONSOLIDATED_METADATA_EXTRA_FIELD`, which reads a group's
288290
`consolidated_metadata` as the envelope the spec fixes. A group's
289291
`consolidated()` reads the documents that member lists, in the group's
290292
own scope.
291293

292294
A v3 model consists of its document and the scope it was read in.
293295
`ZarrV3ArrayMetadata(document, context=None)` reads the document in the
294-
scope (`CORE_AND_EXTENSIONS` when none is given) and raises
296+
scope (`CORE_AND_EXTENSIONS_CONTEXT` when none is given) and raises
295297
`MetadataValidationError` with every problem, so an invalid model is never
296298
built. `to_json` returns the document as it was written, and
297299
`to_key_value` writes that same document to store bytes.
@@ -322,7 +324,7 @@ all of them with `ZarrV3ConsolidatedMetadata`, or with
322324
`read_consolidated_metadata_v3`, which returns problems instead of
323325
raising them. A document with a problem then fails only when it is read.
324326
Pass `context=group.context` to read the documents in the group's own
325-
scope; with no `context`, they are read in `CORE_AND_EXTENSIONS`.
327+
scope; with no `context`, they are read in `CORE_AND_EXTENSIONS_CONTEXT`.
326328

327329
To write consolidated metadata, build a `ZarrV3ConsolidatedMetadata` and
328330
pass it to the group's `update`. The group reads the listed models in its
@@ -332,7 +334,7 @@ models as entries. Such an entry is accepted when its claims refine into
332334
the model's scope, and refused at its path otherwise. To consolidate
333335
children that were read in several scopes, consolidate them in the
334336
`union` of those scopes. Every reader takes `context=None` for the
335-
default scope, `CORE_AND_EXTENSIONS`.
337+
default scope, `CORE_AND_EXTENSIONS_CONTEXT`.
336338

337339
Two models are equal when they represent the same document, even if each
338340
is spelled differently. Values the package interprets (each field, and the
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Add the `ZARR_PYTHON_CONTEXT` scope, which reads what zarr-python writes: `CORE_AND_EXTENSIONS_CONTEXT` plus zarr-python's own data types `null_terminated_bytes` (NumPy's `S`), `raw_bytes` (NumPy's `V`) and `variable_length_bytes`, which no specification registers. `CORE_AND_EXTENSIONS_CONTEXT` now also reads the `fixed_length_utf32` data type (NumPy's `U`) and the `vlen-utf8` and `vlen-bytes` codecs that zarr-extensions registers, so the strings zarr-python writes are checked instead of read as unclaimed.
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Rename the scopes `CORE`, `CORE_AND_EXTENSIONS` and `CORE_V2` to `CORE_CONTEXT`, `CORE_AND_EXTENSIONS_CONTEXT` and `CORE_V2_CONTEXT`, so that a scope's name says it is a context. The old names, imported from `zarr_metadata.v3`, `zarr_metadata.v3.definition`, `zarr_metadata.v2` or `zarr_metadata.v2.definition`, still refer to the same scopes with a `DeprecationWarning`, and will be removed in a later release.

‎packages/zarr-metadata/docs/api/index.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ The package is organized to mirror the structure of the Zarr specifications:
2222
- [`zarr_metadata.v2`](v2.md) — `TypedDict` shapes for Zarr v2 documents
2323
(`.zarray`, `.zgroup`, `.zattrs`, `.zmetadata`), and
2424
`zarr_metadata.v2.definition`: the v2 kinds of metadata, the scope
25-
`CORE_V2`, and the readers of one `dtype` or codec. Each dtype family's
25+
`CORE_V2_CONTEXT`, and the readers of one `dtype` or codec. Each dtype family's
2626
definition is in `zarr_metadata.v2.data_type`, and each codec's in
2727
`zarr_metadata.v2.codec`
2828
- [`zarr_metadata.v3`](v3/index.md) — `TypedDict` shapes for Zarr v3
@@ -35,7 +35,7 @@ The package is organized to mirror the structure of the Zarr specifications:
3535
scope type `Context`. Its module docstring is the guide
3636
- [`zarr_metadata.v3.definition`](v3/definition.md) — the five v3 kinds
3737
of metadata (`CodecDefinition`, `DataTypeDefinition`, ...), the codec
38-
pipeline types, and the v3 scopes `CORE` and `CORE_AND_EXTENSIONS`
38+
pipeline types, and the v3 scopes `CORE_CONTEXT` and `CORE_AND_EXTENSIONS_CONTEXT`
3939

4040
The document types, models, and spec vocabulary — including the store keys —
4141
are re-exported at the top level, so

‎packages/zarr-metadata/docs/api/v3/codec.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,4 +22,8 @@ title: codec
2222

2323
::: zarr_metadata.v3.codec.transpose
2424

25+
::: zarr_metadata.v3.codec.vlen_bytes
26+
27+
::: zarr_metadata.v3.codec.vlen_utf8
28+
2529
::: zarr_metadata.v3.codec.zstd

‎packages/zarr-metadata/docs/api/v3/data_type.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,8 +40,16 @@ title: data_type
4040

4141
::: zarr_metadata.v3.data_type.string
4242

43+
::: zarr_metadata.v3.data_type.fixed_length_utf32
44+
4345
::: zarr_metadata.v3.data_type.numpy_datetime64
4446

4547
::: zarr_metadata.v3.data_type.numpy_timedelta64
4648

4749
::: zarr_metadata.v3.data_type.struct
50+
51+
::: zarr_metadata.v3.data_type.null_terminated_bytes
52+
53+
::: zarr_metadata.v3.data_type.variable_length_bytes
54+
55+
::: zarr_metadata.v3.data_type.zarr_python_raw_bytes

‎packages/zarr-metadata/docs/index.md‎

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ closely model the content of the Zarr specifications, such as:
3939
`TypedDict` with its rules, and the scope that decides which definitions
4040
read a document. `zarr_metadata.v3` and `zarr_metadata.v2` export every
4141
definition and scope of their format, such as `BYTES_CODEC` and
42-
`CORE_AND_EXTENSIONS`.
42+
`CORE_AND_EXTENSIONS_CONTEXT`.
4343
- **JSON type checking** ([`zarr_metadata.typed_json`](api/typed_json.md)):
4444
`check` checks a JSON value against any `TypedDict` and reports every
4545
problem with its location.
@@ -102,7 +102,9 @@ of context-free consistency rules, e.g.:
102102
In a v3 document, validation also checks each extension point: the data
103103
type, the chunk grid, the chunk key encoding, each codec, and each storage
104104
transformer. Each one is checked by the definition that claims its name in
105-
a scope. The scope is `CORE_AND_EXTENSIONS` unless a `context` is passed.
105+
a scope. The scope is `CORE_AND_EXTENSIONS_CONTEXT` unless a `context` is passed.
106+
`ZARR_PYTHON_CONTEXT` adds the data types that zarr-python writes but no
107+
specification registers.
106108
If the definition rejects a configuration, the reader rejects it too,
107109
and a configuration key the definition does not declare is reported as
108110
`unknown_key`. If no definition in the scope claims a name, the reader
@@ -234,13 +236,13 @@ each field:
234236
```python test="true" session="index"
235237
from zarr_metadata.model import read_array_metadata_v3
236238
from zarr_metadata.definition import canonical_of, with_problems
237-
from zarr_metadata.v3.definition import CORE
239+
from zarr_metadata.v3.definition import CORE_CONTEXT
238240

239241
reading = read_array_metadata_v3(raw)
240242
beyond_core = [
241243
loc
242244
for loc, field in reading.fields()
243-
if field.name is not None and CORE.claimant(field.read_as, field.name) is None
245+
if field.name is not None and CORE_CONTEXT.claimant(field.read_as, field.name) is None
244246
]
245247
simplest = {
246248
loc: canonical_of(field, problems)
@@ -285,15 +287,15 @@ scope claims is an `UnclaimedField` too, also when it sits inside another
285287
field, such as a codec inside a shard. A model's `unclaimed_fields` lists
286288
every field nothing in scope claims, at any depth, by its location, and
287289
leaves out the members that waive `must_understand`: a reader can open
288-
the node when it is empty. `CORE` holds
290+
the node when it is empty. `CORE_CONTEXT` holds
289291
`CONSOLIDATED_METADATA_EXTRA_FIELD`, which reads a group's
290292
`consolidated_metadata` as the envelope the spec fixes. A group's
291293
`consolidated()` reads the documents that member lists, in the group's
292294
own scope.
293295

294296
A v3 model consists of its document and the scope it was read in.
295297
`ZarrV3ArrayMetadata(document, context=None)` reads the document in the
296-
scope (`CORE_AND_EXTENSIONS` when none is given) and raises
298+
scope (`CORE_AND_EXTENSIONS_CONTEXT` when none is given) and raises
297299
`MetadataValidationError` with every problem, so an invalid model is never
298300
built. `to_json` returns the document as it was written, and
299301
`to_key_value` writes that same document to store bytes.
@@ -324,7 +326,7 @@ all of them with `ZarrV3ConsolidatedMetadata`, or with
324326
`read_consolidated_metadata_v3`, which returns problems instead of
325327
raising them. A document with a problem then fails only when it is read.
326328
Pass `context=group.context` to read the documents in the group's own
327-
scope; with no `context`, they are read in `CORE_AND_EXTENSIONS`.
329+
scope; with no `context`, they are read in `CORE_AND_EXTENSIONS_CONTEXT`.
328330

329331
To write consolidated metadata, build a `ZarrV3ConsolidatedMetadata` and
330332
pass it to the group's `update`. The group reads the listed models in its
@@ -334,7 +336,7 @@ models as entries. Such an entry is accepted when its claims refine into
334336
the model's scope, and refused at its path otherwise. To consolidate
335337
children that were read in several scopes, consolidate them in the
336338
`union` of those scopes. Every reader takes `context=None` for the
337-
default scope, `CORE_AND_EXTENSIONS`.
339+
default scope, `CORE_AND_EXTENSIONS_CONTEXT`.
338340

339341
Two models are equal when they represent the same document, even if each
340342
is spelled differently. Values the package interprets (each field, and the

‎packages/zarr-metadata/docs/parsing-validation.md‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -161,7 +161,7 @@ This procedure yields one of two values:
161161

162162
`zarr-metadata` does not run each stage as a separate function. Instead, a single read of the document collects the problems from every stage, so a caller gets the full list of problems at once.
163163

164-
The **registry** from the resolve-and-validate stage is called a **scope** in this library. For Zarr V3 it is a [`ZarrV3Context`](api/v3/definition.md) object, with one slot for each kind of field (`data_types`, `codecs`, `chunk_grids`, ...), and every reader takes a `context` argument. The default scope is `CORE_AND_EXTENSIONS`, which holds the data types, codecs, chunk grids, etc. defined in the Zarr V3 spec and in [zarr-extensions](https://github.com/zarr-developers/zarr-extensions/). `CORE` holds only the definitions from the Zarr V3 spec.
164+
The **registry** from the resolve-and-validate stage is called a **scope** in this library. For Zarr V3 it is a [`ZarrV3Context`](api/v3/definition.md) object, with one slot for each kind of field (`data_types`, `codecs`, `chunk_grids`, ...), and every reader takes a `context` argument. The default scope is `CORE_AND_EXTENSIONS_CONTEXT`, which holds the data types, codecs, chunk grids, etc. defined in the Zarr V3 spec and in [zarr-extensions](https://github.com/zarr-developers/zarr-extensions/). `CORE_CONTEXT` holds only the definitions from the Zarr V3 spec. `ZARR_PYTHON_CONTEXT` adds to `CORE_AND_EXTENSIONS_CONTEXT` the data types that zarr-python writes but no specification registers (`null_terminated_bytes` for NumPy's `S`, `raw_bytes` for NumPy's `V`, and `variable_length_bytes`), so a strict check of what zarr-python writes passes it.
165165

166166
`read_array_metadata_v3` reads a JSON document and returns a "reading". `reading.problems` is a tuple of the problems, and `reading.metadata` is the validated model, or `None` if there are problems. Each problem has a `loc`, which is its position in the document as a tuple, a `kind`, like `missing_key` or `invalid_value`, and a human-readable `message`:
167167

@@ -193,10 +193,10 @@ To read one field on its own, such as a codec, use `resolve` from `zarr_metadata
193193

194194
```python test="true" session="parsing-validation"
195195
from zarr_metadata.definition import resolve
196-
from zarr_metadata.v3.definition import CORE_AND_EXTENSIONS, CodecDefinition
196+
from zarr_metadata.v3.definition import CORE_AND_EXTENSIONS_CONTEXT, CodecDefinition
197197

198198
field, problems = resolve(
199-
{"name": "gzip", "configuration": {"level": 12}}, CodecDefinition, CORE_AND_EXTENSIONS
199+
{"name": "gzip", "configuration": {"level": 12}}, CodecDefinition, CORE_AND_EXTENSIONS_CONTEXT
200200
)
201201
for problem in problems:
202202
print(problem.loc, problem.kind)
@@ -214,17 +214,17 @@ To make an unknown data type or codec known, add its definition to the scope's s
214214

215215
```python test="true" session="parsing-validation"
216216
from zarr_metadata.definition import EmptyConfiguration
217-
from zarr_metadata.v3.definition import CORE_AND_EXTENSIONS, CodecDefinition
217+
from zarr_metadata.v3.definition import CORE_AND_EXTENSIONS_CONTEXT, CodecDefinition
218218

219219
# The definition of the codec that was unknown: here, a codec named
220220
# `acme.codec` that takes no configuration.
221221
MY_CODEC = CodecDefinition(
222222
name="acme.codec", configuration=EmptyConfiguration, kind="bytes_bytes", size="dynamic"
223223
)
224-
context = CORE_AND_EXTENSIONS.update(codecs=[*CORE_AND_EXTENSIONS.codecs.values(), MY_CODEC])
224+
context = CORE_AND_EXTENSIONS_CONTEXT.update(codecs=[*CORE_AND_EXTENSIONS_CONTEXT.codecs.values(), MY_CODEC])
225225
reading = read_array_metadata_v3(raw, context=context)
226226
```
227227

228-
`update` replaces each slot it is given and keeps the others. A slot can also be given as a mapping from each definition's name to it, so `CORE_AND_EXTENSIONS.update(codecs={**CORE_AND_EXTENSIONS.codecs, "gzip": MY_GZIP})` replaces the gzip definition with your own.
228+
`update` replaces each slot it is given and keeps the others. A slot can also be given as a mapping from each definition's name to it, so `CORE_AND_EXTENSIONS_CONTEXT.update(codecs={**CORE_AND_EXTENSIONS_CONTEXT.codecs, "gzip": MY_GZIP})` replaces the gzip definition with your own.
229229

230230
[^1]: The one exception is the `r*` data type, which is identified by a templated string with no `configuration` object. This mixes the identifier with the configuration in the same JSON value, which is not good design. So this library normalizes `r*` into the same form as all the other metadata objects: the field `"r16"` keeps its name, and is read by the `r*` definition with the configuration `{"bits": 16}`.

‎packages/zarr-metadata/src/zarr_metadata/__init__.py‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -136,6 +136,16 @@
136136
TransposeCodecMetadata,
137137
TransposeCodecName,
138138
)
139+
from zarr_metadata.v3.codec.vlen_bytes import (
140+
VLEN_BYTES_CODEC_NAME,
141+
VlenBytesCodecMetadata,
142+
VlenBytesCodecName,
143+
)
144+
from zarr_metadata.v3.codec.vlen_utf8 import (
145+
VLEN_UTF8_CODEC_NAME,
146+
VlenUtf8CodecMetadata,
147+
VlenUtf8CodecName,
148+
)
139149
from zarr_metadata.v3.codec.zstd import ZSTD_CODEC_NAME, ZstdCodecMetadata, ZstdCodecName
140150
from zarr_metadata.v3.consolidated import (
141151
ZARR_V3_CONSOLIDATED_METADATA_KEY,
@@ -162,6 +172,11 @@
162172
Complex128DataTypeName,
163173
Complex128FillValue,
164174
)
175+
from zarr_metadata.v3.data_type.fixed_length_utf32 import (
176+
FIXED_LENGTH_UTF32_DATA_TYPE_NAME,
177+
FixedLengthUtf32DataTypeName,
178+
FixedLengthUtf32FillValue,
179+
)
165180
from zarr_metadata.v3.data_type.float16 import (
166181
FLOAT16_DATA_TYPE_NAME,
167182
Float16DataTypeName,
@@ -265,6 +280,7 @@
265280
"DEFAULT_CHUNK_KEY_ENCODING_NAME",
266281
"DEFAULT_CHUNK_KEY_ENCODING_SEPARATOR",
267282
"ENDIANNESS",
283+
"FIXED_LENGTH_UTF32_DATA_TYPE_NAME",
268284
"FLOAT16_DATA_TYPE_NAME",
269285
"FLOAT32_DATA_TYPE_NAME",
270286
"FLOAT64_DATA_TYPE_NAME",
@@ -291,6 +307,8 @@
291307
"UNSET",
292308
"V2_CHUNK_KEY_ENCODING_NAME",
293309
"V2_CHUNK_KEY_ENCODING_SEPARATOR",
310+
"VLEN_BYTES_CODEC_NAME",
311+
"VLEN_UTF8_CODEC_NAME",
294312
"ZARR_V2_ARRAY_DIMENSION_SEPARATOR",
295313
"ZARR_V2_ARRAY_METADATA_STORE_KEY",
296314
"ZARR_V2_ARRAY_ORDER",
@@ -325,6 +343,8 @@
325343
"DefaultChunkKeyEncodingName",
326344
"DefaultChunkKeyEncodingSeparator",
327345
"Endianness",
346+
"FixedLengthUtf32DataTypeName",
347+
"FixedLengthUtf32FillValue",
328348
"Float16DataTypeName",
329349
"Float16FillValue",
330350
"Float32DataTypeName",
@@ -378,6 +398,10 @@
378398
"V2ChunkKeyEncodingName",
379399
"V2ChunkKeyEncodingSeparator",
380400
"ValidationProblem",
401+
"VlenBytesCodecMetadata",
402+
"VlenBytesCodecName",
403+
"VlenUtf8CodecMetadata",
404+
"VlenUtf8CodecName",
381405
"ZarrV2ArrayDimensionSeparator",
382406
"ZarrV2ArrayMetadata",
383407
"ZarrV2ArrayMetadataJSON",
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
"""Scopes this package renamed, which keep their old names for now with a `DeprecationWarning`."""
2+
3+
import warnings
4+
from collections.abc import Mapping
5+
6+
7+
def renamed_scope(module: str, name: str, renames: Mapping[str, tuple[str, object]]) -> object:
8+
"""The scope that `name`, an old name in `renames`, refers to, with a `DeprecationWarning`.
9+
10+
`renames` maps each old name to the scope's name now and the scope. A
11+
module that exported the old names calls this from its `__getattr__`,
12+
so an import of an old name keeps working. Raises `AttributeError`, as
13+
attribute lookup does, when `name` is not an old name.
14+
"""
15+
if name not in renames:
16+
raise AttributeError(f"module {module!r} has no attribute {name!r}")
17+
new, scope = renames[name]
18+
warnings.warn(
19+
f"`{name}` is deprecated and will be removed; use `{new}`, the same scope",
20+
DeprecationWarning,
21+
stacklevel=3,
22+
)
23+
return scope
24+
25+
26+
__all__ = ["renamed_scope"]

0 commit comments

Comments
 (0)