Skip to content

Commit 764cdc9

Browse files
d-v-bclaude
andcommitted
docs(zarr-metadata): the fragments describe the configuration record
Assisted-by: ClaudeCode:claude-fable-5-1 Co-Authored-By: Claude Fable 5.1 <[email protected]>
1 parent 848d700 commit 764cdc9

5 files changed

Lines changed: 66 additions & 43 deletions

File tree

‎packages/zarr-metadata/changes/4379.feature.10.md‎

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,17 @@
11
An entity is the value guarantee, not just the type one. Construction
2-
refuses values the spec disallows, so `BloscCodec(clevel=99)` raises
3-
rather than serializing a document no reader will accept, and there is no
4-
such thing as an invalid entity to hold. Code that wants unvalidated
5-
metadata already has somewhere to put it: `Opaque`, which carries the
6-
JSON verbatim.
2+
refuses values the spec disallows, so `BloscCodec(BloscOptions(clevel=99))`
3+
raises rather than serializing a document no reader will accept, and
4+
there is no such thing as an invalid entity to hold. Code that wants
5+
unvalidated metadata already has somewhere to put it: `Opaque`, which
6+
carries the JSON verbatim, and the configuration record itself, which is
7+
the options as written and not yet judged.
78

8-
One piece: a function of the instance that yields the values the spec
9-
disallows as it finds them, bound on the class as `problems`. The
10-
constructor stops at the first and raises `MetadataValidationError`;
11-
`coerce` runs it to the end and reports every problem in the document
12-
instead of raising; a consumer holding an entity may run it too, and
13-
stop or collect as it likes. `create_unchecked(**members)` builds the
14-
record without the check, for a reader that judges afterwards.
9+
One piece: the configuration record's own `problems`, which yields the
10+
values the spec disallows as it finds them. The entity's constructor
11+
stops at the first and raises `MetadataValidationError`; `coerce` asks
12+
the record before it builds anything and reports every problem in the
13+
document instead of raising; a reader with a record asks it too,
14+
`BloscOptions(...).problems()`, and stops or collects as it likes.
1515

1616
`must_understand` moves to a class variable and out of the configuration
1717
entirely. It is a property of the *kind* of metadata -- a codec is

‎packages/zarr-metadata/changes/4379.feature.7.md‎

Lines changed: 28 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,12 @@ nothing about whether a shard's pipelines are well formed and silencing
2727
them would lose a real judgment. A *required* one stops the entity,
2828
because there is no honest reading of a `blosc` whose level is a string.
2929

30-
The type judgment is read off the entity's own fields rather than
31-
written twice. Which members exist, which may be left out (the type
30+
An entity has the shape of its metadata: its name is the class, and its
31+
configuration, for an entity that adds `Configured` beside its kind, is
32+
a frozen `Configuration` record named in the one field `configuration`
33+
-- `class GzipCodec(BytesBytesCodec, Configured)` with `configuration:
34+
GzipOptions`. The type judgment is read off the record's fields rather
35+
than written twice. Which members exist, which may be left out (the type
3236
admits `UNSET`), and how each is type-checked all follow from the field
3337
annotations: `int`, `float` for any number, `bool`, `str`, a `Literal`
3438
of names, the JSON-value alias, an array homogeneous or fixed, a union
@@ -37,17 +41,17 @@ dataclass, an object of undeclared keys as `Mapping[str, V]`, a
3741
`NewType` as the type it names, and a nested metadata field, written as
3842
its kind with `Opaque`. That covers
3943
every member this package models; an annotation outside them is refused
40-
at registration, and `Annotated[str, FROM_NAME]` marks the one field
41-
carried by the envelope's name rather than a configuration key (`r<N>`),
42-
which `coerce` fills from the envelope. Whether a configuration is required
43-
follows too, since the spec ties the bare-name spelling to whether any member is
44-
required.
44+
at registration, and `Annotated[str, FROM_NAME]` marks the one field of
45+
the entity itself, carried by the envelope's name rather than a
46+
configuration key (`r<N>`), which `coerce` fills from the envelope.
47+
Whether a configuration is required follows too, since the spec ties the
48+
bare-name spelling to whether any member is required.
4549

46-
Nothing is derived from the fields ahead of time: `coerce` parses a
47-
configuration against them as it reads it. Field annotations are
48-
resolved per class, skipping class variables by text, so a `ClassVar`
49-
naming something imported only for the type checker cannot fail
50-
registration.
50+
The record's parser and writer are compiled once per class from its
51+
annotations, and `coerce` runs the parser over the configuration as it
52+
reads it. Field annotations are resolved per class, skipping class
53+
variables by text, so a `ClassVar` naming something imported only for
54+
the type checker cannot fail registration.
5155

5256
One diagnostic became more precise: a malformed `[value, count]` pair in
5357
a rectilinear grid is now reported at the offending element inside the
@@ -57,10 +61,10 @@ An entity that contains other entities declares the field --
5761
`data_type: DataTypeEntity | Opaque`, `codecs: tuple[CodecEntity |
5862
Opaque, ...]`, a record holding one -- and the contained entity is read
5963
through the scope the containing one is read in, at whatever depth the
60-
annotation puts it. Writing it back and putting it in canonical form
61-
are the containing entity's own two lines, `written(self.inner)` in
62-
`to_json` and `replace(self, inner=canonicalized(self.inner))` in
63-
`canonical`, the same way it writes and simplifies its own members.
64+
annotation puts it. Writing it back is the base's, through the contained
65+
entity's own `to_json`; putting it in canonical form is the containing
66+
entity's one line, `self.with_configuration(inner=self.inner.canonical())`
67+
in `canonical`, the same way it simplifies its own members.
6468
`canonical` is the entity itself by default, overridden where two
6569
spellings of its members mean the same -- a rectilinear dimension's
6670
run-length encoding, a `typesize` that `noshuffle` ignores. A field
@@ -69,19 +73,19 @@ refused at registration: no scope could place the one, and the other
6973
lies about what the field holds when the inner name is out of scope.
7074

7175
Everything finer than a type -- a bound, a rule about one member, a
72-
rule that reads two members together -- is a function of the entity's
73-
instance, in plain code, that yields each problem as it finds it,
74-
located relative to the configuration, and is bound on the class as
75-
`problems`. The constructor stops at the first, so `BloscCodec(clevel=99)`
76-
raises `MetadataValidationError`; `coerce` runs it to the end and
77-
reports every problem in the document; a consumer holding an entity may
78-
run it too, stopping or collecting as it needs. The space of
76+
rule that reads two members together -- is the record's own `problems`,
77+
in plain code, yielding each problem as it finds it, located relative
78+
to the configuration. The entity's constructor stops at the first, so
79+
`BloscCodec(BloscOptions(clevel=99))` raises `MetadataValidationError`;
80+
`coerce` asks the record before it builds anything and reports every
81+
problem in the document; a reader with a record asks it too, stopping
82+
or collecting as it needs. The space of
7983
refinements is too wide to capture statically, and reading them in one
8084
place per entity is what will show where a shared form is worth having.
8185

8286
The set of annotation shapes the parser reads is closed: the shapes
8387
JSON takes, and no others. A field annotation outside them is refused
8488
at registration, and the field is written as one of them instead, with
85-
any finer rule in the function bound as `problems`. The parser is one module that knows
89+
any finer rule in the record's `problems`. The parser is one module that knows
8690
nothing of entities; the entity layer hands it the one shape of its own,
8791
a field holding an entity, as a leaf it reads at any depth.

‎packages/zarr-metadata/changes/4379.feature.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,9 +9,10 @@ data type, chunk grid and chunk key encoding — is a class that answers
99
for itself:
1010

1111
- `coerce` reads raw metadata into the entity, or says why it is not one;
12-
- `problems`, a function of the instance bound on the class, yields the
13-
values the spec disallows as it finds them; the constructor stops at
14-
the first and raises, and `coerce` reports every one instead;
12+
- its configuration is a record of its own, `GzipOptions`, whose
13+
`problems` yields the values the spec disallows as it finds them; the
14+
entity's constructor stops at the first and raises, and `coerce`
15+
reports every one instead;
1516
- `to_json` writes it back as it was read, and `canonical` gives its
1617
simplest equivalent spelling;
1718
- `incoming_problems`, `shape_problems`, `fill_value_problems` and

‎packages/zarr-metadata/changes/4379.misc.1.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,9 @@ TypedDicts. The tables cannot drift now, because there is one of each
1212
fact instead of four.
1313

1414
What replaces the drift tests is a correspondence test: an entity's
15-
dataclass fields, its configuration TypedDict, and its member table are
16-
three spellings of one set, and whether the bare-name spelling is
17-
allowed follows from the TypedDict's required keys.
15+
configuration record and its configuration TypedDict are two spellings
16+
of one set of members, with the same requiredness, and whether the
17+
bare-name spelling is allowed follows from that.
1818

1919
One more table went the same way: `zarr_metadata.v3.codec.kind`, which
2020
sorted codec *names* into the spec's three pipeline kinds. A name does

‎packages/zarr-metadata/changes/4379.misc.2.md‎

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,7 @@ registration checks for that are gone. Which of a document's fields
9696
holds which kind is the document's own knowledge, in one place; the
9797
entity layer no longer names a document field.
9898

99-
`to_json` is written once, in the base, from the fields: `writer_for`
99+
`to_json` is written once, in the base, from the configuration record: `writer_for`
100100
is the parser's inverse over the same annotation, so what `coerce` reads
101101
from a document, `to_json` puts back -- the bare name when every member
102102
is absent, the object otherwise, a contained entity through its own
@@ -134,3 +134,21 @@ compiled into it, and the plan for a class, like its resolved field
134134
annotations, is a pure function of the class and cached as one. A
135135
document with a shard, six codecs and a nested pipeline reads in 85 µs
136136
where it took 199, and one codec in 6 µs where it took 21.
137+
138+
An entity has the shape of its metadata. Its name is the class, and its
139+
configuration, for an entity that adds `Configured` beside its kind, is
140+
a frozen `Configuration` record named in the one field `configuration`
141+
-- `class GzipCodec(BytesBytesCodec, Configured)` with `configuration:
142+
GzipOptions`, read as `codec.configuration.level` or through a
143+
`@property` the entity adds for a member it wants at the top level. The
144+
record's fields are the members and its `problems` the rules on them,
145+
so the lift of configuration keys to the entity, which nothing in Python
146+
expressed, is gone with the per-member loop in `coerce` that did it: a
147+
configuration is parsed and written by the parser's and the writer's
148+
record shapes. `with_configuration(**changes)` is the entity with members
149+
of its configuration replaced, checked as any construction is. A record
150+
is the options as written, judged on request; the entity is the value
151+
guarantee, and `coerce` asks the record before it builds one, so no
152+
unchecked construction path exists. The rule a family has about its
153+
name -- `r<N>` a multiple of 8 -- is the entity's `name_problems`, since
154+
a name is not configuration.

0 commit comments

Comments
 (0)