Skip to content

docs(zarr-metadata): say that to_json returns arrays as tuples - #441

Closed
d-v-b wants to merge 5 commits into
mainfrom
docs/zarr-metadata-to-json-tuples
Closed

d-v-b wants to merge 5 commits into
mainfrom
docs/zarr-metadata-to-json-tuples

Conversation

@d-v-b

@d-v-b d-v-b commented Nov 6, 2026 •

Copy link
Copy Markdown
Owner

🤖 AI text below 🤖

Agent summary

The to_json docstrings and the models guide now say what to_json returns: a copy in which each JSON array is a tuple, as the package's JSON types spell an array, so it does not compare equal to what json.loads returns. They point to comparing models instead. The guide had said to_json returns plain containers.

A downstream user (refkv) reported that ZarrV3ArrayMetadata.from_json(doc).to_json() == doc is false and that isinstance(x, list) checks on the result fail. Tuples are deliberate: every JSON TypedDict in the package declares arrays as tuple[...], and to_json returns what its type declares, as pydantic's model_dump() keeps tuple fields as tuples. The docs were what was wrong.

  • The six model to_json docstrings (v3 array, group, consolidated metadata; v2 array, group, .zmetadata) say each array is a tuple and each object a new dict, and point to from_json(document) == model for comparison.
  • docs/models.md replaces "to_json returns plain containers" with the same explanation, noting that model comparison also treats an omitted default as equal to the default written out.

The agent rebased the branch onto main after zarr-developers#4605 split the landing page into guide pages; the sentence it corrects moved from docs/index.md to docs/models.md. just check in packages/zarr-metadata passes.

🤖 Generated with Claude Code

d-v-b added a commit that referenced this pull request Nov 6, 2026
Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
d-v-b and others added 4 commits November 8, 2026 23:11
…cannot open (zarr-developers#4604)

* feat(zarr-metadata): add a reader that reports what a reader cannot open

`read_openable_node_metadata_v3` reads a v3 `zarr.json` as
`read_node_metadata_v3` does and adds a problem of the new kind
`unclaimed_declaration` for each declaration a reader must understand
that nothing in scope claims: an unknown codec, data type or chunk grid,
or an unknown top-level member without `"must_understand": false`. Its
reading has no problems exactly when a reader that understands the scope
can open the node.

The other readers keep accepting unclaimed declarations, so the format
stays open; their docstrings and the parsing guide now point to
`unclaimed_declarations` and the new reader. Prompted by refkv's report
that an unknown v3 member parses without error.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(zarr-metadata): number the changelog fragment for #440

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(zarr-metadata): number the unopenable-declarations fragment after upstream PR 4604

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
… each spec claim (zarr-developers#4605)

* docs(zarr-metadata): split the landing page into guide pages and link each spec claim

Shrink index.md to a short pitch, one example and a reader-by-reader list
of where to go. Move the rest into guide pages: models and readers, what
the readers check, consolidated metadata, repairing known writer bugs,
writing an extension, JSON Schema and Pydantic. Each claim backed by a
Zarr spec links to the spec section that makes it; the consolidated
metadata page says what the specs do and do not settle and links the
zarr-specs PRs and issues and the original zarr-python implementation.
Trim the README so it no longer mirrors the landing page, and say
"context" where the pages said "scope".

Assisted-by: ClaudeCode:claude-fable-5-1
Co-Authored-By: Claude Fable 5.1 <[email protected]>

* docs(zarr-metadata): number the guide-pages fragment after its fork PR

Assisted-by: ClaudeCode:claude-fable-5-1
Co-Authored-By: Claude Fable 5.1 <[email protected]>

* docs(zarr-metadata): say "member" for a JSON name/value pair in the guide pages

The rename branch keeps "member" for a name/value pair of a metadata
object, so the new pages follow: "a top-level member that the spec does
not define is an extra field". "Extra field" and "extension field" stay,
after the names extra_fields and ZarrV3ExtensionField.

Assisted-by: ClaudeCode:claude-fable-5-1
Co-Authored-By: Claude Fable 5.1 <[email protected]>

* docs(zarr-metadata): say "member" for a JSON name/value pair on every page

Sweep the parsing-validation and context pages, which said "field" for a
name/value pair of a metadata object, and state the one exception on the
models page: "extra field" follows the spec's "additional fields" and
the names extra_fields, ExtraFieldDefinition and must_understand_fields;
"field" otherwise means a field of a Python class, or a part of a
structured data type.

Assisted-by: ClaudeCode:claude-fable-5-1
Co-Authored-By: Claude Fable 5.1 <[email protected]>

* docs(zarr-metadata): say "member" in the two must_understand_fields sentences

The models page said "each unclaimed field" and "any such field" where the
landing page and every neighboring sentence say "member". Roborev flagged
the mismatch.

Assisted-by: ClaudeCode:claude-fable-5-1
Co-Authored-By: Claude Fable 5.1 <[email protected]>

* docs(zarr-metadata): say why the additional-fields link points at the latest spec

v3.1.0 has no "Additional fields" heading; PR 373 added it after the
release, so the link cannot be pinned. Say so beside the link.

Assisted-by: ClaudeCode:claude-fable-5-1
Co-Authored-By: Claude Fable 5.1 <[email protected]>

* docs(zarr-metadata): split the additional-fields sentence and date its claim

Roborev found the three-clause sentence hard to parse and its "no tagged
release yet" unanchored in time. Two sentences now, anchored to v3.1.0,
with a note to pin the link once a release carries the heading.

Assisted-by: ClaudeCode:claude-fable-5-1
Co-Authored-By: Claude Fable 5.1 <[email protected]>

* docs(zarr-metadata): fix six wording and example issues roborev found on the guide pages

The pydantic example now builds the manifest with model_validate, so it
type-checks as the next sentence claims. The consolidated page says three
other readers take a context, not two. The equality section quotes the v2
spec on a missing .zattrs in two sentences. The two numbered Sphinx
anchors are replaced by line ranges in the spec source at the v3.1.0 tag.
The landing page says once that the library does no chunk or store I/O.
The extensions page example and the zarr_metadata.definition docstring
example are pinned to each other by a test.

Assisted-by: ClaudeCode:claude-fable-5-1
Co-Authored-By: Claude Fable 5.1 <[email protected]>

* test(zarr-metadata): make the example-drift test say what it could not find

Roborev asked for diagnostics: the test now asserts there is exactly one
extensions-session block and exactly one marker line in the docstring,
each with a message, and compares line lists so a mismatch points at the
differing line.

Assisted-by: ClaudeCode:claude-fable-5-1
Co-Authored-By: Claude Fable 5.1 <[email protected]>

* docs(zarr-metadata): build the extension example with union and say "context" throughout

On main a Context has no update method, so the acme.lz4 example on the
extensions page now adds its definition with union, as the definition
module docstring does. Two leftover "scope"s in the parsing page say
"context", and the models page says "member" for an extra field. The
example-drift test imports at module level.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(zarr-metadata): number the guide-pages fragment after upstream PR 4605

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(zarr-metadata): fence the module docstring examples as python so the API pages highlight them

The examples in the definition, json_schema, pydantic and typed_json
module docstrings were indented code blocks. An indented block names no
language and pymdownx.highlight does not guess one, so the API reference
rendered them as plain text. Each is now a ```python fence, and the
example-drift test reads the definition docstring's fence.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(zarr-metadata): write the guide pages in short sentences and lists

Long comma-chained sentences on the guide pages, README and landing page
are split into short sentences, and enumerations become bulleted or
numbered lists. models, validation-rules, consolidated-metadata, repair,
extensions, json-schema and pydantic are restructured; context and
parsing-validation keep their narrative voice and only have their long
sentences split. No link, claim or code block changes.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
The guide said `to_json` returns plain containers, while another section
said it returns tuples. Each model's `to_json` docstring now says that
every JSON array is a tuple, as the package's JSON types spell one, so
the result does not compare equal to `json.loads` output, and points to
comparing models instead. Prompted by refkv's report.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
@d-v-b
d-v-b force-pushed the docs/zarr-metadata-to-json-tuples branch from 920e7ba to 5dfc78d Compare November 9, 2026 12:31
@d-v-b
d-v-b marked this pull request as draft November 9, 2026 12:32
d-v-b added a commit that referenced this pull request Nov 9, 2026
…developers#4608)

* docs(zarr-metadata): say that to_json returns arrays as tuples

The guide said `to_json` returns plain containers, while another section
said it returns tuples. Each model's `to_json` docstring now says that
every JSON array is a tuple, as the package's JSON types spell one, so
the result does not compare equal to `json.loads` output, and points to
comparing models instead. Prompted by refkv's report.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(zarr-metadata): number the changelog fragment for #441

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(zarr-metadata): number the changelog fragment for zarr-developers#4608

Assisted-by: ClaudeCode:claude-opus-5-5

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
@d-v-b

d-v-b commented Nov 9, 2026

Copy link
Copy Markdown
Owner Author

🤖 AI text below 🤖

Closed as superseded: this branch merged upstream as zarr-developers#4608.

@d-v-b d-v-b closed this Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant