fix(zarr-metadata): fix the problems found by the adversarial review of the readers - #382

🤖 AI text below 🤖

Readers, validators and parsers now stop at 256 levels of nesting (JSON_DEPTH) and report a deeper level as an invalid_value problem. Before, a small document nested a thousand levels deep raised RecursionError from every entry point, and so did json.loads on store bytes made of 100,000 [ (now an invalid_json problem). A chain of groups, each holding the next in its consolidated metadata, was also unbounded; each document is now read from its position in the outer document, so depth is counted from one root.

Other fixes in this PR

• A Read or Unclaimed object placed inside a document passed to a reader is refused as not JSON. Before, it passed validation and let a model write a document the same validator refused.
• Absent members are UNSET, not None: a reading's data_type, chunk_grid and chunk_key_encoding when the key is missing, and Refused.json when the field was not JSON. A field with no name is a missing_key problem.
• consolidated_metadata: null (written by a zarr-python 3.0.x bug) is an invalid_type problem. It is no longer read as absent, and neither JSON Schema admits it.
• Extension names must match ^[a-z][a-z0-9-_.]+$ or be a URI in RFC 3986 characters. Other names are refused before any definition is looked up. A definition built with an invalid name raises TypeError. A raw-bits name with more than 100 digits is an extension name, not a size.
• Definition functions always receive a read-only view of the configuration. An error raised by a canonical function names its definition.
• Messages print an over-long integer by its bit length, a too-deep value as "too deep to show", and a non-string key with Python's repr.
• The pydantic types raise ValidationError with one line error per problem (type, loc, input, ctx) instead of one concatenated value_error.

Where it comes from

Six adversarial review passes over the stack #370 → #381: Hypothesis-based verdicts, spec text against code, public API and docs, hostile input on free-threaded 3.14, zarr-python and numpy as oracles, and mutation testing. Fixes to #381's own commits were amended there; this PR carries the rest. Three later commits fix what further review rounds found: the "too deep" message no longer depends on repr overflowing (which it does not on Python 3.14), a dead RecursionError fallback is removed, and six smaller findings are fixed.

What changes for a caller

Verification

• Tests: 1,954 pass on Python 3.14; 1,952 pass and 2 skip on 3.11. pyright, ruff and the strict docs build are clean.
• New tests pin every row above, plus: the deepest shard nesting the cap admits through every reader, writer and walker, with one level more reporting the depth problem; a v2 document and a v3 fill value at the cap; a chain of consolidated groups bounded at the cap; and the nine behaviors the mutation review found unpinned.
• Differential run: 481,228 records compared between fix(zarr-metadata): close the four gaps a reader of foreign stores hits #381 and this PR. Every difference falls into one of the categories above (missing-name kind, null refused, name syntax, depth cap, documents that crashed before and answer now, fields no longer reached past a refusal, a non-JSON v2 dtype reported where it sits).
• Roborev: twenty review passes. Each finding is fixed and pinned by a test or recorded as a documented choice. The session log has the list.

Design choices to look at

1. A depth cap of 256, reported as a problem, instead of iterative walkers. Every value the package reads passes through refine_json, so one cap there protects every walker after it, json.dumps included. 256 leaves about half of Python's default recursion limit to the caller, because the costliest operations (copy.deepcopy, and pickle before 3.12) use two frames per level. Real documents nest below 30 levels.
2. A hand-built Read inside a model is still trusted. dataclasses.replace(model, codecs=(Read(...),)) takes the field as read, as feat(zarr-metadata)!: check each model when it is built, and write it as it is #379 documents. Only documents and update's members are checked. Having Read judge its own configuration is the follow-up feat(zarr-metadata)!: check each model when it is built, and write it as it is #379 named. (Superseded in feat(zarr-metadata)!: make v3 models a pair of document and scope #393, where models can no longer be built from fields.)
3. Integer fill values past 2^53 round twice, as zarrs and tensorstore do. A number is read as float64 first, then converted to the type. Both readers do this; the docstring cites them and a test pins the example 2**60 + 2**36 + 1.
4. The read-only view covers the members only. Deep-freezing would copy every configuration on every hook call.

Left out

• Read judging its configuration when built by hand (choice 2; moot after feat(zarr-metadata)!: make v3 models a pair of document and scope #393).
• 161 surviving mutants from the mutation run, not triaged beyond the nine adopted.
• Caching a model's equality key (done in feat(zarr-metadata)!: make v3 models a pair of document and scope #393).
• vlen-utf8 and vlen-bytes definitions; is_/parse_node_metadata_v3; validate_consolidated_*.
• zarr-python bugs found by the interop review (reserved names written, KeyError on bad consolidated paths, NaN payloads lost, OverflowError on int fills), to be filed separately.

🤖 Generated with Claude Code