Repository navigation
fix(s1): correct the emitted STAC items and the data model that rejects them - #261
Merged
Merged
Conversation
…ts them Stacked on `s1-tiling`; this is the STAC-builder and data-model half of the #216 remediation, split out so the catalogue-facing changes can be reviewed on their own. ## The data model rejected the stores this repo's own writer produces `S1RtcRoot.from_zarr` failed with 23 `extra_forbidden` errors on any real store: the member TypedDicts are `closed=True` and omitted `x`/`y`/`spatial_ref` at every level, plus `time` on the overviews (written since #192). Every model test stayed green because they only ever validated a static fixture that predated both. Adding the keys is not the fix on its own. Both TypedDicts are `closed=True, total=False`, so every key is NotRequired: adding keys only relaxes `extra_forbidden` and could never detect a writer that *stops* emitting a coordinate — exactly the #192 regression class. So each dataset class also gets a `model_validator` asserting the coordinates are present. `Required[...]` was not used; it would break the documented incremental-construction pattern. The fixture is regenerated from real writer output by a committed script (`tests/_test_data/s1_rtc_examples/regenerate.py`) that dumps with the *generic* `GroupSpec`, never `S1RtcRoot` — dumping through the model would record only what the model already accepts and close the loop falsely. `TestWriterRoundTrip` validates a freshly ingested store in both directions and is the check that cannot go stale; a further test asserts the fixture still matches writer output, so a hand-edit cannot reopen the drift one level down. Also relaxes the conditions group from requiring `gamma_area_*` to requiring any condition array: `ingest_s1tiling_conditions` gates only on "at least one condition path" and all three CLI flags default to None, so a lia-only group is writable today and was then unopenable. ## Every emitted STAC item failed the render extension it declared `_rgb_render` omitted `assets`, the one required field of a Render Object — and the block was emitted under `properties`, while render v1.0.0's Item branch requires `renders` at the item ROOT (its own examples/item-landsat8.json agrees; only the README's "Item Properties" table says otherwise). Both are fixed. Verified by running stac-validator against emitted items; no test pins it, because `jsonschema` is not installed and is in no dependency group — a test that imported it would error rather than skip, and marking it `network` would be silently deselected by CI's `-m "not network"`. The tests assert the required fields and their placement directly. `sat:relative_orbit` / `sat:absolute_orbit` are declared `{"type": "integer", "minimum": 1}`, and both coordinates are created with `fill_value=0`, so a torn append — or the known upstream bug putting 0 on live slices — made a validating STAC API refuse the item. `0` now reads as "not recorded" rather than orbit zero. A cube may only claim a single-valued orbit field when every slice agrees AND every slice is recorded. Values are pooled only from orbit groups that carry the coordinate, so a dual-orbit cube with one group written before those coordinates existed would otherwise assert one track for acquisitions from both orbit directions — while correctly omitting `sat:orbit_state`. Other fixes: a malformed tile id is now rejected rather than minted into `grid:code`, which is what tile-filtered searches and the cube-to-acquisition cross-links join on; `providers` and `created` are emitted (a STAC API search returns bare items, so an item without `providers` loses its attribution); the inert timestamps extension is dropped; `_utm_to_wgs84` densifies with `transform_bounds(densify_pts=21)`, so a UTM zone 1/60 tile gets a 1.75-degree crossing bbox instead of a 358-degree one that spanned the globe, with `_bbox_to_geometry` splitting it into a two-part MultiPolygon at the antimeridian and `_union_wgs84` unwrapping before taking extremes; and `to_epsg()` returning None falls back to WKT instead of emitting a null reference system. ## Cross-repo follow-ups (data-pipeline, not fixed here) - `tests/unit/test_build_s1_rtc_collections.py:184` compares `_rgb_render` output with `assets` stripped from one side only; it breaks on the next pin bump. - `_reorient_item_to_orbit` (`scripts/register_v1_s1_rtc.py`) rewrites the render `expression` but not the new `assets`, so a dual-orbit cube pinned to a descending preview ships a self-contradicting render block. Rendering is unaffected (`_render_to_query` allow-lists fields), so it is silent metadata corruption. - `run_ingest_register.py` builds the store path from an unvalidated `--tile-id` and registers after ingesting, so a typo now leaves an orphaned S3 cube plus a failed step. Validate the tile id at argparse time. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_011TEYQYdyBLTXuEQJwgMZEC
The move of `renders` from `properties` to the item root is correct --
render v1.0.0's Item branch requires it at the root -- but it is a silent
breaking change for three data-pipeline consumers on that repo's `main`:
register_per_acquisition.py:132 d["properties"]["renders"]["rgb"] KeyError
register_v1_s1_rtc.py:57 props.get("renders", {}).get("rgb") -> None
register_v1.py:191 item.properties.get("renders") -> None
The two `.get` sites fail silently, which turns `_reorient_item_to_orbit`
into a no-op: every dual-orbit cube would ship the preferred orbit's render
regardless of which slice the preview is pinned to. All three break the
moment the data-model pin bumps.
Emit `renders` in both places until those consumers read the root. The root
copy stays authoritative; the mirror is built from the same object so the
two cannot drift.
On per-acquisition items both copies must be re-pointed: `props` is cloned
from the cube base, so mirroring without the override would leave the
*preferred* orbit's render in `properties` and the run orbit's at the root
-- reintroducing the same defect one level down. Test added for exactly
that (verified failing without the fix).
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EzfLgkdrTVoY6YQKDDNuQr
lhoupert
marked this pull request as ready for review
September 10, 2026 12:26
This was referenced Sep 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #216 — targets
s1-tiling, notmain, so this diff reviews on its own.Fixes two production defects in the S1 RTC catalogue path.
S1RtcRoot.from_zarrrejected every store this repo's own writer produces (23extra_forbiddenerrors: the fixture it validated against predated thex/y/spatial_refcoordinates and the per-leveltimeadded in #192). And every emitted STAC item failed the render extension it declared —renderssat underpropertiesrather than the item root, and Render Objects omitted their one required field,assets. Also: malformed store names now raise instead of minting a badgrid:code,sat:*_orbit == 0reads as "not recorded" rather than tripping the extension'sminimum: 1, and_utm_to_wgs84densifies so a UTM zone 1/60 tile gets a 1.75° antimeridian-crossing bbox instead of a 358° one spanning the globe.For reviewers
rendersis emitted twice, deliberately. The item root is authoritative (what render v1.0.0 requires); the copy underpropertiesis a temporary compatibility mirror, because three data-pipeline consumers still read the old location andregister_per_acquisition.py:132subscripts it directly — it would raiseKeyErrorthe moment the data-model pin bumps. Both code sites carry aCOMPATIBILITY MIRRORcomment naming all three; remove the mirror once they read the root.Not pinned by a test:
jsonschemais in no dependency group, so the render/sat schema claims were verified by runningstac-validatorby hand.Author attestation
AI-assisted: implementation and review with Claude Code, including an adversarial review pass that found three of the defects fixed here. Reviewed commit by commit.