Skip to content

fix(s1): correct the emitted STAC items and the data model that rejects them - #261

Merged
lhoupert merged 2 commits into
s1-tilingfrom
fix/216-stac-and-data-model
Sep 10, 2026
Merged

lhoupert merged 2 commits into
s1-tilingfrom
fix/216-stac-and-data-model

Conversation

@lhoupert

@lhoupert lhoupert commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #216 — targets s1-tiling, not main, so this diff reviews on its own.

Fixes two production defects in the S1 RTC catalogue path. S1RtcRoot.from_zarr rejected every store this repo's own writer produces (23 extra_forbidden errors: the fixture it validated against predated the x/y/spatial_ref coordinates and the per-level time added in #192). And every emitted STAC item failed the render extension it declared — renders sat under properties rather than the item root, and Render Objects omitted their one required field, assets. Also: malformed store names now raise instead of minting a bad grid:code, sat:*_orbit == 0 reads as "not recorded" rather than tripping the extension's minimum: 1, and _utm_to_wgs84 densifies so a UTM zone 1/60 tile gets a 1.75° antimeridian-crossing bbox instead of a 358° one spanning the globe.

For reviewers

renders is emitted twice, deliberately. The item root is authoritative (what render v1.0.0 requires); the copy under properties is a temporary compatibility mirror, because three data-pipeline consumers still read the old location and register_per_acquisition.py:132 subscripts it directly — it would raise KeyError the moment the data-model pin bumps. Both code sites carry a COMPATIBILITY MIRROR comment naming all three; remove the mirror once they read the root.

Not pinned by a test: jsonschema is in no dependency group, so the render/sat schema claims were verified by running stac-validator by hand.

Author attestation

  • I am a human, these are my changes, and I have reviewed and understood every change and can explain why each is correct.

AI-assisted: implementation and review with Claude Code, including an adversarial review pass that found three of the defects fixed here. Reviewed commit by commit.

lhoupert and others added 2 commits September 9, 2026 16:16
…ts them

Stacked on `s1-tiling`; this is the STAC-builder and data-model half of the #216 remediation,
split out so the catalogue-facing changes can be reviewed on their own.

## The data model rejected the stores this repo's own writer produces

`S1RtcRoot.from_zarr` failed with 23 `extra_forbidden` errors on any real store: the member
TypedDicts are `closed=True` and omitted `x`/`y`/`spatial_ref` at every level, plus `time` on the
overviews (written since #192). Every model test stayed green because they only ever validated a
static fixture that predated both.

Adding the keys is not the fix on its own. Both TypedDicts are `closed=True, total=False`, so
every key is NotRequired: adding keys only relaxes `extra_forbidden` and could never detect a
writer that *stops* emitting a coordinate — exactly the #192 regression class. So each dataset
class also gets a `model_validator` asserting the coordinates are present. `Required[...]` was
not used; it would break the documented incremental-construction pattern.

The fixture is regenerated from real writer output by a committed script
(`tests/_test_data/s1_rtc_examples/regenerate.py`) that dumps with the *generic* `GroupSpec`,
never `S1RtcRoot` — dumping through the model would record only what the model already accepts
and close the loop falsely. `TestWriterRoundTrip` validates a freshly ingested store in both
directions and is the check that cannot go stale; a further test asserts the fixture still
matches writer output, so a hand-edit cannot reopen the drift one level down.

Also relaxes the conditions group from requiring `gamma_area_*` to requiring any condition
array: `ingest_s1tiling_conditions` gates only on "at least one condition path" and all three CLI
flags default to None, so a lia-only group is writable today and was then unopenable.

## Every emitted STAC item failed the render extension it declared

`_rgb_render` omitted `assets`, the one required field of a Render Object — and the block was
emitted under `properties`, while render v1.0.0's Item branch requires `renders` at the item
ROOT (its own examples/item-landsat8.json agrees; only the README's "Item Properties" table says
otherwise). Both are fixed. Verified by running stac-validator against emitted items; no test
pins it, because `jsonschema` is not installed and is in no dependency group — a test that
imported it would error rather than skip, and marking it `network` would be silently deselected
by CI's `-m "not network"`. The tests assert the required fields and their placement directly.

`sat:relative_orbit` / `sat:absolute_orbit` are declared `{"type": "integer", "minimum": 1}`, and
both coordinates are created with `fill_value=0`, so a torn append — or the known upstream bug
putting 0 on live slices — made a validating STAC API refuse the item. `0` now reads as "not
recorded" rather than orbit zero.

A cube may only claim a single-valued orbit field when every slice agrees AND every slice is
recorded. Values are pooled only from orbit groups that carry the coordinate, so a dual-orbit
cube with one group written before those coordinates existed would otherwise assert one track for
acquisitions from both orbit directions — while correctly omitting `sat:orbit_state`.

Other fixes: a malformed tile id is now rejected rather than minted into `grid:code`, which is
what tile-filtered searches and the cube-to-acquisition cross-links join on; `providers` and
`created` are emitted (a STAC API search returns bare items, so an item without `providers` loses
its attribution); the inert timestamps extension is dropped; `_utm_to_wgs84` densifies with
`transform_bounds(densify_pts=21)`, so a UTM zone 1/60 tile gets a 1.75-degree crossing bbox
instead of a 358-degree one that spanned the globe, with `_bbox_to_geometry` splitting it into a
two-part MultiPolygon at the antimeridian and `_union_wgs84` unwrapping before taking extremes;
and `to_epsg()` returning None falls back to WKT instead of emitting a null reference system.

## Cross-repo follow-ups (data-pipeline, not fixed here)

- `tests/unit/test_build_s1_rtc_collections.py:184` compares `_rgb_render` output with `assets`
  stripped from one side only; it breaks on the next pin bump.
- `_reorient_item_to_orbit` (`scripts/register_v1_s1_rtc.py`) rewrites the render `expression`
  but not the new `assets`, so a dual-orbit cube pinned to a descending preview ships a
  self-contradicting render block. Rendering is unaffected (`_render_to_query` allow-lists
  fields), so it is silent metadata corruption.
- `run_ingest_register.py` builds the store path from an unvalidated `--tile-id` and registers
  after ingesting, so a typo now leaves an orphaned S3 cube plus a failed step. Validate the tile
  id at argparse time.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_011TEYQYdyBLTXuEQJwgMZEC
The move of `renders` from `properties` to the item root is correct --
render v1.0.0's Item branch requires it at the root -- but it is a silent
breaking change for three data-pipeline consumers on that repo's `main`:

  register_per_acquisition.py:132  d["properties"]["renders"]["rgb"]  KeyError
  register_v1_s1_rtc.py:57         props.get("renders", {}).get("rgb") -> None
  register_v1.py:191               item.properties.get("renders")     -> None

The two `.get` sites fail silently, which turns `_reorient_item_to_orbit`
into a no-op: every dual-orbit cube would ship the preferred orbit's render
regardless of which slice the preview is pinned to. All three break the
moment the data-model pin bumps.

Emit `renders` in both places until those consumers read the root. The root
copy stays authoritative; the mirror is built from the same object so the
two cannot drift.

On per-acquisition items both copies must be re-pointed: `props` is cloned
from the cube base, so mirroring without the override would leave the
*preferred* orbit's render in `properties` and the run orbit's at the root
-- reintroducing the same defect one level down. Test added for exactly
that (verified failing without the fix).

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EzfLgkdrTVoY6YQKDDNuQr
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant