Skip to content

S1 RTC STAC builder fails on non-consolidated cube store (same-orbit time-slice append) #188

Closed

Description

@lhoupert

Symptom

build_s1_rtc_stac_item() (src/eopf_geozarr/stac/s1_rtc.py) opens the cube with zarr.open_consolidated(zarr_store, zarr_format=3), which raises:

ValueError: Consolidated metadata requested with 'use_consolidated=True' but not found in ''.

In the live S1 GRD RTC pipeline this fails STAC registration (register_v1_s1_rtc.py / register_per_acquisition.py), so the cube item never registers / the previous item is left deleted.

Trigger (reproduced)

The per-tile cube (s1-rtc-{tile}.zarr) accumulates one time slice per acquisition. The failure occurs specifically when a slice is appended to an existing same-orbit group (e.g. the 2nd+ ascending acquisition for a tile):

Scenario Result
Fresh single-acquisition store ✅ builds
Append a new orbit group (descending into an ascending store) ✅ builds
Append a slice to an existing same-orbit group (2nd ascending) ❌ ValueError: Consolidated metadata ... not found

Observed live on staging: 30TWM (descending appended into an ascending store) built fine; 31TEH (ascending appended into an existing ascending store) failed; a clean-slate 31TEH (fresh store) built fine.

Why it happens

The data-pipeline ingest does call consolidate_s1_store() after every acquisition (ingest_v1_s1_rtc.py:189,195 → s1_ingest.py:688-700, which consolidates the orbit group then the root). Yet after a same-orbit append on the S3 store, the root is read back without consolidated metadata, so open_consolidated raises. titiler reads the very same stores fine (its /info returns 200) because it does not require consolidated metadata.

Fix (done)

The builder must not require consolidated metadata — fall back to a direct zarr.open_group(zarr_store, mode="r", zarr_format=3) when it's absent (re-raising any other ValueError). Shipped in PR #180 (commit bbd18e3) with a regression test (tests/test_s1_stac.py::test_builds_from_non_consolidated_store: a 2-slice same-orbit store built with consolidate=False — fails before the fix, passes after).

Open sub-question (separate, not blocking)

Why does consolidate_s1_store() not leave the root consolidated after an S3 same-orbit append, despite being called? Likely S3 zarr-store consolidation behavior (note geozarr.py:1235 checks store.supports_consolidated_metadata). Worth tracking separately — the builder-tolerant fix unblocks production regardless, but losing consolidated metadata on appends forfeits its read optimization for all consumers that rely on it.

Activity

  1. added
    bugSomething isn't working
    sentinel-1Sentinel-1 products (GRD, SLC, OCN, GRD RTC)
    stacSTAC metadata and items
    on Sep 28, 2026
  2. emmanuelmathot commented on Sep 28, 2026

    @emmanuelmathot
    Contributor

    Closing: the fix is on main since #180. The STAC builder opens the cube with zarr.open_group(..., use_consolidated=None), so it uses consolidated metadata when present and lists the store otherwise. tests/test_s1_stac.py::test_builds_from_non_consolidated_store covers the same-orbit append case.

    @lhoupert, the side question remains: why consolidate_s1_store() does not leave the root consolidated after a same-orbit append on S3. Do you still see it in production? If yes, please open a separate issue for it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingsentinel-1Sentinel-1 products (GRD, SLC, OCN, GRD RTC)stacSTAC metadata and items

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions