Sentinel-1 GRD γ0T RTC GeoZarr V3 — Implementation Tracker

Overview

Implementation of Sentinel-1 GRD γ0T RTC time-series GeoZarr V3 stores in the data-model codebase. S1Tiling (CNES/OTB) produces γ0T RTC GeoTIFFs on the S2 MGRS grid; this work converts those GeoTIFFs into GeoZarr V3 stores following the EOPF hierarchy.

Design document: .github/prompts/s1-grd-rtc-implementation-plan-v2.md (on s1-tiling branch)
Working branch: s1-tiling

Implementation Phases

Phase 0 — Design and prototyping (commit 1ae1b99, faad006)
Phase 1 — Pydantic-zarr V3 models (PR feat: Pydantic-zarr V3 model for Sentinel-1 GRD γ0T RTC stores #138, commit 4bb1367)
Phase 2 — GeoTIFF ingestion (PR feat: phase 2 — S1 GRD RTC GeoTIFF ingestion pipeline #146)
Phase 3 — Conditions ingestion and discovery (PR feat: Phase 3 — S1Tiling conditions ingestion and discovery #153)
Phase 4 — CLI and S3 (PR feat: Phase 4 — CLI and S3 (#139) #155)
Phase 5 — STAC and Argo

Devlog

2026-03-23 — Phase 1 complete

Delivered: Pydantic-zarr V3 schema for S1 GRD RTC stores, aligned with the S2 model pattern.

Files:

src/eopf_geozarr/data_api/s1_rtc.py — 316 lines, model hierarchy (Root → OrbitGroup → Resolution → Arrays)
tests/_test_data/s1_rtc_examples/s1-grd-rtc-31TCH.json — realistic fixture (MGRS 31TCH, 3 timesteps, sharding codecs)
tests/test_data_api/test_s1_rtc.py — 11 tests (round-trip, structure, 5 negative cases)

Key decisions:

Uses pyz.v3 (Zarr V3) instead of pyz.v2 (Zarr V2) used by S2
extra="allow" on attrs models for forward compatibility
Conditions group uses dict[str, ArraySpec] (dynamic keys per orbit)
Both ascending/descending optional with at-least-one validator

PR #138 opened as draft for review by pydantic-zarr schema maintainers.

Open questions for reviewers:

Should attrs models inherit from a shared base with S2?
Is extra="allow" the right choice, or lock down all fields?
Should multiscales use the typed zarr_cm model instead of dict[str, Any]?
Should conditions enforce a regex pattern beyond gamma_area_ prefix?

2026-03-22 — Phase 0 complete

Delivered: End-to-end prototype from S1Tiling Docker → GeoTIFF → GeoZarr V3.

S1Tiling 1.4.0 ran on 3 acquisitions over MGRS 31TCH (orbits 008, 037, 110)
Discovered EODAG 4.0.0 breaking changes (5 issues, monkey-patch created)
Real-data conversion validated: 10980×10980 sharded arrays, border_mask, gamma_area conditions
Inner chunk divisibility constraint: 10980 % 512 ≠ 0, solution: best_chunk_size() → 366
Store size: ~1.8 GB for 3 timesteps across all overview levels
xarray readback verified for all levels

Should multiscales use the typed zarr_cm model instead of dict[str, Any]?

IMO yes, this will reduce the likelihood of small errors in the multiscales convention declaration

Is extra="allow" the right choice, or lock down all fields?

This depends on how dynamic the source attributes are, and whether we see the geozarr conversion as purely additive. If the goal is to guarantee a strict set of attributes in the output, which means removing attributes from the input, then extra="allow" is probably not what we want. but this depends on the needs / expectations of the consumers.

docs: update Phase 1 checklist, add Phase 1 findings, link tracking i…

feat: Phase 2 — S1 GRD RTC GeoTIFF ingestion pipeline

2025-03-24 — Phase 2 complete: GeoTIFF → Zarr V3 ingestion pipeline

Delivered files

File	Description
`src/eopf_geozarr/conversion/s1_ingest.py` (~500 lines)	Production ingestion pipeline
`tests/test_s1_rtc_ingest.py` (27 tests)	Full test coverage

Public API

ingest_s1tiling_acquisition(vv, vh, mask, store, orbit) — create-or-append one acquisition
consolidate_s1_store(store, orbit) — post-batch consolidation
discover_s1tiling_acquisitions(input_dir) — file discovery with grouping
extract_geotiff_metadata(path) — rasterio-based metadata extraction

Test results: 27/27 passed ✅

Key design decisions

Convention metadata built from zarr_cm CMO dicts (UUID, schema_url, spec_url)
Private _downsample_2d helper for overview generation (mean downsampling)
structlog for structured logging throughout
CRS + shape validation on append (rejects mismatched acquisitions)
1D spatial coordinate arrays (x, y) at every resolution level via np.linspace
Lowercase dimensions ["y", "x"] / ["time", "y", "x"] required by titiler-eopf

Known issues

Pre-existing Phase 1 test failure: test_s1_rtc_orbit_attrs — Multiscales model not subscriptable (Phase 1 model needs TypedDict x/y members)

Branch: feat/s1-rtc-geotiff-ingestion (based on feat/s1-rtc-pydantic-model)
PR ready: feat/s1-rtc-pydantic-model...feat/s1-rtc-geotiff-ingestion

@d-v-b

Devlog update — zcm.Multiscales adoption

Context: Reviewer @d-v-b suggested replacing dict[str, Any] with the typed zcm.Multiscales Pydantic model.

Changes (commit 8cc7df1 on PR #138):

S1RtcOrbitGroupAttrs.multiscales field type changed from dict[str, Any] to Multiscales (imported from eopf_geozarr.data_api.geozarr.multiscales.zcm)
Removed all inline replacement models (MultiscalesTransform, MultiscalesScaleLevel, Multiscales) that were a workaround for Python 3.11 incompatibility
spatial:dimensions validator updated to lowercase ['y', 'x'] to align with Python 3.12 conventions
Test assertions updated to use Pydantic attribute access instead of dict access

Blocker resolved: zcm.py was incompatible with Python 3.11 due to typing.TypedDict vs typing_extensions.TypedDict in Pydantic 2.12. PR #141 bumped minimum Python to 3.12, unblocking the import.

Status: PR #138 rebased on main, force-pushed, 11/11 tests passing.

docs: update Phase 1 checklist, add Phase 1 findings, link tracking i…

feat: Phase 2 — S1 GRD RTC GeoTIFF ingestion pipeline

docs: update Phase 1 checklist, add Phase 1 findings, link tracking i…

feat: Phase 2 — S1 GRD RTC GeoTIFF ingestion pipeline

feat: phase 2 — S1 GRD RTC GeoTIFF ingestion pipeline (#146)

2026-04-02 — Phase 3: Conditions ingestion and discovery

PR: #153 (targets s1-rtc)

Delivered files

File	Description
`src/eopf_geozarr/conversion/s1_ingest.py` (+200 lines)	Conditions ingestion + discovery
`src/eopf_geozarr/cli.py` (+90 lines)	3 new CLI subcommands
`tests/test_s1_rtc_ingest.py` (+20 tests)	Conditions test coverage

Public API additions

ingest_s1tiling_conditions(store, orbit_direction, relative_orbit, gamma_area_path, lia_path, incidence_angle_path) — write per-orbit condition arrays into a conditions group with proj:/spatial: attributes
discover_s1tiling_conditions(input_dir) — discover and group condition GeoTIFFs by tile + orbit

CLI commands

ingest-s1 — ingest a single acquisition (VV + VH + border mask)
ingest-s1-conditions — ingest condition arrays for a given orbit
consolidate-s1 — post-batch Zarr metadata consolidation

Test results: 47/47 passed ✅ (27 existing + 20 new)

Key design decisions

Conditions are time-invariant (Y, X) arrays per orbit (no time dimension), stored as gamma_area_037, lia_037, etc.
Conditions group carries its own proj:code, spatial:dimensions, spatial:transform, spatial:shape attributes
CRS consistency validated against the existing store
Overwrite semantics: re-ingesting the same orbit condition replaces data in-place
Schema validation confirmed: stores with conditions validate against S1RtcRoot Pydantic model

2026-04-02 — Phase 4: CLI and S3

PR: #155 (targets s1-rtc)

Changes

File	Description
`src/eopf_geozarr/cli.py` (+55 lines)	`validate-s1` CLI subcommand
`src/eopf_geozarr/conversion/s1_ingest.py` (~10 lines changed)	S3 output support via `open_zarr_group`/`path_exists`
`tests/test_cli_e2e.py` (+130 lines)	3 E2E CLI tests

New CLI subcommand

validate-s1 --store <path> [--verbose] — validates store against S1RtcRoot Pydantic schema, reports orbit directions, data arrays, coordinates, conditions status

S3 support

ingest_s1tiling_acquisition, ingest_s1tiling_conditions, create_s1_store now use open_zarr_group() and path_exists() from fs_utils — S3 paths work transparently
consolidate_s1_store follows the S2 pattern (from geozarr.py): opens store, passes root.store to zarr.consolidate_metadata()

E2E test pipeline

ingest-s1 → ingest-s1-conditions → consolidate-s1 → validate-s1 → xr.open_zarr() readback — all via subprocess.run against the CLI

Test results: 57/57 passed ✅ (54 existing + 3 new E2E)

https://api.explorer.eopf.copernicus.eu/rstaging/collections/sentinel-1-grd-rtc-staging/items/s1-rtc-31TCH/viewer

@lhoupert

cc @lhoupert. Let me know if you want to take this over together?

fix: support s3:// URIs in S1Tiling discovery and ingest

fix: support s3:// URIs in S1Tiling discovery and ingest (#175)

feat(stac): add build_s1_rtc_stac_item and generate-stac-s1 CLI (#173)

feat(s1-rtc): STAC item builders + ingest fixes for TiTiler rendering (…

feat(s1): Sentinel-1 GRD RTC GeoZarr ingestion, data model and STAC b…

Closing this tracker: all data-model phases are merged.

Phase 0–1: design, prototype and pydantic-zarr V3 models (feat: Pydantic-zarr V3 model for Sentinel-1 GRD γ0T RTC stores #138)
Phase 2: GeoTIFF ingestion (feat: phase 2 — S1 GRD RTC GeoTIFF ingestion pipeline #146)
Phase 3: conditions ingestion and discovery (feat: Phase 3 — S1Tiling conditions ingestion and discovery #153)
Phase 4: CLI and S3 (feat: Phase 4 — CLI and S3 (#139) #155, fix: support s3:// URIs in S1Tiling discovery and ingest #175)
Phase 5, data-model side: STAC item builders, CF time and metadata fixes (feat(s1-rtc): STAC item builders + ingest fixes for TiTiler rendering #180, fix(s1-rtc): CF-encode time at every level for datetime-based rendering #193, fix(s1-rtc): enrich STAC items & consolidate construction into the library (#195) #196, fix(s1-rtc): heal a multiscale level missing time on append (robust writer) #200, feat(s1): Sentinel-1 GRD RTC GeoZarr ingestion, data model and STAC builders #216, fix(s1): correct the emitted STAC items and the data model that rejects them #261)

The remaining work is outside this repository: STAC registration and the Argo pipeline in EOPF-Explorer/data-pipeline#185, and publishing the products in EOPF-Explorer/coordination#288. The ingestion commands are documented in the standalone converter guide (#289).

Navigation Menu

Repository navigation

Sentinel-1 GRD γ0T RTC GeoZarr V3 — Implementation Tracker #139

Overview

Implementation Phases

Devlog

2026-03-23 — Phase 1 complete

2026-03-22 — Phase 0 complete

2025-03-24 — Phase 2 complete: GeoTIFF → Zarr V3 ingestion pipeline

Devlog update — zcm.Multiscales adoption

1 remaining item

2026-04-02 — Phase 3: Conditions ingestion and discovery

2026-04-02 — Phase 4: CLI and S3

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Sentinel-1 GRD γ0T RTC GeoZarr V3 — Implementation Tracker #139

Description

Overview

Implementation Phases

Devlog

2026-03-23 — Phase 1 complete

2026-03-22 — Phase 0 complete

Activity

d-v-b commented on Mar 24, 2026

emmanuelmathot commented on Mar 24, 2026

2025-03-24 — Phase 2 complete: GeoTIFF → Zarr V3 ingestion pipeline

emmanuelmathot commented on Mar 24, 2026

Devlog update — zcm.Multiscales adoption

1 remaining item

emmanuelmathot commented on Apr 2, 2026

2026-04-02 — Phase 3: Conditions ingestion and discovery

emmanuelmathot commented on Apr 2, 2026

2026-04-02 — Phase 4: CLI and S3

emmanuelmathot commented on Apr 2, 2026

emmanuelmathot commented on Apr 21, 2026

emmanuelmathot commented on Sep 28, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions