Repository navigation
Closed
Labels
cpm-driverGeoZarr driver for EOPF CPM (eopf convert-geozarr, engine geozarr)GeoZarr driver for EOPF CPM (eopf convert-geozarr, engine geozarr)questionFurther information is requestedFurther information is requestedstacSTAC metadata and itemsSTAC metadata and items
Description
Problem
The converter copies the input product's root attributes through to the output verbatim (write_store_root_stac_metadata, called from simple_root_consolidation). For CPM products that means other_metadata, processing_history and stac_discovery land in the GeoZarr output unchanged.
None of it is specified anywhere. We do not state what we carry through, what we drop, what we rewrite, or what a consumer may rely on. The result is metadata that is stale by construction.
Evidence
EODC's 31 August S2 L2A product (built from the cpm-geozarr-eodc branch):
- the output root carries
stac_discoverywith 83 assets, each{href, title}only; - every href points at an input path, e.g.
/conditions/geometry/sun_angles; - none references the pyramid levels the conversion creates —
r120m,r360m,r720mappear nowhere in it.
So the store ships a STAC object describing a layout the store no longer has, with no marker saying so. A consumer reading it gets a wrong answer rather than no answer.
Why it matters now
- Correctness. A copied-through object that contradicts the store is worse than an absent one.
- Source of truth. This is the same question as EOPF-Explorer/coordination#302 — whether the in-store object, the producer's catalogue, or our catalogue is authoritative. Copying an unmodified input object into a restructured output quietly asserts the first, while making it untrue.
- A latent collision.
write_store_root_stac_metadatadoes an unconditionalroot.attrs.update()afterwrite_store_root_geo_metadatahas written the minispec footprint. Current CPM inputs carry nospatial:bbox/proj:code/zarr_conventionsat the root, so nothing collides today — but we have asked CPM to start declaringzarr_conventionsthere (https://gitlab.eopf.copernicus.eu/cpm/eopf-cpm/-/issues/1189). When that lands, this silently overwrites our computed footprint.
What to decide
- Which input root attributes we carry through, drop, or rewrite — written down, not implicit.
- For
stac_discoveryspecifically: rewrite the assets to describe the output layout (pyramid groups, per the multiscales convention), replace it with astac:linkto the authoritative catalogue, or drop it. Copying it unchanged is the one option that is definitely wrong. - Make the root-attribute merge explicit about precedence, so computed GeoZarr metadata is never silently overwritten by passthrough.
Related
- Cpm geozarr eodc changes #247 introduced the passthrough — flagged there, deliberately not blocking it.
- EOPF-Explorer/coordination#302 — STAC source-of-truth decision.
- https://gitlab.eopf.copernicus.eu/cpm/eopf-cpm/-/issues/1189 — asking CPM to adopt the STAC Zarr Convention.
- There are also three overlapping root-writing helpers now (
write_store_root_geo_metadata,write_store_root_bboxwhich merely wraps it, andwrite_store_root_stac_metadata); worth consolidating while doing this.
Metadata
Metadata
Assignees
Labels
cpm-driverGeoZarr driver for EOPF CPM (eopf convert-geozarr, engine geozarr)GeoZarr driver for EOPF CPM (eopf convert-geozarr, engine geozarr)questionFurther information is requestedFurther information is requestedstacSTAC metadata and itemsSTAC metadata and items
Activity
lhoupert commentedon Sep 21, 2026
unchanged on the 20 Sep product (S2C_MSIL2A_20260920T134841_N0512_R110_T27WWV_20260920T170508, sentinel-2-l2a-zarr3).
The store has the pyramid r10m/r20m/r60m/r120m/r360m/r720m all resolve. But the in-store stac_discovery doesn't mention it: 84 assets, every href an input path, zero referencing r120m/r360m/r720m.
shollendEODC commentedon Sep 21, 2026
Thanks for noticing. We have not finished this discussion during the meetings yet, and then the issue slipped past me during the recent release (the code is implemented, also for NOT overwriting theoretical cpm based zarr_conventions, but not executed on conversion).
Do some functionalities on your end depend on the inclusion of this metadata in STAC or is does it solely impact the completeness of the assets?
shollendEODC commentedon Oct 14, 2026
This is closed as per #292
Future PRs will have to address the same issue for other products, but the functionality is in place and can be utilized as such