Skip to content

Specify which CPM root metadata is carried through to GeoZarr output (stac_discovery is stale by construction) #257

Closed

Description

@emmanuelmathot

Problem

The converter copies the input product's root attributes through to the output verbatim (write_store_root_stac_metadata, called from simple_root_consolidation). For CPM products that means other_metadata, processing_history and stac_discovery land in the GeoZarr output unchanged.

None of it is specified anywhere. We do not state what we carry through, what we drop, what we rewrite, or what a consumer may rely on. The result is metadata that is stale by construction.

Evidence

EODC's 31 August S2 L2A product (built from the cpm-geozarr-eodc branch):

  • the output root carries stac_discovery with 83 assets, each {href, title} only;
  • every href points at an input path, e.g. /conditions/geometry/sun_angles;
  • none references the pyramid levels the conversion creates — r120m, r360m, r720m appear nowhere in it.

So the store ships a STAC object describing a layout the store no longer has, with no marker saying so. A consumer reading it gets a wrong answer rather than no answer.

Why it matters now

  1. Correctness. A copied-through object that contradicts the store is worse than an absent one.
  2. Source of truth. This is the same question as EOPF-Explorer/coordination#302 — whether the in-store object, the producer's catalogue, or our catalogue is authoritative. Copying an unmodified input object into a restructured output quietly asserts the first, while making it untrue.
  3. A latent collision. write_store_root_stac_metadata does an unconditional root.attrs.update() after write_store_root_geo_metadata has written the minispec footprint. Current CPM inputs carry no spatial:bbox / proj:code / zarr_conventions at the root, so nothing collides today — but we have asked CPM to start declaring zarr_conventions there (https://gitlab.eopf.copernicus.eu/cpm/eopf-cpm/-/issues/1189). When that lands, this silently overwrites our computed footprint.

What to decide

  • Which input root attributes we carry through, drop, or rewrite — written down, not implicit.
  • For stac_discovery specifically: rewrite the assets to describe the output layout (pyramid groups, per the multiscales convention), replace it with a stac:link to the authoritative catalogue, or drop it. Copying it unchanged is the one option that is definitely wrong.
  • Make the root-attribute merge explicit about precedence, so computed GeoZarr metadata is never silently overwritten by passthrough.

Related

Activity

  1. lhoupert commented on Sep 21, 2026

    @lhoupert
    Contributor

    unchanged on the 20 Sep product (S2C_MSIL2A_20260920T134841_N0512_R110_T27WWV_20260920T170508, sentinel-2-l2a-zarr3).

    The store has the pyramid r10m/r20m/r60m/r120m/r360m/r720m all resolve. But the in-store stac_discovery doesn't mention it: 84 assets, every href an input path, zero referencing r120m/r360m/r720m.

  2. shollendEODC commented on Sep 21, 2026

    @shollendEODC
    Collaborator

    Thanks for noticing. We have not finished this discussion during the meetings yet, and then the issue slipped past me during the recent release (the code is implemented, also for NOT overwriting theoretical cpm based zarr_conventions, but not executed on conversion).

    Do some functionalities on your end depend on the inclusion of this metadata in STAC or is does it solely impact the completeness of the assets?

  3. added
    questionFurther information is requested
    cpm-driverGeoZarr driver for EOPF CPM (eopf convert-geozarr, engine geozarr)
    stacSTAC metadata and items
    on Sep 28, 2026
  4. shollendEODC commented on Oct 14, 2026

    @shollendEODC
    Collaborator

    This is closed as per #292
    Future PRs will have to address the same issue for other products, but the functionality is in place and can be utilized as such

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

cpm-driverGeoZarr driver for EOPF CPM (eopf convert-geozarr, engine geozarr)questionFurther information is requestedstacSTAC metadata and items

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions