No way to tell which assets are renderable: AOT/SCL/WVP 500 on one route, empty 200 on the other

I was going through the assets of an S2 L2A item on /rstaging and hit two failures that may have the same cause.

Only reflectance carries bands; AOT_10m, SCL_20m and WVP_10m point at a Zarr Array, not a Group.

B=https://api.explorer.eopf.copernicus.eu/rstaging
P=$B/collections/sentinel-2-l2a/items/S2C_MSIL2A_20260903T135731_N0512_R010_T26WNE_20260903T171314

curl -s "$P/assets"                                                    # ["AOT_10m","SCL_20m","WVP_10m","reflectance"]
curl -s -o /dev/null -w '%{http_code}\n' "$P/assets/reflectance/info"  # 200
curl -s -o /dev/null -w '%{http_code}\n' "$P/assets/SCL_20m/info"      # 500
curl -s "$P/info?assets=SCL_20m"                                       # 200, body is {}

A problem I see is that /assets advertises four assets and nothing says three of them are unusable. Asking for one make the API answers two different ways

Route	Now
`…/assets/SCL_20m/info`	500 `{"error":"Internal server error"}`	unhandled, no reason given
`…/info?assets=SCL_20m`	200 `{}`	success with nothing in it

The 200 is something that worries me a bit, can we find another code to say to a client that an asset is not supported ?

@lhoupert

Thanks @lhoupert

The main issue is that the /assets return a list of assets which have valid media type

titiler-eopf/titiler/eopf/stac.py

Lines 24 to 31 in f8a07ef

    
           _VALID_TYPE = { 
        
               *DEFAULT_VALID_TYPE, 
        
               "application/x-zarr", 
        
               "application/vnd.zarr", 
        
               "application/vnd+zarr", 
        
               "application/vnd.zarr; version=3", 
        
               "application/vnd.zarr; version=3; profile=multiscales", 
        
           }

titiler-eopf won't perform any data check before the users tries to open the dataset, which happens on /info

I can look at the SCL_20m to understand what's the issue with this asset.

for /info?assets= we choose to discard the error

titiler-eopf/titiler/eopf/stac.py

Lines 287 to 298 in f8a07ef

    
           # Some STAC assets (e.g. AOT/SCL/WVP) point at a single Zarr Array rather 
        
           # than a Group, and `GeoZarrReader` opens every asset as a `DataTree`. 
        
           # Opening an Array path as a DataTree raises `ContainsArrayError` (xarray 
        
           # tries `zarr.open_group`/`open_consolidated` on a path that is actually 
        
           # an Array). Since `.info()` fans out over *all* assets, allow that 
        
           # exception so those non-group assets are skipped instead of failing 
        
           # the whole request. 
        
           allowed_exceptions = kwargs.pop("allowed_exceptions", ()) 
        
           allowed_exceptions += (zarr.errors.ContainsArrayError,) 
        
           infos = super().info( 
        
               assets=assets, **kwargs, allowed_exceptions=allowed_exceptions 
        
           )

but we could revert

curl https://api.explorer.eopf.copernicus.eu/rstaging/collections/sentinel-2-l2a/items/S2C_MSIL2A_20260903T135731_N0512_R010_T26WNE_20260903T171314/assets/SCL_20m/info

{
  "detail": "An array already exists in store object_store://HTTPStore(\"https://s3.explorer.eopf.copernicus.eu/esa-zarr-sentinel-explorer-fra/tests-output/sentinel-2-l2a/S2C_MSIL2A_20260903T135731_N0512_R010_T26WNE_20260903T171314.zarr/conditions/mask/l2a_classification/r20m/scl\") at path ."
}

This means that SCL_20m asset is not a Group but an array, which is not supported by the GeoZarrReader

ref: pydata/xarray#2660

@vincentsarago

Thanks @vincentsarago . Not sure reverting is a good option. I also dug in this and Claude found a few things we could fix to solve this issue

AOT_10m / WVP_10m — our bug, no titiler change needed. GeoZarrReader opens quality/atmosphere fine (/r10m:aot, /r10m:wvp, …); the assets just point one level too deep. We repoint them at the group with variables=.

SCL_20m — our converter. conditions/mask/l2a_classification/r20m|r60m have empty attrs, no spatial_ref, no grid_mapping on scl, because we only write geo metadata for /measurements/ and /quality/ paths. Needs a data-model fix + re-registration.

Question fo you @vincentsarago , could we make the error code more informative for single-asset route instead of 500 ( 400, or 404 like InvalidAssetName?)

@emmanuelmathot could you validate the point 2. SCL part . Do you think that writing the spatial/proj attrs on the conditions/mask groups is the right fix in data-model?

fix(stac): point S2 AOT/WVP at the store root, not the atmosphere group

@emmanuelmathot

Did you have to look at this one @emmanuelmathot and maybe this EOPF-Explorer/data-model#262 ? it would help close https://github.com/EOPF-Explorer/coordination/issues/304

@emmanuelmathot

@emmanuelmathot could you validate the point 2. SCL part . Do you think that writing the spatial/proj attrs on the conditions/mask groups is the right fix in data-model?

l2a_classification needs prj/spatial and multiscales without adding overviews

@emmanuelmathot

can you review this PR EOPF-Explorer/data-model#262 when you have a minute @emmanuelmathot ? Thank you!

@vincentsarago

Status, tested today with the EODC Zarr sample service as reference data (sentinel-2-l2a-zarr3, new asset layout, item S2C_MSIL2A_20261109T135351_N0513_R110_T26VLR_20261109T153305), on current main:

Asset	href	`/assets/{asset}/info`	`/info?assets={asset}`
`reflectance`	`/measurements/reflectance` (multiscales)	200	200, b01…b8a
`scl`	`/conditions/mask/l2a_classification` (multiscales)	200	200, `scl`
`mask`	`/quality/probability` (multiscales)	200	200, `cld`, `snw`
`product`	`/`	200	200
`atmosphere`	`/quality/atmosphere/r10m`	200, empty	200, empty

Data side is fixed. feat!: reintegrate cpm-geozarr into main data-model#277 and #318 write the geo metadata and multiscales on conditions/mask/l2a_classification, and the EODC scl asset now points to that group. An SCL preview renders. On our side, fix(stac): point S2 AOT/WVP assets at the store root (titiler-eopf#163) data-pipeline#412 and #477 align the asset hrefs.
Products in sentinel-2-l2a and sentinel-2-l2a-staging that were converted before these fixes still return 500 for SCL_20m until they are reprocessed.
New case of the "empty 200" problem: atmosphere. Its href points to the single-level group quality/atmosphere/r10m. That group has proj:code and spatial:* but no multiscales, and GeoZarrReader finds no variables in it, so both routes return an empty 200 with no error. On the parent multiscale group quality/atmosphere, the reader finds aot and wvp. Either the reader must support a single-level GeoZarr group, or the asset must point to quality/atmosphere. In both cases, an asset with no readable variable must not return an empty 200.
API side is fix: raise proper 400 error when store point to an array #165: 400 with a clear message on the single-asset route, and an error (not 200 {}) when an asset is requested explicitly in /info?assets=. Only :all: skips it. @vincentsarago, fix: raise proper 400 error when store point to an array #165 needs a rebase and this change before the next review. The atmosphere case above should get the same treatment.

This issue will close when #165 merges.

@emmanuelmathot

@emmanuelmathot are you sure quality/atmosphere/r10m is a group?

The GeoZarrReader should not fail to open Groups (as DataTree). It should only fail for Arrays.

@vincentsarago

@vincentsarago yes, it is a group, and the reader does not fail. It opens it with no data variables: only the multiscale groups (quality/atmosphere, measurements/reflectance, …) carry consolidated_metadata, not the level groups (…/r10m), and the HTTP store cannot list. measurements/reflectance/r10m gives the same empty result.

Options: point the asset to quality/atmosphere, or fall back to the root consolidated metadata for a subgroup. In both cases, no variables must give an error, not an empty 200.

ok, so this is all going back to my original comment. I think we should help the reader by providing a valid GeoZarr Media-Type so we avoid to list assets that are not compatible with the GeoXarrayReader.

#163 (comment)

@vincentsarago

@vincentsarago agreed. On the data side, this is fixed in eopf-stac 1.1.0 (EOPF-Sample-Service/eopf-stac#88): atmosphere now points to quality/atmosphere with profile=multiscales, like scl and mask. EODC has not deployed it yet; the newest sentinel-2-l2a-zarr3 items still have the old asset.

@lhoupert can you track the EODC deployment of eopf-stac 1.1.0 and test this case on the first new items (/assets/atmosphere/info and /info?assets=atmosphere must list aot and wvp)?

fix: raise 400 when an asset points to a Zarr array

Dry run on main (9093d24), against the EODC STAC:

S2A_MSIL2A_20261108T134311_N0513_R067_T26WMA_20261108T175402 (1.1.0 shape, quality/atmosphere): both routes return 200 with aot and wvp.
S2C_MSIL2A_20261109T135351_N0513_R110_T26VLR_20261109T153305 (current shape, quality/atmosphere/r10m): both routes still return 200 {}. fix: raise proper 400 error when store point to an array #165 covers hrefs that point at an array; this one is a group with no readable variable.

EODC is still writing 1.1.0-rc2 items. The check will be re-run on the first 1.1.0 item.

I am opening a followup isue for the empty-200 case

Navigation Menu

Repository navigation

No way to tell which assets are renderable: AOT/SCL/WVP 500 on one route, empty 200 on the other #163

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	_VALID_TYPE = {
	*DEFAULT_VALID_TYPE,
	"application/x-zarr",
	"application/vnd.zarr",
	"application/vnd+zarr",
	"application/vnd.zarr; version=3",
	"application/vnd.zarr; version=3; profile=multiscales",
	}

	# Some STAC assets (e.g. AOT/SCL/WVP) point at a single Zarr Array rather
	# than a Group, and `GeoZarrReader` opens every asset as a `DataTree`.
	# Opening an Array path as a DataTree raises `ContainsArrayError` (xarray
	# tries `zarr.open_group`/`open_consolidated` on a path that is actually
	# an Array). Since `.info()` fans out over all assets, allow that
	# exception so those non-group assets are skipped instead of failing
	# the whole request.
	allowed_exceptions = kwargs.pop("allowed_exceptions", ())
	allowed_exceptions += (zarr.errors.ContainsArrayError,)
	infos = super().info(
	assets=assets, **kwargs, allowed_exceptions=allowed_exceptions
	)

No way to tell which assets are renderable: AOT/SCL/WVP 500 on one route, empty 200 on the other #163

Description

Activity

vincentsarago commented on Sep 10, 2026

vincentsarago commented on Sep 10, 2026

lhoupert commented on Sep 10, 2026

lhoupert commented on Sep 22, 2026

emmanuelmathot commented on Sep 23, 2026

lhoupert commented on Sep 23, 2026

emmanuelmathot commented on Nov 10, 2026

vincentsarago commented on Nov 10, 2026

emmanuelmathot commented on Nov 10, 2026

vincentsarago commented on Nov 10, 2026

emmanuelmathot commented on Nov 10, 2026

lhoupert commented on Nov 10, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions