Skip to content

Filter metrics_metadata by destination-existence#617

Merged
riyazsh merged 1 commit into
mainfrom
riyaz/HAMR-392-metrics-metadata-dest-existence
Jul 9, 2026
Merged

Filter metrics_metadata by destination-existence#617
riyazsh merged 1 commit into
mainfrom
riyaz/HAMR-392-metrics-metadata-dest-existence

Conversation

@riyazsh

@riyazsh riyazsh commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Skip metrics_metadata PUT when the metric doesn't exist on destination. The legacy dogweb handler returns 400 {"errors":["error updating metric metadata"]} for missing metrics, wasting ~700 API calls / 25-30min per DR dispatch.

Change

Before PUT, GET /api/v1/metrics/{name} (same base_path):

GET result Action
200 Proceed to PUT (existing behavior)
404 Raise SkipResource (logged at DEBUG); PUT skipped
Other Propagate to existing retry layer

5 unit tests cover 200/404/500/403 + create_resource delegation. Mirrors the host_tags.py 404-skip pattern.

Notes

  • Uses /api/v1/metrics/{name} — the v2 bare-name route isn't registered in dd-source governance (only v2 list + subresources).
  • No CLI opt-out flag added; 404-skip strictly reduces failure noise vs. current behavior.
  • Unblocks restoring metrics_metadata to Vault allowlist (Jul8-T19 dropped it temporarily).

Ref: HAMR-392 Jul8-T20. Root cause at team-compass HAMR-392/T-metrics-metadata/STATUS.md@d22a7ce222.

@riyazsh
riyazsh requested a review from a team as a code owner July 9, 2026 15:15
metrics_metadata can only attach metadata (description, unit, integration
hints) to a metric that already exists on destination. The legacy dogweb
handler returns 400 {"errors":["error updating metric metadata"]} when the
metric key is missing on destination.

Under DR replication into an org that hasn't yet received the source's
custom metric emissions, this fails constantly and adds 25-30 minutes of
wall-clock time per dispatch across 14+ batches of ~700 records.

Fix: probe destination for the metric via GET /api/v2/metrics/{name}
before the PUT. On 404, raise SkipResource so the sync framework logs it
as skipped (not a failure). Other HTTP errors from the probe (5xx, 403)
propagate to the existing retry layer unchanged.

Adds 5 unit tests covering 200/404/500/403 GET outcomes plus the
create_resource → update_resource delegation.

Ref: HAMR-392 Jul8-T20. Root-cause analysis at team-compass
HAMR-392/T-metrics-metadata/STATUS.md commit d22a7ce222.
@riyazsh
riyazsh force-pushed the riyaz/HAMR-392-metrics-metadata-dest-existence branch from 3b0f25e to c06c92a Compare July 9, 2026 15:27
@riyazsh
riyazsh merged commit 974810e into main Jul 9, 2026
17 of 18 checks passed
@riyazsh
riyazsh deleted the riyaz/HAMR-392-metrics-metadata-dest-existence branch July 9, 2026 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants