Skip to content

fix(roles): surface source ids on sync failure + paginated-list truncation#614

Merged
riyazsh merged 1 commit into
mainfrom
riyaz/HAMR-392-roles-phase2-visibility-phase6-remap
Jul 9, 2026
Merged

fix(roles): surface source ids on sync failure + paginated-list truncation#614
riyazsh merged 1 commit into
mainfrom
riyaz/HAMR-392-roles-phase2-visibility-phase6-remap

Conversation

@riyazsh

@riyazsh riyazsh commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Three linked observability fixes so operators can correlate cross-process cascade failures when sync-cli is invoked once per resource type as separate processes:

  1. Roles.create_resource / update_resource — log source id + role name at ERROR before re-raising on unhandled failure. Previously the module logged nothing before propagating the HTTP error, so a downstream resource type that later couldn't remap the role UUID had no way to see which specific role failed to sync.

  2. Counter — track failed and missing-dependency source ids by resource type. apply_resources emits a per-type summary at end so a subsequent sync-cli invocation can find the exact ids to search for in cascade logs. Ids are chunked at 50 per line to stay under common log-shipper truncation caps. Backwards-compatible: existing callers that pass no ids still increment the numeric counter as before.

  3. paginated_request — elevate the non-5xx mid-pagination failure log to say TRUNCATED explicitly and mention downstream cascade risk. A 403 Forbidden on one page previously produced a benign-looking Error during paginated request ... line and silently truncated the returned resource list; downstream syncs saw a partial source list without knowing why.

Motivation

When sync-cli is orchestrated as one process per resource type (each reading state from a shared store), a failure in an earlier process can cascade into "missing connections" errors in later processes referencing ids that never appeared in any prior log. These three changes make that failure mode debuggable from logs alone.

No behavior change for callers that

  • don't hit an unhandled error in Roles.create/update (2xx and handled 4xx paths unchanged)
  • don't experience non-5xx mid-pagination failures
  • don't consume the new counter attributes

Test plan

  • pytest tests/unit/ — 829 passing (19 new across 4 new test files).
  • New coverage includes:
    • Counter data-structure semantics (backwards-compat guards).
    • Handler-level summary emission: chunking, insertion-order preservation, silence-on-clean-run, exception propagation.
    • paginated_request truncation log wording (403 path) + regression guard for 5xx isolation path.
    • Roles create/update ERROR logging on source id + role name.

@riyazsh
riyazsh requested a review from a team as a code owner July 9, 2026 14:05
@riyazsh
riyazsh force-pushed the riyaz/HAMR-392-roles-phase2-visibility-phase6-remap branch from e9343b9 to c46aa9b Compare July 9, 2026 14:43
…ation

Three linked observability fixes for cross-process cascade correlation:

1. roles.create/update: log source id + role name at ERROR on failure
   before re-raising, so downstream sync-cli invocations for other
   resource types can correlate missing-dependency cascades.
2. Counter: track failed / missing-deps source ids by resource type.
   apply_resources emits a per-type summary at end so operators see
   which specific ids to search for in downstream cascade logs.
3. paginated_request: elevate non-5xx mid-pagination failure log to
   say TRUNCATED explicitly and mention downstream cascade risk. A
   403 Forbidden on one page silently truncated the returned list
   before; now the truncation is unambiguous in logs.

11 new unit tests. 821 total (was 810).

Co-Authored-By: Claude Opus 4.7 <[email protected]>
@riyazsh
riyazsh force-pushed the riyaz/HAMR-392-roles-phase2-visibility-phase6-remap branch from c46aa9b to 8e49fca Compare July 9, 2026 15:10
@riyazsh
riyazsh merged commit 443d51e into main Jul 9, 2026
10 checks passed
@riyazsh
riyazsh deleted the riyaz/HAMR-392-roles-phase2-visibility-phase6-remap branch July 9, 2026 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants