feat: Trust-system Arc 3 (first wave) — timed half-open circuit breaker + consolidation#172
Merged
hugocorreia90 merged 1 commit intomainfrom Apr 20, 2026
Merged
Conversation
…er + adapter consolidation - `rocky_core::circuit_breaker::CircuitBreaker` gains a three-state machine (Closed / Open / HalfOpen). New `with_recovery_timeout` constructor enables timed auto-recovery: after `circuit_breaker_recovery_timeout_secs` elapses while Open, the next `check` transitions to HalfOpen and lets one trial request through. Success closes the breaker, failure re-opens. Wired to `RetryConfig::build_circuit_breaker()` so every adapter picks it up. - `rocky-databricks` and `rocky-snowflake` drop their inline `consecutive_failures: AtomicU32` and share the consolidated breaker via `Arc<CircuitBreaker>`. Replaces the "orphan breaker module" drift — there's now one implementation in the tree. - Adapters emit `circuit_breaker_tripped` / `circuit_breaker_recovered` `PipelineEvent`s on state transitions. `TransitionOutcome` lets the caller forward the signal without the breaker depending on `rocky-observe`. `HookEvent::CircuitBreakerTripped` is intentionally deferred to wave 2 (needs event→hook bridge; parallels `statement_retry`'s existing adapter-event-without-hook precedent). `rocky branch promote` (atomic shadow→prod swap) moved to its own plan doc after advisor-surfaced deploy-semantics questions (cross-table atomicity, UC rename privileges, dry-run output shape).
3 tasks
hugocorreia90
added a commit
that referenced
this pull request
Apr 20, 2026
Closes the first wave of every trust-system arc (Arcs 1-7) plus the two wave-2 follow-ups landed the same day. Nine feature PRs since v1.10.0. - Arc 1 (#170): rocky lineage --downstream, rocky branch, rocky run --branch, rocky replay - Arc 2 (#171): per-run cost attribution, [budget] block, budget_breach hook - Arc 3 (#172): three-state CircuitBreaker, adapter consolidation - Arc 4 (#173): rocky trace Gantt + feature-gated OTLP metrics export - Arc 5 (#174): schema-grounded rocky ai prompt + project-aware validator - Arc 6 wave 1 (#184): --target-dialect P001 portability lint (12 constructs) - Arc 7 wave 1 (#185): blast-radius P002 SELECT * lint (semantic-graph aware) - Arc 6 wave 2 (#186): [portability] config block + per-model rocky-allow pragma - Arc 7 wave 2 wave-1 (#187): --with-seed source-schema inference Plus #169 fix: install scripts pick latest engine version by semver. Version bump: 20 Cargo.toml files (all workspace members except rocky-bigquery, which tracks its own version). Wave 2/3 work for every arc remains in the deferred backlog — see the changelog Deferred section for the full carry-forward.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First wave of Arc 3 from the trust-system direction. Focused scope after advisor flagged
rocky branch promoteas a separate design exercise.CircuitBreaker. Three-state machine (Closed/Open/HalfOpen). NewRetryConfig.circuit_breaker_recovery_timeout_secspromotes the breaker from "manual-reset only" to auto-recovery: after timeout elapses while Open, the nextchecktransitions to HalfOpen and lets one trial through. Success closes the breaker (emitscircuit_breaker_recovered); failure re-opens (emitscircuit_breaker_trippedagain). CAS on both trip paths guarantees exactly one event per transition even under concurrent failing trials.rocky-databricksandrocky-snowflakedrop their inlineconsecutive_failures: AtomicU32and shareArc<CircuitBreaker>viaRetryConfig::build_circuit_breaker(). Removes the orphan-module drift advisor surfaced — there's now one implementation.rocky-fivetranhas no inline breaker (source adapter, retry shape different);rocky-bigquerylikewise has no inline breaker.circuit_breaker_tripped/_recoveredon transitions with adapter-name target + error class.TransitionOutcomereturn value decouples the breaker fromrocky-observe— callers choose whether to publish.HookEvent::CircuitBreakerTrippedis deferred to wave 2 since adapter-emitted events don't currently have a hook bridge (mirrorsstatement_retry's existing precedent).Deferred to later Arc 3 waves
rocky branch promote <name>(atomic shadow→prod swap). Advisor flagged real deploy-semantics questions — cross-table atomicity, Unity Catalog rename privileges, dry-run output shape, behavior when branch/prod disagree on table set. Separate plan doc + dedicated PR.[hook.on_circuit_breaker_tripped]fires.Test plan
cargo test -p rocky-core— 976 tests (9 newcircuit_breaker::tests::*covering half-open transitions, concurrent-trial CAS, timeout-elapsed check).cargo test -p rocky-databricks— 93 tests pass, wiremock suite clean.cargo test -p rocky-snowflake— 98 tests pass.cargo clippy --workspace --all-targets -- -D warningsclean.cargo fmt --checkclean.just codegen— adapter_config + rocky_project schemas picked up the newcircuit_breaker_recovery_timeout_secsfield; Pydantic + TypeScript regenerated.just regen-fixtures— no RunOutput changes, fixture corpus unchanged.uv run pytestinintegrations/dagster/— 307 tests pass.🤖 Generated with Claude Code