Skip to content

refactor(engine/rocky-core): dedup compute_backoff across adapter crates#217

Merged
hugocorreia90 merged 1 commit intomainfrom
refactor/compute-backoff-dedup
Apr 22, 2026
Merged

refactor(engine/rocky-core): dedup compute_backoff across adapter crates#217
hugocorreia90 merged 1 commit intomainfrom
refactor/compute-backoff-dedup

Conversation

@hugocorreia90
Copy link
Copy Markdown
Contributor

Summary

  • Hoists the three identical compute_backoff copies from rocky-core::state_sync, rocky-databricks::connector, and rocky-snowflake::connector into a single shared helper at rocky_core::retry::compute_backoff.
  • Adapter crates already depend on rocky-core; no Cargo.toml changes needed.
  • Zero behaviour change — bodies were already behaviourally identical (only trivial cosmetic diffs: SystemTime::UNIX_EPOCH vs UNIX_EPOCH, inline comments, doc wording).
  • Follow-up to feat(rocky-core): wire state backend retry + circuit-breaker #213, which flagged the third copy as a deliberate duplicate to be deduped separately.

Before / after layout

Before:

rocky-core/src/state_sync.rs        fn compute_backoff(..) { .. }   // private
rocky-databricks/src/connector.rs   fn compute_backoff(..) { .. }   // private
rocky-snowflake/src/connector.rs    fn compute_backoff(..) { .. }   // private

After:

rocky-core/src/retry.rs             pub fn compute_backoff(..) { .. }   // single source
rocky-core/src/state_sync.rs        use crate::retry::compute_backoff;
rocky-databricks/src/connector.rs   use rocky_core::retry::compute_backoff;
rocky-snowflake/src/connector.rs    use rocky_core::retry::compute_backoff;

Net: 5 files changed, +97 / -196 lines (the new retry.rs ships 90 lines of impl + consolidated tests; the three duplicates and their per-crate test blocks are deleted).

Consolidated tests in retry.rs (covering exponential-no-jitter, capped, with-jitter-in-range) are the superset of what the adapter crates had; the two state_sync tests covered a proper subset.

Test plan

  • cargo test -p rocky-core -p rocky-databricks -p rocky-snowflake — all green (991 + 30 + 20 + supporting suites)
  • cargo clippy --workspace --all-targets -- -D warnings — clean
  • cargo fmt --all --check — clean
  • CI green on engine-ci.yml

Hoist the three identical `compute_backoff` copies (rocky-core::state_sync,
rocky-databricks::connector, rocky-snowflake::connector) into a single
shared `rocky_core::retry::compute_backoff` helper. All three call sites
and their existing retry loops are unchanged; only the source of the
function moves.

Zero behaviour change. The tests from the adapter crates (exponential
without jitter, capped, with jitter in range) are consolidated into the
new module; duplicate tests in state_sync are removed.

Follow-up to #213.
@hugocorreia90 hugocorreia90 merged commit f1d044f into main Apr 22, 2026
12 checks passed
@hugocorreia90 hugocorreia90 deleted the refactor/compute-backoff-dedup branch April 22, 2026 12:18
hugocorreia90 added a commit that referenced this pull request Apr 22, 2026
* chore: release engine-v1.14.0 + dagster-v1.10.0 + vscode-v1.6.4

Bumps all three artifacts to cover the 16-PR cascade since engine-v1.13.0
/ dagster-v1.9.0 / vscode-v1.6.3. Details in each CHANGELOG.

Engine headlines (12 PRs):
- Arc 7 wave 2 wave-2 complete — cached DESCRIBE end-to-end
  (#223 infra, #228 reads, #230 write tap, #231 discover warm-up,
  #232 state controls + --cache-ttl override)
- Arc 2 wave 3 complete — bytes_scanned / bytes_written on
  MaterializationOutput (#219 BQ, #221 Databricks, #220 Snowflake
  deferred doc, #222 docstring cascade). Real $ on rocky cost for
  BQ + Databricks
- FR-005 Unity Catalog workspace-binding reconcile (#226)
- FR-002 Fivetran connector metadata via SourceOutput.metadata (#225)
- Housekeeping: compute_backoff dedup into rocky_core::retry (#217)

Dagster headlines (4 PRs):
- FR-001 RockyComponent Pipes execution mode + FR-006 strict doctor
  on RockyResource startup (#224)
- FR-003 RockyResource.state_health() (#227) + FR follow-up threading
  doctor(check=state_rw) for sub-second probes (#229)
- RockyResource.cost() wiring + fixture (#218)

VS Code: regenerated TS bindings for engine 1.14.0 type additions.
No extension feature changes.

* chore(integrations/dagster): regenerate test fixtures for engine 1.14.0

36 fixtures picked up the new engine version string in their top-level
"version" field. No schema changes — just the version bump.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant