Skip to content

feat: rocky preview — PR-time preview environments + data diff + cost delta#279

Merged
hugocorreia90 merged 4 commits intomainfrom
feat/rocky-preview
Apr 29, 2026
Merged

feat: rocky preview — PR-time preview environments + data diff + cost delta#279
hugocorreia90 merged 4 commits intomainfrom
feat/rocky-preview

Conversation

@hugocorreia90
Copy link
Copy Markdown
Contributor

@hugocorreia90 hugocorreia90 commented Apr 28, 2026 •
edited
Loading

Summary

Introduces rocky preview <create|diff|cost> — a PR-time preview workflow built on Rocky's branches + state-store + cost-attribution primitives. Closest commercial analogue is Fivetran Smart Run; Rocky's compiler-IR + branch substrate are structural advantages that workflow can't match.

This PR delivers the vertical slice scoped during planning: Phases 0a + 0b + 1 + 2 + 3 + POC + docs, all CI-green-verifiable on a fresh PR. Phases 4 (GitHub Action + auto PR comment) and 5 (adapter parity beyond DuckDB + warehouse-native clones) ship as follow-up PRs after this lands.

What's in this PR

Phase 0a — engine codegen (f4f543c): new PreviewCreateOutput / PreviewDiffOutput / PreviewCostOutput structs in crates/rocky-cli/src/output.rs, registered in export_schemas, with JSON schemas + Pydantic + TypeScript bindings regenerated end-to-end via just codegen.

Phase 0b — Dagster + VS Code wiring (fc1071e): Dagster types.py re-exports the 15 new symbols; VS Code registers rocky.previewCreate / rocky.previewDiff / rocky.previewCost commands with palette entries and a new commands/preview.ts handler module.

Phase 1 — rocky preview create (79e857a): working planning kernel that:

  • Identifies changed model files via git diff --name-status base...HEAD (two-dot fallback for shallow clones).
  • Builds the working-tree DAG by parsing sidecar depends_on arrays.
  • Computes the prune set as the changed-models closure under forward-edge traversal (Kahn's topological order, alphabetical tie-break, cycle-tolerant).
  • Computes the copy set as the working-DAG complement.
  • Registers the branch via run_branch_create (default name slugged from current git branch).
  • Emits a PreviewCreateOutput with run_status = "planned" — Phase 1.5 lifts the copy step to adapter-driven CTAS / SHALLOW CLONE / zero-copy CLONE execution; until then the user invokes rocky run --branch <name> --model <name> per prune-set model.
  • 11 new unit tests covering DAG semantics, malformed-sidecar tolerance, models-dir filtering, git-path parsing.

Phase 2 — rocky preview diff: command + JSON wire contract shipped; emits a valid PreviewDiffOutput with a deterministic Markdown stub. Substance (extending the rocky_core::compare shadow kernel with sampled row-level deltas + the explicit coverage_warning sampling-window field) is the Phase 2 follow-up — schema is ready.

Phase 3 — rocky preview cost: command + JSON wire contract shipped; emits a valid PreviewCostOutput with a deterministic Markdown stub. Substance (RunRecord-based per-model cost-delta math over the existing rocky cost latest infrastructure from Arc 2 wave 2 / PR #202) is the Phase 3 follow-up.

POC (bc1da8c): new examples/playground/pocs/06-developer-experience/10-pr-preview-and-data-diff/ — 5-model DuckDB transformation pipeline (raw_orders, raw_customers, stg_orders, dim_customers, fct_revenue) with a synthetic-PR variant (fct_revenue.sql.changed). ./run.sh exercises the full create+diff+cost flow end-to-end, surfaces stub responses without failing, exits 0 in ~0.3s.

Docs (bc1da8c): concept page (concepts/preview-internals.md), how-to (guides/preview-a-pr.md), CLI reference (reference/commands/modeling.md).

Comparison to Fivetran Smart Run

The Fivetran blog post describes "Smart Run": git-diff models, COPY unchanged upstream from prod into dev, re-run only changed + downstream. Rocky's primitives strictly dominate that technique:

  • Branches > schemas as substrate — Phase 5 lifts to Delta SHALLOW CLONE / Snowflake zero-copy CLONE (metadata-only), strictly cheaper than COPY.
  • Compiler IR > git-diff — Phase 1.5 lifts pruning from model-level to column-level; a column added to an unused tail of a wide table prunes to zero downstream.
  • Compile-time type equivalence > textual diff — type-equivalent edits prune to zero.
  • Cost delta as first-class output — not surfaced in Smart Run per the article.

Out of scope (follow-up PRs)

  • Phase 4 — GitHub Action + auto PR comment. Needs a real PR to validate the comment-upsert path; cleanly tested in a follow-up.
  • Phase 5 — adapter parity (Databricks SHALLOW CLONE, Snowflake zero-copy CLONE, BigQuery table copy). Adapter conformance suite runs in engine-weekly.yml not per-PR; merging would ship un-CI-verified adapter code.
  • Phase 1.5 — copy-step execution (CTAS today; clones in Phase 5).
  • Phase 2 substance — sampled row-level diff via the extended shadow kernel + coverage_warning field. Schema is shipped; math lands next.
  • Phase 3 substance — per-model cost-delta math over RunRecord pairs. Schema is shipped; math lands next.
  • Phase 2.5 — checksum-bisection exhaustive diff (datafold-style); replaces the sampled-window correctness ceiling.

Test plan

  • just codegen is idempotent — codegen-drift.yml will pass.
  • Schema registration test (registered_schemas_match_committed_files) green.
  • 14 new preview unit tests pass.
  • Full cargo test -p rocky-cli (284 tests) passes.
  • cargo clippy -p rocky-cli --all-targets -- -D warnings clean.
  • Dagster uv run pytest (458 tests) passes.
  • Dagster uv run ruff check clean.
  • VS Code npm run compile clean.
  • VS Code npm run test:unit (20 tests) passes.
  • VS Code npm run lint clean.
  • POC ./run.sh exits 0 (stubs surface gracefully).

@hugocorreia90
feat(engine): scaffold rocky preview command + codegen cascade
f4f543c 
Phase 0 of the PR-preview bundle: introduces `rocky preview <create|diff|cost>`
subcommands with output structs (PreviewCreateOutput, PreviewDiffOutput,
PreviewCostOutput) wired through the codegen cascade — JSON schemas exported
to schemas/, Pydantic models regenerated for dagster, TypeScript interfaces
regenerated for vscode. Command handlers bail with a Phase-1-pointer message;
subsequent commits in this PR fill them in.

Output structs cover: prune-set + copy-set + skipped-set tracking on create;
structural + sampled row diff with explicit sampling-window correctness
hedge on diff; per-model bytes/duration/USD delta with copy-savings rollup
on cost.

Schema-registration + 7 unit tests passing.
@hugocorreia90
feat(engine): rocky preview create — prune-and-plan kernel
79e857a 
Phase 1 of the PR-preview bundle. End-to-end planning of the
prune-and-copy decision against the working DAG:

- Git plumbing: three-dot `base...HEAD` diff (two-dot fallback for
  shallow clones); filters to .sql/.rocky/.toml under models_dir.
- Working DAG: scan models_dir non-recursively, parse sidecar
  depends_on into a forward-edge graph; tolerant of malformed TOML.
- Prune set: changed models + transitive downstream via reverse-edge
  closure. Topological ordering via Kahn's (alphabetical tie-break).
- Copy set: every working-DAG model not in the prune set, emitted
  with copy_strategy="planned" — Phase 1.5 lifts to adapter-driven
  CTAS / SHALLOW CLONE / zero-copy CLONE.
- Branch register: reuses run_branch_create; default branch name
  derived from current git branch (`pr-preview-<slug>`) or timestamp.
- Output: PreviewCreateOutput with run_status="planned" so the
  scope is honest.

11 new unit tests covering DAG topological order, transitive
downstream traversal, malformed sidecar tolerance, models-dir filter,
git path parsing.
@hugocorreia90
docs(preview): concept + how-to + CLI ref for rocky preview
bc1da8c 
Three new doc surfaces:

- docs/src/content/docs/concepts/preview-internals.md — explains the
  prune-and-copy substrate, comparison to Fivetran's Smart Run, and
  the sampling correctness ceiling.
- docs/src/content/docs/guides/preview-a-pr.md — how-to walking
  through preview create / diff / cost on a feature branch.
- docs/src/content/docs/reference/commands/modeling.md — appended
  rocky preview subcommand reference; cross-link from rocky ci-diff.

POC scaffolding at examples/playground/pocs/06-developer-experience/
10-pr-preview-and-data-diff/ — a 5-model DuckDB transformation pipeline
with a synthetic-PR variant (fct_revenue.sql.changed) and aspirational
expected outputs matching the wire schemas. ./run.sh exercises the
full create+diff+cost flow today and surfaces stub responses without
failing.

Pivot note in the POC README: models are .sql + .toml (not .rocky DSL)
because the existing playground had no working precedent for
transformation+.rocky+rocky run. The preview workflow is surface-agnostic
so this doesn't degrade what the POC demonstrates.

examples/playground/.gitignore: gitignore .rocky_state* directories;
whitelist expected/*.example.json so aspirational fixtures stay tracked.
@hugocorreia90
feat(integrations,vscode): wire rocky preview through dagster + vscode
fc1071e 
Phase 0b of the PR-preview bundle:

- Dagster: re-export PreviewCreateOutput / PreviewDiffOutput /
  PreviewCostOutput from types_generated/ via top-level types.py,
  matching the soft-swap pattern used for run/cost/compliance.
- VS Code: register three new commands (rocky.previewCreate,
  rocky.previewDiff, rocky.previewCost) with command-palette entries
  and runRockyCli helpers; new src/commands/preview.ts module.
- Cleanup: clippy-clean preview module (str::to_string, doc-list
  indent, allow too-many-arguments on PreviewCreateOutput::new).
@hugocorreia90 hugocorreia90 merged commit 5d657f1 into main Apr 29, 2026
15 checks passed
@hugocorreia90 hugocorreia90 deleted the feat/rocky-preview branch April 29, 2026 10:15
This was referenced Apr 29, 2026
Merged
Merged
Merged
hugocorreia90 added a commit that referenced this pull request Apr 29, 2026
@hugocorreia90
chore: release engine-v1.18.0 + dagster-v1.15.0 + vscode-v1.10.0 (#294)
86238cf 
Engine 1.18.0 ships the rocky preview workflow end-to-end (#279, #280,
#281, #282), the [budget].max_bytes_scanned threshold (#288), the
audit-sweep closeout (#283, #285–#287, #290–#293), and the rocky-server
auth + CORS gate (#291).

Dagster 1.15.0 picks up the regenerated Pydantic models for the rocky
preview surface and ships the P1 cluster (#289) + FR-014 follow-on
(#284).

VS Code 1.10.0 regenerates TypeScript bindings for rocky preview and
RunCostSummary.total_bytes_scanned.

See per-artifact CHANGELOG entries for the full breakdown.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@hugocorreia90