Skip to content

docs: drift sweep against engine-v1.20.0 (catalog scoping + Phase 5 + lineage-diff)#341

Merged
hugocorreia90 merged 4 commits intomainfrom
doc-poc-drift-audit-v1.20
May 1, 2026
Merged

docs: drift sweep against engine-v1.20.0 (catalog scoping + Phase 5 + lineage-diff)#341
hugocorreia90 merged 4 commits intomainfrom
doc-poc-drift-audit-v1.20

Conversation

@hugocorreia90
Copy link
Copy Markdown
Contributor

@hugocorreia90 hugocorreia90 commented May 1, 2026

Summary

Doc + POC drift audit against the engine v1.20.0 / dagster v1.18.0 / vscode v1.11.0 line. Four scoped commits, each addressing one drift class:

  • Strip internal phase-number leaks from public docs and POCs (f89cdd3) — eight sites in docs/concepts/preview-internals.md, docs/guides/preview-a-pr.md, and two playground POCs were referencing internal phase numbers. Neutralised to topic-only language.
  • Refresh POC catalog counts in docs/guides/playground.md (f552b74) — Developer Experience 10→11 and Adapters 5→6; add entries for 11-lineage-diff and 06-rust-native-adapter-skeleton shipped in v1.19.0.
  • Add rocky lineage-diff to CLI index surfaces (d367295) — was missing from docs/reference/cli.md Modeling row, docs/features/all-features.md Modeling chip, and engine/README.md "CLI at a glance".
  • Document MaterializationOutput.job_ids (82aa19c) — new field shipped in PR feat(rocky-cli): emit warehouse job_ids on materialization output + cost cross-check smoke #337 was missing from docs/reference/json-output.md.

Validation

  • npm run build from docs/ produces 72 pages clean.
  • ./run.sh re-run on both edited POCs (01-quality/06-quality-pipeline-standalone, 06-developer-experience/10-pr-preview-and-data-diff) — both pass.

Out of scope (flagged for follow-up)

  • rocky lineage-diff has no full reference page under docs/reference/commands/modeling/. Index entries only here; design call on whether to add a full page belongs elsewhere.
  • engine/README.md adapter status table still says BigQuery + Snowflake are "Beta"; v1.21.0 dropped the BigQuery experimental override. Positioning call.
  • 10-pr-preview-and-data-diff POC triggers invalid SQL identifier 'branch__pr-preview-poc-10' WARN logs because the POC's --name contains hyphens. POC still exits 0 but engine refuses to copy unchanged tables on this path. Engine source change needed.

Test plan

  • `npm run build` clean from `docs/`
  • Both edited POCs `./run.sh` exit 0

Removes "Phase 1/2/2.5/3/4" internal-roadmap references from public-
facing surfaces where they leaked through. Doc-comment / comment-only
edits with no behaviour change.

- docs/concepts/preview-internals.md — "Phase 1 substrate" /
  "Phase 1 sampling rule" / "Phase 2.5 lift" → neutral wording
- docs/guides/preview-a-pr.md — "Phase 2.5 checksum-bisection" →
  "planned checksum-bisection"
- pocs/01-quality/06-quality-pipeline-standalone (rocky.toml + run.sh)
  — "Phase 1/2/3/4a/4b" comments → topic-only headings
- pocs/06-developer-experience/10-pr-preview-and-data-diff
  (README.md + run.sh) — "Phases 1, 1.5, 2, 3 merged" / "Phase N not
  yet wired" / Phase 3-4 prose → reflect actual production status
…d 06-rust-native-adapter-skeleton

The Developer Experience and Adapters tables in the playground guide
lagged behind the POC directory. Update the POC counts (10→11,
5→6) and add the two missing entries shipped in engine-v1.19.0.
`rocky lineage-diff` shipped in engine-v1.19.0 but was missing from
the index lists. One-line additive entries on each surface:

- docs/reference/cli.md — Modeling category line
- docs/features/all-features.md — Modeling & Compilation chip row
- engine/README.md — "CLI at a glance"

The full command reference page under docs/reference/commands/modeling/
is left for a follow-up — flagged in the audit report.
Adds the new top-level `job_ids` field to the MaterializationOutput
reference table. Shipped in engine-v1.21.0 (#337) — surfaces warehouse-
side job IDs so consumers can cross-check rocky-reported bytes against
the warehouse console without scraping stderr.
@hugocorreia90 hugocorreia90 merged commit d761a99 into main May 1, 2026
12 checks passed
@hugocorreia90 hugocorreia90 deleted the doc-poc-drift-audit-v1.20 branch May 1, 2026 21:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant