Skip to content

Commit 39d7d94

Browse files
sahrizviclaude
andauthored
feat(review): [R20 S1] grain-key not_null completeness detector (+11 findings on 5-PR corpus) (#1029)
* feat(review): [R20 S1] grain-key `not_null` completeness detector For every column named in a `dbt_utils.unique_combination_of_columns` test's `combination_of_columns`, require `not_null` coverage on the same model. Otherwise a NULL grain-key value silently passes the uniqueness test — the guardrail is toothless. Cited as a hard rule in `docs/DBT_GUIDELINES.md` in the corpus repo; kilo catches it, we didn't. Coverage sources: - `constraints: [{type: not_null}]` — only counted when `contract.enforced == true` (per codex R20 S1 high #3). On views / non-contracted models, `constraints:` is documentation-only, not enforced, so we don't miss a real gap. - Column-level `data_tests: [not_null]` / `tests: [not_null]` — always counted; dbt's test runner enforces regardless of contract state. Recommendation flips between `constraints:` (contracted model) and `data_tests:` (view / non-contracted) based on the model's contract state — matches the adapter-semantics discussion in the corpus study (PR D×2 + PR A×2). Supported YAML shapes: - `unique_combination_of_columns` and `dbt_utils.unique_combination_of_columns` (exact match per codex high #1; `endsWith` would over-match). - pre-1.9 flat args + dbt 1.9+ `arguments:` nesting. - top-level `contract:` and nested `config.contract:` for enforcement flag. False-positive guards: - Column-name comparison is case-folded (Snowflake identifiers, per codex high #2) so `WORKSPACE_ID` in `combination_of_columns` matches `workspace_id` in `columns:`. - Skipped on `status === "deleted"` files (no current grain to guard). - Only runs on files in the PR diff, not repo-wide. ### Validation on 5-PR internal corpus (recall improvement) vs S4 baseline (the tier-promotion PR this branch stacks on): | Variant | S4 baseline | S1 result | Delta | |---|---:|---:|---:| | A1 (`--pure --no-ai=true`) | 118 | 129 | **+11 grain-key gaps** | | A2 (`--pure`, LLM lane) | 143 | 150 | +7 (net; LLM run-to-run noise) | Grain-key gaps caught: - PR B (#1111) ×1: `mrt_all_purpose_auto_tune_recommendation.rec_type` - PR C (#862) ×10: workspace_id x3, job_id x2, period_start_time x2, period_end_time x2, task_key x1 - PR A, D, E: 0 (fixes already landed in HEAD state per corpus study) Strong matches to human blocker findings on PR C: - `int_job_run_billing.workspace_id`, `int_job_run_cost_carrier.workspace_id`, `int_job_task_run_cost_carrier.workspace_id` → PR C human F11 (critical: "workspace_id missing from carrier identity") - `mrt_job_run_timeline.period_end_time`, `mrt_job_task_run_timeline.period_end_time` → PR C human F10 (critical: "dbt mart grain includes period_end_time") ### Tests - 71 pass / 0 fail in `review-dbt-patterns.test.ts` (58 pre-existing + 13 new S1 tests). - 3797 pass / 640 skip / 0 fail in the full altimate review suite (132 files). - Codex-reviewed diff, 3 highs addressed: - endsWith → exact-name match - column-name case-folding - constraints only count when contract enforced - Codex minor #4 addressed (test fixture for top-level `contract:`) - Codex minor #5 addressed (test fixture for non-contracted constraints ≠ coverage) Depends on PR #1028 (feat/review-r20-s4-triage-promotion), which in turn stacks on PR #1027 (feat/review-r18-observability-recall). Co-Authored-By: Claude Opus 4.7 <[email protected]> Claude-Session: https://claude.ai/code/session_017zXDXMiNFh4qDPxPCfa2of * fix(review): [R20 S1] address consensus-review findings on grain-key detector Bundle addresses PR #1029's consensus review: - MAJOR #1 — model-level and column-level primary_key / not_null constraints now count as coverage. dbt 1.5+ supports model-level constraints via `constraints: [{type: primary_key, columns: [a, b]}]`, and a primary_key inherently enforces NOT NULL on Postgres/Snowflake/ BigQuery/Databricks. Grain columns declared via a model-level PK were previously falsely flagged as missing not_null. Fix scans both `mm.constraints` (model-level, honouring the `columns:` list) and column-level `constraints: [{type: primary_key}]`. - MINOR #3 — contract-precedence bug. Earlier ternary short-circuited when `cfg.contract` was any object (e.g. `config: {contract: {alias: X}}` with no `enforced` key), masking a top-level `contract: {enforced: true}`. Now evaluated independently at both locations and OR'd. - MINOR #4 — `dbt.` prefix on namespaced test names (`dbt.not_null` in dbt 1.8+) is stripped in `testName()` before matching, so it counts as coverage. - MINOR #6 — `{name: <alias>, test_name: not_null}` alternative object form is recognised. Reading `Object.keys(t)[0]` returned `name` (the alias) rather than the underlying test type. Now `test_name` wins when present, falling back to the first key. - NIT #7 — `norm()` hoisted from per-model to function scope. Consensus items NOT addressed this round: - NIT #8 (dedup GrainKeyGap for a column listed twice / multiple grain tests) — collapses downstream via the global finding fingerprint; cosmetic rather than correctness. - NIT #9 (model-level `data_tests:` scanned as a grain-test source) — reviewer noted "harmless (no false match)"; no action. - NIT #10 (documentation of fallback-skip) — no code change needed. - MINOR #5 (contract resolved from dbt_project.yml or SQL config) — cross-file / cross-context, deferred. Regression tests (all pass; 82 pass / 0 fail in review-dbt-patterns suite, 3828 pass / 0 fail in full altimate suite — up from 3785): - Model-level primary_key constraint covers grain columns - Model-level not_null constraint with `columns:` list covers named cols - Column-level primary_key counts as coverage - Precision guard: PK missing cols still flagged - Model-level constraints on non-contracted model don't count - `config.contract` without `enforced` does NOT mask top-level `contract: {enforced: true}` (MINOR #3) - `dbt.not_null` covers (MINOR #4) - `{name:, test_name: not_null}` alternative form covers (MINOR #6) Co-Authored-By: Claude Opus 4.7 <[email protected]> Claude-Session: https://claude.ai/code/session_017zXDXMiNFh4qDPxPCfa2of * chore(review): [R20 S1] genericize internal-vocab leaks in dbt-patterns comments Companion cleanup to PR #1028's FinOps strip. The grain-key detector's docstrings named a specific internal column (`workspace_id`) as the Snowflake case-folding example and referenced internal corpus PR labels (`PR D×2, PR A×2`). Neither carries product meaning outside the altimate-ingestion codebase. - `dbt-patterns.ts:962` — replace `WORKSPACE_ID` / `workspace_id` example with `ORDER_ID` / `order_id`. Same illustrative point, no internal name. - `dbt-patterns.ts:1443` — replace `(PR D×2, PR A×2)` corpus-label citation with `(four instances across the sample)`. Same count, no internal reference. Tests: 250/250 green. * fix(review): [R20 S1] address altimate-harness-bot review findings on grain-key detector Two substantive findings on PR #1029: 1. `dbt-patterns.ts:1099` — grain detector was not change-scoped. `extractGrainKeyGaps(newDoc)` ran against every entity in the file, so a housekeeping edit (description bump, meta tag) surfaced all pre-existing grain gaps on unrelated models. Real precision cost: reviewers seeing findings on a PR they didn't intend suppress the whole rule. Fix: compare the `combination_of_columns` set per entity between `oldDoc` and `newDoc`; only emit gaps for entities whose grain declaration actually changed (added, removed, or column set diff). Added entities on the new side count as changed; the "added file" case (no oldDoc) unconditionally treats every entity as changed so newly-shipped grain declarations are still guarded. New helpers: `extractGrainDeclarations` returns `Map<entityName, Set<column>>` and `grainDeclChangedEntities` diffs two docs into a set of entity names whose declaration moved. Existing test that pinned the old steady-state-fires behavior (`R20 S1: unique_combination_of_columns with grain col missing not_null → warning finding`) rewritten to test the newly-added case; added a companion test that pins the new precision guarantee (`steady-state grain gap on unchanged model is NOT re-surfaced`), plus a `grain declaration changed (column added to combination_of_columns) does fire gap` test that keeps the regression case covered. 2. `dbt-patterns.ts:963` — `extractGrainKeyGaps` iterated only `d.models`, silently skipping `snapshots`, `sources`, and `seeds`. `unique_combination_of_columns` on a snapshot is a real SCD-2 grain declaration; sources declare per-table `columns:` + tests; seeds carry the model shape. Fix: new `iterateGrainEntities(d)` helper walks all four sections (mirrors `extractTestOccurrences`). Sources descend one level to iterate their `tables[]` entries which carry the model shape. Three new tests: `grain detector also covers snapshots`, `... source tables`, `... seeds`. Tests: 255/255 green (8 review-* files, +6 new grain-key tests). * fix(review): [R20 S1] qualify source-table entity names as `<source>.<table>` (cubic-review P2) Two sources both containing a table named `orders` (e.g. `raw.orders` and `legacy.orders`) previously conflated in `iterateGrainEntities`: - Grain-change detection collapsed them into a single map entry, so a change in one source's grain declaration could surface or suppress gaps for the other. - Finding fingerprint uses the entity name; two distinct source tables with the same table name would dedupe to a single finding. Fix: `iterateGrainEntities` now returns `{name, body}` pairs. Source tables are qualified as `${sourceName}.${tableName}` while models, snapshots, and seeds retain their unqualified name (they live in a flat namespace already). Tests: `grain detector also covers source tables` updated to assert the qualified name; new test `same source-table name in two sources does NOT conflate` locks in the fix. 257/257 review-* tests pass. --------- Co-authored-by: Haider <[email protected]> Co-authored-by: Claude Opus 4.7 <[email protected]>
1 parent f1389dc commit 39d7d94

2 files changed

Lines changed: 1120 additions & 1 deletion

File tree

0 commit comments

Comments
 (0)