Skip to content

Commit 7d74cec

Browse files
sahrizviclaude
andcommitted
feat(review): [R20 S1] grain-key not_null completeness detector
For every column named in a `dbt_utils.unique_combination_of_columns` test's `combination_of_columns`, require `not_null` coverage on the same model. Otherwise a NULL grain-key value silently passes the uniqueness test — the guardrail is toothless. Cited as a hard rule in `docs/DBT_GUIDELINES.md` in the corpus repo; kilo catches it, we didn't. Coverage sources: - `constraints: [{type: not_null}]` — only counted when `contract.enforced == true` (per codex R20 S1 high #3). On views / non-contracted models, `constraints:` is documentation-only, not enforced, so we don't miss a real gap. - Column-level `data_tests: [not_null]` / `tests: [not_null]` — always counted; dbt's test runner enforces regardless of contract state. Recommendation flips between `constraints:` (contracted model) and `data_tests:` (view / non-contracted) based on the model's contract state — matches the adapter-semantics discussion in the corpus study (PR D×2 + PR A×2). Supported YAML shapes: - `unique_combination_of_columns` and `dbt_utils.unique_combination_of_columns` (exact match per codex high #1; `endsWith` would over-match). - pre-1.9 flat args + dbt 1.9+ `arguments:` nesting. - top-level `contract:` and nested `config.contract:` for enforcement flag. False-positive guards: - Column-name comparison is case-folded (Snowflake identifiers, per codex high #2) so `WORKSPACE_ID` in `combination_of_columns` matches `workspace_id` in `columns:`. - Skipped on `status === "deleted"` files (no current grain to guard). - Only runs on files in the PR diff, not repo-wide. ### Validation on 5-PR internal corpus (recall improvement) vs S4 baseline (the tier-promotion PR this branch stacks on): | Variant | S4 baseline | S1 result | Delta | |---|---:|---:|---:| | A1 (`--pure --no-ai=true`) | 118 | 129 | **+11 grain-key gaps** | | A2 (`--pure`, LLM lane) | 143 | 150 | +7 (net; LLM run-to-run noise) | Grain-key gaps caught: - PR B (#1111) ×1: `mrt_all_purpose_auto_tune_recommendation.rec_type` - PR C (#862) ×10: workspace_id x3, job_id x2, period_start_time x2, period_end_time x2, task_key x1 - PR A, D, E: 0 (fixes already landed in HEAD state per corpus study) Strong matches to human blocker findings on PR C: - `int_job_run_billing.workspace_id`, `int_job_run_cost_carrier.workspace_id`, `int_job_task_run_cost_carrier.workspace_id` → PR C human F11 (critical: "workspace_id missing from carrier identity") - `mrt_job_run_timeline.period_end_time`, `mrt_job_task_run_timeline.period_end_time` → PR C human F10 (critical: "dbt mart grain includes period_end_time") ### Tests - 71 pass / 0 fail in `review-dbt-patterns.test.ts` (58 pre-existing + 13 new S1 tests). - 3797 pass / 640 skip / 0 fail in the full altimate review suite (132 files). - Codex-reviewed diff, 3 highs addressed: - endsWith → exact-name match - column-name case-folding - constraints only count when contract enforced - Codex minor #4 addressed (test fixture for top-level `contract:`) - Codex minor #5 addressed (test fixture for non-contracted constraints ≠ coverage) Depends on PR #1028 (feat/review-r20-s4-triage-promotion), which in turn stacks on PR #1027 (feat/review-r18-observability-recall). Co-Authored-By: Claude Opus 4.7 <[email protected]> Claude-Session: https://claude.ai/code/session_017zXDXMiNFh4qDPxPCfa2of
1 parent 4544cf8 commit 7d74cec

2 files changed

Lines changed: 542 additions & 1 deletion

File tree

‎packages/opencode/src/altimate/review/dbt-patterns.ts‎

Lines changed: 212 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -922,6 +922,162 @@ function extractTestOccurrences(doc: unknown): Set<string> {
922922
return out
923923
}
924924

925+
/**
926+
* R20 S1 — grain-key `not_null` completeness.
927+
*
928+
* A `dbt_utils.unique_combination_of_columns` test names N columns as the
929+
* declared grain. If any grain column lacks `not_null` coverage, a NULL
930+
* grain-key silently passes the uniqueness test — the guardrail is toothless.
931+
* `DBT_GUIDELINES.md` in the corpus repo states this as a hard rule; kilo
932+
* catches it, we didn't.
933+
*
934+
* Coverage sources (either counts):
935+
* - `constraints: [{type: not_null}]` on the column — enforced by the
936+
* database when the model has `contract: {enforced: true}` on Trino /
937+
* Databricks-with-contracts / Postgres / Snowflake.
938+
* - `data_tests: [not_null]` / `tests: [not_null]` on the column —
939+
* enforced by dbt test runner regardless of contract.
940+
*
941+
* Returns one gap per uncovered column per grain declaration.
942+
*/
943+
export interface GrainKeyGap {
944+
/** Model / entity name. */
945+
model: string
946+
/** Column name in the uncovered `combination_of_columns`. */
947+
column: string
948+
/** True when `config.contract.enforced == true` — coverage should be a
949+
* `constraints:` entry rather than a `data_tests:` entry, since the
950+
* constraint enforces at write-time and the test enforces at CI-time. */
951+
contractEnforced: boolean
952+
}
953+
954+
function extractGrainKeyGaps(doc: unknown): GrainKeyGap[] {
955+
const gaps: GrainKeyGap[] = []
956+
if (!doc || typeof doc !== "object") return gaps
957+
const d = doc as Record<string, unknown>
958+
if (!Array.isArray(d.models)) return gaps
959+
960+
// Small local helper: pull the string test-name from a YAML test entry.
961+
// Bare form: `- not_null`, block form: `- not_null: {config: {severity: warn}}`.
962+
const testName = (t: unknown): string | undefined => {
963+
if (typeof t === "string") return t.toLowerCase()
964+
if (t && typeof t === "object") {
965+
const k = Object.keys(t as Record<string, unknown>)[0]
966+
return k ? k.toLowerCase() : undefined
967+
}
968+
return undefined
969+
}
970+
971+
for (const m of d.models) {
972+
if (!m || typeof m !== "object") continue
973+
const mm = m as Record<string, unknown>
974+
const mname = typeof mm.name === "string" ? mm.name : undefined
975+
if (!mname) continue
976+
977+
// Contract enforcement: either at model-level `config.contract.enforced`
978+
// OR at top-level `contract:` (dbt supports both shapes).
979+
const cfg = mm.config && typeof mm.config === "object" ? (mm.config as Record<string, unknown>) : {}
980+
const contractCfg =
981+
cfg.contract && typeof cfg.contract === "object"
982+
? (cfg.contract as Record<string, unknown>)
983+
: mm.contract && typeof mm.contract === "object"
984+
? (mm.contract as Record<string, unknown>)
985+
: {}
986+
const contractEnforced = contractCfg.enforced === true
987+
988+
// Normalize column names for coverage comparison. dbt YAML often uses
989+
// adapter-cased column names (Snowflake folds unquoted identifiers to
990+
// uppercase; other adapters differ). Lowercase both sides so
991+
// `WORKSPACE_ID` in `combination_of_columns` matches `workspace_id`
992+
// in `columns:`. Original spelling is preserved for display in findings
993+
// (populated below from the grain-combo entry).
994+
const norm = (s: string): string => s.toLowerCase()
995+
996+
// Coverage per column, split by mechanism (per codex R20 S1 review):
997+
// - constraint coverage counts ONLY when `contract.enforced == true`.
998+
// On non-contracted models, `constraints: [{type: not_null}]` is
999+
// documentation-only and not enforced by the database. Requiring
1000+
// contract-enforced means we don't miss a real gap on views.
1001+
// - test coverage (`tests:` / `data_tests: [not_null]`) always counts,
1002+
// since dbt's test runner enforces it independent of contract state.
1003+
const coveredByConstraint = new Set<string>()
1004+
const coveredByTest = new Set<string>()
1005+
if (Array.isArray(mm.columns)) {
1006+
for (const col of mm.columns) {
1007+
if (!col || typeof col !== "object") continue
1008+
const c = col as Record<string, unknown>
1009+
const cname = typeof c.name === "string" ? c.name : undefined
1010+
if (!cname) continue
1011+
const key = norm(cname)
1012+
if (Array.isArray(c.constraints)) {
1013+
for (const cn of c.constraints) {
1014+
if (cn && typeof cn === "object") {
1015+
const type = (cn as Record<string, unknown>).type
1016+
if (typeof type === "string" && type.toLowerCase() === "not_null") {
1017+
coveredByConstraint.add(key)
1018+
}
1019+
}
1020+
}
1021+
}
1022+
for (const testsKey of ["tests", "data_tests"] as const) {
1023+
const tests = c[testsKey]
1024+
if (!Array.isArray(tests)) continue
1025+
for (const t of tests) {
1026+
if (testName(t) === "not_null") coveredByTest.add(key)
1027+
}
1028+
}
1029+
}
1030+
}
1031+
const hasCoverage = (col: string): boolean => {
1032+
const k = norm(col)
1033+
return coveredByTest.has(k) || (contractEnforced && coveredByConstraint.has(k))
1034+
}
1035+
1036+
// Find grain declarations in this model's model-level tests / data_tests.
1037+
// Restrict to `unique_combination_of_columns` and
1038+
// `dbt_utils.unique_combination_of_columns` exactly (per codex R20 S1
1039+
// review) — `endsWith` would over-match `not_unique_combination_of_columns`
1040+
// and third-party macros that share the suffix.
1041+
// Supports both dbt shapes:
1042+
// pre-1.9: `- dbt_utils.unique_combination_of_columns: {combination_of_columns: [...]}`
1043+
// 1.9+: `- dbt_utils.unique_combination_of_columns: {arguments: {combination_of_columns: [...]}}`
1044+
const isGrainTestName = (name: string): boolean => {
1045+
const n = name.toLowerCase()
1046+
return n === "unique_combination_of_columns" || n === "dbt_utils.unique_combination_of_columns"
1047+
}
1048+
for (const testsKey of ["tests", "data_tests"] as const) {
1049+
const tests = mm[testsKey]
1050+
if (!Array.isArray(tests)) continue
1051+
for (const t of tests) {
1052+
if (!t || typeof t !== "object") continue
1053+
const entry = t as Record<string, unknown>
1054+
for (const k of Object.keys(entry)) {
1055+
if (!isGrainTestName(k)) continue
1056+
const args = entry[k]
1057+
if (!args || typeof args !== "object") continue
1058+
const argsObj = args as Record<string, unknown>
1059+
const nested =
1060+
argsObj.arguments && typeof argsObj.arguments === "object"
1061+
? (argsObj.arguments as Record<string, unknown>)
1062+
: undefined
1063+
const combo =
1064+
(nested?.combination_of_columns as unknown) ?? (argsObj.combination_of_columns as unknown)
1065+
if (!Array.isArray(combo)) continue
1066+
for (const col of combo) {
1067+
if (typeof col !== "string") continue
1068+
if (!hasCoverage(col)) {
1069+
// Preserve original grain-column spelling in the finding.
1070+
gaps.push({ model: mname, column: col, contractEnforced })
1071+
}
1072+
}
1073+
}
1074+
}
1075+
}
1076+
}
1077+
1078+
return gaps
1079+
}
1080+
9251081
/**
9261082
* Fallback removed-test detector for callers that supply only the diff
9271083
* (no old/new content). Kept intentionally conservative: matches removed
@@ -985,6 +1141,8 @@ export function detectSchemaYmlPatterns(
9851141
testTag?: string
9861142
}> = []
9871143
let usedStructural = false
1144+
// R20 S1 — populated from the structural NEW-side parse below.
1145+
let grainGaps: GrainKeyGap[] = []
9881146

9891147
// Deleting a whole schema.yml removes every test declared in it — arguably
9901148
// a bigger removal than dropping a single test. Treat the new side as empty
@@ -1069,6 +1227,16 @@ export function detectSchemaYmlPatterns(
10691227
removals.push({ model, column, test, testTag })
10701228
}
10711229
usedStructural = true
1230+
// R20 S1 — grain-key `not_null` completeness. Only fire on
1231+
// non-deleted files: a deleted schema.yml has no current grain to
1232+
// guard. Extract from the NEW side so we flag the current state of
1233+
// the file, whether the grain declaration was newly added in this
1234+
// PR or pre-existing (a broken grain-guard is a real risk either
1235+
// way, and reviewers can suppress if the pre-existing case is by
1236+
// design). Uses the `not_null-missing` gap emitted below.
1237+
if (!isDeletedFile && newDoc !== undefined) {
1238+
grainGaps = extractGrainKeyGaps(newDoc)
1239+
}
10721240
}
10731241
}
10741242

@@ -1100,7 +1268,7 @@ export function detectSchemaYmlPatterns(
11001268
}
11011269
}
11021270

1103-
if (!removals.length) return []
1271+
if (!removals.length && !grainGaps.length) return []
11041272

11051273
const findings: Finding[] = []
11061274
const isMartLayer = /(^|\/)(marts?|reporting)\//.test(file.path)
@@ -1220,5 +1388,48 @@ export function detectSchemaYmlPatterns(
12201388
}),
12211389
)
12221390
}
1391+
1392+
// R20 S1 — grain-key `not_null` completeness. For every column in a
1393+
// `unique_combination_of_columns` test's `combination_of_columns` that
1394+
// lacks `not_null` coverage on the same model, emit one finding.
1395+
// Recommendation flips between `constraints:` (contracted model) and
1396+
// `data_tests:` (view / non-contracted model) based on the model's
1397+
// contract state — matches the adapter-semantics discussion in the
1398+
// corpus study (PR D×2, PR A×2).
1399+
for (const g of grainGaps) {
1400+
const filename = file.path.split("/").pop()
1401+
const recommendation = g.contractEnforced
1402+
? `Add \`constraints: [{type: not_null}]\` to \`${g.column}\` on \`${g.model}\` (contract is \`enforced: true\` so the constraint is enforced at write-time).`
1403+
: `Add \`not_null\` to \`${g.column}\`'s \`data_tests:\` on \`${g.model}\` (contract is not enforced, so a \`constraints:\` entry would be inert — use a data_test).`
1404+
findings.push(
1405+
makeFinding({
1406+
severity: clampSeverity("test_coverage", "warning", "high"),
1407+
category: "test_coverage",
1408+
title: `${filename}: grain column \`${g.column}\` in unique_combination_of_columns lacks \`not_null\` on \`${g.model}\``,
1409+
body:
1410+
`The \`unique_combination_of_columns\` test on \`${g.model}\` names \`${g.column}\` as a grain key, but no \`not_null\` ` +
1411+
`coverage is declared for it (either as a \`constraints:\` entry on a contracted model, or as a \`data_tests: [not_null]\` ` +
1412+
`on a view). A NULL grain-key value silently passes the uniqueness test, so a fan-out or duplicate bug can ship without ` +
1413+
`any test catching it. ${recommendation}`,
1414+
file: file.path,
1415+
model: g.model,
1416+
column: g.column,
1417+
confidence: "high",
1418+
evidence: {
1419+
tool: "dbt-patterns",
1420+
result: {
1421+
rule: "grain_key_not_null_missing",
1422+
model: g.model,
1423+
column: g.column,
1424+
contractEnforced: g.contractEnforced,
1425+
},
1426+
},
1427+
// Per-column ruleKey so the global fingerprint dedupe keeps distinct
1428+
// grain-column gaps on the same model as separate findings.
1429+
ruleKey: `test_coverage:grain-key-not-null:${g.model}.${g.column}`,
1430+
}),
1431+
)
1432+
}
1433+
12231434
return findings.filter((x) => !exclusionReason(x, rubric))
12241435
}

0 commit comments

Comments
 (0)