Skip to content

Commit 00d0a02

Browse files
anandgupta42claude
andcommitted
fix: sync remaining engine-output consumers flagged by consensus review
Round-2 contract sync for PR #1090, fixing every Critical/Major finding from the 4-model consensus review (all verified against the live 0.7.0 binary; all latent since the Python-engine elimination, not 0.7.0 breakage): Dead gates (silent false-passes): - `check --checks validate`: gated on envelope `success`, which the handler sets even for invalid SQL — every file passed. Now gates on `data.valid`, maps `ValidationError` (`location.line/column`, `suggestions`) and fails closed on engine failure. - `check --checks semantic` + `altimate_core_semantics` tool: engine returns `valid:true` WITH `findings` (cartesian product) — `valid` means "plannable", not "clean". Both consumers now read `findings` and never gate on `valid`; tool formatter/title/telemetry follow the findings list. - `check --checks grade`: read `grade`/`score`/`recommendations`, none of which `evaluate()` returns — no grade or finding ever surfaced. Now reads `overall_grade`/`scores.overall`/`lint.findings` and fails closed on the failure envelope. The check-e2e mock enshrined the fictional shape — fixed. - `schema.detect_pii` (pii-detector): read `piiData.findings`; engine `PiiReport` is `{ columns, pii_count, … }` — schema PII scanning returned zero findings for every scan. Shared `piiColumnsFromReport` now filters `classification !== "None"` on both the cache and live paths. Wrong shapes / crashes: - `altimate-core-compare` tool: read `differences` (engine: `identical`/ `diff_count`/`diffs`, `DiffEntry.change_type`) — different queries rendered "Compare: IDENTICAL". Error-gated title added. - `altimate-core-policy` tool: titled on `pass` (engine: `allowed`) — clean SQL always rendered "VIOLATIONS FOUND"; `metadata.success` now reflects the envelope and error output no longer contradicts the ERROR title. - Empty-dialect coercion centralized: new `dialectHint()` in `native/engine-coerce.ts` applied to all 7 dialect-forwarding handlers (columnLineage, formatSql, extractMetadata, compareQueries, importDdl + the two already fixed) — the compare/column-lineage/extract-metadata/ import-ddl tools crashed with `unknown dialect ''` whenever `dialect` was omitted (the common invocation path). - `{ Custom: string }` PII classifications rendered `[object Object]` in classify-pii, the composite check renderer, and the review runner's signed verdict — shared `classificationToString()` used everywhere; classify-pii also no longer counts `classification: "None"` rows and error-gates its title/output; query-pii treats engine `parse_error` as an abstention (previously rendered CLEAN for unparseable SQL). - Composite `altimate_core.check` now computes query PII (fail-safe) — the tool's "=== PII ===" section previously always printed "No PII detected" because the handler never populated it. - `ThreatFinding.location` is `[byteOffset, byteLength]` — surfaced as a `(chars a-b)` range in `check --checks safety` (was dropped entirely). Tests: real-engine "consumer contract sync (round 2)" block (semantics valid-gate, compare, policy pass/violation, classify-pii None-filter, no-dialect tool invocations, query-pii/classify-pii/policy error gating, composite-check PII, grade contract); CLI-shape tests for validate/semantic/ grade incl. fail-closed cases; pii-detector unit tests against the live engine + a DuckDB-gated e2e; legacy formatter tests updated to real shapes. Also filed upstream: stale `SafetyRule` union in the engine's index.d.ts (altimate-core-internal#764). Codex-verified twice: all shape corrections match the installed 0.7.0 engine; its five review findings (parse_error abstention, grade fail-closed, byte-length location, contradictory failure outputs, untracked files) are addressed in this commit. Co-Authored-By: Claude Fable 5 <[email protected]>
1 parent e3db0a4 commit 00d0a02

16 files changed

Lines changed: 680 additions & 117 deletions

‎packages/opencode/src/altimate/native/altimate-core.ts‎

Lines changed: 23 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
*/
1010

1111
import * as core from "@altimateai/altimate-core"
12+
import { dialectHint } from "./engine-coerce"
1213
import { register } from "./dispatcher"
1314
import { schemaOrEmpty, resolveSchema } from "./schema-resolver"
1415
import type { AltimateCoreResult } from "./types"
@@ -181,10 +182,20 @@ export function registerAll(): void {
181182
)
182183
: core.lint(params.sql, schema)
183184
const safety = core.scanSql(params.sql)
185+
// PII exposure for the composite check — the tool has always rendered a
186+
// PII section; previously nothing populated it. Additive: a PII failure
187+
// must not fail the whole composite.
188+
let pii: Record<string, unknown> = {}
189+
try {
190+
pii = toData(core.checkQueryPii(params.sql, schema))
191+
} catch {
192+
// validation/lint above already report unparseable SQL
193+
}
184194
const data: Record<string, unknown> = {
185195
validation: toData(validation),
186196
lint: toData(lintResult),
187197
safety: toData(safety),
198+
pii,
188199
}
189200
return ok(true, data)
190201
} catch (e) {
@@ -323,10 +334,10 @@ export function registerAll(): void {
323334
// Pass the optional dialect hint so dialect-specific compiled warehouse SQL
324335
// (e.g. Snowflake semi-structured `col:field`) parses and the pair is
325336
// decidable instead of abstaining on a syntax error. Supported since
326-
// [email protected]. Use `|| undefined` (not `??`) so the default empty
327-
// string from ReviewConfig.dialect coerces to "no hint": the engine throws
328-
// on an unknown dialect "", and "" must mean auto-detect, not a real dialect.
329-
const raw = await core.checkEquivalence(params.sql1, params.sql2, schema, params.dialect || undefined)
337+
// [email protected]. dialectHint coerces "" (the ReviewConfig default)
338+
// to undefined: the engine throws on an unknown dialect "", and "" must
339+
// mean auto-detect, not a real dialect.
340+
const raw = await core.checkEquivalence(params.sql1, params.sql2, schema, dialectHint(params.dialect))
330341
const data = toData(raw)
331342
return ok(true, data)
332343
} catch (e) {
@@ -337,10 +348,9 @@ export function registerAll(): void {
337348
// 12. altimate_core.migration
338349
register("altimate_core.migration", async (params) => {
339350
try {
340-
// Build schema from old_ddl, analyze new_ddl against it.
341-
// `|| undefined` (not `??`): "" must mean auto-detect — the engine
342-
// throws on an unknown dialect "" (same coercion as equivalence above).
343-
const schema = core.Schema.fromDdl(params.old_ddl, params.dialect || undefined)
351+
// Build schema from old_ddl, analyze new_ddl against it. dialectHint
352+
// coerces "" to auto-detect (the engine throws on an unknown dialect "").
353+
const schema = core.Schema.fromDdl(params.old_ddl, dialectHint(params.dialect))
344354
const raw = core.analyzeMigration(params.new_ddl, schema)
345355
const data = toData(raw)
346356
return ok(true, data)
@@ -434,7 +444,7 @@ export function registerAll(): void {
434444
register("altimate_core.column_lineage", async (params) => {
435445
try {
436446
const schema = resolveSchema(params.schema_path, params.schema_context)
437-
const raw = core.columnLineage(params.sql, params.dialect ?? undefined, schema ?? undefined)
447+
const raw = core.columnLineage(params.sql, dialectHint(params.dialect), schema ?? undefined)
438448
return ok(true, toData(raw))
439449
} catch (e) {
440450
return fail(e)
@@ -455,7 +465,7 @@ export function registerAll(): void {
455465
// 22. altimate_core.format
456466
register("altimate_core.format", async (params) => {
457467
try {
458-
const raw = core.formatSql(params.sql, params.dialect ?? undefined)
468+
const raw = core.formatSql(params.sql, dialectHint(params.dialect))
459469
const data = toData(raw)
460470
return ok(true, data)
461471
} catch (e) {
@@ -466,7 +476,7 @@ export function registerAll(): void {
466476
// 23. altimate_core.metadata
467477
register("altimate_core.metadata", async (params) => {
468478
try {
469-
const raw = core.extractMetadata(params.sql, params.dialect ?? undefined)
479+
const raw = core.extractMetadata(params.sql, dialectHint(params.dialect))
470480
return ok(true, toData(raw))
471481
} catch (e) {
472482
return fail(e)
@@ -476,7 +486,7 @@ export function registerAll(): void {
476486
// 24. altimate_core.compare
477487
register("altimate_core.compare", async (params) => {
478488
try {
479-
const raw = core.compareQueries(params.left_sql, params.right_sql, params.dialect ?? undefined)
489+
const raw = core.compareQueries(params.left_sql, params.right_sql, dialectHint(params.dialect))
480490
return ok(true, toData(raw))
481491
} catch (e) {
482492
return fail(e)
@@ -530,7 +540,7 @@ export function registerAll(): void {
530540
// 29. altimate_core.import_ddl — returns Schema, must serialize
531541
register("altimate_core.import_ddl", async (params) => {
532542
try {
533-
const schema = core.importDdl(params.ddl, params.dialect ?? undefined)
543+
const schema = core.importDdl(params.ddl, dialectHint(params.dialect))
534544
const jsonObj = schema.toJson()
535545
return ok(true, { success: true, schema: toData(jsonObj) })
536546
} catch (e) {
Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
/**
2+
* Shared coercions for altimate-core engine values.
3+
*
4+
* The engine's napi surface has two recurring foot-guns for consumers:
5+
* - `PiiClassification` is a string OR `{ Custom: string }` — naive string
6+
* interpolation renders `[object Object]`.
7+
* - dialect parameters throw on the empty string (`unknown dialect ''`);
8+
* `""` must mean "auto-detect" and be passed as undefined.
9+
*/
10+
11+
/** PiiClassification is 'Email' | … | { Custom: string } | 'None'. */
12+
export function classificationToString(c: unknown, fallback = "PII"): string {
13+
if (typeof c === "string") return c
14+
if (c && typeof c === "object" && typeof (c as { Custom?: unknown }).Custom === "string") {
15+
return (c as { Custom: string }).Custom
16+
}
17+
return fallback
18+
}
19+
20+
/** Coerce an optional dialect to an engine-safe hint: "" and null mean auto-detect. */
21+
export function dialectHint(dialect: string | undefined | null): string | undefined {
22+
return dialect || undefined
23+
}

‎packages/opencode/src/altimate/native/schema/pii-detector.ts‎

Lines changed: 41 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
*/
55

66
import * as core from "@altimateai/altimate-core"
7+
import { classificationToString } from "../engine-coerce"
78
import { getCache } from "./cache"
89
import * as Registry from "../connections/registry"
910
import type {
@@ -12,6 +13,19 @@ import type {
1213
PiiFinding,
1314
} from "../types"
1415

16+
/**
17+
* Extract the real PII rows from an engine PiiReport.
18+
*
19+
* The engine returns `{ columns, pii_count, risk_level, total_columns }` with
20+
* a row for EVERY column — classification "None" means not PII. (The previous
21+
* consumer read a nonexistent `findings` field, making detection a silent
22+
* no-op.) Exported for tests.
23+
*/
24+
export function piiColumnsFromReport(piiData: unknown): Array<Record<string, any>> {
25+
const columns = ((piiData as Record<string, any>)?.columns ?? []) as Array<Record<string, any>>
26+
return columns.filter((c) => c.classification !== "None")
27+
}
28+
1529
/**
1630
* Detect PII in cached schema metadata by running altimate-core's
1731
* classifyPii() on column names and types.
@@ -72,19 +86,20 @@ export async function detectPii(params: PiiDetectParams): Promise<PiiDetectResul
7286
const result = core.classifyPii(schema)
7387
const piiData = JSON.parse(JSON.stringify(result))
7488

75-
if (piiData && piiData.findings && piiData.findings.length > 0) {
76-
for (const finding of piiData.findings) {
77-
findings.push({
78-
warehouse: col.warehouse,
79-
schema: col.schema_name,
80-
table: col.table,
81-
column: col.name,
82-
data_type: col.data_type,
83-
pii_category: finding.category || finding.pii_type || "UNKNOWN",
84-
confidence: finding.confidence || "medium",
85-
})
86-
tablesWithPii.add(`${col.warehouse}.${col.schema_name}.${col.table}`)
87-
}
89+
// Engine PiiReport: { columns, pii_count, risk_level, total_columns };
90+
// every column gets a row — classification "None" means not PII.
91+
const piiColumns = piiColumnsFromReport(piiData)
92+
for (const piiCol of piiColumns) {
93+
findings.push({
94+
warehouse: col.warehouse,
95+
schema: col.schema_name,
96+
table: col.table,
97+
column: col.name,
98+
data_type: col.data_type,
99+
pii_category: classificationToString(piiCol.classification, "UNKNOWN"),
100+
confidence: piiCol.confidence ?? "medium",
101+
})
102+
tablesWithPii.add(`${col.warehouse}.${col.schema_name}.${col.table}`)
88103
}
89104
} catch {
90105
// classifyPii may not find PII — that is expected
@@ -165,19 +180,19 @@ async function detectPiiLive(params: PiiDetectParams): Promise<PiiDetectResult>
165180
const result = core.classifyPii(schema)
166181
const piiData = JSON.parse(JSON.stringify(result))
167182

168-
if (piiData?.findings) {
169-
for (const finding of piiData.findings) {
170-
findings.push({
171-
warehouse: params.warehouse!,
172-
schema: schemaName,
173-
table: tableInfo.name,
174-
column: finding.column || "",
175-
data_type: finding.data_type,
176-
pii_category: finding.category || finding.pii_type || "UNKNOWN",
177-
confidence: finding.confidence || "medium",
178-
})
179-
tablesWithPii.add(`${params.warehouse}.${schemaName}.${tableInfo.name}`)
180-
}
183+
// Engine PiiReport: { columns, … } with a row per column; "None" = not PII.
184+
const piiColumns = piiColumnsFromReport(piiData)
185+
for (const piiCol of piiColumns) {
186+
findings.push({
187+
warehouse: params.warehouse!,
188+
schema: schemaName,
189+
table: tableInfo.name,
190+
column: piiCol.column || "",
191+
data_type: columns.find((c) => c.name === piiCol.column)?.data_type,
192+
pii_category: classificationToString(piiCol.classification, "UNKNOWN"),
193+
confidence: piiCol.confidence ?? "medium",
194+
})
195+
tablesWithPii.add(`${params.warehouse}.${schemaName}.${tableInfo.name}`)
181196
}
182197
} catch {
183198
// ignore

‎packages/opencode/src/altimate/review/runner.ts‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
import { Dispatcher } from "../native"
2+
import { classificationToString } from "../native/engine-coerce"
23
import { parseManifest } from "../native/dbt/manifest"
34
import type { CheckResult, EquivalenceResult, GradeResult, ImpactResult, ReviewRunner } from "./orchestrate"
45
import { buildReviewSchemaContext, type SchemaContext } from "./schema-context"
@@ -400,7 +401,8 @@ export function createDispatcherRunner(opts: DispatcherRunnerOptions): ReviewRun
400401
.filter((c) => c?.classification && c.classification !== "None")
401402
.map((c) => ({
402403
column: String(c.column ?? ""),
403-
classification: String(c.classification ?? ""),
404+
// classification can be { Custom: string } — String() would emit "[object Object]"
405+
classification: classificationToString(c.classification, ""),
404406
confidence: typeof c.confidence === "number" ? c.confidence : 0,
405407
masking: c.suggested_masking ?? undefined,
406408
}))

‎packages/opencode/src/altimate/tools/altimate-core-check.ts‎

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
import z from "zod"
22
import { Tool } from "../../tool/tool"
33
import { Dispatcher } from "../native"
4+
import { classificationToString } from "../native/engine-coerce"
45
import type { Telemetry } from "../telemetry"
56

67
export const AltimateCoreCheckTool = Tool.define("altimate_core_check", {
@@ -32,7 +33,7 @@ export const AltimateCoreCheckTool = Tool.define("altimate_core_check", {
3233
for (const t of data.safety?.threats ?? []) {
3334
findings.push({ category: t.rule ?? t.type ?? "safety_threat" })
3435
}
35-
for (const p of data.pii?.findings ?? []) {
36+
for (const p of data.pii?.pii_columns ?? data.pii?.findings ?? []) {
3637
findings.push({ category: "pii_detected" })
3738
}
3839
// altimate_change end
@@ -62,7 +63,7 @@ export function formatCheckTitle(data: Record<string, any>): string {
6263
if (!data.validation?.valid) parts.push("validation errors")
6364
if (!data.lint?.clean) parts.push(`${data.lint?.findings?.length ?? 0} lint findings`)
6465
if (!data.safety?.safe) parts.push("safety threats")
65-
if (data.pii?.findings?.length) parts.push("PII detected")
66+
if (data.pii?.pii_columns?.length || data.pii?.findings?.length) parts.push("PII detected")
6667
return parts.length ? parts.join(", ") : "PASS"
6768
}
6869

@@ -98,11 +99,15 @@ export function formatCheck(data: Record<string, any>): string {
9899
}
99100

100101
lines.push("\n=== PII ===")
101-
if (!data.pii?.findings?.length) {
102+
// Engine PiiQueryResult: { accesses_pii, pii_columns, risk_level, … }
103+
const piiCols = (data.pii?.pii_columns ?? data.pii?.findings ?? []) as any[]
104+
if (!piiCols.length) {
102105
lines.push("No PII detected.")
103106
} else {
104-
for (const p of data.pii?.findings ?? []) {
105-
lines.push(` ${p.column ?? "unknown"}: ${p.category ?? "PII"} (${p.confidence ?? "unknown"} confidence)`)
107+
for (const p of piiCols) {
108+
const cls = classificationToString(p.classification ?? p.category)
109+
const where = [p.table, p.column ?? "unknown"].filter(Boolean).join(".")
110+
lines.push(` ${where}: ${cls}${p.suggested_masking ? ` (masking: ${p.suggested_masking})` : ""}`)
106111
}
107112
}
108113

‎packages/opencode/src/altimate/tools/altimate-core-classify-pii.ts‎

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,13 @@
11
import z from "zod"
22
import { Tool } from "../../tool/tool"
33
import { Dispatcher } from "../native"
4+
import { classificationToString } from "../native/engine-coerce"
5+
6+
/** Engine PiiReport lists EVERY column; classification "None" means not PII. */
7+
function realPiiColumns(data: Record<string, any>): any[] {
8+
const columns = (data.columns ?? data.findings ?? []) as any[]
9+
return columns.filter((c) => c.classification !== "None")
10+
}
411

512
export const AltimateCoreClassifyPiiTool = Tool.define("altimate_core_classify_pii", {
613
description:
@@ -16,13 +23,14 @@ export const AltimateCoreClassifyPiiTool = Tool.define("altimate_core_classify_p
1623
schema_context: args.schema_context,
1724
})
1825
const data = (result.data ?? {}) as Record<string, any>
19-
const piiColumns = data.columns ?? data.findings ?? []
26+
const piiColumns = realPiiColumns(data)
2027
const findingCount = piiColumns.length
2128
const error = result.error ?? data.error
2229
return {
23-
title: `PII Classification: ${findingCount} finding(s)`,
30+
// Never render a finding count when the engine call itself failed.
31+
title: error ? "PII Classification: ERROR" : `PII Classification: ${findingCount} finding(s)`,
2432
metadata: { success: result.success, finding_count: findingCount, ...(error && { error }) },
25-
output: formatClassifyPii(data),
33+
output: error ? `Error: ${error}` : formatClassifyPii(data),
2634
}
2735
} catch (e) {
2836
const msg = e instanceof Error ? e.message : String(e)
@@ -37,15 +45,15 @@ export const AltimateCoreClassifyPiiTool = Tool.define("altimate_core_classify_p
3745

3846
function formatClassifyPii(data: Record<string, any>): string {
3947
if (data.error) return `Error: ${data.error}`
40-
const piiColumns = data.columns ?? data.findings ?? []
48+
const piiColumns = realPiiColumns(data)
4149
if (!piiColumns.length) return "No PII columns detected."
4250
const lines: string[] = []
4351
if (data.risk_level) lines.push(`Risk level: ${data.risk_level}`)
4452
if (data.pii_count != null) lines.push(`PII columns: ${data.pii_count} of ${data.total_columns}`)
4553
lines.push("")
4654
lines.push("PII columns found:")
4755
for (const f of piiColumns) {
48-
const classification = f.classification ?? f.category ?? "PII"
56+
const classification = classificationToString(f.classification ?? f.category)
4957
const confidence = f.confidence ?? "high"
5058
const table = f.table ?? "unknown"
5159
const column = f.column ?? "unknown"

‎packages/opencode/src/altimate/tools/altimate-core-compare.ts‎

Lines changed: 17 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -18,12 +18,21 @@ export const AltimateCoreCompareTool = Tool.define("altimate_core_compare", {
1818
dialect: args.dialect ?? "",
1919
})
2020
const data = (result.data ?? {}) as Record<string, any>
21-
const diffCount = data.differences?.length ?? 0
21+
// Engine CompareResult: { identical, diff_count, diffs } — `differences`
22+
// never existed, so every comparison used to render IDENTICAL.
23+
const diffs = (data.diffs ?? data.differences ?? []) as any[]
24+
const diffCount = data.diff_count ?? diffs.length
2225
const error = result.error ?? data.error
26+
// Never render IDENTICAL when the engine call itself failed.
27+
const title = error
28+
? "Compare: ERROR"
29+
: data.identical === false || diffCount > 0
30+
? `Compare: ${diffCount} difference(s)`
31+
: "Compare: IDENTICAL"
2332
return {
24-
title: `Compare: ${diffCount === 0 ? "IDENTICAL" : `${diffCount} difference(s)`}`,
33+
title,
2534
metadata: { success: result.success, difference_count: diffCount, ...(error && { error }) },
26-
output: formatCompare(data),
35+
output: error ? `Error: ${error}` : formatCompare(data),
2736
}
2837
} catch (e) {
2938
const msg = e instanceof Error ? e.message : String(e)
@@ -38,10 +47,12 @@ export const AltimateCoreCompareTool = Tool.define("altimate_core_compare", {
3847

3948
function formatCompare(data: Record<string, any>): string {
4049
if (data.error) return `Error: ${data.error}`
41-
if (!data.differences?.length) return "Queries are structurally identical."
50+
// Engine DiffEntry: { change_type, description }.
51+
const diffs = (data.diffs ?? data.differences ?? []) as any[]
52+
if (!diffs.length) return "Queries are structurally identical."
4253
const lines = ["Structural differences:\n"]
43-
for (const d of data.differences) {
44-
lines.push(` [${d.type ?? "change"}] ${d.description ?? d.message ?? d}`)
54+
for (const d of diffs) {
55+
lines.push(` [${d.change_type ?? d.type ?? "change"}] ${d.description ?? d.message ?? d}`)
4556
}
4657
return lines.join("\n")
4758
}

0 commit comments

Comments
 (0)