Skip to content

Commit 7aa18cd

Browse files
anandgupta42claude
andauthored
fix: redact non-SQL content echoed in safety threat messages (main sanity failure) (AltimateAI#1111)
* fix: redact non-SQL content echoed in safety threat messages The main-only Sanity (Verdaccio) security test 6/15 (`altimate check ../../../../etc/passwd` must not reflect file contents) fails since AltimateAI#1090: surfacing real `ThreatFinding` messages exposed that the engine's `multi_statement` rule quotes the raw "statement type" token verbatim — for non-SQL input that is the file's content (`Disallowed statement type: ROOT:X:0:0:…`), an information-disclosure echo. The sanity job is skipped on PRs, so AltimateAI#1090's CI could not catch it. Fix at the dispatcher boundary (covers CLI, tools, and review): - `EngineCoerce.redactThreatText`: keeps SQL-keyword-like statement types (`DROP`, `ALTER TABLE`) and replaces arbitrary content with `<non-SQL content redacted>` in threat message/detail. - `multi_statement` threats also redact `matched_pattern` — for that rule the pattern IS the raw input line. Injection rules keep their SQL-shaped patterns. Diff-scoping keys stay consistent (base and head redact identically). - Applied in the `altimate_core.safety` handler and the composite check. Tests: unit coverage for the redaction (keyword kept / content redacted / unrelated messages untouched) plus real-engine tests proving passwd-like input no longer echoes through either handler while `DROP` survives in messages. Verified the exact sanity scenario locally: CLI output now shows `<non-SQL content redacted>`. Filed upstream for the engine-side echo: to be linked. Closes AltimateAI#1110 Co-Authored-By: Claude Fable 5 <[email protected]> * fix: diff-scope safety on raw scans, redact after subtraction Cursor review catch: redacting the head scan BEFORE the base subtraction rewrote `multi_statement` matched_patterns to "<redacted>", so base keys (raw) never matched and pre-existing threats resurfaced as newly introduced. Redaction now runs after diff-scoping — subtraction compares raw engine patterns on both sides. Regression test: identical base/head passwd-like input yields zero threats and safe:true. Co-Authored-By: Claude Fable 5 <[email protected]> * fix: harden threat redaction — keyword allowlist, greedy quotes, all consumers Round-2 review hardening on the redaction: - Keyword ALLOWLIST, not shape: the token is kept only when its first word is a known SQL statement keyword — shape alone passed content like "TOP SECRET PASSWORD" or "AKIAIOSFODNN7EXAMPLE". - Greedy quoted match: an embedded apostrophe in the raw token ("Statement type 'DROP'ROOT:X:0' is not…") previously closed the match early and leaked the remainder; greedy capture spans to the last quote and embedded quotes fail the check, redacting everything inside. - All scanSql consumers covered: `redactScan` moved to `EngineCoerce` (pure, no NAPI import) and applied in `sql.analyze` (copied raw message/detail into issues) and the `altimate_core.grade` handler (EvalResult embeds a full safety scan that the CLI grade check renders). - Tests: allowlist/apostrophe cases; composite + safety tests assert success AND non-empty threats AND the redaction marker (previously vacuous if threats were missing); sql.analyze + grade echo tests; Dispatcher.reset() and telemetry-env restore in afterAll; shared dispatcher import hoisted into beforeAll. Co-Authored-By: Claude Fable 5 <[email protected]> * fix: whole-phrase keyword allowlist + non-vacuous consumer tests - `isSqlStatementType` now requires EVERY word of the preserved phrase to come from the SQL keyword vocabulary (statement keywords first, auxiliary words like TABLE/INDEX/IF/EXISTS after) — checking only the first word preserved attacker tails such as "SET SECRET PASSWORD" verbatim. Since preserved text is a subset of the fixed vocabulary, no attacker-specific content can survive. Tests cover keyword-prefixed tails and compound types ("DROP INDEX IF EXISTS" stays useful). - `sql.analyze` test asserts non-empty safety issues + the redaction marker (was vacuous if the scan produced nothing). - Grade test corrected to the REAL EvalResult shape: its `safety` section is `{ method, score }` — there are no nested threats to echo (the handler redaction stays as defense-in-depth). The test now asserts the whole result is echo-free minus the by-contract input-echo fields (`sql`, `lint.sql`), which the CLI never renders. Co-Authored-By: Claude Fable 5 <[email protected]> * fix: exact single-keyword statement-type allowlist (engine-faithful) Two review comments pulled opposite directions — cubic wanted compound types like "DELETE FROM" preserved; Codex wanted exact statement types instead of any keyword-vocabulary ordering ("SELECT DELETE UPDATE" leaked verbatim). Live-probing the 0.7.0 engine across DDL/DML/DCL/TCL settles both: the engine emits SINGLE-WORD statement types only ({ALTER, BEGIN, CALL, CREATE, DELETE, DROP, EXPLAIN, GRANT, INSERT, SET, TRUNCATE, UPDATE}). The allowlist is now an exact single-keyword match — the strictest rule that never redacts anything the engine actually produces: multi-word input can only be non-engine content, so keyword-prefixed tails and keyword permutations both redact. Vocabulary-ordering and multi-word tests updated with the probe rationale inline. Co-Authored-By: Claude Fable 5 <[email protected]> --------- Co-authored-by: Claude Fable 5 <[email protected]>
1 parent d9292ec commit 7aa18cd

4 files changed

Lines changed: 237 additions & 3 deletions

File tree

‎packages/opencode/src/altimate/native/altimate-core.ts‎

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -114,7 +114,7 @@ export function registerAll(): void {
114114
register("altimate_core.safety", async (params) => {
115115
try {
116116
const raw = core.scanSql(params.sql)
117-
const data = toData(raw)
117+
const data = EngineCoerce.redactScan(toData(raw))
118118
return ok(true, data)
119119
} catch (e) {
120120
return fail(e)
@@ -229,6 +229,10 @@ export function registerAll(): void {
229229
// Unscannable base — keep the full head scan (fail open to MORE findings).
230230
}
231231
}
232+
// Redact AFTER diff-scoping: subtraction must compare raw engine
233+
// matched_patterns on both sides (redacting first would rewrite head
234+
// multi_statement keys to "<redacted>" and never match the base).
235+
safety = EngineCoerce.redactScan(safety)
232236
// PII exposure for the composite check — the tool has always rendered a
233237
// PII section; previously nothing populated it. Additive: a PII failure
234238
// must not fail the whole composite.
@@ -470,7 +474,13 @@ export function registerAll(): void {
470474
try {
471475
const schema = schemaOrEmpty(params.schema_path, params.schema_context)
472476
const raw = await core.evaluate(params.sql, schema)
473-
return ok(true, toData(raw))
477+
const data = toData(raw)
478+
// EvalResult embeds a full safety scan — redact its threat echoes too
479+
// (the CLI grade check renders nested threat messages).
480+
if (data.safety && typeof data.safety === "object") {
481+
data.safety = EngineCoerce.redactScan(data.safety as Record<string, unknown>)
482+
}
483+
return ok(true, data)
474484
} catch (e) {
475485
return fail(e)
476486
}

‎packages/opencode/src/altimate/native/engine-coerce.ts‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,4 +59,67 @@ export function piiColumnsFromReport(piiData: unknown): Array<Record<string, any
5959
return columns.filter((c) => c && c.classification !== "None")
6060
}
6161

62+
/**
63+
* Redact raw input echoed inside engine threat messages.
64+
*
65+
* The engine's `multi_statement` rule quotes the offending "statement type"
66+
* verbatim — for non-SQL input (e.g. `altimate check /etc/passwd`) that
67+
* reflects arbitrary file content back into CLI/tool output. A token is kept
68+
* only when it BOTH looks like a short keyword phrase AND starts with a known
69+
* SQL statement keyword — shape alone would pass content like
70+
* "TOP SECRET PASSWORD" or "AKIAIOSFODNN7EXAMPLE".
71+
*/
72+
const REDACTED = "<non-SQL content redacted>"
73+
const SQL_TOKEN_SHAPE = /^[A-Za-z_][A-Za-z0-9_$ ]{0,31}$/
74+
const SQL_STATEMENT_KEYWORDS = new Set([
75+
"SELECT", "INSERT", "UPDATE", "DELETE", "DROP", "CREATE", "ALTER", "TRUNCATE",
76+
"GRANT", "REVOKE", "MERGE", "WITH", "BEGIN", "COMMIT", "ROLLBACK", "SET",
77+
"USE", "CALL", "EXPLAIN", "ANALYZE", "VACUUM", "COPY", "SHOW", "DESCRIBE",
78+
"DESC", "EXECUTE", "EXEC", "REPLACE", "VALUES", "LOCK", "UNLOCK", "COMMENT",
79+
"RENAME", "START", "SAVEPOINT", "RELEASE", "PREPARE", "DEALLOCATE",
80+
])
81+
// The engine emits SINGLE-WORD statement types only — probed against the
82+
// live 0.7.0 binary across DDL/DML/DCL/TCL second statements, the full
83+
// observed set is {ALTER, BEGIN, CALL, CREATE, DELETE, DROP, EXPLAIN, GRANT,
84+
// INSERT, SET, TRUNCATE, UPDATE}. Exact single-keyword matching is therefore
85+
// both engine-faithful and the strictest rule: multi-word input can only be
86+
// non-engine content (kills keyword-permutation leaks like
87+
// "SELECT DELETE UPDATE" without redacting anything the engine produces).
88+
function isSqlStatementType(token: string): boolean {
89+
const t = token.trim()
90+
if (!SQL_TOKEN_SHAPE.test(t)) return false
91+
return SQL_STATEMENT_KEYWORDS.has(t.toUpperCase())
92+
}
93+
export function redactThreatText(text: string): string {
94+
return (
95+
text
96+
.replace(/(Disallowed statement type: )(.+)$/i, (_m, prefix: string, tok: string) =>
97+
isSqlStatementType(tok) ? `${prefix}${tok}` : `${prefix}${REDACTED}`,
98+
)
99+
// GREEDY quoted match — an attacker apostrophe inside the token would
100+
// otherwise close the match early and leak the remainder
101+
// (e.g. "Statement type 'DROP'ROOT:X:0' is not…"). Greedy capture spans
102+
// to the LAST quote; embedded quotes fail the shape check and redact.
103+
.replace(/(Statement type ')(.*)(')/i, (_m, a: string, tok: string, c: string) =>
104+
isSqlStatementType(tok) ? `${a}${tok}${c}` : `${a}${REDACTED}${c}`,
105+
)
106+
)
107+
}
108+
109+
/**
110+
* Redact raw-input echoes from a scan result's threats. For `multi_statement`
111+
* the matched pattern IS the raw input line, so it is always redacted;
112+
* injection rules keep their SQL-shaped patterns. Pure record manipulation —
113+
* safe to use from any consumer without loading the NAPI binding.
114+
*/
115+
export function redactScan(scan: Record<string, unknown>): Record<string, unknown> {
116+
const threats = (scan.threats as any[] | undefined)?.map((t: any) => ({
117+
...t,
118+
message: typeof t.message === "string" ? redactThreatText(t.message) : t.message,
119+
detail: typeof t.detail === "string" ? redactThreatText(t.detail) : t.detail,
120+
matched_pattern: t.rule === "multi_statement" ? "<redacted>" : t.matched_pattern,
121+
}))
122+
return threats ? { ...scan, threats } : scan
123+
}
124+
62125
export * as EngineCoerce from "./engine-coerce"

‎packages/opencode/src/altimate/native/sql/register.ts‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,8 @@ export function registerAllSql(): void {
3838

3939
const lint = JSON.parse(JSON.stringify(lintRaw))
4040
const semantics = JSON.parse(JSON.stringify(semanticsRaw))
41-
const safety = JSON.parse(JSON.stringify(safetyRaw))
41+
// Redact raw-input echoes (multi_statement quotes the input verbatim).
42+
const safety = EngineCoerce.redactScan(JSON.parse(JSON.stringify(safetyRaw))) as any
4243

4344
const issues: SqlAnalyzeIssue[] = []
4445

Lines changed: 160 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,160 @@
1+
import { describe, expect, test, beforeAll, afterAll } from "bun:test"
2+
import { EngineCoerce } from "../../src/altimate/native/engine-coerce"
3+
4+
let coreAvailable = false
5+
try {
6+
require.resolve("@altimateai/altimate-core")
7+
coreAvailable = true
8+
} catch {}
9+
const describeIf = coreAvailable ? describe : describe.skip
10+
11+
describe("redactThreatText", () => {
12+
test("keeps single SQL-statement-keyword types (the only shape the engine emits)", () => {
13+
expect(EngineCoerce.redactThreatText("Disallowed statement type: DROP")).toBe("Disallowed statement type: DROP")
14+
expect(EngineCoerce.redactThreatText("Statement type 'ALTER' is not in the allowed list")).toBe(
15+
"Statement type 'ALTER' is not in the allowed list",
16+
)
17+
})
18+
19+
test("redacts non-SQL content (e.g. /etc/passwd lines)", () => {
20+
const msg = "Disallowed statement type: ROOT:X:0:0:ROOT:/ROOT:/BIN/BASH"
21+
expect(EngineCoerce.redactThreatText(msg)).toBe("Disallowed statement type: <non-SQL content redacted>")
22+
const detail = "Statement type 'ROOT:X:0:0:ROOT:/ROOT:/BIN/BASH' is not in the allowed list: [\"SELECT\"]"
23+
expect(EngineCoerce.redactThreatText(detail)).toContain("'<non-SQL content redacted>'")
24+
expect(EngineCoerce.redactThreatText(detail)).not.toContain("ROOT:X:0")
25+
})
26+
27+
test("redacts keyword-shaped but non-SQL content (allowlist, not shape)", () => {
28+
// Shape alone would pass these — the first word must be a SQL statement keyword.
29+
expect(EngineCoerce.redactThreatText("Disallowed statement type: TOP SECRET PASSWORD")).toBe(
30+
"Disallowed statement type: <non-SQL content redacted>",
31+
)
32+
expect(EngineCoerce.redactThreatText("Disallowed statement type: AKIAIOSFODNN7EXAMPLE")).toBe(
33+
"Disallowed statement type: <non-SQL content redacted>",
34+
)
35+
expect(EngineCoerce.redactThreatText("Statement type 'PRIVATE NOTE' is not in the allowed list")).toBe(
36+
"Statement type '<non-SQL content redacted>' is not in the allowed list",
37+
)
38+
// ANY multi-word phrase redacts — the engine only emits single-word
39+
// statement types (live-probed), so multi-word input is never engine
40+
// vocabulary: keyword-prefixed tails and keyword permutations both die.
41+
expect(EngineCoerce.redactThreatText("Disallowed statement type: SET SECRET PASSWORD")).toBe(
42+
"Disallowed statement type: <non-SQL content redacted>",
43+
)
44+
expect(EngineCoerce.redactThreatText("Disallowed statement type: DROP DEAD FRED")).toBe(
45+
"Disallowed statement type: <non-SQL content redacted>",
46+
)
47+
expect(EngineCoerce.redactThreatText("Disallowed statement type: SELECT DELETE UPDATE")).toBe(
48+
"Disallowed statement type: <non-SQL content redacted>",
49+
)
50+
})
51+
52+
test("embedded apostrophes cannot close the quoted match early", () => {
53+
// Greedy capture spans to the LAST quote; the embedded quote fails the
54+
// shape check and everything inside is redacted.
55+
const attack = "Statement type 'DROP'ROOT:X:0:0' is not in the allowed list"
56+
const out = EngineCoerce.redactThreatText(attack)
57+
expect(out).not.toContain("ROOT:X:0")
58+
expect(out).toContain("<non-SQL content redacted>")
59+
})
60+
61+
test("leaves unrelated messages untouched", () => {
62+
expect(EngineCoerce.redactThreatText("Tautology attack detected (numeric constant comparison)")).toBe(
63+
"Tautology attack detected (numeric constant comparison)",
64+
)
65+
})
66+
})
67+
68+
// Sanity security test [6/15] regression: `altimate check /etc/passwd` must not
69+
// reflect file contents in threat messages (found by the main-only Verdaccio
70+
// sanity job after PR #1090 surfaced real threat messages).
71+
describeIf("safety consumers redact raw-input echoes (real engine)", () => {
72+
let D: typeof import("../../src/altimate/native/dispatcher")
73+
let priorTelemetry: string | undefined
74+
75+
beforeAll(async () => {
76+
priorTelemetry = process.env.ALTIMATE_TELEMETRY_DISABLED
77+
process.env.ALTIMATE_TELEMETRY_DISABLED = "true"
78+
const { registerAll } = await import("../../src/altimate/native/altimate-core")
79+
const { registerAllSql } = await import("../../src/altimate/native/sql/register")
80+
registerAll()
81+
registerAllSql()
82+
D = await import("../../src/altimate/native/dispatcher")
83+
})
84+
85+
afterAll(async () => {
86+
if (priorTelemetry === undefined) delete process.env.ALTIMATE_TELEMETRY_DISABLED
87+
else process.env.ALTIMATE_TELEMETRY_DISABLED = priorTelemetry
88+
// registerAll mutates shared Dispatcher state — reset for test isolation.
89+
;(await import("../../src/altimate/native/dispatcher")).reset()
90+
})
91+
92+
const PASSWD = "root:x:0:0:root:/root:/bin/bash\ndaemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin\n"
93+
94+
test("altimate_core.safety does not echo non-SQL file content", async () => {
95+
const r = await D.call("altimate_core.safety", { sql: PASSWD })
96+
expect(r.success).toBe(true)
97+
const threats = ((r.data as any).threats ?? []) as any[]
98+
expect(threats.length).toBeGreaterThan(0)
99+
const text = JSON.stringify(threats)
100+
expect(text.toLowerCase()).not.toContain("root:x:0")
101+
expect(text).toContain("redacted")
102+
})
103+
104+
test("composite check does not echo non-SQL file content in threats", async () => {
105+
const r = await D.call("altimate_core.check", { sql: PASSWD })
106+
expect(r.success).toBe(true)
107+
// The sanitizer must actually be exercised — an empty threat list would
108+
// make the not-contains assertion vacuous.
109+
const threats = (((r.data as any).safety?.threats ?? []) as any[])
110+
expect(threats.length).toBeGreaterThan(0)
111+
const text = JSON.stringify(threats)
112+
expect(text.toLowerCase()).not.toContain("root:x:0")
113+
expect(text).toContain("redacted")
114+
})
115+
116+
test("sql.analyze does not echo non-SQL file content in issues", async () => {
117+
const r = await D.call("sql.analyze", { sql: PASSWD })
118+
const safetyIssues = (((r as any).issues ?? []) as any[]).filter((i) => i.type === "safety")
119+
// The sanitizer must actually be exercised — no safety issues would make
120+
// the not-contains assertion vacuous.
121+
expect(safetyIssues.length).toBeGreaterThan(0)
122+
const text = JSON.stringify(safetyIssues)
123+
expect(text.toLowerCase()).not.toContain("root:x:0")
124+
expect(text).toContain("redacted")
125+
})
126+
127+
test("altimate_core.grade result does not echo non-SQL file content", async () => {
128+
// EvalResult's safety section is { method, score } — no threat scan to
129+
// echo (the handler's redactScan is defense-in-depth if that changes).
130+
// Assert the whole result is echo-free, minus the `sql` field, which by
131+
// contract carries back the caller's own input.
132+
const r = await D.call("altimate_core.grade", { sql: PASSWD })
133+
expect(r.success).toBe(true)
134+
// `sql` (EvalResult) and `lint.sql` (LintResult) carry back the caller's
135+
// own input by contract and are never rendered by the CLI grade check.
136+
const { sql: _echoedInput, ...rest } = r.data as any
137+
if (rest.lint) rest.lint = { ...rest.lint, sql: undefined }
138+
expect(JSON.stringify(rest).toLowerCase()).not.toContain("root:x:0")
139+
})
140+
141+
test("diff-scoping still filters pre-existing multi_statement threats despite redaction", async () => {
142+
// Regression (cursor review on #1111): redacting BEFORE base subtraction
143+
// rewrote head matched_patterns to "<redacted>" so base keys never matched
144+
// and pre-existing threats resurfaced as introduced. Redaction now runs
145+
// after subtraction: identical base/head must yield zero threats.
146+
const r = await D.call("altimate_core.check", { sql: PASSWD, base_sql: PASSWD })
147+
expect((((r.data as any).safety?.threats ?? []) as any[]).length).toBe(0)
148+
expect((r.data as any).safety.safe).toBe(true)
149+
})
150+
151+
test("real SQL statement types are preserved in messages", async () => {
152+
const r = await D.call("altimate_core.safety", { sql: "SELECT 1; DROP TABLE users;" })
153+
const threats = ((r.data as any).threats ?? []) as any[]
154+
const messages = threats.map((t) => `${t.message} ${t.detail}`).join(" ")
155+
// SQL-keyword statement types stay useful in messages/details
156+
// (matched_pattern is always redacted for multi_statement by design).
157+
expect(messages).toContain("DROP")
158+
expect(messages).not.toContain("redacted")
159+
})
160+
})

0 commit comments

Comments
 (0)