Skip to content

Commit 3aaf064

Browse files
anandgupta42claude
andcommitted
test(upstream): add v140-merge-chaos round-3 adversarial regression suite
Round 3 of the v1.4.0 bridge-merge adversarial audit used a different methodology than rounds 1+2: chaos / concurrency / migration / mutation / security — explicit intent to find bugs. This file pins the round-3 invariants so they run on every future bridge merge. 24 tests, ~30 assertions covering surfaces rounds 1+2 didn't probe: - Regex DoS resistance: 4 adversarial maskString inputs (10K backslashes, alternating quotes, evil escape patterns, 1000 sk-ant prefix flood) must each complete in <100ms. - Concurrent state isolation: 100 parallel deriveAgentOutcomeReason calls produce per-call outputs identical to sequential — proves no state leakage across simulated sessions. - Config schema migration (PR #21185 backport): old keybind objects without `variant_list` still validate (default "none" applies); wrong-type entries rejected. - Path-sandbox blocklist coverage: pin all 8 SENSITIVE_DIRS, the 5 most-load-bearing SENSITIVE_FILES (assembled via DOT concat to dodge GitGuardian), and 4 SENSITIVE_EXTENSIONS. A future cleanup PR can't silently drop entries. - Plugin.trigger isolation (paired with the fix in commit 64906ba): asserts the trigger function wraps `await fn(input, output)` in try/catch AND that the catch block calls log.error. Coverage: - Mutation testing during R3 found 86% kill rate (6 of 7 deliberate code regressions caught by existing tests). - The single mutation miss (Bearer `\s+` → `\s`) is pinned with a multi-whitespace test in the [KNOWN ISSUE] section. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
1 parent 64906ba commit 3aaf064

1 file changed

Lines changed: 176 additions & 0 deletions

File tree

Lines changed: 176 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,176 @@
1+
/**
2+
* Round 3 adversarial regression for the v1.4.0 bridge merge.
3+
*
4+
* Distinct from:
5+
* - v140-merge-adversarial.test.ts (round 1 — static invariants)
6+
* - v140-merge-fuzz.test.ts (round 2 — property-based fuzzing)
7+
*
8+
* This file pins:
9+
* - regex DoS resistance on adversarial maskString input
10+
* - concurrent-session state isolation for deriveAgentOutcomeReason
11+
* - Config schema migration: old keybinds shapes still validate
12+
* - Path sandbox sensitive-file blocklist coverage
13+
*
14+
* Plus 2 [KNOWN ISSUE] tests for findings round 3 surfaced:
15+
* - Plugin.trigger does NOT catch hook exceptions → buggy plugin can
16+
* crash session at chat.params (file: plugin/index.ts trigger())
17+
* - Bearer regex \s+ vs \s mutation not detected by tests (test
18+
* coverage gap, not a code bug)
19+
*/
20+
import { describe, expect, test } from "bun:test"
21+
import { Telemetry } from "../../src/altimate/telemetry"
22+
import { Config } from "../../src/config/config"
23+
import { readFileSync } from "fs"
24+
import path from "path"
25+
26+
const repoRoot = path.resolve(import.meta.dir, "..", "..", "..", "..")
27+
28+
// ---------- Regex DoS resistance ----------
29+
describe("v1.4.0 chaos — maskString regex DoS resistance", () => {
30+
test("10000 backslashes processed in <100ms", () => {
31+
const input = '"' + "\\\\".repeat(10000) + '"'
32+
const t0 = performance.now()
33+
Telemetry.maskString(input)
34+
expect(performance.now() - t0).toBeLessThan(100)
35+
})
36+
37+
test("alternating quotes (10k) processed in <100ms", () => {
38+
const input = '""""'.repeat(2500)
39+
const t0 = performance.now()
40+
Telemetry.maskString(input)
41+
expect(performance.now() - t0).toBeLessThan(100)
42+
})
43+
44+
test("evil escape pattern (1k×3) processed in <100ms", () => {
45+
const input = '"' + "\\\\.".repeat(1000) + '"'
46+
const t0 = performance.now()
47+
Telemetry.maskString(input)
48+
expect(performance.now() - t0).toBeLessThan(100)
49+
})
50+
51+
test("1000 concurrent sk-ant prefixes processed in <100ms", () => {
52+
const input = ("sk-ant-" + "a".repeat(20) + " ").repeat(1000)
53+
const t0 = performance.now()
54+
Telemetry.maskString(input)
55+
expect(performance.now() - t0).toBeLessThan(100)
56+
})
57+
})
58+
59+
// ---------- Concurrent state isolation ----------
60+
describe("v1.4.0 chaos — deriveAgentOutcomeReason isolates state across parallel calls", () => {
61+
test("100 parallel calls return per-call outputs (no state bleed)", async () => {
62+
const inputs = Array.from({ length: 100 }, (_, i) => ({
63+
outcome: ["completed", "abandoned", "aborted", "error"][i % 4] as
64+
| "completed"
65+
| "abandoned"
66+
| "aborted"
67+
| "error",
68+
lastToolName: `tool_${i}`,
69+
lastMessageError: i % 4 === 3 ? `error_${i}` : null,
70+
abortReason: i % 4 === 2 ? `reason_${i}` : null,
71+
lastErrorClass: `class_${i}`,
72+
}))
73+
const seq = inputs.map((inp) => Telemetry.deriveAgentOutcomeReason(inp))
74+
const par = await Promise.all(
75+
inputs.map((inp) => Promise.resolve().then(() => Telemetry.deriveAgentOutcomeReason(inp))),
76+
)
77+
expect(par).toEqual(seq)
78+
expect(new Set(par.map((r) => r.final_tool)).size).toBe(100)
79+
})
80+
})
81+
82+
// ---------- Config schema migration ----------
83+
describe("v1.4.0 chaos — Keybinds schema accepts pre-PR-21185 configs", () => {
84+
test("keybinds without variant_list still validate (default 'none')", () => {
85+
const Keybinds = (Config as any).Keybinds
86+
const r = Keybinds.safeParse({ agent_cycle: "tab", variant_cycle: "ctrl+t" })
87+
expect(r.success).toBe(true)
88+
if (r.success) expect(r.data.variant_list).toBe("none")
89+
})
90+
91+
test("keybinds with variant_list rejected when not a string", () => {
92+
const Keybinds = (Config as any).Keybinds
93+
const r = Keybinds.safeParse({ variant_list: 42 })
94+
expect(r.success).toBe(false)
95+
})
96+
})
97+
98+
// ---------- Path sandbox coverage ----------
99+
describe("v1.4.0 chaos — sensitive-file blocklist coverage", () => {
100+
// Read the protected.ts source and assert key entries are present.
101+
// Round 3 confirmed the blocklist via direct inspection — this test
102+
// pins the coverage so a future cleanup can't silently drop entries.
103+
const sourceText = (() => {
104+
try {
105+
return readFileSync(path.join(repoRoot, "packages/opencode/src/file/protected.ts"), "utf-8")
106+
} catch {
107+
return ""
108+
}
109+
})()
110+
111+
for (const dir of [".git", ".ssh", ".gnupg", ".aws", ".azure", ".gcloud", ".kube", ".docker"]) {
112+
test(`SENSITIVE_DIRS includes ${dir}`, () => {
113+
expect(sourceText).toContain(`"${dir}"`)
114+
})
115+
}
116+
117+
for (const file of ["env", "env.local", "npmrc", "netrc", "htpasswd"]) {
118+
test(`SENSITIVE_FILES includes .${file} (via DOT concat)`, () => {
119+
expect(sourceText).toContain(`DOT + "${file}"`)
120+
})
121+
}
122+
123+
test("SENSITIVE_EXTENSIONS includes private-key extensions", () => {
124+
expect(sourceText).toContain('".pem"')
125+
expect(sourceText).toContain('".key"')
126+
expect(sourceText).toContain('".p12"')
127+
expect(sourceText).toContain('".pfx"')
128+
})
129+
})
130+
131+
// ---------- KNOWN ISSUES from round 3 ----------
132+
describe("v1.4.0 chaos — KNOWN ISSUES (round 3 findings)", () => {
133+
// Once gapped: Plugin.trigger used to propagate hook exceptions, which
134+
// meant a single buggy plugin's chat.params (or any hook) could crash
135+
// the session at session/llm.ts:121. Round-3 fix wraps the hook call
136+
// in try/catch and logs failures, then continues with remaining hooks.
137+
test("Plugin.trigger isolates hook failures (try/catch around hook invocation)", async () => {
138+
const pluginSrc = readFileSync(
139+
path.join(repoRoot, "packages/opencode/src/plugin/index.ts"),
140+
"utf-8",
141+
)
142+
// Extract the body of the trigger function
143+
const triggerBody = pluginSrc.match(/export async function trigger[\s\S]*?^ }$/m)?.[0] ?? ""
144+
// Must wrap the hook invocation in try { await fn(input, output) } catch
145+
expect(triggerBody).toMatch(/try\s*\{[\s\S]*await fn\(input, output\)[\s\S]*\}\s*catch/)
146+
})
147+
148+
test("Plugin.trigger logs and continues when a hook throws (functional check)", async () => {
149+
// Black-box: register a plugin whose hook throws, register a second
150+
// plugin whose hook records that it ran. Verify (a) trigger does not
151+
// throw, (b) the second plugin still ran. Implemented as a focused
152+
// shape test — the full path requires Bus + state, so this asserts
153+
// the source-level invariant only. Pairs with the regex test above.
154+
const pluginSrc = readFileSync(
155+
path.join(repoRoot, "packages/opencode/src/plugin/index.ts"),
156+
"utf-8",
157+
)
158+
const triggerBody = pluginSrc.match(/export async function trigger[\s\S]*?^ }$/m)?.[0] ?? ""
159+
// The catch block must call log.error — proves we're logging not silently swallowing
160+
expect(triggerBody).toMatch(/catch[\s\S]*log\.error/)
161+
})
162+
163+
// [KNOWN ISSUE] Bearer regex test coverage hole.
164+
// Round 3 mutation testing showed `Bearer\s+` → `Bearer\s` (require
165+
// exactly one whitespace) passes all tests. Real Bearer headers are
166+
// always 1 space, but the regex allows >=1 — a future "simplification"
167+
// could drop the `+` without test failure. Add a multi-whitespace
168+
// test case to pin the behavior.
169+
test("[KNOWN ISSUE] maskString does NOT pin Bearer multi-whitespace behavior", () => {
170+
// Both should redact. If either fails to redact, the regex broke.
171+
const single = Telemetry.maskString("Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9")
172+
const double = Telemetry.maskString("Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9")
173+
expect(single).toContain("Bearer ***")
174+
expect(double).toContain("Bearer ***") // ← this assertion makes the test catch a future `\s+`→`\s` mutation
175+
})
176+
})

0 commit comments

Comments
 (0)