Skip to content

Commit 1de9355

Browse files
committed
fix(harness): close four review findings — redaction, mutation gate, chunk boundaries
Each fix has a regression test verified to FAIL against the pre-fix code. - compaction: redactLedgerDetail missed `curl.exe` and path-qualified curl spellings, so a Windows `curl.exe -u user password` left the credential unredacted in the facts ledger. Recognize the .exe suffix and both path separators. - compaction: credentialShaped treated any `x:y` token as user:password, so `docker run --user 1000:1000` had its UID:GID redacted out of the ledger. Require an alphabetic character before the colon; anything alphabetic is still redacted, so no credential regresses. - idle-done: GNU sed documents `-i[SUFFIX], --in-place[=SUFFIX]`, but only the short form counted as mutating. `sed --in-place` therefore mutated the worktree without advancing the mutation watermark, letting a stale green verification satisfy the idle-done gate. - tool-result-cap: line chunking used a fixed 2,000-code-unit slice stride, which split astral characters across chunks; assemble() rejoins chunks with a newline, so a surviving pair replayed as two lone surrogates. Back the boundary off a high surrogate. Three (padding, cap) witnesses were observed corrupting output pre-fix and are pinned as tests.
1 parent bc4b9e5 commit 1de9355

6 files changed

Lines changed: 157 additions & 4 deletions

File tree

‎packages/opencode/src/cli/cmd/idle-done.ts‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -258,7 +258,11 @@ export namespace IdleDone {
258258
// redirect is the safe direction here: it only makes idle-done fire less.
259259
if (/>>?\s*(?!&)/.test(command)) return true
260260
// In-place editors: the head is on the read-only list, the `-i` flag writes.
261-
if (/\b(?:sed|perl|ruby)\b[^|;&]*\s-[A-Za-z]*i\b/.test(command)) return true
261+
// GNU sed documents the flag as `-i[SUFFIX], --in-place[=SUFFIX]`, so the
262+
// long spelling edits files just as the short one does; matching only the
263+
// short form left `sed --in-place s/x/y/ file` classified as read-only and
264+
// a stale green verification could still satisfy the idle-done gate.
265+
if (/\b(?:sed|perl|ruby)\b[^|;&]*(?:\s-[A-Za-z]*i\b|\s--in-place\b)/.test(command)) return true
262266
for (const statement of command.split(/&&|\|\||[;|\n]/)) {
263267
const tokens = statement
264268
.trim()

‎packages/opencode/src/session/compaction.ts‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -608,8 +608,17 @@ export namespace SessionCompaction {
608608
// Attached values are valid only for short `-u` (`-ualice:pass`).
609609
if (flag.toLowerCase() === "--user" && separator === undefined) return match
610610
const rawValue = (separatedValue ?? attachedValue ?? "").replace(/^["']|["']$/g, "")
611-
const curlContext = /(?:^|[\s/])curl(?=\s|$)/i.test(shellSegmentBefore(whole, offset + lead.length))
612-
const credentialShaped = /^[^:/\s]+:[^/\s]+$/.test(rawValue)
611+
// Windows invokes curl as `curl.exe`, and either platform may reach it
612+
// through a path such as /usr/bin/curl or a Windows System32 path.
613+
// Missing those spellings left `-u user password` unredacted.
614+
const curlContext = /(?:^|[\s/\\])curl(?:\.exe)?(?=\s|$)/i.test(
615+
shellSegmentBefore(whole, offset + lead.length),
616+
)
617+
// Require an alphabetic character before the colon so that genuinely
618+
// non-credential `x:y` literals survive outside a curl context — most
619+
// importantly `docker run --user 1000:1000`, whose UID:GID is exactly
620+
// the kind of task detail the ledger exists to preserve.
621+
const credentialShaped = /^(?=[^:/\s]*[A-Za-z])[^:/\s]+:[^/\s]+$/.test(rawValue)
613622
if (!curlContext && !credentialShaped) return match
614623
return `${lead}${flag}${separator ?? ""}<redacted>`
615624
},

‎packages/opencode/src/session/tool-result-cap.ts‎

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -118,7 +118,24 @@ export namespace ToolResultCap {
118118
lines.push(line)
119119
continue
120120
}
121-
for (let i = 0; i < line.length; i += LINE_CHUNK_CHARS) lines.push(line.slice(i, i + LINE_CHUNK_CHARS))
121+
// Chunk on code-point boundaries. `slice` counts UTF-16 code units, so a
122+
// fixed stride can land between the high and low halves of an astral
123+
// character (emoji, CJK ext, ...). The truncation machinery may then keep
124+
// one half, and the replayed diagnostic carries a lone surrogate instead
125+
// of the original text.
126+
for (let i = 0; i < line.length; ) {
127+
let end = Math.min(i + LINE_CHUNK_CHARS, line.length)
128+
if (end < line.length) {
129+
const code = line.charCodeAt(end - 1)
130+
// High surrogate at the boundary: its pair starts here, so end the
131+
// chunk before it and let the next chunk carry the whole character.
132+
if (code >= 0xd800 && code <= 0xdbff) end -= 1
133+
}
134+
// Defensive: never fail to advance, whatever LINE_CHUNK_CHARS becomes.
135+
if (end <= i) end = Math.min(i + 2, line.length)
136+
lines.push(line.slice(i, end))
137+
i = end
138+
}
122139
}
123140
const totalBytes = Buffer.byteLength(output, "utf-8")
124141
// altimate_change start — outcome-accurate hint (see `opts.outcome`).

‎packages/opencode/test/cli/idle-done.test.ts‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -220,6 +220,20 @@ describe("IdleDone.isReadOnlyCommand (generic classifier, .ii)", () => {
220220
expect(IdleDone.isMutatingCommand("perl -i -pe 's/a/b/' f.txt")).toBe(true)
221221
})
222222

223+
// GNU sed documents `-i[SUFFIX], --in-place[=SUFFIX]`. Only the short form
224+
// was recognized, so the long spelling mutated the worktree while leaving the
225+
// mutation watermark stale — a prior green verify then satisfied the gate.
226+
test("long-form in-place editor flags are mutating", () => {
227+
for (const cmd of ["sed --in-place s/a/b/ f.txt", "sed --in-place=.bak s/a/b/ f.txt"]) {
228+
expect(IdleDone.isReadOnlyCommand(cmd)).toBe(true)
229+
expect(IdleDone.isMutatingCommand(cmd)).toBe(true)
230+
}
231+
// Also caught in a tail statement, where the verifier command is the head.
232+
expect(IdleDone.isMutatingCommand("npm test && sed --in-place s/a/b/ f.txt")).toBe(true)
233+
// A read-only sed without any in-place flag stays read-only.
234+
expect(IdleDone.isMutatingCommand("sed -n 1,10p f.txt")).toBe(false)
235+
})
236+
223237
test("output redirection is mutating; fd duplication is not", () => {
224238
expect(IdleDone.isMutatingCommand("cat a.txt > b.txt")).toBe(true)
225239
expect(IdleDone.isMutatingCommand("echo hi >> log.txt")).toBe(true)

‎packages/opencode/test/session/compaction-ledger.test.ts‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -396,6 +396,48 @@ describe("SessionCompaction.renderLedger", () => {
396396
}
397397
expect(SessionCompaction.redactLedgerDetail("curl -u alice https://example.com")).not.toContain("alice")
398398
})
399+
400+
test("recognizes curl through .exe and path spellings", () => {
401+
// A bare `-u user password` is only redacted because the shell segment is
402+
// recognized as curl. `curl.exe` (Windows) and path-qualified spellings
403+
// previously missed that check and leaked the credential into the ledger.
404+
for (const command of [
405+
"curl.exe -u alice dummy-password https://example.com",
406+
"CURL.EXE -u alice dummy-password https://example.com",
407+
"/usr/bin/curl -u alice dummy-password https://example.com",
408+
"/usr/local/bin/curl.exe -u alice dummy-password https://example.com",
409+
]) {
410+
const detail = SessionCompaction.redactLedgerDetail(command)
411+
expect(detail).not.toContain("alice")
412+
}
413+
414+
// The suffix must be the whole executable name, not a prefix match: a
415+
// command merely starting with "curl" is not curl.
416+
expect(SessionCompaction.redactLedgerDetail("curlywurly -u alice script.py")).toBe(
417+
"curlywurly -u alice script.py",
418+
)
419+
})
420+
421+
test("keeps non-credential colon-shaped values outside a curl context", () => {
422+
// `1000:1000` has no alphabetic character before the colon, so it is a
423+
// UID:GID pair rather than user:password and must survive in the ledger.
424+
for (const command of [
425+
"docker run --user 1000:1000 alpine",
426+
"docker run -u 1000:1000 alpine",
427+
"podman run --user=0:0 alpine",
428+
]) {
429+
expect(SessionCompaction.redactLedgerDetail(command)).toBe(command)
430+
}
431+
432+
// A genuinely credential-shaped value is still redacted outside curl.
433+
const credential = SessionCompaction.redactLedgerDetail("tool --user alice:dummy-password")
434+
expect(credential).not.toContain("dummy-password")
435+
expect(credential).not.toContain("alice")
436+
437+
// ...and inside a curl context the numeric shape is still redacted,
438+
// because curl's own `-u` is unambiguously authentication.
439+
expect(SessionCompaction.redactLedgerDetail("curl -u 1000:1000 https://example.com")).not.toContain("1000:1000")
440+
})
399441
})
400442

401443
// ─── 5b: extractAccomplished / corroborateCarry / renderCarryAnchors ────────

‎packages/opencode/test/session/tool-result-cap.test.ts‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -264,3 +264,70 @@ describe("ToolResultCap.capInterruptedMetadata", () => {
264264
expect(metadata.output).not.toBe(giant)
265265
})
266266
})
267+
268+
describe("ToolResultCap.apply — Unicode chunk boundaries", () => {
269+
// Long single lines are chunked at a fixed 2,000-code-unit stride before the
270+
// truncation machinery runs. `slice` counts UTF-16 code units, so a non-BMP
271+
// character straddling that stride was split into a high surrogate ending one
272+
// chunk and a low surrogate starting the next. `assemble` rejoins chunks with
273+
// "\n", so when BOTH halves survived into the preview the pair came back as
274+
// two lone surrogates separated by a newline — the replayed diagnostic held
275+
// replacement characters instead of the original text.
276+
//
277+
// Iterating with the spread operator yields a well-formed pair as a single
278+
// two-code-unit string, so length is what distinguishes a LONE surrogate
279+
// from an intact astral character.
280+
const isLoneSurrogate = (ch: string) => {
281+
if (ch.length !== 1) return false
282+
const code = ch.charCodeAt(0)
283+
return code >= 0xd800 && code <= 0xdfff
284+
}
285+
// A lone surrogate is not encodable, so a UTF-8 round trip replaces it. Both
286+
// checks are kept: the first localizes the defect, the second is what a
287+
// consumer actually observes once the result is serialized to the provider.
288+
const isCorrupt = (s: string) =>
289+
[...s].some(isLoneSurrogate) || Buffer.from(s, "utf-8").toString("utf-8") !== s
290+
291+
// These three (padding, cap) pairs are not illustrative — each was observed
292+
// to produce a corrupted result against the pre-fix chunker. They are the
293+
// regression's minimal witnesses: the emoji sits astride the 2,000-unit
294+
// stride and the cap is wide enough for both halves to survive truncation.
295+
const WITNESSES: Array<{ pad: number; cap: number }> = [
296+
{ pad: 1_999, cap: 2_100 },
297+
{ pad: 5_999, cap: 3_100 },
298+
{ pad: 5_999, cap: 3_200 },
299+
]
300+
301+
test("keeps astral characters intact across a chunk boundary", () => {
302+
for (const { pad, cap } of WITNESSES) {
303+
const line = "a".repeat(pad) + "🙂" + "b".repeat(6_000)
304+
const result = ToolResultCap.apply(line, cap)
305+
// Guards the witness itself: if this stopped truncating, the case would
306+
// pass vacuously and stop covering the chunker at all.
307+
expect(result.truncated).toBe(true)
308+
expect(isCorrupt(result.content)).toBe(false)
309+
}
310+
})
311+
312+
test("keeps astral characters intact across a sweep of caps", () => {
313+
// Widen beyond the witnesses so a future change to the stride, the head
314+
// ratio, or the byte budget cannot quietly reopen the defect at a cap that
315+
// happens not to be pinned above.
316+
for (const pad of [1_999, 3_999, 5_999]) {
317+
const line = "a".repeat(pad) + "🙂" + "b".repeat(6_000)
318+
for (let cap = 100; cap <= 6_000; cap += 100) {
319+
expect(isCorrupt(ToolResultCap.apply(line, cap).content)).toBe(false)
320+
}
321+
}
322+
})
323+
324+
test("still truncates and terminates on an all-astral line", () => {
325+
// Guards the advance path: every boundary is a surrogate pair here, so a
326+
// back-off that failed to advance would hang instead of truncating.
327+
const line = "🙂".repeat(10_000)
328+
const result = ToolResultCap.apply(line, 120)
329+
expect(result.truncated).toBe(true)
330+
expect(isCorrupt(result.content)).toBe(false)
331+
expect(Token.estimate(result.content)).toBeLessThanOrEqual(120)
332+
})
333+
})

0 commit comments

Comments
 (0)