Skip to content

Commit 54e93b7

Browse files
anandgupta42claude
andcommitted
fix(harness): sixth-pass — correct four defects the bots found in the fifth pass
The review bots re-reviewed the previous commit and caught real problems in it. Four were regressions I introduced; two were weaknesses in how I fixed things. Regressions in the fifth pass, now corrected: - The per-result dispatch cap was extended to FAILED tool results, but `ToolResultCap.apply` appends a hint that says *"The tool call succeeded"*. An oversized error was therefore persisted with success wording, so the model could read a real failure as a truncated success. `apply()` now takes an `outcome` and emits an error-specific hint that states the failure is real. - The nudge arbiter was changed to replace pending directives by SOURCE alone. That is wrong: four independent detectors register under `starvation_breaker` (`doom_loop_nudge`, `doom_loop_status_check`, `repeat_signature`, `starvation`) and they fire at different points in a step, so a later WEAKER detector could clobber a stronger directive that had already fired — the opposite of the intended fix. Replacement is back to source+kind, and `take()` now ranks kinds by explicit strength within the winning source. - The prompt-retry acceptance probe failed OPEN: if the probe itself threw, the message was treated as absent and the task was resent — exactly the duplicate execution the mechanism exists to prevent. Acceptance is now three-valued. Only a definitive 404 from a reachable server permits a retry; an unknown state fails the run with a clear message rather than risking a second execution or a silent hang. - Stripping `ALTIMATE_RUN_MODE` from bash child environments also removed an explicit opt-out (`=0`/`false`), so a nested `run` would re-apply the default and turn run mode back ON. Only an ACTIVE marker is stripped now. Weaknesses in the fifth pass, now addressed: - `ToolResultCap.resolve` resolved the safety fraction AFTER the unknown-model branch returned, so a configured fraction never scaled that fallback. - The run-mode child-env test asserted on `bash.ts` source text, which cannot catch the strip being relocated, applied to a different object, or skipped — the same false-confidence problem flagged elsewhere in this review. The logic is now the exported `stripRunModeMarkers()` and the test exercises it. Tests: strongest-wins and weaker-does-not-displace ordering for every `starvation_breaker` kind; error vs success truncation wording (and that the error hint still respects the cap); configured fraction scaling the unknown-model fallback; and behavioural coverage of the marker strip including opt-out survival. Gates: typecheck clean, marker check clean, 2411 pass / 0 fail. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01VqnuBDGkh1ZT65Ti7e6DHZ
1 parent a6b6c6d commit 54e93b7

8 files changed

Lines changed: 241 additions & 59 deletions

File tree

‎packages/opencode/src/cli/cmd/run.ts‎

Lines changed: 34 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1094,12 +1094,26 @@ You are speaking to a non-technical business executive. Follow these rules stric
10941094
...(audienceSystem ? { system: audienceSystem } : {}),
10951095
})
10961096
}
1097-
/** True when the server already persisted this attempt's user message. */
1098-
const alreadyAccepted = async () => {
1099-
const res = await sdk.session
1100-
.message({ sessionID, messageID: sendMessageID })
1101-
.catch(() => undefined)
1102-
return Boolean((res as { data?: { info?: unknown } } | undefined)?.data?.info)
1097+
/** Did the server persist this attempt's user message?
1098+
* Three-valued ON PURPOSE — a retry may only proceed on definitive
1099+
* evidence that the message did NOT land. Treating an unreachable
1100+
* server as "absent" would resend a task that may already be running,
1101+
* which is the duplication this whole mechanism exists to prevent. */
1102+
const acceptanceState = async (): Promise<"accepted" | "absent" | "unknown"> => {
1103+
try {
1104+
const res = (await sdk.session.message({ sessionID, messageID: sendMessageID })) as {
1105+
data?: { info?: unknown }
1106+
error?: unknown
1107+
response?: { status?: number }
1108+
}
1109+
if (res?.data?.info) return "accepted"
1110+
// A definitive 404 from a reachable server is the only proof of absence.
1111+
if (res?.response?.status === 404) return "absent"
1112+
return "unknown"
1113+
} catch {
1114+
// The probe itself failed — the server is unreachable, so we cannot tell.
1115+
return "unknown"
1116+
}
11031117
}
11041118
// altimate_change end
11051119
type SendResult = {
@@ -1122,10 +1136,14 @@ You are speaking to a non-technical business executive. Follow these rules stric
11221136
if (!RunAccounting.isRetryableThrown(e)) throw e
11231137
reason = e instanceof Error ? e.message : String(e)
11241138
}
1125-
// altimate_change start — never re-send a prompt the server already
1126-
// accepted: that duplicates the task. The failure was on the response
1127-
// path, so fall through and let the event loop drain to idle instead.
1128-
if (await alreadyAccepted()) {
1139+
// altimate_change start — a retry may only proceed on definitive
1140+
// evidence that the message did NOT land. Re-sending an accepted prompt
1141+
// duplicates the task; re-sending on an UNKNOWN state risks the same,
1142+
// so that case fails the run loudly instead of guessing.
1143+
const acceptance = await acceptanceState()
1144+
if (acceptance === "accepted") {
1145+
// The failure was on the response path only — the run is in flight,
1146+
// so fall through and let the event loop drain to idle.
11291147
if (!emit("retry_skipped", { reason, messageID: sendMessageID })) {
11301148
UI.println(
11311149
UI.Style.TEXT_WARNING_BOLD + "!",
@@ -1134,6 +1152,12 @@ You are speaking to a non-technical business executive. Follow these rules stric
11341152
}
11351153
break
11361154
}
1155+
if (acceptance === "unknown") {
1156+
throw new Error(
1157+
`prompt failed and the server could not be reached to determine whether it was accepted; ` +
1158+
`not retrying to avoid running the task twice — ${reason}`,
1159+
)
1160+
}
11371161
// altimate_change end
11381162
if (sendAttempt >= retryMax) throw new Error(`prompt failed after ${retryMax} retries: ${reason}`)
11391163
const delay = RunAccounting.retryDelayMs(retryBaseMs, sendAttempt)

‎packages/opencode/src/session/nudge.ts‎

Lines changed: 31 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -49,16 +49,33 @@ export namespace NudgeArbiter {
4949
return b
5050
}
5151

52+
// altimate_change start — STRENGTH ordering within a source. Several
53+
// independent detectors register under `starvation_breaker`
54+
// (`doom_loop_nudge`, `doom_loop_status_check`, `repeat_signature`,
55+
// `starvation`), so neither "earliest wins" nor "latest wins" is correct:
56+
// the first delivered a stale nudge when the same generation had already
57+
// escalated to a status check, and the second let a later, weaker detector
58+
// clobber a stronger directive that fired earlier in the same step.
59+
// Rank the kinds explicitly instead — highest rank wins, and equal ranks
60+
// fall back to the latest registration (a re-fire of the same rung is
61+
// current information).
62+
const KIND_STRENGTH: Record<string, number> = {
63+
doom_loop_status_check: 3,
64+
repeat_signature: 2,
65+
starvation: 1,
66+
doom_loop_nudge: 1,
67+
}
68+
69+
function strength(kind: string): number {
70+
return KIND_STRENGTH[kind] ?? 0
71+
}
72+
5273
/** Register a candidate directive for the session's next injected turn.
53-
* altimate_change start — replace by SOURCE, not source+kind. Only one
54-
* directive per source is ever delivered, and `take()` picked the EARLIEST
55-
* match, so a single generation that crossed two rungs of the doom-loop
56-
* ladder (nudge, then the stronger status_check) delivered the stale nudge
57-
* and dropped the escalation with the rest of the bucket. The latest
58-
* registration from a source is the current one, so it wins. */
74+
* Registrations from the same source+kind replace; different kinds from one
75+
* source coexist and are ranked by strength at `take()` time. */
5976
export function register(sessionID: string, directive: Directive): void {
6077
const b = bucket(sessionID)
61-
const existing = b.findIndex((d) => d.source === directive.source)
78+
const existing = b.findIndex((d) => d.source === directive.source && d.kind === directive.kind)
6279
if (existing >= 0) b[existing] = directive
6380
else b.push(directive)
6481
}
@@ -76,7 +93,13 @@ export namespace NudgeArbiter {
7693
if (!b || b.length === 0) return undefined
7794
let winner: Directive | undefined
7895
for (const source of PRECEDENCE) {
79-
winner = b.find((d) => d.source === source)
96+
// altimate_change start — strongest directive within the winning source,
97+
// not merely the first registered one.
98+
for (const d of b) {
99+
if (d.source !== source) continue
100+
if (!winner || strength(d.kind) >= strength(winner.kind)) winner = d
101+
}
102+
// altimate_change end
80103
if (winner) break
81104
}
82105
pendingBySession.delete(sessionID)

‎packages/opencode/src/session/processor.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -633,7 +633,7 @@ export namespace SessionProcessor {
633633
const toolErrorText = (() => {
634634
const raw = (value.error as any).toString()
635635
if (typeof raw !== "string") return raw
636-
const capped = ToolResultCap.apply(raw, toolResultCapTokens)
636+
const capped = ToolResultCap.apply(raw, toolResultCapTokens, { outcome: "error" })
637637
if (capped.truncated)
638638
log.info("tool error capped at dispatch", {
639639
tool: match.tool,

‎packages/opencode/src/session/tool-result-cap.ts‎

Lines changed: 28 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -33,8 +33,11 @@ export namespace ToolResultCap {
3333
// the model had the smallest window this cap protects (64K, scaled by the
3434
// default safety fraction) rather than trusting the byte-derived cap
3535
// (~17K tokens), which can overwhelm a small window on its own.
36+
/** The smallest window this cap protects; the unknown-model fallback is sized against it. */
37+
export const UNKNOWN_MODEL_CONTEXT = 65_536
38+
3639
export const UNKNOWN_MODEL_CAP_TOKENS = Math.floor(
37-
Math.floor(65_536 * DEFAULT_SAFETY_FRACTION) * DEFAULT_LIMIT_FRACTION,
40+
Math.floor(UNKNOWN_MODEL_CONTEXT * DEFAULT_SAFETY_FRACTION) * DEFAULT_LIMIT_FRACTION,
3841
)
3942

4043
/**
@@ -58,25 +61,31 @@ export namespace ToolResultCap {
5861
const maxBytes = input.config?.tool_output?.max_bytes ?? TruncateCore.MAX_BYTES
5962
const existingCapTokens = Math.ceil(maxBytes / MIN_CHARS_PER_TOKEN)
6063

61-
const base = input.model?.limit?.input ?? input.model?.limit?.context ?? 0
62-
if (base <= 0) return Math.min(existingCapTokens, UNKNOWN_MODEL_CAP_TOKENS)
63-
6464
// Default to the estimator safety fraction, not 1: an omitted fraction must
6565
// fail conservative (tool outputs are estimate-domain), never fail open.
6666
// altimate_change start — `config.compaction.context_safety_fraction` was
6767
// declared on this input and never read, so a caller that passed only the
6868
// config (every caller except processor.ts) silently got the default
6969
// instead of the configured fraction. Honour it as the second choice.
70+
// Resolved BEFORE the unknown-model branch so the conservative fallback is
71+
// scaled by the configured fraction too, not only the known-limit path.
7072
const configuredFraction = input.config?.compaction?.context_safety_fraction
7173
const fraction =
7274
input.safetyFraction ??
7375
(typeof configuredFraction === "number" && Number.isFinite(configuredFraction) && configuredFraction > 0
7476
? configuredFraction
7577
: DEFAULT_SAFETY_FRACTION)
78+
// Same shape as UNKNOWN_MODEL_CAP_TOKENS, but at the resolved fraction; with
79+
// the default fraction the two are identical.
80+
const unknownCapTokens = Math.floor(Math.floor(UNKNOWN_MODEL_CONTEXT * fraction) * DEFAULT_LIMIT_FRACTION)
7681
// altimate_change end
82+
83+
const base = input.model?.limit?.input ?? input.model?.limit?.context ?? 0
84+
if (base <= 0) return Math.min(existingCapTokens, unknownCapTokens)
85+
7786
const effectiveLimit = Math.floor(base * fraction)
7887
const limitCapTokens = Math.floor(effectiveLimit * DEFAULT_LIMIT_FRACTION)
79-
if (limitCapTokens <= 0) return Math.min(existingCapTokens, UNKNOWN_MODEL_CAP_TOKENS)
88+
if (limitCapTokens <= 0) return Math.min(existingCapTokens, unknownCapTokens)
8089
return Math.min(existingCapTokens, limitCapTokens)
8190
}
8291

@@ -86,7 +95,15 @@ export namespace ToolResultCap {
8695
* marker as the tool-level truncation service) with a notice telling the model
8796
* the output was truncated.
8897
*/
89-
export function apply(output: string, capTokens: number): { content: string; truncated: boolean } {
98+
export function apply(
99+
output: string,
100+
capTokens: number,
101+
// altimate_change start — the hint must match the OUTCOME. The cap is now
102+
// applied to failed tool results too, and the success wording would have
103+
// told the model a real failure was a truncated success.
104+
opts?: { outcome?: "success" | "error" },
105+
// altimate_change end
106+
): { content: string; truncated: boolean } {
90107
if (capTokens <= 0) return { content: output, truncated: false }
91108
if (Token.estimate(output) <= capTokens) return { content: output, truncated: false }
92109

@@ -99,8 +116,12 @@ export namespace ToolResultCap {
99116
for (let i = 0; i < line.length; i += LINE_CHUNK_CHARS) lines.push(line.slice(i, i + LINE_CHUNK_CHARS))
100117
}
101118
const totalBytes = Buffer.byteLength(output, "utf-8")
119+
// altimate_change start — outcome-accurate hint (see `opts.outcome`).
102120
const hint =
103-
"The tool call succeeded but the output exceeded the per-result context budget and was truncated before dispatch. Re-run the tool with a narrower query (filters, LIMIT, offset/limit) to view specific sections."
121+
opts?.outcome === "error"
122+
? "The tool call FAILED and its error output exceeded the per-result context budget, so the error text below was truncated before dispatch. The failure is real — do not treat this as a successful result. Re-run with a narrower scope if you need the full error."
123+
: "The tool call succeeded but the output exceeded the per-result context budget and was truncated before dispatch. Re-run the tool with a narrower query (filters, LIMIT, offset/limit) to view specific sections."
124+
// altimate_change end
104125
const frame = (bodyBytes: number) => {
105126
const preview = TruncateCore.preview(lines, totalBytes, {
106127
maxLines: Number.MAX_SAFE_INTEGER,

‎packages/opencode/src/tool/bash.ts‎

Lines changed: 25 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,30 @@ import { Truncate } from "./truncation"
1919
import { Plugin } from "@/plugin"
2020
import { Global } from "@/global"
2121

22+
// altimate_change start — run-mode markers must not reach bash child processes.
23+
// `run` sets ALTIMATE_RUN_MODE on its own process to arm run-mode-only
24+
// mechanisms (DONE-termination gate, starvation directives, doom-loop
25+
// escalation ladder). A nested `serve`/TUI launched through the bash tool
26+
// inherited it and armed those mechanisms in an interactive session,
27+
// contradicting the invariant documented in session/processor.ts. A nested
28+
// `run` re-applies the default itself (cli/cmd/run/run-mode.ts), so nothing
29+
// that should be in run mode loses it.
30+
//
31+
// Only an ACTIVE marker is stripped: an explicit opt-out
32+
// (ALTIMATE_RUN_MODE=0/false) must SURVIVE into the child, because deleting it
33+
// would let a nested `run` re-apply the default and turn run mode back on —
34+
// the opposite of what the operator asked for.
35+
//
36+
// Exported so the contract is tested behaviourally rather than by reading this
37+
// file's source text.
38+
export function stripRunModeMarkers(env: Record<string, string | undefined>) {
39+
const active = env["ALTIMATE_RUN_MODE"]?.trim().toLowerCase()
40+
if (active === "1" || active === "true") delete env["ALTIMATE_RUN_MODE"]
41+
delete env["ALTIMATE_RUN_RESUMED"]
42+
return env
43+
}
44+
// altimate_change end
45+
2246
const MAX_METADATA_LENGTH = 30_000
2347
const DEFAULT_TIMEOUT = Flag.OPENCODE_EXPERIMENTAL_BASH_DEFAULT_TIMEOUT_MS || 2 * 60 * 1000
2448

@@ -178,15 +202,7 @@ export const BashTool = Tool.define("bash", async () => {
178202
delete mergedEnv["ALTIMATE_NON_INTERACTIVE"]
179203
// altimate_change end
180204
// altimate_change start — strip the run-mode markers for the same reason.
181-
// `run` sets ALTIMATE_RUN_MODE on its own process to arm run-mode-only
182-
// mechanisms (DONE-termination gate, starvation directives, doom-loop
183-
// escalation). A nested `serve`/TUI launched through this tool inherited
184-
// it and armed those mechanisms in an interactive session, contradicting
185-
// the invariant that they never apply outside run mode. A nested `run`
186-
// re-applies the default itself (cli/cmd/run/run-mode.ts), so nothing
187-
// that should be in run mode loses it.
188-
delete mergedEnv["ALTIMATE_RUN_MODE"]
189-
delete mergedEnv["ALTIMATE_RUN_RESUMED"]
205+
stripRunModeMarkers(mergedEnv)
190206
// altimate_change end
191207
const sep = process.platform === "win32" ? ";" : ":"
192208
const basePath = mergedEnv.PATH ?? mergedEnv.Path ?? ""

‎packages/opencode/test/cli/run/run-mode.test.ts‎

Lines changed: 31 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
import { afterEach, beforeEach, describe, expect, test } from "bun:test"
22
import { applyRunModeDefault } from "@/cli/cmd/run/run-mode"
33
import { Flag } from "@/flag/flag"
4+
// altimate_change — behavioural coverage of the child-env marker strip
5+
import { stripRunModeMarkers } from "@/tool/bash"
46

57
// ─── `altimate-code run` implies run mode ───────────────────────
68
// External drivers (harbor, CI) invoke `run` without exporting
@@ -150,21 +152,37 @@ describe("Flag.parseRunModeValue (strict trimmed boolean parser)", () => {
150152
// ALTIMATE_NON_INTERACTIVE. A nested `serve`/TUI therefore inherited run mode
151153
// and armed run-mode-only mechanisms in an interactive session.
152154
describe("run-mode markers do not leak into bash child processes", () => {
153-
test("bash tool strips ALTIMATE_RUN_MODE and ALTIMATE_RUN_RESUMED from child env", async () => {
154-
const source = await Bun.file(new URL("../../../src/tool/bash.ts", import.meta.url)).text()
155-
expect(source).toContain('delete mergedEnv["ALTIMATE_RUN_MODE"]')
156-
expect(source).toContain('delete mergedEnv["ALTIMATE_RUN_RESUMED"]')
157-
// the pre-existing sibling strip must remain
158-
expect(source).toContain('delete mergedEnv["ALTIMATE_NON_INTERACTIVE"]')
159-
expect(source).toContain("env: mergedEnv")
160-
})
161-
162-
test("a nested run re-arms run mode for itself, so stripping loses nothing", () => {
163-
// applyRunModeDefault is what `run` calls at handler startup; a child that
164-
// should be in run mode sets it again from an empty environment.
165-
const childEnv: Record<string, string | undefined> = {}
155+
test("an active marker is stripped from the child environment", () => {
156+
for (const value of ["1", "true", " 1 ", "TRUE"]) {
157+
const env = stripRunModeMarkers({ ALTIMATE_RUN_MODE: value, ALTIMATE_RUN_RESUMED: "1", PATH: "/bin" })
158+
expect(env["ALTIMATE_RUN_MODE"]).toBeUndefined()
159+
expect(env["ALTIMATE_RUN_RESUMED"]).toBeUndefined()
160+
// unrelated variables are untouched
161+
expect(env["PATH"]).toBe("/bin")
162+
}
163+
})
164+
165+
test("an explicit opt-out SURVIVES — deleting it would let a nested run re-arm run mode", () => {
166+
for (const value of ["0", "false"]) {
167+
const env = stripRunModeMarkers({ ALTIMATE_RUN_MODE: value })
168+
expect(env["ALTIMATE_RUN_MODE"]).toBe(value)
169+
// and a nested run must therefore stay opted out
170+
applyRunModeDefault(env)
171+
expect(Flag.parseRunModeValue(env["ALTIMATE_RUN_MODE"]!)).toBe(false)
172+
}
173+
})
174+
175+
test("a nested run re-arms run mode for itself, so stripping an active marker loses nothing", () => {
176+
const childEnv = stripRunModeMarkers({ ALTIMATE_RUN_MODE: "1" })
177+
expect(childEnv["ALTIMATE_RUN_MODE"]).toBeUndefined()
178+
// applyRunModeDefault is what `run` calls at handler startup
166179
applyRunModeDefault(childEnv)
167180
expect(childEnv["ALTIMATE_RUN_MODE"]).toBe("1")
168181
})
182+
183+
test("an absent marker stays absent", () => {
184+
const env = stripRunModeMarkers({})
185+
expect("ALTIMATE_RUN_MODE" in env).toBe(false)
186+
})
169187
})
170188
// altimate_change end

0 commit comments

Comments
 (0)