Skip to content

Commit fb5b107

Browse files
committed
fix(shell): throttle streaming output metadata updates to cut CPU
The shell tool called ctx.metadata({ output: last }) on every stdout chunk. Each call flows through updateToolCall -> session.updatePart -> a durable message.part.updated event, which commits a synchronous SQLite IMMEDIATE transaction (schema encode + full-snapshot row in the event table + projector write) plus GlobalBus emits and per-SSE-client serialization. A long-running command emitting output over time therefore fired one synchronous SQLite transaction per chunk on the server event loop, pinning CPU at 100% and bloating the event table. Coalesce the intermediate preview updates to at most one every METADATA_THROTTLE_MS (100ms) while streaming, and flush the final preview once when the command completes. Correctness is preserved: the rolling `last` preview is still updated on every chunk, the tool's return value still carries the complete final metadata (persisted durably by completeToolCall), and the final flush publishes the complete preview promptly. Only the frequency of intermediate durable writes changes; what the user ultimately sees does not. Add a test that paces stdout across many chunks (output spread over time, since instantaneous output is coalesced by the OS pipe) and asserts the number of metadata updates is far below the number of chunks while the final update still carries the complete output. Fixes anomalyco#31831 Related: anomalyco#33399, anomalyco#33356
1 parent 4438f69 commit fb5b107

2 files changed

Lines changed: 100 additions & 26 deletions

File tree

‎packages/opencode/src/tool/shell.ts‎

Lines changed: 47 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,11 @@ import { BashArity } from "@/permission/arity"
2525
export { Parameters } from "./shell/prompt"
2626

2727
const MAX_METADATA_LENGTH = 30_000
28+
// Throttle interval for streaming output preview updates. Each update is a durable
29+
// `message.part.updated` event (synchronous SQLite IMMEDIATE transaction + full-snapshot
30+
// row), so emitting one per stdout chunk pins the server event loop at 100% CPU for
31+
// chatty commands. Coalesce intermediate previews; the final state is always flushed.
32+
const METADATA_THROTTLE_MS = 100
2833
const CWD = new Set(["cd", "chdir", "popd", "pushd", "push-location", "set-location"])
2934
const FILES = new Set([
3035
...CWD,
@@ -446,6 +451,7 @@ export const ShellTool = Tool.define(
446451
let cut = false
447452
let expired = false
448453
let aborted = false
454+
let dirty = false
449455

450456
const closeSink = Effect.fnUntraced(function* () {
451457
const stream = sink
@@ -499,34 +505,40 @@ export const ShellTool = Tool.define(
499505

500506
if (file) {
501507
sink?.write(chunk)
502-
} else {
503-
full += chunk
504-
if (Buffer.byteLength(full, "utf-8") > limits.maxBytes) {
505-
return trunc.write(full).pipe(
506-
Effect.andThen((next) =>
507-
Effect.sync(() => {
508-
file = next
509-
cut = true
510-
sink = createWriteStream(next, { flags: "a" })
511-
full = ""
512-
}),
513-
),
514-
Effect.andThen(
515-
ctx.metadata({
516-
metadata: {
517-
output: last,
518-
},
519-
}),
520-
),
521-
)
522-
}
508+
dirty = true
509+
return Effect.void
510+
}
511+
full += chunk
512+
if (Buffer.byteLength(full, "utf-8") > limits.maxBytes) {
513+
return trunc.write(full).pipe(
514+
Effect.andThen((next) =>
515+
Effect.sync(() => {
516+
file = next
517+
cut = true
518+
sink = createWriteStream(next, { flags: "a" })
519+
full = ""
520+
dirty = true
521+
}),
522+
),
523+
)
523524
}
525+
// Mark output dirty; the forked flusher below publishes it at most once per
526+
// METADATA_THROTTLE_MS. Unlike a leading-edge throttle, this guarantees the latest
527+
// output is published within one window even if the command stops emitting chunks
528+
// (e.g. prints then blocks/sleeps), so live output never goes stale.
529+
dirty = true
530+
return Effect.void
531+
}),
532+
)
524533

525-
return ctx.metadata({
526-
metadata: {
527-
output: last,
528-
},
529-
})
534+
yield* Effect.forkScoped(
535+
Effect.gen(function* () {
536+
while (true) {
537+
yield* Effect.sleep(`${METADATA_THROTTLE_MS} millis`)
538+
if (!dirty) continue
539+
dirty = false
540+
yield* ctx.metadata({ metadata: { output: last } })
541+
}
530542
}),
531543
)
532544

@@ -558,6 +570,15 @@ export const ShellTool = Tool.define(
558570
}),
559571
).pipe(Effect.orDie)
560572

573+
// Flush the final streamed preview now: the throttle above may have skipped the
574+
// last chunk's update, so publish the complete preview promptly instead of waiting
575+
// for completeToolCall to persist the final metadata at tool completion.
576+
yield* ctx.metadata({
577+
metadata: {
578+
output: last,
579+
},
580+
})
581+
561582
const meta: string[] = []
562583
if (expired) {
563584
meta.push(

‎packages/opencode/test/tool/shell.test.ts‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1131,6 +1131,59 @@ describe("tool.shell abort", () => {
11311131
)
11321132
})
11331133

1134+
// Emit `n` lines paced `intervalMs` apart so the output arrives as many temporally
1135+
// separate stdout chunks. Instantaneous output would be coalesced by the OS pipe into a
1136+
// couple of reads regardless of throttling; spacing it out reproduces the real
1137+
// long-running-command scenario where each chunk previously triggered its own durable
1138+
// `message.part.updated` event.
1139+
const paced = (n: number, intervalMs: number) => {
1140+
// No quote characters in the code: evalarg wraps it in shell quotes and does not escape
1141+
// nested ones (see fill). Emit the bare counter per line so the final line is `n`.
1142+
const code =
1143+
"let i=0;const t=setInterval(()=>{if(i>=Number(Bun.argv[1])){clearInterval(t);return}i++;process.stdout.write(i+String.fromCharCode(10))},Number(Bun.argv[2]))"
1144+
const text = `${bin} -e ${evalarg(code)} ${n} ${intervalMs}`
1145+
if (PS.has(sh())) return `& ${text}`
1146+
return text
1147+
}
1148+
1149+
describe("tool.shell streaming throttle", () => {
1150+
it.live(
1151+
"coalesces streaming output into far fewer metadata updates than chunks",
1152+
() =>
1153+
runIn(
1154+
projectRoot,
1155+
Effect.gen(function* () {
1156+
// 400 lines, 5ms apart => ~2s of output across many chunks (~20 throttle
1157+
// windows). Without throttling this would emit hundreds of updates; the
1158+
// throttle bounds it to roughly one per 100ms.
1159+
const lineCount = 400
1160+
const updates: string[] = []
1161+
const result = yield* run(
1162+
{
1163+
command: paced(lineCount, 5),
1164+
},
1165+
{
1166+
...ctx,
1167+
metadata: (input) =>
1168+
Effect.sync(() => {
1169+
const output = (input.metadata as { output?: string })?.output
1170+
if (output) updates.push(output)
1171+
}),
1172+
},
1173+
)
1174+
// Coalescing happened: far fewer updates than emitted lines/chunks.
1175+
expect(updates.length).toBeGreaterThan(1)
1176+
expect(updates.length).toBeLessThan(100)
1177+
// Final state is not lost: the last update and the result carry the last line.
1178+
expect(updates.at(-1)).toContain(String(lineCount))
1179+
expect(result.output.trim().endsWith(String(lineCount))).toBe(true)
1180+
expect(result.metadata.exit).toBe(0)
1181+
}),
1182+
),
1183+
20_000,
1184+
)
1185+
})
1186+
11341187
describe("tool.shell truncation", () => {
11351188
it.live("truncates output exceeding line limit", () =>
11361189
runIn(

0 commit comments

Comments
 (0)