Skip to content

Commit cab10da

Browse files
committed
feat(v1.17.7): patched.6 — default-off per-boot project-copy refresh (wvv2) + bound step-end summary diff (0lik, upstream #29762)
Two patches from the headless-serve wedge investigation: - project-copy-debounce.patch (#12, EXTENDED — bead workstation-wvv2): gate refreshAfterBoot behind OPENCODE_PROJECT_COPY_REFRESH_ON_BOOT. Default (unset / anything but "1") DISABLES the per-boot O(#dirs) scan (~136 dirs / ~300 git subprocesses pegging a core on every cold boot). Safe headless: source dirs self-register via Project.saveProjectDirectory on open, the Move Session dialog refreshes on demand, project identity is git-derived (not directory-index-derived). The explicit refresh() API (HTTP endpoint + Move Session dialog) is untouched — gate is only at the boot entry. Set OPENCODE_PROJECT_COPY_REFRESH_ON_BOOT=1 to restore the upstream every-boot behavior. +2 it.live tests. - step-end-diff-bound.patch (#13, NEW — bead workstation-0lik, upstream anomalyco/opencode#29762): bound Snapshot.diffFull's jsdiff structuredPatch (was context:MAX_SAFE_INTEGER with no edit/time limit), which pinned one JS thread at 100% CPU (+~1GB alloc) on a full-file rewrite of a sub-2MB file at step-end summary time. New boundedFilePatch() composes a 1 MiB size pre-guard with jsdiff {maxEditLength:4000, timeout:1000ms}; on trip it omits the optional FileDiff.patch field (counts-only — consumers already guard typeof patch!=="string"). Only touches snapshot/index.ts + new test. Does NOT address the distinct, uncharacterized flat-RSS read-only-subagent spin variant (#32965). +3 unit tests (R0 micro-repro). apply.sh: registers step-end-diff-bound as #13 (PATCHES array + doc block); project-copy-debounce gets a #12 sub-note documenting the wvv2 extension. Verified on a fresh v1.17.7 worktree: apply.sh clean (all 13 patches), project-copy.test.ts 18/0, bounded-diff.test.ts 3/0 (+1 known node:sqlite dev-harness afterAll fail, harmless — CI/deployed Bun has node:sqlite), tsgo --noEmit clean (packages/core 0 errors; packages/opencode only the pre-existing session.ts(944,7) TS2719 from createnext-readback). Also lands the swarm design docs: projectcopy-refresh-cost, step-end-spin, and the event fan-out verdict (qjk4 — parked, the K-serve pool already structurally defuses the storm).
1 parent 229d76e commit cab10da

6 files changed

Lines changed: 1156 additions & 7 deletions
Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
# GET /event O(N×M) fan-out: verdict + minimal-fix design
2+
3+
**Date:** 2026-06-23
4+
**Author:** swarm worker 2 (qjk4 territory)
5+
**Round constraint:** DESIGN/VERIFY ONLY — no implementation, no patch edits.
6+
**Scope:** the `/event` fan-out half of `workstation-qjk4` (M3.1 filter-before-queue,
7+
M3.3 setMaxListeners). The ProjectCopy half (M3.2) is owned by worker 1.
8+
9+
---
10+
11+
## Verdict: RESIDUAL GAP (structurally defused, not eliminated)
12+
13+
The catastrophic single-core O(N×M) event storm from
14+
`OPENCODE-SERVE-MULTICORE-INVESTIGATION.md §1.A` is **structurally defused** by the
15+
K-serve pool, but the two handler-level fixes qjk4 scoped (M3.1 filter-before-queue,
16+
M3.3 setMaxListeners) were **NOT implemented**. They remain real (if smaller and
17+
currently-dormant) gaps.
18+
19+
### Evidence
20+
21+
**The deployed `/event` handler still offers every event to every client's queue.**
22+
After `event-session-scope.patch` (#7) and `event-cold-start-directory.patch` (#11),
23+
the listener line is byte-for-byte the base v1.17.7 line:
24+
25+
```ts
26+
const unsubscribe = yield* events.listen((event) => Effect.sync(() => Queue.offerUnsafe(queue, event)))
27+
```
28+
29+
Both the directory filter AND the `?session_ids=` filter run **post-queue**, inside
30+
`Stream.fromQueue(queue).pipe(Stream.filter(...))`. The cold-start patch merely *merged*
31+
the directory filter into the same post-queue `Stream.filter` and added session-aggregate
32+
gating — it did not move anything ahead of the queue offer. So per published event, every
33+
connected client still:
34+
1. fires its `events.listen` callback (EventV2 keeps listeners in a plain array, iterated
35+
for all N on every publish — `core/src/event.ts:421`), and
36+
2. allocates a `Queue.offerUnsafe`,
37+
…and only *then* is the event filtered before SSE serialization. The doc's §5 Step 1
38+
("filter **before** offering to the queue", reduce to O(M_local)) was **not done**.
39+
40+
**`?session_ids=` is DEPLOYED-BUT-UNUSED by real clients.** The TUI SSE path
41+
(`tui/src/context/sdk.tsx`, via `attach-route-resolve.patch`) and the bash clients
42+
(`oc-auto-attach`, `opencode-launch`) all use pigeon `GET /route` only to extract
43+
`.apiBase` and *attach to the owning serve*; the actual subscription is a bare
44+
`sdk.global.event({ signal, sseMaxRetryAttempts: 0 })` with **no** `session_ids` param.
45+
The only `?session_ids=` occurrences in 1.1M log lines are manual `probe`/grep commands,
46+
never TUI traffic. Session isolation is therefore achieved **structurally** (each serve is
47+
a separate process whose in-memory GlobalBus only carries its own sessions' events), not by
48+
the filter. The filter is dead weight on the hot path today — and being post-queue, it
49+
wouldn't kill the fan-out even if clients did use it.
50+
51+
**`setMaxListeners` is never raised.** Zero `setMaxListeners` references in base v1.17.7 or
52+
in any patch; `bus/global.ts` is unmodified (default limit = 10). Each `/event` client
53+
registers one `GlobalBus.on("event")` listener for `server.instance.disposed` detection
54+
(`event.ts` `disposed` sub-stream), so a single serve with ≥11 concurrent clients emits
55+
`MaxListenersExceededWarning`. Live evidence: **0 occurrences in 1.1M log lines** → on the
56+
current 4-serve pool (ports 4096–4099, confirmed running + pigeon) no single serve has hit
57+
11 concurrent clients. The warning is **dormant, not fixed** — a serve owning 11+
58+
sessions/TUIs would still emit it.
59+
60+
### Why the storm is defused (the structural win, for the record)
61+
62+
| | Pre-pool (the §1.A scenario) | Deployed K=4 pool |
63+
|---|---|---|
64+
| Processes / event loops | 1 | 4 (separate GlobalBus each) |
65+
| N (clients per bus) | all TUIs (11–20+) | only that serve's owners (single digits) |
66+
| M (events per bus) | events from all active sessions | only that serve's sessions' events |
67+
| Fan-out work | O(N×M) on one core | ~O((N/K)×(M/K)) spread over K cores |
68+
69+
Worst-case fan-out drops ~K² *and* moves off the single hot core. That is the real reason
70+
the production warning/CPU-spin is gone — `serve-lease.patch` (HRW session sharding) +
71+
`attach-route-resolve.patch` (TUI → owning serve), **not** the `?session_ids=` filter.
72+
73+
---
74+
75+
## Minimal fix design (the two still-open qjk4 items)
76+
77+
Both are small, low-risk, and independent. Recommend shipping M3.3 unconditionally and
78+
M3.1 as a cheap defense-in-depth for "hot serve" cases.
79+
80+
### M3.3 — `GlobalBus.setMaxListeners` (trivial, unambiguous)
81+
82+
`packages/opencode/src/bus/global.ts`, one line after construction:
83+
84+
```ts
85+
export const GlobalBus = new GlobalBusEmitter()
86+
GlobalBus.setMaxListeners(0) // 0 = unlimited; or 100 per the investigation doc
87+
```
88+
89+
Rationale: the disposed-listener-per-client pattern is legitimate fan-out, not a leak. `0`
90+
(unlimited) is preferable to `100` because a busy multi-session serve has a genuinely
91+
unbounded client count and we never want this false-positive to mask a *real* leak warning
92+
elsewhere. Matches doc §5 Step 2. New patch file (e.g. `globalbus-maxlisteners.patch`),
93+
applied any time after base; touches only `global.ts`.
94+
95+
### M3.1 — move the existing predicate ahead of the queue offer (yl00-safe)
96+
97+
The key correctness constraint: do **not** re-introduce the yl00 cold-start directory race.
98+
That race is already handled by the cold-start predicate (session-scoped subs gate on
99+
session-aggregate membership, directory-independent). The fix is simply to evaluate that
100+
**same** predicate in the `listen` callback instead of in a downstream `Stream.filter`:
101+
102+
```ts
103+
const shouldForward = (event: EventV2.Payload) => {
104+
const sid = (event.data as Record<string, unknown> | undefined)?.["sessionID"]
105+
if (sessionIds !== undefined && typeof sid === "string") return sessionIds.has(sid)
106+
return matchesDirectory(event)
107+
}
108+
const unsubscribe = yield* events.listen((event) =>
109+
shouldForward(event) ? Effect.sync(() => Queue.offerUnsafe(queue, event)) : Effect.void,
110+
)
111+
// ...the downstream Stream.filter becomes redundant; drop it (or keep as defense-in-depth).
112+
```
113+
114+
Why this is safe and sufficient:
115+
- It is the **identical predicate** already in `event-cold-start-directory.patch`, just
116+
evaluated earlier — so yl00 semantics are preserved exactly (session subs stay
117+
directory-independent; non-session subs keep directory scoping they always had).
118+
- Non-matching events no longer allocate a queue offer or wake the client's stream fiber →
119+
per-serve fan-out becomes O(M_local) instead of O(N_serve × M_serve). This matters for a
120+
*hot serve* (e.g. a swarm coordinator + workers colocated on one serve via HRW).
121+
- The `disposed` GlobalBus sub-stream is a separate path and is untouched.
122+
123+
This would land as an edit to `event-cold-start-directory.patch` (it owns the final shape of
124+
that block) **or** a new follow-on patch that must apply after it. No client change required.
125+
126+
---
127+
128+
## Open decision for the coordinator / human
129+
130+
M3.3 is unambiguous, ~free, and currently-dormant-but-unmitigated → recommend ship.
131+
132+
M3.1 is the judgment call: the pool already defuses the *catastrophic* case, so the
133+
remaining win is bounded (hot-serve only) and the change, while small, edits a patch that
134+
two beads (qjk4 + yl00) already depend on. Decision to escalate: **ship M3.1 now as cheap
135+
insurance, or close qjk4-M3.1 as "obviated by the pool" and keep only M3.3?** I lean
136+
ship-both (both are low-risk and qjk4 is still OPEN scoping all three), but the
137+
"obviated by pool" close is defensible given zero production warnings today.
138+
139+
No implementation performed this round.

0 commit comments

Comments
 (0)