Skip to content

Commit 12a17a5

Browse files
committed
test(ci): quarantine the 50 MiB sideband relay case into its own lane
sideband GET /v1/live/{callId} relays a 50 MiB WebSocket frame end to end against a hard 15s deadline while sharing a process with the rest of its --shard=N/2 half, so its result measures the whole process rather than the relay. On dev it lands in shard 1 and its echo leg alone spends 7.4s of that budget. Three test files added elsewhere in this branch made Bun repartition the halves, the case moved to shard 2, and the echo leg went past 15s twice with the peer never receiving the frame -- with nothing on the sideband path changed. SERIAL_FULL_SUITE_FILES is the mechanism this repository already has for that category; its own guard describes it as quarantining load-sensitive files into one-worker lanes. The deadline, the assertion and the macOS leg are unchanged; the case simply stops sharing a process, which also keeps it from breaking the next branch that adds a test file anywhere in the tree.
1 parent a9ee112 commit 12a17a5

2 files changed

Lines changed: 34 additions & 0 deletions

File tree

‎devlog/_plan/260920_meaning_preservation_batch/050_lane_d_account_cache_generation.md‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -179,6 +179,31 @@ introduced by the carried #4793 columns:
179179

180180
## Verification
181181

182+
### The macOS sideband failure was shard composition, not the relay
183+
184+
`macos 2/2` then failed twice on `sideband GET /v1/live/{callId} relays the exact frame
185+
ceiling bidirectionally`, and it is worth being precise about why, because retrying it would
186+
not have helped and neither would touching its deadline.
187+
188+
The case relays a 50 MiB WebSocket frame end to end against a hard 15s deadline. It is not in
189+
`SERIAL_FULL_SUITE_FILES`, so it runs inside `bun test --shard=N/2` sharing one process with
190+
the rest of that half. On `dev` at `043aa435f` it lands in shard 1 and its echo leg alone
191+
takes **7.4s of the 15s budget**. This branch adds three test files in unrelated directories,
192+
Bun repartitioned the halves, `tests/server/server-live.test.ts` moved to shard 2, and the
193+
echo leg went past 15s on both attempts while the peer never received the frame
194+
(`recv=13 progress=5 moving=no`). Nothing in this lane's diff touches the sideband relay, the
195+
live route or WebSocket handling, and the delta between the run that passed every shard and
196+
the run that failed this one is three GUI test files and a devlog page.
197+
198+
So the test has been passing by accident: its result was a property of which half it drew.
199+
The remedy is the mechanism the repository already has for this exact category —
200+
`SERIAL_FULL_SUITE_FILES`, described in its own guard as quarantining *load-sensitive* files
201+
into one-worker lanes. Adding `server/server-live.test.ts` there keeps the 15s deadline, keeps
202+
the assertion, and keeps macOS in the matrix; it only stops the case from sharing a process.
203+
It also takes the landmine out of the path of the next lane that adds a test file anywhere in
204+
the tree. If the coordinator would rather own that change centrally, it is one line in
205+
[scripts/test.ts](../../../scripts/test.ts) and can be lifted out of this branch.
206+
182207
Per batch rules, no local suites, individual tests, typecheck, build,
183208
install or live `ocx` execution. Verification is static source review plus
184209
exact-head hosted CI.

‎scripts/test.ts‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -352,6 +352,15 @@ export const SERIAL_FULL_SUITE_FILES = [
352352
"adapters/openai/openai-provider-option-e2e.test.ts",
353353
"ci-workflows/release-helper.test.ts",
354354
"update/update-stop-first.test.ts",
355+
// Relays a 50 MiB WebSocket frame end to end against a 15s deadline, so its result is a
356+
// measurement of the whole process, not of the relay. On a healthy 3-CPU macOS runner the
357+
// echo leg alone spends 7.4s of that budget; whichever half of `--shard=N/2` it lands in
358+
// decides whether it finishes. It has been passing by accident: it sat in the lighter half
359+
// until three unrelated test files were added elsewhere in the tree, Bun repartitioned, and
360+
// it went from 7.4s to over 15s twice in a row without anything on the sideband path
361+
// changing. Quarantining it here is what keeps it a test of the relay instead of a test of
362+
// its neighbours.
363+
"server/server-live.test.ts",
355364
] as const;
356365

357366
type SerialLaneBasename = (typeof SERIAL_FULL_SUITE_FILES)[number] extends infer P

0 commit comments

Comments
 (0)