Skip to content

test(browser): serve the diffs and CLI reload fixtures outside the navigation budget - #3445

Merged
matt2e merged 1 commit into
mainfrom
mtoohey/flake-cold-dev-server
Nov 14, 2026
Merged

test(browser): serve the diffs and CLI reload fixtures outside the navigation budget#3445
matt2e merged 1 commit into
mainfrom
mtoohey/flake-cold-dev-server

Conversation

@matt2e

@matt2e matt2e commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

Why

Two Chromium journeys time out at page.goto in the Flake check lane because each test starts its own cold Vite dev server and navigates at once.

Test Sightings
diffs.spec.mjs, "diff preview expands in both layouts and keeps focus and scroll containment" Flake check, Chromium: 11 Nov, 9/12, 11 Nov, 3/10, 12 Nov, 2/9, 12 Nov, 3/9
plugin-cli-reload.spec.mjs, "CLI reload replaces an enabled folder plugin without a Settings action" Flake check, Chromium: 10 Nov, 5/9, passed on re-run

Root cause

Both specs created a Vite dev server inside each test. The diffs fixture reaches the bundled plugins, the message row, the thread panel, the conversation service, the relay session and the global stylesheet, about 1,800 modules. The conversation page is about 1,900. The worker-level warm-up from #3055 lives in source-fixture.mjs and only covers specs that take their base URL from it.

The Flake check lane runs --repeat-each=10 with two workers, and every repeat is a fresh worker. Each repeat therefore paid for a cold Node process, a cold Tailwind build and a cold module graph. In the failing traces 1,671 to 1,707 module requests were still in flight when the 15 s navigation timeout fired, with the global stylesheet alone taking about 9 s. The page itself was requested once, so this was not a dependency re-optimisation.

Fix

  • diffs.spec.mjs: build the page once per worker with previewFixtures and drop the per-test dev server, as the typeahead and voice-notes specs do. The spec has no server middleware and no pixel assertions, so the static bundle is enough. previewFixtures keeps development React. Most of the diff is the test body moving out of the server's try block by one indent level.
  • plugin-cli-reload.spec.mjs: it needs its proof middleware, so it keeps the dev server and warms up the conversation entry right after listen(), as conversation.spec.mjs does.

The navigation timeout is unchanged. No assertions changed and no browser cases were added or removed.

Evidence

Local runs on an idle 32-core Linux host, Chromium, a fresh worker per repeat, bin/pnpm test:browser:ci --project chromium --no-deps --reporter=list.

Scope Result
diffs.spec.mjs, --workers=1 --repeat-each=5 5/5 passed, 2.9 s each, 32.7 s total
diffs.spec.mjs, --workers=2 --repeat-each=4 4/4 passed, 14.9 s total
plugin-cli-reload.spec.mjs, --workers=1 --repeat-each=5 5/5 passed, 12.6 to 14.5 s each
plugin-cli-reload.spec.mjs, --workers=2 --repeat-each=4 4/4 passed, 13.4 to 13.8 s each

The native fixture for the reload spec was built first with bin/cargo build --locked -p buzzodz-plugins --example fixture-bridge --bin buzzodz.

Before the change, during triage on the same host: the unchanged diffs spec passed 5/5 on a quiet host, failed 5/5 at a load average near 25, and failed 4/4 with two workers. A standalone cold navigation took 5.4 to 7.7 s idle, 14.1 s on two cores and 15.3 s on one core. Passing repeats of the diffs test in the failing CI jobs took about 17.8 s; hosted timings after the change are only known once CI runs this PR.

Not reproduced, not run

  • The cold-navigation timeout reproduced only under load or with two concurrent cold workers, not on an idle host. The runs on this branch were on an idle host, so they guard against regression rather than reproduce CI timing.
  • WebKit was not run on this branch.
  • No human has tested this yet, so buzz-review-completed is not set.

Left alone

Other specs still serve fixture pages from a per-test cold dev server, including window-chrome, composer-attachments, image-strip, image-zoom and message-edit. They have not shown this timeout in CI and are candidates for the same treatment in their own change. Open draft #2746 edits other parts of plugin-cli-reload.spec.mjs; a trial merge against this change is clean.

🤖 Generated with Claude Code

@matt2e @claude
test(browser): serve the diffs and CLI reload fixtures outside the na…
e262000 
…vigation budget

Both specs started their own Vite dev server inside each test and
navigated at once. tests/fixtures/diffs.tsx reaches bundledPlugins,
MessageRow, ThreadPanel, ConversationService, createRelaySession and
globals.css, about 1,800 modules; conversation.html is about 1,900. The
worker-level warm-up from #3055 lives in source-fixture.mjs and only
covers specs that take baseURL from it. The flake-check lane runs
--repeat-each=10 with two workers, and every repeat is a fresh worker, so
each repeat paid a cold Node, cold Tailwind and cold module graph. In the
failing traces 1,671–1,707 module requests were still in flight when the
15 s navigation timeout fired, with globals.css alone taking ~9 s.

diffs.spec.mjs: build the page once per worker with previewFixtures and
drop the per-test dev server, as typeahead and voice-notes do. The spec
has no server middleware and no pixel assertions, so the static bundle is
enough; previewFixtures keeps development React.

plugin-cli-reload.spec.mjs: it needs its /__proof middleware, so keep the
dev server and warmUp the conversation entry right after listen(), as
conversation.spec.mjs does.

Verified on Chromium with a fresh worker per repeat
(`bin/pnpm test:browser:ci --project chromium --no-deps --reporter=list`):
- diffs: --workers=1 --repeat-each=5 → 5/5 (2.9 s each, 32.7 s total);
  --workers=2 --repeat-each=4 → 4/4 (14.9 s total).
- plugin-cli-reload: --workers=1 --repeat-each=5 → 5/5 (12.6–14.5 s);
  --workers=2 --repeat-each=4 → 4/4 (13.4–13.8 s). The native fixture was
  built first with `bin/cargo build --locked -p buzzodz-plugins --example
  fixture-bridge --bin buzzodz`.
The triage reproduced the cold-goto timeout only under load (~25) and
with two concurrent cold workers, not on an idle host, so these runs on an
idle 32-core host guard against regression rather than reproduce CI
timing.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Signed-off-by: mtoohey <[email protected]>
Signed-off-by: Matt Toohey <[email protected]>
@matt2e
matt2e requested a review from a team as a code owner November 13, 2026 02:49
kazemek21
kazemek21 approved these changes Nov 13, 2026
View reviewed changes

@kazemek21 kazemek21 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kyle reviewed this PR and the Nexus review-tracker notes and approved with no comments.

Posted by Codex (gpt-6.1-sol) on behalf of kylez

wesbillman
wesbillman reviewed Nov 13, 2026
View reviewed changes

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Star Lord automated source review via Wes’s account.

No actionable findings at head e262000f2bd577c430b78837b218f5e165e2126a, base/merge-base 7f03764beb8eb0ddcb4ff9ec0b8bda3e07699716. Reviewed the preview-fixture lifecycle, static assets/development defines, CLI warm-up ordering and middleware boundary, and preservation of assertions, case counts and timeouts. The changes reuse the existing fixture helpers without changing production code.

Read-only hosted evidence from run 43354824243, checked out at merge 125fd749bd3b984f9f2334a0552b091badd19a21 with those exact parents: each engine passed both changed journeys ten times (20/20 Chromium; 20/20 WebKit). Timing summaries report 185.1s / 198.2s elapsed wall time and 252.3s / 277.5s summed test execution respectively. The reload case was slowest at 23.1s / 22.8s; summed file execution was 204.7s / 210.1s for reload and 47.6s / 67.4s for diffs.

Source-only review; no local tests, builds or apps were run. These hosted results do not establish a matched before/after speedup or reproduce the original forced-load timeout. This is a COMMENT review, not approval.

@matt2e
matt2e merged commit 407c36c into main Nov 14, 2026
47 checks passed
@matt2e
matt2e deleted the mtoohey/flake-cold-dev-server branch November 14, 2026 00:47
wpfleger96 added a commit that referenced this pull request Nov 14, 2026
@wpfleger96
Merge remote-tracking branch 'origin/main' into duncan/cross-communit…
3cec1f1 
…y-unreads

* origin/main: (24 commits)
  [1/2] feat(agents): add a recommended agent setup (core) (#3344)
  test(browser): survive the live-stream restart a page-clock jump causes (#3442)
  chore(renovate): hold rusqlite below 0.41 for mesh-llm (#3438)
  test(browser): serve the diffs and CLI reload fixtures outside the navigation budget (#3445)
  test(browser): play the draft clip at rate 0 instead of looping it before the Play click (#3446)
  fix(status): reopen a status whose preset deadlines coincide as Today (#3449)
  fix(messages): tolerate a cancelled reaction-name chunk load (#3450)
  fix(pairing): tolerate /proc/net/tcp rows printed twice or missed (#3451)
  test(browser): wait on the signal each test asserts in local-agent-activity and message-navigation (#3444)
  Expose native channel headings to Solo sources (#3523)
  test(browser-host): run the 100,000-channel broker scale check alone in the measurements lane (#3447)
  fix(solo): keep every contributed destination action (#3516)
  fix(identity): report an issuer that keeps rate-limiting an agent as a limit (#2365)
  fix(identity): let a stuck at-launch sign-in copy its link (#2453)
  fix(profiles): open avatar previews on hover in compact density (#3510)
  feat(solo): smooth chat recency reordering (#3502)
  Reject self-mentions that cannot wake another session (#3513)
  feat(status): offer don't clear for every status (#3515)
  Let plugin panels name their tab with panels.useTabTitle (#3251)
  fix(agents): report a conversation wake that fails before its turn starts (#2192)
  ...

# Conflicts:
#	tests/browser/message-management.spec.mjs
wpfleger96 added a commit that referenced this pull request Nov 14, 2026
@wpfleger96
Merge remote-tracking branch 'origin/main' into duncan/clock-skew-2min
7c356a2 
* origin/main: (27 commits)
  [2/2] feat(agents): offer Automatic (Recommended) in Create agent and Identity (#3345)
  fix(solo): preserve and resume unsent chat drafts (#3525)
  fix(builderlab): give refused app sign-ins and Identity links a route to Organization sign-in (#3078)
  fix(agents): say why Import an agent can't import here, and link Manage plugins (#3073)
  [1/2] feat(agents): add a recommended agent setup (core) (#3344)
  test(browser): survive the live-stream restart a page-clock jump causes (#3442)
  chore(renovate): hold rusqlite below 0.41 for mesh-llm (#3438)
  test(browser): serve the diffs and CLI reload fixtures outside the navigation budget (#3445)
  test(browser): play the draft clip at rate 0 instead of looping it before the Play click (#3446)
  fix(status): reopen a status whose preset deadlines coincide as Today (#3449)
  fix(messages): tolerate a cancelled reaction-name chunk load (#3450)
  fix(pairing): tolerate /proc/net/tcp rows printed twice or missed (#3451)
  test(browser): wait on the signal each test asserts in local-agent-activity and message-navigation (#3444)
  Expose native channel headings to Solo sources (#3523)
  test(browser-host): run the 100,000-channel broker scale check alone in the measurements lane (#3447)
  fix(solo): keep every contributed destination action (#3516)
  fix(identity): report an issuer that keeps rate-limiting an agent as a limit (#2365)
  fix(identity): let a stuck at-launch sign-in copy its link (#2453)
  fix(profiles): open avatar previews on hover in compact density (#3510)
  feat(solo): smooth chat recency reordering (#3502)
  ...

# Conflicts:
#	src-tauri/src/federated_identity/agent/tests.rs
#	src-tauri/src/federated_identity/mod.rs
#	src-tauri/src/federated_identity/protocol.rs
joahg added a commit to joahg/buzz-app that referenced this pull request Nov 14, 2026
@joahg
Merge remote-tracking branch 'origin/main' into joah/agents-solo-not-dm
0a821dc 
* origin/main: (133 commits)
  Expose ordinary chats without workspace headings (block#3530)
  [2/2] feat(agents): offer Automatic (Recommended) in Create agent and Identity (block#3345)
  fix(solo): preserve and resume unsent chat drafts (block#3525)
  fix(builderlab): give refused app sign-ins and Identity links a route to Organization sign-in (block#3078)
  fix(agents): say why Import an agent can't import here, and link Manage plugins (block#3073)
  [1/2] feat(agents): add a recommended agent setup (core) (block#3344)
  test(browser): survive the live-stream restart a page-clock jump causes (block#3442)
  chore(renovate): hold rusqlite below 0.41 for mesh-llm (block#3438)
  test(browser): serve the diffs and CLI reload fixtures outside the navigation budget (block#3445)
  test(browser): play the draft clip at rate 0 instead of looping it before the Play click (block#3446)
  fix(status): reopen a status whose preset deadlines coincide as Today (block#3449)
  fix(messages): tolerate a cancelled reaction-name chunk load (block#3450)
  fix(pairing): tolerate /proc/net/tcp rows printed twice or missed (block#3451)
  test(browser): wait on the signal each test asserts in local-agent-activity and message-navigation (block#3444)
  Expose native channel headings to Solo sources (block#3523)
  test(browser-host): run the 100,000-channel broker scale check alone in the measurements lane (block#3447)
  fix(solo): keep every contributed destination action (block#3516)
  fix(identity): report an issuer that keeps rate-limiting an agent as a limit (block#2365)
  fix(identity): let a stuck at-launch sign-in copy its link (block#2453)
  fix(profiles): open avatar previews on hover in compact density (block#3510)
  ...

Signed-off-by: Joah Gerstenberg <[email protected]>

# Conflicts:
#	src/bundled/codex/prompt.test.ts
#	src/bundled/goose/runtime.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wesbillman wesbillman wesbillman left review comments
@kazemek21 kazemek21 kazemek21 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@matt2e @wesbillman @kazemek21