Skip to content

AIDA-2282: Reject legacy CLI calls to the desktop app - #2014

Open
ravarora2 wants to merge 7 commits into
mainfrom
rarora/aida-2282-reject-legacy-cli
Open

ravarora2 wants to merge 7 commits into
mainfrom
rarora/aida-2282-reject-legacy-cli

Conversation

@ravarora2

@ravarora2 ravarora2 commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Why

Old scripts call /Applications/Buzz.app/Contents/MacOS/buzz. On case-insensitive macOS, that path resolves to the graphical Buzz executable. A CLI call can foreground the app and report success without doing its work. The shortcut cleanup in #1914 does not cover direct bundle paths.

What

Recognized CLI calls now exit 1 before native startup. The static error directs scripts to the CLI bundled by #2027 at Buzz.app/Contents/Resources/bin/buzz on macOS. Exit 1 matches the CLI's bad-input status; exit 2 means a relay/network failure.

How

Check only the first argument against the pinned CLI's commands and global options, including gifs, auth, huddles, sidebar, and --option=value forms. Deep links and OS launch arguments retain the existing startup path. The error never echoes command payloads.

The bundled-CLI harness derives command names and root options from the actual CLI's help and checks the desktop executable against them. Root options are checked both bare and as --option=value. Executable tests use disposable data directories and an existing startup refusal before identity access if recognition regresses. Six pass-through controls protect first-argument routing. The CI contract stages resources and builds the workspace test graph before the full Rust test run and FI configuration variants. No production bypass flag was added.

Risk

This affects recognized CLI calls to the desktop executable. Windows GUI-subsystem launches may lack stderr; exit status 1 remains the failure signal. Full GUI startup, cold/warm OS-delivered deep links, and human acceptance remain pending at this head.

The bundled CLI builder isolates both Cargo build outputs and install metadata by revision, target and profile. Its new cache namespace avoids previously staged binaries from the old shared build cache.

Testing

  • Reconciled with main 6614b8bb2 at 9b5c4d3c1: preserved shared worktree cache location, revision/target/profile isolation and both regression tests. Six CLI tooling tests and Biome passed; push hooks including workspace Clippy passed. Related-unit-test push lane excluded under the existing local-test direction; broad local execution was deferred to hosted CI. Hosted CI completed successfully on attempt 1 at 9b5c4d3c1. Rust/tool integration, all six JavaScript shards, all browser journeys, both browser flake checks, CI required and DCO passed. Windows native validation and Vitest flake checks were skipped. This does not clear the disclosed local macOS library crash or configured GUI/OS-link/human acceptance requirements; code-owner approval remains outstanding. Earlier local results below remain scoped to their recorded heads.
  • At 8037f71ff, validation uses the actual pinned c8d73664 CLI built in an empty isolated cache. Its help advertises 26 commands, including sidebar; staged and installed binary hashes match the freshly built binary. The old layout had copied the previous revision's binary into a newly labelled install. Earlier local 25-command/33-probe claims for c8d73664 were incorrect.
  • Before adding sidebar, the clean staged CLI reproduced the hosted diagnostic assertion failure. The fix adds positive classification and actual-executable payload coverage. Final 34-probe contract and Node integration results: all 34 probes passed at 8037f71ff; the complete Node integration suite passed 283 tests with one existing skip.
  • The staging regression covers revision/profile/target changes and old install invalidation. Independently restoring either the shared build output directory or the old install namespace makes it fail. Independent review found no code blockers.
  • The complete local macOS Buzz package run passed the classifier, build-config and executable tests plus doctests. Its library process still SIGSEGVs at the unchanged window-state boundary; the full local suite is not green.
  • The complete panel-motion.spec.mjs passed all 10 Chromium/WebKit cases at a42c60ddc. Its existing seam assertions now wait for dock/grid animations, preserving precision and real motion; a temporary 1px defect failed in both engines. No cases added/removed. Local final run was 25.2s, versus an unmodified 36.3s baseline; this does not establish Linux timing. Hosted WebKit 14/14 subsequently passed.
  • The previous native timeouts led to reusing the workspace test graph for the CLI contract; the rebase includes main's cold-cache budget fix ci: give the native job room for a cold Rust cache #2867 (30 minutes). No checks were removed. Hosted CI at the prior 8037f71ff head and DCO passed; the transcript cleanup timeout passed on one rerun without code or timeout changes.
  • Earlier packaged checks at 289e60b3e passed 78 legacy invocations through both macOS filename spellings with static, payload-free stderr. These predate the latest command additions. Full configured GUI startup, community loading, cold/warm OS-delivered deep-link navigation/focus, and human acceptance remain pending.

Bigger picture

Scripts must update their executable path or PATH symlink to use the bundled CLI. This guard rejects the obsolete path; it does not execute or forward commands. It does not address standalone CLI authentication failures.

Generated with Codex

@ravarora2
ravarora2 marked this pull request as ready for review October 30, 2026 20:05
@ravarora2
ravarora2 requested a review from a team as a code owner October 30, 2026 20:05

@ravarora2 ravarora2 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Reviewed f687135c016aa34dba965bb220cf0a3ad15672df: no blocking correctness or security findings in the three-file diff. Minimalness, elegance, and correctness: 9/10 each for the stated scope.

The recognized command/global-option list matches the legacy CLI. Rejection happens before identity access, single-instance activation, or deep-link registration; stderr is static and later payload arguments are not inspected. The regression tests exercise the actual executable and bound its startup time.

Independent local validation on macOS at this head:

  • cargo test -p Buzz --locked --no-fail-fast -- --test-threads=2: 721 passed, two existing tests ignored.
  • 72 actual-executable invocations through both target/debug/Buzz and its case-insensitive lowercase buzz path: all recognized commands/switches, separate-value and equals-form global options, and a non-Unicode later payload. Every invocation returned 2, empty stdout, and the exact static stderr message.
  • git diff --check passed; current required CI checks passed when inspected.

Remaining acceptance gap: normal cold/warm GUI startup and OS-delivered deep links were not exercised in this review. As documented in docs/deep-links.md, those require a coordinated bundle test because launching it changes the installed app's URL handler. The classifier tests establish argument pass-through, not complete GUI/OS acceptance. Human acceptance remains separate under AGENTS.md.

@wpfleger96 wpfleger96 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Blocking: one correctness gap must be fixed before merge. Reviewed at f687135c016aa34dba965bb220cf0a3ad15672df.

Blocking: the legacy gifs command is not recognized

is_legacy_cli_arg in src-tauri/src/legacy_cli_invocation.rs lists every top-level command of the legacy CLI except gifs. The legacy CLI defines Gifs(GifsCmd) (search, share) in crates/buzz-cli/src/lib.rs in block/buzz. It exists at this app's pinned Buzz revision 7e646124 and in the last old-desktop release desktop-v0.5.28, which bundles the CLI as binaries/buzz. So …/Contents/MacOS/buzz gifs search cats still falls through to native startup. With an instance already running, it is foregrounded and exits 0 without doing anything. That is the silent success this PR is meant to stop, and it contradicts the claim that the recognized list matches the legacy CLI.

Fix: add "gifs" to the classifier, add it to recognizes_legacy_commands_and_global_options, and add a representative gifs invocation to the real-binary test in src-tauri/tests/legacy_cli_invocation.rs.

Checked and fine

  • All other top-level commands and all four global options (--relay, --private-key, --auth-tag, --format, including =value forms) match the legacy CLI. --version/-V don't exist there, but rejecting them is harmless.
  • No legitimate launch argument is caught. Windows ("<exe>" "%1") and Linux (%u) deep-link registration, single-instance forwarding, the updater handoff, and the macOS relauncher (src-tauri/src/relaunch.rs) all pass through the original or buzz:// arguments, and checking only argv[1] keeps link contents and later values out of the match.
  • main() -> ExitCode is valid under windows_subsystem = "windows". Note that a release GUI-subsystem launch on Windows may have no stderr handle, so the message can be dropped there; exit status 2 is the reliable signal, so don't promise visible stderr on every Windows launch.

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Star Lord’s automated source review via Wes’s account, integrating Rocket (startup/lifecycle), Mantis (correctness/security), and Groot (CLI behavior).

Reviewed head: f687135
Base tip: 6ee523c
PR diff merge base: 859c995

[P2] Remove the internal tracker hyperlink from the public description. The “Why” paragraph publishes a private workspace’s Linear issue URL in this public repository. Remove that hyperlink and retain the self-contained problem statement (or replace it with a public issue). This is a public-material boundary finding about the coordination reference itself; it does not assert exposure of the ticket contents or credentials.

No actionable code defects found in the three-file patch. The first-argument guard precedes native initialization, uses a static diagnostic, and leaves the existing launch/deep-link owners in place. Legitimate commit attribution is not a finding.

Validation: source-only review; no tests, builds or app launches executed. One hosted-check snapshot reported CI required successful, with Windows native validation skipped. Contributor-reported packaged checks were not independently reproduced; native diagnostics, cold/warm OS deep-link behavior and human acceptance remain unverified. The linked issue could not be read (403); the description and commit message were inspected and there were no attached images. This COMMENT is not an approval or merge authorization.

@ravarora2
ravarora2 force-pushed the rarora/aida-2282-reject-legacy-cli branch from f687135 to f5f28e0 Compare November 2, 2026 16:23

@kalvinnchau kalvinnchau left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Three actionable concerns: migration exit-code semantics, command-inventory drift, and executable-test isolation. Native/package acceptance remains unverified.

Comment thread src-tauri/src/main.rs Resolved
Comment thread src-tauri/src/legacy_cli_invocation.rs Resolved

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Star Lord’s automated source review via Wes’s account, integrating Rocket (startup/lifecycle), Mantis (correctness/security), and Groot (CLI behavior).

No additional changes requested in this follow-up. The earlier internal tracker hyperlink is removed, gifs now matches the pinned CLI command inventory, and the diagnostic/help point to the bundled CLI path. The original guard patch is unchanged by the rebase; the follow-up extends the existing executable regression with GIF payload suppression and the replacement path.

The existing exit-code, inventory-drift and test-isolation comments remain visible. We did not establish a concrete failing caller for the exit-code concern; inventory synchronization and regression-path isolation remain optional improvements, not newly demonstrated defects in this follow-up.

Reviewed head: f5f28e0
Base tip: e4177d0
PR diff merge base: 7f793e9

Source-only: no tests, builds or app launches executed, and CI was not assessed this cycle. Contributor-reported checks were not independently reproduced. Installed-package behavior, cold/warm OS deep links, rendered retry/focus behavior and human acceptance remain unverified. This COMMENT is not an approval.

@wpfleger96 wpfleger96 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review at f5f28e047426f9fe85461bbcfefa05c912e6428f (base e4177d0a46c68a221bedf5df6a46a8b6ea50d60c): no blocking findings.

Source review (two independent reviewers):

  • The 23 command names in src-tauri/src/legacy_cli_invocation.rs match the top-level Cmd enum of crates/buzz-cli exactly. This holds at the block/buzz revision this app pins and bundles (7e646124, from the buzz-audio-client rev used by scripts/build-buzz-cli.mjs) and at block/buzz main. Neither defines aliases. The four root options match Cli. -V/--version go beyond that CLI, but matching them is harmless.
  • The guard runs before buzz_foundation_lib::run(), so it precedes native-MCP setup, identity initialization and single-instance IPC. Only argv[1] is checked, so buzz:// links, -psn_*, Cocoa launch options and literal -- keep the normal startup path. The locked deep-link and single-instance plugins pass the URL as argv[1] on Windows and Linux, and parse it after this guard. The diagnostic is static and never echoes payloads.

Native executable check (debug build of this head, headless):

  • These each exited 2, with empty stdout and the exact diagnostic: messages send …, gifs search …, --help, --relay=x, --private-key=… messages list and --format compact channels list.
  • These each passed the guard and entered run(): an unknown argument, a buzz:// link, -psn_0_12345, -- messages send, an unknown argument followed by messages, --help=something, bare send, and no arguments. This was shown with a pre-window startup exit. Bare send is correctly not rejected, since the CLI nests it under messages.
  • Not covered: full app startup, OS-delivered deep links and packaged-app behavior.

MINOR (non-blocking): src-tauri/tests/legacy_cli_invocation.rs checks only the arguments that should be rejected at the executable level. A pass-through control would guard the "argv[1] only" wiring against a future change that scans every argument. For example, run an unknown first argument followed by messages, with a startup tripwire such as BUZZ_NATIVE_MCP_PORT that exits before the window opens.

CI: the failing browser journeys (Chromium 6/9 and WebKit 9/14 for thread navigation history, and WebKit nested-reply focus) fail the same way on main at this PR's base e4177d0a. Later main commits are green, so a rebase should clear them. Rust and tool integration, including the new tests, passed.

@ravarora2
ravarora2 force-pushed the rarora/aida-2282-reject-legacy-cli branch from f5f28e0 to 289e60b Compare November 3, 2026 22:50

@kalvinnchau kalvinnchau left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 One non-blocking inventory-coverage gap remains; no current runtime defect demonstrated.

Comment thread scripts/test-buzz-cli.mjs Resolved

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Star Lord’s automated source review via Wes’s account, integrating Rocket (startup/lifecycle), Mantis (correctness/security), and Groot (CLI behavior).

No additional changes requested in this follow-up. Exit status 1 now matches the bundled CLI’s bad-input contract. The executable tests stop at the existing pre-identity refusal if recognition regresses, and the bundled-CLI harness checks every advertised root command. The earlier guard and GIF repair remain intact through the rebase.

The existing suggestion to derive root-option coverage from help remains optional: the current four options match the pinned CLI, and no present runtime defect was demonstrated. No duplicate inline comment added.

Reviewed head: 289e60b
Base tip: f557d0a
PR diff merge base: f759d75

Source-only: no tests, builds or app launches executed. One hosted-check snapshot reported CI required successful, with Windows native validation and flake checks skipped. The author reports the local full macOS suite still crashes in window_state::tests::onboarding_snapshot_is_kept_until_next_launch_restore; we did not reproduce or diagnose that failure. Installed-package behavior, full GUI startup, cold/warm OS deep links, focus behavior and human acceptance remain unverified. Public description and commit messages were inspected; no attached images were present. This COMMENT is not an approval or merge authorization.

@wpfleger96 wpfleger96 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review at 289e60b3eff61e1cf1927f604869232e5c1a609b (base f557d0a50486cd90bc0a377f8f80a06dcba5bf6a): no blocking findings.

I reviewed the one commit added since f5f28e04, "refine CLI migration failures". The other two commits are the same as before apart from a docs rebase.

  • Exit status 1. This now matches the bundled CLI's bad-input status. main.rs, docs/buzz-help.md and both executable tests agree on it. The pass-through test still separates the two exit-1 paths by checking stderr: the native-MCP refusal versus the migration message.
  • Pass-through controls. These cover the earlier suggestion. ordinary_launch_arguments_reach_the_pre_identity_startup_refusal sends no arguments, a buzz:// link, -psn_*, a Cocoa option followed by messages, an unknown argument followed by messages, and -- messages send. Each must reach the BUZZ_NATIVE_MCP_PORT=invalid refusal. That refusal runs at the top of run() (src-tauri/src/lib.rs), right after the static app context loads, so a regression stops before sign-out erase work, identity access and single-instance IPC. The HOME/XDG/APPDATA overrides plus that tripwire also cover the test-isolation concern. tempfile is already a regular dependency of the Buzz crate.
  • Bundled-CLI contract. scripts/test-buzz-cli.mjs takes the root commands from the staged CLI's own --help output and confirms that the desktop executable rejects each one with status 1, empty stdout and the migration message. It guards against an empty list. It reuses the same isolated environment and tripwire. The script is the existing CI step "Bundled buzz CLI".

CI at this head: green. In the Rust and tool integration job log, both executable tests and the three unit tests pass, and the contract step prints Desktop rejects all 23 bundled CLI commands.

MINOR (non-blocking): I agree with the open comment on scripts/test-buzz-cli.mjs. The four root options (--relay, --private-key, --auth-tag, --format) are still a handwritten list that nothing checks against the bundled CLI. Taking --long names from the Options: section of the same help output would close that drift path.

Not covered: full GUI startup, cold and warm OS-delivered deep links, and installed-package behavior.

@kalvinnchau kalvinnchau left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 One integration blocker against current main; the previous root-option coverage gap is addressed.

Comment thread src-tauri/src/legacy_cli_invocation.rs Resolved
@ravarora2
ravarora2 force-pushed the rarora/aida-2282-reject-legacy-cli branch 2 times, most recently from 7d38301 to a42c60d Compare November 6, 2026 19:42

@kalvinnchau kalvinnchau left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 One remaining command-inventory defect at a42c60d; add sidebar before merging.

Comment thread src-tauri/src/legacy_cli_invocation.rs Resolved
ravarora2 and others added 7 commits November 6, 2026 23:21
Return a visible failure before native startup when old scripts reach the desktop executable instead of the legacy CLI.

Co-authored-by: BuzzPRReviewAssistant1 <a581a7e8115222acdc32ffb14bfce55d48c24478d98e155f8eba126b69b83c7b@buzz.block.builderlab.xyz>
Signed-off-by: Ravneet Arora <[email protected]>
Recognize the shipped gifs command and direct old scripts to the CLI now bundled in app resources.

Co-authored-by: BuzzPRReviewAssistant1 <a581a7e8115222acdc32ffb14bfce55d48c24478d98e155f8eba126b69b83c7b@buzz.block.builderlab.xyz>
Signed-off-by: Ravneet Arora <[email protected]>
Use the CLI bad-input status for obsolete desktop invocations. Check the bundled command inventory and stop executable regression tests before identity startup.

Co-authored-by: BuzzPRReviewAssistant1 <a581a7e8115222acdc32ffb14bfce55d48c24478d98e155f8eba126b69b83c7b@buzz.block.builderlab.xyz>
Signed-off-by: Ravneet Arora <[email protected]>
Derive root options from bundled CLI help and check both bare and equals-value migration invocations.

Co-authored-by: BuzzPRReviewAssistant1 <a581a7e8115222acdc32ffb14bfce55d48c24478d98e155f8eba126b69b83c7b@buzz.block.builderlab.xyz>
Signed-off-by: Ravneet Arora <[email protected]>
Reject auth and huddles before desktop startup. Reuse the workspace test build for the bundled CLI contract before FI variants change native build inputs.

Co-authored-by: BuzzPRReviewAssistant1 <a581a7e8115222acdc32ffb14bfce55d48c24478d98e155f8eba126b69b83c7b@buzz.block.builderlab.xyz>
Signed-off-by: Ravneet Arora <[email protected]>
Await the dock and grid animations before checking settled header geometry.
Keep the existing seam precision and real transition behavior.

Co-authored-by: appetizer <a581a7e8115222acdc32ffb14bfce55d48c24478d98e155f8eba126b69b83c7b@buzz.block.builderlab.xyz>
Signed-off-by: Ravneet Arora <[email protected]>
Recognize sidebar before native startup and cover payload suppression. Isolate Cargo build outputs alongside revision-specific installs and invalidate the old cache layout to prevent staging a stale named binary.

Co-authored-by: appetizer <a581a7e8115222acdc32ffb14bfce55d48c24478d98e155f8eba126b69b83c7b@buzz.block.builderlab.xyz>
Signed-off-by: Ravneet Arora <[email protected]>
@ravarora2
ravarora2 force-pushed the rarora/aida-2282-reject-legacy-cli branch from 8037f71 to 9b5c4d3 Compare November 7, 2026 04:24
matt2e added a commit that referenced this pull request Nov 14, 2026
…in the measurements lane (#3447)

## Why

`browser-host/relay-broker-live.test.mjs`, "real broker keeps 100,000
joined channels and delivers outside the first batch", times out
intermittently in the shared JavaScript shard.

| Sighting | Lane |
|---|---|
| [7 Nov, PR, passed on
re-run](https://github.com/block/buzz-app/actions/runs/42415071330/job/128531273095)
| JavaScript (2/6) |
| [9 Nov, PR, passed on
re-run](https://github.com/block/buzz-app/actions/runs/42600556170/job/129103660721)
| JavaScript (2/6) |

**This PR edits `.github/workflows/ci.yml`.** It adds one step to the
existing Browser measurements job. The required gate is unchanged: no
job was added or removed, and the measurements result that `CI required`
already checks now carries the scale check.

## Root cause

The test asserts wall-clock time: about 20 s of single-threaded CPU has
to finish inside a 30 s `vi.waitFor` budget. Vitest hashes files to
shards, so the test always shared shard 2/6 with `InboxPage.test.tsx`
and `vite-config.test.mjs`, under two workers.

- `main` passed at 22.5 s.
- The two failing jobs reached the budget at 33.9 s and 32.8 s, with the
broker connected and no channels reported joined yet.
- Their whole shard ran 440 to 446 s against 366 s on `main`.
- Locally the test took 18 s unthrottled and failed 2 of 3 at about 35 s
under CPU throttling.

The 30 s budget is a documented performance contract, so it is not
raised.

## Change

The test body moves into a helper shared by two tests in the same file.

- **"real broker delivers outside the first joined batch"** joins one
channel more than a live batch and runs in the shared shards. It keeps
the behavioural contract: one socket, the first live-only REQ names a
full batch that excludes the last channel, an event in that channel is
delivered, and swapping it for a new channel establishes the new one and
retires the old route. The lone overflow channel replays inside its own
REQ, so this case expects one live-only batch REQ, not two.
- **The 100,000-channel check** keeps its 49 live-only REQs, its
open-wire ceiling and its 30 s and 60 s budgets. It is gated with
`test.skipIf(!process.env.BUZZ_BROKER_SCALE)`, the same pattern the
Claude Code protection live test uses.

The gated check runs as a step of the Browser measurements job, which
already runs serially on an isolated runner. The step sets
`BUZZ_BROKER_SCALE`, filters by title through `ci-test-report.mjs`, and
uploads its report with the job's evidence artifact. A title filter that
matches nothing is a green run of zero tests, so the step then requires
exactly one passed test. `docs/contributing.md` lists the step under the
lane.

No budget, assertion or failure path was weakened, and no engine was
skipped.

## Evidence

Measured locally on an idle 32-core Linux host, Vitest 5.0.4.

| Measurement | Before | After |
|---|---|---|
| Scale check alone | 14.8 s test time (`vitest run -t "100,000"`, 15.3
s total) | 15.0 s test time, 16.6 s elapsed with the exact lane command;
1 passed, 55 skipped, guard exit 0 |
| Whole file, as a shared shard runs it | 22.2 s, 55 tests | 7.5 s, 55
passed and 1 skipped |
| New behavioural case | | 10 of 10 fresh runs passed, 231 to 277 ms |
| Guard with the gate unset, `-t "100,000"` | | vitest exits 0 with 56
skipped; the guard exits 1 |

Hosted, before, from a green `main` run: Browser measurements 10 m 34 s,
JavaScript (2/6) 7 m 31 s. Biome is clean on the test file and the
workflow parses. The pre-push type check and the file's related unit
tests passed on this branch's head.

`tests/integration/browser-ci.test.mjs` pins the upload paths of the
**Measurement evidence** step, in the case "classic-scrollbar cases run
exactly once, after the measurements, without clearing their evidence".
The second commit adds `test-results/broker-scale` to that expected
list. It changes the test expectation only; the workflow is as in the
first commit.

| Integration test file | Before the second commit | After |
|---|---|---|
| `./bin/node --test tests/integration/browser-ci.test.mjs`, macOS, Node
v24.18.0 | 6 passed, 1 failed on the upload-path assertion | 7 passed, 0
failed |

That assertion is the only pin of the measurements job's steps or paths.
The lane's other integration test files and the Linux container the lane
uses in CI were not run locally.

## Not confirmed, not run

- Hosted timings after the change are not known yet. The measurements
lane gains about the scale check's run plus Vitest startup. The drop in
shard 2 is confirmed only by the job summary once CI runs this PR.
Reviewers should compare both there.
- The local timings are from Linux and are not hosted CI numbers.
- No human has tested this yet, so `buzz-review-completed` is not set.

## Left alone

- `InboxPage.test.tsx` shares the same slow shard with no race found. It
needs fixture-cost profiling rather than a timeout change.
- Open PRs #3050, #3046, #2883, #2836, #2014, #1140, #1105 and #873 also
edit `ci.yml`; #2604 and #427 edit the test file; #1547, #1140 and #427
edit `docs/contributing.md`. A trial merge of each against this change
is clean.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: mtoohey <[email protected]>
Signed-off-by: Matt Toohey <[email protected]>
Co-authored-by: Claude Fable 5.1 <[email protected]>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants