fix(search): hold an early ⌘K Enter until results land - #1910

…r paging

Addresses the PR #1910 review of 857ecd6.

Show more messages: a held Enter now treats the "Show more messages" row
as an action, like the scope action. If a full page of message hits is
all filtered out while the open-channel directory is still reading, that
row is the only result. Enter then chose it, or "Search in <conversation>"
above it, at once, and the open channel that landed later never opened.
Now Enter waits for the lookups and opens the open channel, or the first
row once they settle. The predicate is renamed isSearchAction. Choosing
the row still runs it at once once the user has highlighted it.

Paging cancels a DM: choosing "Show more messages" now cancels a DM that
is still opening, like every other choice. Before, the DM's completion
closed the palette while the user was paging.

Closing cancels a DM: SearchResults cancels a DM that is still opening
when `open` becomes false, and clears its status line. Before, closing
with Escape and reopening during the close animation kept the retained
attempt, and isOpen() was true again, so the old DM navigated out of the
fresh palette. PageSearch's isOpen() check stays.

Tests (each fails with its fix reverted):
- SearchResults.test.tsx: new "holds an early Enter for an open channel
  over Show more messages %s", alone and with the scope action (without
  the fix, page 2 is read or the search is scoped). "lets the newest
  choice win over a DM that is still opening" serves a full first page
  and adds a pending DM, then Show more, then late resolution.
- PageSearch.test.tsx: the close-animation test is now "does not open a
  DM that finishes starting after search closed and reopened". Escape
  aborts the open, the palette reopens before the animation ends, and
  the DM resolving does not navigate. The old assertion that the status
  line stays during the close animation is gone, since closing now
  clears it.

Validation: vitest src/app/shell, src/features/search,
src/features/direct-messages, src/bundled/mentions, src/bundled/repos
and ChannelMembersDialog (28 files, 637 tests) pass. `pnpm typecheck`,
`pnpm lint`, biome check and `git diff --check` pass. Not exercised in
the running app.

Line counts: this commit production +15/-5 (SearchResults.tsx), tests
+124/-7.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Matt Toohey <[email protected]>

…r paging

Addresses the PR #1910 review of 857ecd6.

Show more messages: a held Enter now treats the "Show more messages" row
as an action, like the scope action. If a full page of message hits is
all filtered out while the open-channel directory is still reading, that
row is the only result. Enter then chose it, or "Search in <conversation>"
above it, at once, and the open channel that landed later never opened.
Now Enter waits for the lookups and opens the open channel, or the first
row once they settle. The predicate is renamed isSearchAction. Choosing
the row still runs it at once once the user has highlighted it.

Paging cancels a DM: choosing "Show more messages" now cancels a DM that
is still opening, like every other choice. Before, the DM's completion
closed the palette while the user was paging.

Closing cancels a DM: SearchResults cancels a DM that is still opening
when `open` becomes false, and clears its status line. Before, closing
with Escape and reopening during the close animation kept the retained
attempt, and isOpen() was true again, so the old DM navigated out of the
fresh palette. PageSearch's isOpen() check stays.

Tests (each fails with its fix reverted):
- SearchResults.test.tsx: new "holds an early Enter for an open channel
  over Show more messages %s", alone and with the scope action (without
  the fix, page 2 is read or the search is scoped). "lets the newest
  choice win over a DM that is still opening" serves a full first page
  and adds a pending DM, then Show more, then late resolution.
- PageSearch.test.tsx: the close-animation test is now "does not open a
  DM that finishes starting after search closed and reopened". Escape
  aborts the open, the palette reopens before the animation ends, and
  the DM resolving does not navigate. The old assertion that the status
  line stays during the close animation is gone, since closing now
  clears it.

Validation: vitest src/app/shell, src/features/search,
src/features/direct-messages, src/bundled/mentions, src/bundled/repos
and ChannelMembersDialog (28 files, 637 tests) pass. `pnpm typecheck`,
`pnpm lint`, biome check and `git diff --check` pass. Not exercised in
the running app.

Line counts: this commit production +15/-5 (SearchResults.tsx), tests
+124/-7.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Matt Toohey <[email protected]>

…isibility

* origin/main:
  feat(identity): first-run setup store, step table and pages (composition follows in the next PR) (#1819)
  perf(identity-names): re-render a names consumer only when a label it resolved changes (#2100)
  fix(search): hold an early ⌘K Enter until results land (#1910)
  ci: 14 WebKit and 4 Vitest shards; retry preview publish and tool fetch (#1974)
  perf(channels): re-render working-agent rows only for their agents' profiles (#2101)
  test(messages): measure coordination text relative to its row (#2096)
  test: serve artwork in reaction preview geometry fixture (#2068)
  feat(desktop): send app version as user-agent on native relay requests (#2111)
  test: stabilize reaction overlay fixture media (#2106)
  [1/5] Keep protected launches on the startup Agents root (#2044)
  fix(mcp): tag community emoji in agent messages and edits (#2102)
  feat(identity): show when the organization sign-in ends in settings (#2097)
  feat(channels): simplify access choices in the members dialog (#2030)
  fix(profiles): show nip-fi verified name read-only in profile settings (#2099)

…undetermined WebKit flakes (#2118)

## Summary

**These are diagnostics and unproven mitigations, not fixes.** Three
WebKit journeys failed once on GitHub Actions between 31 Oct and 2 Nov
and passed on re-run. None of their causes is established, and none
reproduces reliably locally. This PR adds assertions and instrumentation
so the next failure reports where it went wrong. It also adds two
readiness waits that may avoid one suspected ordering. No product code
changes.

- `agent-search-chat.spec.mjs` and `global-search.spec.mjs`: assert the
typed query right after `fill()`, and wait for rendered content before
opening the palette.
- `messages.spec.mjs`: when the inline video does not pause, report the
media element's state and recent events. The instrumentation is
temporary and should be removed once the failure is diagnosed.

This is one of three independent PRs split from the
`flakey-tests-monday` branch: #2116 (ProfileAgentIdentity unit test) and
#2117 (browser root-cause fixes). Each is cut from `main` and can merge
in any order; no file is shared between them.

### `agent-search-chat.spec.mjs:19`: assert the typed value; wait for
the shell's Solo chats first

Test: "search opens a new Solo chat with one of the viewer's agents".

- **CI:** [WebKit shard 1/14, run 41286046125 (job
124787613179)](https://github.com/block/buzz-app/actions/runs/41286046125/job/124787613179).
The Agents result was never found.
- **Evidence:** reproduced once locally in 10 runs, with a trace. The
palette input stayed mounted and focused. `fill("honey")` resolved 70 ms
after the Cmd+K press while the palette was still entering, yet the
input value was `""` 14 ms later. The palette kept showing Recent
activity, so no query ran.
- Playwright's `fill()` selects and focuses in the page, then inserts
the text through `Page.insertText` into whatever is focused and selected
at that moment, without checking the value. Something in that window
loses the text.
- Base UI's deferred initial focus targets the already focused input.
There is no composer on this page, and the input is not remounted (same
Base UI id). The trigger is not determined.
- 0/120 with an instrumented copy of the spec, 0/40 with product-side
logging, 30/30 under CPU load and 30/30 in a Linux WebKit container did
not reproduce it.
- **Change:**
- The test asserts `toHaveValue("honey")` right after `fill()`, so the
next failure points at the typed query instead of a missing result. This
is a diagnostic boundary.
- Before opening the palette, it waits for the sidebar's Solo
conversations section to list "Earlier with Honey": the fixture's data
has loaded and rendered in the shell. This is the nearest equivalent to
the rendered-conversation wait that the public search destination test
in `global-search.spec.mjs` already uses. This page opens on Channels
("Select a channel to read it.") and renders no conversation. The code
comment says the mechanism is unproven.
- **Verification:** 5/5 locally on WebKit with `--repeat-each 5`.

### `global-search.spec.mjs:602`: assert the typed operator query; wait
for a rendered message first

Test: "combines author, channel and inclusive/exclusive day boundaries
in the rendered flow".

- **CI:** [WebKit shard 2/14, run 41272526754 (job
124748254345)](https://github.com/block/buzz-app/actions/runs/41272526754/job/124748254345).
The crew-search result never became visible.
- **Evidence:** no trace survived the re-run. The attempt-1 log carries
no detail beyond the timeout and a 429 that belongs to the neighbouring
history-loading quota test. The test fills the operator query
immediately after opening the palette, the same shape as the
agent-search case. The `in:` operand resolves and re-runs once the
channel list arrives, so no product cause was found. 30/30 in a Linux
WebKit container did not reproduce it.
- **Change:**
- The same value assertion follows its first `fill()`, to attribute the
next failure to the typed query or to the search.
- The test waits for a rendered `[data-message-id]` before opening the
palette. This is the same unproven ordering mitigation as the public
search destination test in this file.
- **Verification:** 5/5 locally on WebKit with `--repeat-each 5`.

### `messages.spec.mjs:176`: report media state when the pause assertion
fails

Test: "inline video controls follow playback, hover and keyboard focus".

- **CI:** [WebKit shard 7/14, run 41331319960 (job
124922069284)](https://github.com/block/buzz-app/actions/runs/41331319960/job/124922069284).
`expect(video).toHaveJSProperty("paused", true)` at `:255` received
`false` for 10 s after clicking "Pause video". The click resolved to the
button with no retries, yet the label stayed "Pause video".
- **Trace evidence** (the trace's slider snapshots record the media
clock):
- From the Play click (11.25 s into the test), the clock ran at about
1.0x to 1.167 s, then jumped to 0.288 s at 12.38 s. That is a restart
from zero well short of the 2 s loop boundary.
- It ran to 0.828 s at the Pause click (12.93 s) and read 0.001 s right
after it: a second off-boundary restart instead of a pause.
- Over the following 10 s it advanced only to 0.269 s, while screencast
frames stopped at about 13.9 s. The media pipeline stalled after the
Pause click, and no `pause` event reached the product.
- WebKit's `pause()` sets `paused` synchronously except for early
returns (suspension, interrupted media session, user-gesture policy).
The trace can neither confirm nor rule out any of them.
- The clip is a 2 s, 12 fps, silent h264 file. No path in
`MediaAttachment`, `VideoPlayer` or the media read recovery calls
`play()` after a pause in this fixture.
- **Not reproduced:** 13/13 full runs on WebKit. A probe that paused the
looping clip 12 times at `currentTime` 1.85–2.00 s paused immediately
every time. A stalled media pipeline on Linux WebKit is consistent with
the frozen frames but is not established.
- **Change (diagnostic only; behaviour unchanged):**
- Once the video appears, the test records the element's last 24
`loadstart`, `abort`, `play`, `pause`, `playing`, `waiting`, `stalled`,
`seeking`, `seeked`, `ended`, `emptied` and `error` events.
- It polls the pause assertion against a state object: `paused`,
`currentTime`, `readyState`, `seeking`, `ended`, `networkState`,
`error`, `currentSrc`, the instrumented element's `isConnected`, and the
recent events.
- The instrumented element is held through an `ElementHandle` while the
poll reads the live locator, so a reload, a seek and a remount are
distinguishable.
- Same expect timeout and semantics as `toHaveJSProperty("paused",
true)`. `expect.objectContaining` prints the whole received object on
failure.
- **Remove this instrumentation once the CI failure is diagnosed;** the
code comment says so.
- **Verification:** a must-fail control copy printed the full state and
event list, with `currentSrc` as the fixture URL. WebKit `--repeat-each
5`: 5/5. Chromium `--repeat-each 3`: 3/3.

### Browser coverage

- Browser cases added or removed: none.
- Changed: the two search journeys gain a readiness precondition and a
typed-value assertion. The video journey's pause assertion now polls a
state object with the same pass condition. No assertion is relaxed.

### Verification (local macOS)

- Full files, one single-worker Playwright process per engine, on the
source branch at 5e9fc0db5 (based on 7edaec0). `agent-search-chat`,
`global-search` and `messages` ran in the same run as the other branch
files: Chromium 78 passed, 1 skipped; WebKit 78 passed, 1 skipped. The
Chromium skip is the existing one at `messages.spec.mjs:893`.
- This PR's tree is `origin/main` ac8495a plus these three files. Main
has not changed them since that base, and each is byte-identical to the
source branch. `biome check` on the three files and `git diff --check`
pass.
- Agent review: two review passes on the source branch. The readiness
waits and the extra video state (`loadstart`/`abort`, `currentSrc`,
`isConnected`) came from the first. No blocking findings remain.

### Remaining gaps

- Hosted CI has not run on this head yet.
- None of the three causes is determined. The lost WebKit `fill()`
reproduced once in about 230 local attempts (once in the first 10 runs,
then not in the next 220). The global-search and inline-video failures
did not reproduce locally at all. The readiness waits are unproven
mitigations, and the video change is diagnostic only.
- Since the source branch's base, main gained #1910, which holds an
early ⌘K Enter until results land. These search journeys were not re-run
against it.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Matt Toohey <[email protected]>
Co-authored-by: Claude Fable 5.1 <[email protected]>

## Problem

⌘K's Settings group indexed only three hard-coded controls (Color mode,
Sound, Notify while viewing). Typing a section name such as `shortcuts`,
`profile`, `identity` or `plugins` found no Settings row; plugin on/off
switches were the only hits, and Enter toggled focus onto one of them.
The palette never read `settingsSections()`.

## Fix

Three commits, each reviewed:

1. **List Settings sections in ⌘K.** `useSearchSettings` builds
`settingsSections()` from the visible settings cards and the selected
membership, the same way `SettingsSidebar` does, and adds one row per
section (label, group as detail, icon). Section rows sort ahead of their
own controls on a tie. The hook retains card visibility while the
palette is open, because the cards snapshot lists only cards whose
visibility has settled visible. `PageSearch` passes `settingsCards` and
`communities` through and opens a section via `onSelect("settings", {
section })`, so navigation's `select()` adds the community scope that
sections like Leave or remove need.
2. **Rank group-label matches behind name matches.** The app group's
label is "Buzz", so `buzz` had put eight exact-rank Settings rows ahead
of a channel that merely starts with "buzz". A group-label match now
ranks one step behind a section's own label (as plugin categories
already do), and the Settings and Plugins inline groups rank by their
best row rather than their own label, which keeps Manage plugins first
for `plugins`.
3. **Drop section matches looser than `MatchRank.wordRuns`** (the
threshold `ChannelCompletion.tsx` already uses). Letters-in-order
matches had made `wes` → Software updates, `ops` → Software updates,
`help` → Channel templates, which mattered because an inline row present
before directory results arrive takes an early Enter (#1910).

Unchanged: the three control rows, the generic Settings page row, plugin
row matching, and the #1910 Enter hold.

## Before / after

Same keys, same timing on both builds (browser-test fixture, headless
Chromium, 1024×720). Left is the base `ac8495a8b`; right is this branch.


https://github.com/user-attachments/assets/da2b52b9-a7cf-43d8-a510-88814ea22e64

| Query | Before | After |
|---|---|---|
| `shortcuts` | Plugins only: Notes reader, Attention | **Shortcuts**
(Buzz) first, then the same switches |
| `identity` | Plugins only | **Identity** (Buzz) first |
| `plugins` | Plugin switches only; no Manage plugins | **Manage
plugins** and the Plugins group's sections, then the switches |
| `notifications` | Sound and Notify while viewing switches |
**Notifications** section first, then the two switches |
| `shortcuts` + Enter | Focus moves to the Notes reader switch; palette
stays open | Settings opens at Shortcuts |

Also checked in the after build: dark mode, 390 px width, and 1.5× text.
Rows fit, nothing overlaps, icons and match underlines render.

## Tests

15 new Vitest cases across `SearchSettings.test.tsx`,
`PageSearch.test.tsx` and `SearchResults.test.tsx`. Fail-then-pass
evidence per commit:

- Commit 1: with only the two production files reverted, all 9 new tests
fail and the 23 existing ones still pass.
- Commit 2: 5 new `SearchResults` cases (`buzz` and `community` with a
channel and with a person; a contributed `Labs` group) run the real hook
against real channel/person ranking. Each fails without the ranking
change.
- Commit 3: 1 new `SearchResults` case with a directory-only person and
Enter pressed before the lookup resolves. Fails on commit 2
(`openSection` called once), passes here. The query is `champ` (→
Channel templates) because plugin descriptions already match almost
every real first name at the letters-in-order rank, so no realistic name
isolates the section change.

`vitest run src/app/shell`: 15 files, 267 tests pass. `pnpm typecheck`
and `pnpm lint` clean. `git diff --check` clean. No browser (Playwright)
cases added; the hook and ranking are covered in jsdom and the behavior
was exercised in the app for the recording.

## Known follow-ups (not in this PR)

- **Early Enter can still land on an inline row.** Since #1910 the hold
applies only while every visible row is an action; inline rows
(controls, plugin switches, section rows) aren't. Plugin descriptions
match nearly every first name, so a fast Enter on a directory-only
person can still pick a plugin switch today and, where a section row
leads, now navigates to Settings instead of focusing a switch. The
complete fix (treat inline rows as actions for the hold) changes #1910's
behavior for every inline row and should be its own PR.
- **Narrow width: opening Settings at a section leaves the section below
the fold.** At 390 px the section list fills the screen; `Dialog.tsx`
scrolls content into view only on a nav click. Pre-existing;
`SoloRegions.tsx` and `CommunityPicker.tsx` open sections the same way.
- Ranking side effects: `settings` no longer scores the inline Settings
group at 0, so a channel like `settings-sync` now leads it (the Settings
page row still wins). On ties, Settings now sorts before Plugins.

## Deferred checks

- Human test (AGENTS.md step 3). Steps: run the browser fixture or
`BUZZ_DEV_VIEWER= just web`, press ⌘K, type `shortcuts`, `identity`,
`plugins`, `notifications`; each should list its section under Settings
first, and Enter should open Settings at that section. In Personal
space, `community` should list no Community sections.
- WebKit and the native desktop app were not exercised.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Matt Toohey <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>

## What

Search gets **All** and **Channels** tabs (Ctrl+Tab / Ctrl+Shift+Tab
switch from the field). Channels is how someone finds open channels they
haven't joined yet.

- **Nothing typed:** lists the open channels you can join, A–Z.
- **Typed text:** fuzzy-matches those plus your own channels, which are
marked "Joined" so a familiar name is never a dead end.
- **Choosing a row:** a joined channel opens. An unjoined one opens its
read-only preview, where Join lives.
- **Member counts:** each row ends with a person icon and the channel's
member count.

Builds on the fuzzy matching from #1986.

## Screenshots

Taken in the browser test fixture with made-up channels.

| | Light | Dark |
|---|---|---|
| Browsing open channels, A–Z with member counts | ![Channels tab
browsing,
light](https://raw.githubusercontent.com/block/buzz-app/2fa5d92c9a061058125abffd969a34bd22c5b081/pr-2164--channels-browse-light.png)
| ![Channels tab browsing,
dark](https://raw.githubusercontent.com/block/buzz-app/2fa5d92c9a061058125abffd969a34bd22c5b081/pr-2164--channels-browse-dark.png)
|
| Typing "des": matches with matched letters underlined | ![Typed
matches,
light](https://raw.githubusercontent.com/block/buzz-app/2fa5d92c9a061058125abffd969a34bd22c5b081/pr-2164--channels-typed-light.png)
| ![Typed matches,
dark](https://raw.githubusercontent.com/block/buzz-app/2fa5d92c9a061058125abffd969a34bd22c5b081/pr-2164--channels-typed-dark.png)
|
| Typing "al": a joined channel leads, marked Joined | ![Joined match
first,
light](https://raw.githubusercontent.com/block/buzz-app/2fa5d92c9a061058125abffd969a34bd22c5b081/pr-2164--channels-joined-light.png)
| ![Joined match first,
dark](https://raw.githubusercontent.com/block/buzz-app/2fa5d92c9a061058125abffd969a34bd22c5b081/pr-2164--channels-joined-dark.png)
|

## How it feels

- **Fast first paint.** The directory read starts as search opens, so
the Channels tab usually has it by the time it's chosen. Each page shows
as it lands. Later pages join below the rows already on screen rather
than reshuffling them, and the list is fully A–Z the next time it opens.
- **Counts without jumpiness.** Counts are read only for rows in or near
view, joined channels answer from local state, and the first batch goes
out right away. The count's slot (8ch, tabular numbers) is held from the
start, so its arrival doesn't move the row or trim the description
again.
- **Returns where you left it.** For five minutes after closing, search
keeps its tab, text, highlighted row and scroll place; Escape means
done, and the next open starts fresh. Kept text opens selected, so
typing something new replaces it. A highlight is kept only while its row
is in view, so Enter never chooses a row nobody can see. If the order
changed while search was closed (a partial list became the full A–Z), it
reopens on the same rows rather than the same pixel offset.
- Main's "held Enter while search closes chooses nothing" (#1910)
applies to both tabs.

## Notable code

- `relay/store.ts`: progressive naming of the directory during a first
read (`refresh` distinguishes a refresh of a finished read),
`readDirectory`, `directoryComplete`, and `readMemberCounts`. Counts are
batched `#d` reads of kind 39002 at 100 ids each, cached per epoch, and
counted as distinct `p` tags. This is merged with #2023's incremental
public refresh.
- `SearchChoices.tsx`: a `trailing` slot per row, `onRowsShown` (an
IntersectionObserver, with a report-all fallback), and a `SearchPlace`
that holds an anchor-based scroll restore.
- `use-search-highlight.ts`: `initial` (where `""` highlights none) and
`clear()`.
- `docs/buzz-help.md`: describes the Channels tab.
- `switch-tab-request.js`: the macOS Show Next/Previous Tab menu items
now reach a modal's own tabs (search) instead of doing nothing.

## Testing

- **Checks:** `tsc`, `biome check --error-on-warnings` and `pnpm run
design:check` are clean.
- **Vitest, merged with main:** across `src/app/shell`,
`src/features/{search,channels,relay,conversation,panels,direct-messages}`
and `src/bundled`, 7,476 tests passed and 1 failed. The failure was
`codex-sh.test.ts › lets go of the hand-over lock`, a timing case this
change doesn't touch; it passes when that file runs alone.
- **New and updated coverage:**
- `ChannelDirectoryResults.test.tsx`: browse vs. typed, the joined
label, counts with singular and plural, retry, progressive pages
appended in place then A–Z on reopen, and reopening on the highlighted
row.
- `SearchChoices.test.tsx`: scrolling away clears a kept highlight, a
visible highlight is kept, and a reordered list reopens on the same rows
with no selection.
  - `use-search-highlight.test.ts`: `initial` and `clear()`.
  - `store-discovery.test.ts`: count reads.
- **Fail-then-pass:** each review blocker below got a test that failed
before the fix and passes after it. They are a hidden highlighted row
after a scroll restore, a stale highlight after dismissing, and pixel
restore landing on the wrong rows after the order changed.
- **Journey fix:** `navigation-sidebar.spec.mjs` › "keyboard page search
focuses main" failed in both engines because kept text was appended to.
It now passes in Chromium and WebKit locally.
- **Review:** agent review (Crack Catcher) found three blockers, all
fixed and re-approved. A visual pass (Xela) found nothing that must
change.
- **In the app:** the human author tested the feature in the desktop
app, and retested it after the final merge with main: progressive
loading, counts, typed matching, and reopening on the same rows.
- **Agent exercise:** not done. Screen capture of the native app wasn't
available to the agent, so no agent drove the app. The human author
chose to skip this step.

## Deferred

- Visual check: whether the 14px person icon reads heavy at the default
stroke; `stroke={1.75}` is the suggested fallback.
- No browser (Playwright) journey was added; the coverage above is
unit-level with jsdom geometry stubs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: tulsi <[email protected]>
Co-authored-by: BasilBot <399581b6165ab01343e3a1324774b1024d76942aea563468f93c975e5ee5193c@buzz.block.builderlab.xyz>
Co-authored-by: Claude Opus 5.5 <[email protected]>

…2607)

## Problem

The PR flake check repeated only the tests in files a pull request
edited. If a product change made an unedited test racy, the check
selected nothing. #2000 repeated one emoji journey and skipped the seven
other specs that load `emoji.html`. Five of its flaky journeys were in
those specs.

## What changes

`scripts/flake-check.mjs select` now selects in two tiers:

- **Tier 1, edited tests.** This is exactly the old rule: same
repetitions, both engines. The budget never trims it, even when tier 1
alone goes over.
- **Tier 2, related tests**, run as whole files:
- Browser specs that import a changed helper, or that open a
`tests/fixtures/*.html` page whose module graph holds a changed source
file, stylesheet or patched package. One rolldown build of every spec
and fixture page supplies the graph and writes nothing (1,027 modules in
about 1 s).
- Vitest files related to the changed modules through Vitest's module
graph. This is the graph the pre-push `vitest related` uses.

Picks are ranked by specificity: the smallest route that holds a changed
file comes first, and the cheaper test wins a tie. A test named after a
changed module ranks ahead of that order (`ComposerAttachments.tsx` →
`composer-attachments.spec.mjs`). The step summary marks those rows as a
naming heuristic, not graph evidence. Tier 2 adds picks in rank order
while each lane's predicted wall time (`flake_browser` and
`flake_vitest`) stays within **9 minutes** (`LANE_BUDGET_MINUTES`;
`--budget=<minutes>` overrides it). The 40-journey and 20-Vitest-file
caps still apply, and the pick that crosses a cap still runs.

**Cost table.** `scripts/flake-check-costs.json` is checked in and
written by the new `flake-check costs [run-id…]` subcommand. By default
it reads the latest three green `main` push runs. It reads the required
lanes' Playwright and Vitest reports and stores median passing seconds
per journey title as [Chromium, WebKit], plus a median per Vitest file.
A test missing from the table falls back to its file's median, then to
the engine-wide median or the median of all Vitest files. I checked in
the table instead of fetching timings in CI for three reasons: `select`
stays reproducible from the commit, `flake_select` needs no token or
network, and the artifacts are about 100 MB per run and expire after 7
days. The current table comes from runs 41990511962, 41980968364 and
41979442279 (Nov 5): 1,049 journeys and 964 Vitest files in 181 KB.

**Lane model.** I fit the model to hosted PR runs from Nov 4–5. The
median predicted/actual ratio is 0.95 for the browser lane and 1.02 for
Vitest.

- Browser shard: 105 s of overhead + 1.11 × the shard's summed test time
× 10 repetitions / 2 workers, then × 1.22 for imbalance when there is
more than one shard.
- Vitest batch: 26 s + 5 × (5.7 s + 0.72 × max(sum / 2, slowest file)).

The step summary gains a Tier column. It also shows each lane's
predicted time against the budget, with the edited-only time beside it,
and the related picks dropped for time and for the cap.

`docs/contributing.md` describes the tiers and the budget. It also
corrects "at most eight" shards to sixteen (`MAX_SHARDS`).

## For reviewers: the budget takes the longest fitting prefix

Tier 2 does **not** stop at the first pick that would go over the
budget. It takes the longest rank-order prefix whose predicted lane
fits. Browser shards grow with the selection (one shard per five
journeys), so a longer prefix can predict a *shorter* lane than a
shorter prefix. In the 248-PR replay, stopping at the first overrun
would have trimmed browser picks in 45 PRs. In 44 of those, the cap-only
selection already fit, and 40 predicted a slower lane after trimming.
#2000 would have kept 2 related journeys at 8.6 minutes instead of 46 at
6.2, and lost both of its flaky specs. Ranks are never skipped. The
Vitest lane only grows with each pick, so both rules give the same
result there. A unit test covers the non-monotonic case, and that test
fails under a first-overrun loop.

## Validation

Replays of `select <merge>^1 <merge>` in detached checkouts of each
merge commit. Selection before this PR → after:

| PR | Edited Vitest / journeys | Related journeys | Related Vitest |
Browser lane (min) | Vitest lane (min) | Flaky targets kept |
| --- | --- | --- | --- | --- | --- | --- |
| #2000 0944c1d | 1 / 1 | 0 → 46 (12 specs, incl. reactions-polish,
message-overlays) | 0 → 4 | 6.2 | 1.5 | yes |
| #1361 5992e69 | 2 / 0 | 0 → 40 (incl. composer-attachments, by name)
| 0 → 0 (budget) | 8.0 | 9.4 (edited alone 9.4) | yes |
| #2291 83b72a3 | 2 / 153 | 0 → 0 (budget) | 0 → 18 | 15.3 (edited
alone 15.3) | 6.6 | n/a |
| #1910 e7ae7b1 | 3 / 0 | 0 → 53 | 0 → 10 | 5.7 | 9.0 | n/a |

Lane times are this PR's predictions. Without the budget (the first
commit alone), the Vitest lane would have been 9.8 min for #2000, 16.3
min for #1910 and 6.6 min for #2291, and #2291's browser lane 18.3 min.
`--budget=Infinity` reproduces the first commit's output byte for byte
on all four source PRs. In the first commit's replay, a spec-only PR
(#2557) and a docs-only PR (#1694) selected exactly what the old rule
selected.

- `node --test tests/integration/flake-check.test.mjs`: 20 pass. Biome
is clean, including the table.
- Local `select` time on a laptop: #2000 takes 49 s with a cold Vitest
cache and 6 s warm (13 s before); #1361 takes 63 s cold and 8 s warm (19
s before). CI runs cold, within the select job's 5-minute limit. These
are local measurements, not hosted ones.

## Cost and known gaps

- **Runner-minutes grow about 7.5×.** In a replay of the 248 PRs merged
Nov 2–4, flake runner-minutes per run averaged about 11.4 under the old
rule and about 84 with related selection. Almost every source PR fills
the 40/20 caps, and the caps are what bound the cost. The budget limits
wall time only: against the first commit it moves the mean from 84.6 to
84.0 and the max from 381 to 319 (#2291). Lane wall time stays about 6–7
minutes for most PRs, and the highest predicted lanes fall from 16.7 to
13.7 minutes (browser) and from 16.3 to 10.4 (Vitest). All 11 Vitest
lanes still over 9 minutes are over on edited tests alone. The lanes
stay advisory, and repetitions, timeouts and workers are unchanged.
- **Not yet run in hosted CI.** This PR's own run will be the first.
- **Cost table drift.** Timings go stale until someone reruns
`flake-check costs`.
- **Model error.** Predicted/actual ranges from 0.73 to 1.23 (p10–p90)
for the browser lane and from 0.75 to 1.58 for Vitest. A Vitest
`file:line` filter is charged for its whole file.
- **Full-app journeys.** Journeys that load the full app rather than a
fixture page are reached only through helpers. This misses #1729, #2108
and #1969, which a coverage-map follow-up would address. A patched
package reaches browser specs but not Vitest files.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Matt Toohey <[email protected]>
Co-authored-by: Claude Fable 5.1 <[email protected]>