Skip to content

Commit 02c6309

Browse files
authored
fix(ci): select runtime suites from PR changes only (#7843)
A documentation-only update to #7809 selected desktop builds and E2E tests because path detection compared an old PR base SHA with GitHub's newer synthetic merge commit. The [failing run's path-detection log](https://github.com/block/buzz/actions/runs/35789957773/job/106955810630) includes four unrelated desktop files from `main`; `VISION_MOBILE.md` did not match a runtime filter. Use GitHub's PR file list for pull requests so unrelated changes in the synthetic merge commit cannot select runtime suites. The existing directory filters are unchanged: root and docs/ Markdown do not select runtime suites, while Markdown under runtime directories still selects its affected suites. Mixed code/documentation changes retain normal coverage. Always-on security, policy, and source-contract checks and full push-to-main coverage are unchanged. Added regression coverage runs the pinned paths-filter action against real fixture repositories, including a synthetic merge containing unrelated upstream desktop code. All 35 selection scenarios and 26 required-check scenarios pass; the new regressions failed before the fix. Existing required-context isolation, file-size policy, and security-review contract checks pass, as do script lint and workflow syntax validation. Full workflow lint reports the same two pre-existing shell-quoting findings in the untouched dead-token guard. A desktop E2E build was run to diagnose the unrelated required smoke failure. Related: #7809 (incident, unchanged) and #5756 (shared-input path coverage, separate scope). No matching issue found. PRs with at least 3,000 changed files now select every runtime suite, avoiding GitHub's PR-file-list ceiling. Boundary tests cover 2,999, 3,000, and 3,001 files, including a runtime file omitted beyond the API cap; disabling the safeguard makes the latter two regressions fail. Required checks now run and fail when path selection fails, is cancelled, or is skipped. Regression coverage exercises the workflow conditions and shell checks for all 13 required wrappers, plus an API-denial case against the pinned action. Mutating either the scheduling guard or the result check makes all 13 failure regressions fail. The required mention-settings smoke test expected a pin after explicitly disabling automatic mentions. Waiting for its old avatar to exit reproduced the CI failure 3/3; the test now checks that subsequent mentions remain manual and retains outgoing recipient-tag assertions. Corrected browser regression: 20/20 repeated runs; related picker unit tests: 18/18. No desktop production behavior changed. The always() requirement is now pinned in required-wrapper regression tests; deleting it fails all 26 gate cases. --------- Signed-off-by: Tom Brow <[email protected]>
1 parent a1fb664 commit 02c6309

5 files changed

Lines changed: 411 additions & 57 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 56 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -22,11 +22,11 @@ jobs:
2222
contents: read
2323
pull-requests: read
2424
outputs:
25-
rust: ${{ steps.filter.outputs.rust }}
26-
desktop: ${{ steps.filter.outputs.desktop }}
27-
desktop-rust: ${{ steps.filter.outputs.desktop-rust }}
28-
web: ${{ steps.filter.outputs.web }}
29-
mobile: ${{ steps.filter.outputs.mobile }}
25+
rust: ${{ steps.selection.outputs.rust }}
26+
desktop: ${{ steps.selection.outputs.desktop }}
27+
desktop-rust: ${{ steps.selection.outputs.desktop-rust }}
28+
web: ${{ steps.selection.outputs.web }}
29+
mobile: ${{ steps.selection.outputs.mobile }}
3030
steps:
3131
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
3232
with:
@@ -35,7 +35,11 @@ jobs:
3535
- uses: dorny/paths-filter@ceb8a2b8f2d89434be7ff52d3de7ec3738c5cc9d # v4.0.3
3636
id: filter
3737
with:
38-
token: ''
38+
# PR files come from GitHub's PR diff. A tokenless comparison uses the
39+
# checked-out merge commit and can include newer, unrelated base changes.
40+
token: ${{ github.token }}
41+
# Root and docs/ Markdown do not select runtime suites. Markdown under
42+
# runtime directories remains covered by the directory filters below.
3943
filters: |
4044
rust:
4145
- 'crates/**'
@@ -84,8 +88,13 @@ jobs:
8488
- '.github/workflows/mobile-release-candidate.yml'
8589
- '.github/workflows/ci.yml'
8690
- '.github/workflows/_ci-*.yml'
91+
- name: Select runtime suites conservatively at the PR file-list limit
92+
id: selection
93+
env:
94+
FILTER_OUTPUTS: ${{ toJSON(steps.filter.outputs) }}
95+
run: node scripts/ci-runtime-selection.mjs
8796
- name: Validate PostgreSQL test discovery
88-
if: github.event_name == 'push' || steps.filter.outputs.rust == 'true'
97+
if: github.event_name == 'push' || steps.selection.outputs.rust == 'true'
8998
run: |
9099
scripts/test-postgres-test-discovery.sh
91100
scripts/test-postgres-test-wrapper.sh
@@ -116,7 +125,7 @@ jobs:
116125
- name: CI required-context isolation contract
117126
run: scripts/test-ci-required-context-isolation.sh
118127
- name: CI path selection regression tests
119-
run: node --test scripts/ci-selection.test.mjs
128+
run: node --test scripts/ci-selection.test.mjs scripts/ci-required-gates.test.mjs
120129
- name: File size policy
121130
run: just file-size-check
122131

@@ -244,169 +253,182 @@ jobs:
244253

245254
rust-lint:
246255
name: Rust Lint
247-
if: always() && needs.changes.result == 'success' && (github.event_name == 'push' || needs.changes.outputs.rust == 'true' || needs.changes.outputs.desktop-rust == 'true')
256+
if: always() && (needs.changes.result != 'success' || ((github.event_name == 'push' || needs.changes.outputs.rust == 'true' || needs.changes.outputs.desktop-rust == 'true')))
248257
needs: [changes, rust]
249258
runs-on: ubuntu-latest
250259
timeout-minutes: 5
251260
permissions: {}
252261
steps:
253262
- name: Check Rust Lint result
254263
env:
264+
SELECTION_RESULT: ${{ needs.changes.result }}
255265
RESULT: ${{ needs.rust.outputs.rust_lint_result }}
256-
run: test "$RESULT" = success
266+
run: test "$SELECTION_RESULT" = success && test "$RESULT" = success
257267

258268
unit-tests:
259269
name: Unit Tests
260-
if: always() && needs.changes.result == 'success' && (github.event_name == 'push' || needs.changes.outputs.rust == 'true')
270+
if: always() && (needs.changes.result != 'success' || ((github.event_name == 'push' || needs.changes.outputs.rust == 'true')))
261271
needs: [changes, rust]
262272
runs-on: ubuntu-latest
263273
timeout-minutes: 5
264274
permissions: {}
265275
steps:
266276
- name: Check Unit Tests result
267277
env:
278+
SELECTION_RESULT: ${{ needs.changes.result }}
268279
RESULT: ${{ needs.rust.outputs.unit_tests_result }}
269-
run: test "$RESULT" = success
280+
run: test "$SELECTION_RESULT" = success && test "$RESULT" = success
270281

271282
windows-rust:
272283
name: Windows Rust (x86_64-pc-windows-msvc)
273-
if: always() && needs.changes.result == 'success' && (github.event_name == 'push' || needs.changes.outputs.rust == 'true' || needs.changes.outputs.desktop-rust == 'true')
284+
if: always() && (needs.changes.result != 'success' || ((github.event_name == 'push' || needs.changes.outputs.rust == 'true' || needs.changes.outputs.desktop-rust == 'true')))
274285
needs: [changes, rust]
275286
runs-on: ubuntu-latest
276287
timeout-minutes: 5
277288
permissions: {}
278289
steps:
279290
- name: Check Windows Rust result
280291
env:
292+
SELECTION_RESULT: ${{ needs.changes.result }}
281293
RESULT: ${{ needs.rust.outputs.windows_rust_result }}
282-
run: test "$RESULT" = success
294+
run: test "$SELECTION_RESULT" = success && test "$RESULT" = success
283295

284296
desktop:
285297
name: Desktop
286-
if: always() && needs.changes.result == 'success' && (github.event_name == 'push' || needs.changes.outputs.desktop == 'true' || needs.changes.outputs.desktop-rust == 'true' || needs.changes.outputs.rust == 'true')
298+
if: always() && (needs.changes.result != 'success' || ((github.event_name == 'push' || needs.changes.outputs.desktop == 'true' || needs.changes.outputs.desktop-rust == 'true' || needs.changes.outputs.rust == 'true')))
287299
needs: [changes, desktop-domain]
288300
runs-on: ubuntu-latest
289301
timeout-minutes: 5
290302
permissions: {}
291303
steps:
292304
- name: Check Desktop result
293305
env:
306+
SELECTION_RESULT: ${{ needs.changes.result }}
294307
RESULT: ${{ needs.desktop-domain.outputs.desktop_result }}
295-
run: test "$RESULT" = success
308+
run: test "$SELECTION_RESULT" = success && test "$RESULT" = success
296309

297310
desktop-build-macos:
298311
name: Desktop Build (macOS)
299-
if: always() && needs.changes.result == 'success' && (github.event_name == 'push' || needs.changes.outputs.desktop == 'true' || needs.changes.outputs.desktop-rust == 'true' || needs.changes.outputs.rust == 'true')
312+
if: always() && (needs.changes.result != 'success' || ((github.event_name == 'push' || needs.changes.outputs.desktop == 'true' || needs.changes.outputs.desktop-rust == 'true' || needs.changes.outputs.rust == 'true')))
300313
needs: [changes, desktop-macos-domain]
301314
runs-on: ubuntu-latest
302315
timeout-minutes: 5
303316
permissions: {}
304317
steps:
305318
- name: Check Desktop Build (macOS) result
306319
env:
320+
SELECTION_RESULT: ${{ needs.changes.result }}
307321
RESULT: ${{ needs.desktop-macos-domain.outputs.desktop_macos_result }}
308-
run: test "$RESULT" = success
322+
run: test "$SELECTION_RESULT" = success && test "$RESULT" = success
309323

310324
desktop-e2e-relay:
311325
name: Desktop E2E Relay
312-
if: always() && needs.changes.result == 'success' && (github.event_name == 'push' || needs.changes.outputs.desktop == 'true' || needs.changes.outputs.desktop-rust == 'true' || needs.changes.outputs.rust == 'true')
326+
if: always() && (needs.changes.result != 'success' || ((github.event_name == 'push' || needs.changes.outputs.desktop == 'true' || needs.changes.outputs.desktop-rust == 'true' || needs.changes.outputs.rust == 'true')))
313327
needs: [changes, relay-artifacts-domain]
314328
runs-on: ubuntu-latest
315329
timeout-minutes: 5
316330
permissions: {}
317331
steps:
318332
- name: Check Desktop E2E Relay result
319333
env:
334+
SELECTION_RESULT: ${{ needs.changes.result }}
320335
RESULT: ${{ needs.relay-artifacts-domain.outputs.desktop_e2e_relay_result }}
321-
run: test "$RESULT" = success
336+
run: test "$SELECTION_RESULT" = success && test "$RESULT" = success
322337

323338
desktop-e2e-integration:
324339
name: Desktop E2E Integration
325-
if: always() && needs.changes.result == 'success' && needs.relay-artifacts-domain.result == 'success' && (github.event_name == 'push' || needs.changes.outputs.desktop == 'true' || needs.changes.outputs.desktop-rust == 'true' || needs.changes.outputs.rust == 'true')
340+
if: always() && (needs.changes.result != 'success' || (needs.relay-artifacts-domain.result == 'success' && (github.event_name == 'push' || needs.changes.outputs.desktop == 'true' || needs.changes.outputs.desktop-rust == 'true' || needs.changes.outputs.rust == 'true')))
326341
needs: [changes, relay-artifacts-domain, relay-domain]
327342
runs-on: ubuntu-latest
328343
timeout-minutes: 5
329344
permissions: {}
330345
steps:
331346
- name: Check Desktop E2E Integration result
332347
env:
348+
SELECTION_RESULT: ${{ needs.changes.result }}
333349
RESULT: ${{ needs.relay-domain.outputs.desktop_e2e_integration_result }}
334-
run: test "$RESULT" = success
350+
run: test "$SELECTION_RESULT" = success && test "$RESULT" = success
335351

336352
backend-integration:
337353
name: Backend Integration (relay e2e)
338-
if: always() && needs.changes.result == 'success' && needs.relay-artifacts-domain.result == 'success' && (github.event_name == 'push' || needs.changes.outputs.rust == 'true')
354+
if: always() && (needs.changes.result != 'success' || (needs.relay-artifacts-domain.result == 'success' && (github.event_name == 'push' || needs.changes.outputs.rust == 'true')))
339355
needs: [changes, relay-artifacts-domain, relay-domain]
340356
runs-on: ubuntu-latest
341357
timeout-minutes: 5
342358
permissions: {}
343359
steps:
344360
- name: Check Backend Integration result
345361
env:
362+
SELECTION_RESULT: ${{ needs.changes.result }}
346363
RESULT: ${{ needs.relay-domain.outputs.backend_integration_result }}
347-
run: test "$RESULT" = success
364+
run: test "$SELECTION_RESULT" = success && test "$RESULT" = success
348365

349366
postgres-tests:
350367
name: PostgreSQL Tests
351-
if: always() && needs.changes.result == 'success' && needs.relay-artifacts-domain.result == 'success' && (github.event_name == 'push' || needs.changes.outputs.rust == 'true')
368+
if: always() && (needs.changes.result != 'success' || (needs.relay-artifacts-domain.result == 'success' && (github.event_name == 'push' || needs.changes.outputs.rust == 'true')))
352369
needs: [changes, relay-artifacts-domain, postgres-domain]
353370
runs-on: ubuntu-latest
354371
timeout-minutes: 5
355372
permissions: {}
356373
steps:
357374
- name: Check PostgreSQL Tests result
358375
env:
376+
SELECTION_RESULT: ${{ needs.changes.result }}
359377
RESULT: ${{ needs.postgres-domain.outputs.postgres_tests_result }}
360-
run: test "$RESULT" = success
378+
run: test "$SELECTION_RESULT" = success && test "$RESULT" = success
361379

362380
relay-e2e:
363381
name: Relay E2E
364-
if: always() && needs.changes.result == 'success' && needs.relay-artifacts-domain.result == 'success' && (github.event_name == 'push' || needs.changes.outputs.rust == 'true')
382+
if: always() && (needs.changes.result != 'success' || (needs.relay-artifacts-domain.result == 'success' && (github.event_name == 'push' || needs.changes.outputs.rust == 'true')))
365383
needs: [changes, relay-artifacts-domain, relay-domain]
366384
runs-on: ubuntu-latest
367385
timeout-minutes: 5
368386
permissions: {}
369387
steps:
370388
- name: Check Relay E2E result
371389
env:
390+
SELECTION_RESULT: ${{ needs.changes.result }}
372391
RESULT: ${{ needs.relay-domain.outputs.relay_e2e_result }}
373-
run: test "$RESULT" = success
392+
run: test "$SELECTION_RESULT" = success && test "$RESULT" = success
374393

375394
web:
376395
name: Web
377-
if: always() && needs.changes.result == 'success' && (github.event_name == 'push' || needs.changes.outputs.web == 'true')
396+
if: always() && (needs.changes.result != 'success' || ((github.event_name == 'push' || needs.changes.outputs.web == 'true')))
378397
needs: [changes, clients]
379398
runs-on: ubuntu-latest
380399
timeout-minutes: 5
381400
permissions: {}
382401
steps:
383402
- name: Check Web result
384403
env:
404+
SELECTION_RESULT: ${{ needs.changes.result }}
385405
RESULT: ${{ needs.clients.outputs.web_result }}
386-
run: test "$RESULT" = success
406+
run: test "$SELECTION_RESULT" = success && test "$RESULT" = success
387407

388408
mobile:
389409
name: Mobile
390-
if: always() && needs.changes.result == 'success' && (github.event_name == 'push' || needs.changes.outputs.mobile == 'true')
410+
if: always() && (needs.changes.result != 'success' || ((github.event_name == 'push' || needs.changes.outputs.mobile == 'true')))
391411
needs: [changes, clients]
392412
runs-on: ubuntu-latest
393413
timeout-minutes: 5
394414
permissions: {}
395415
steps:
396416
- name: Check Mobile result
397417
env:
418+
SELECTION_RESULT: ${{ needs.changes.result }}
398419
RESULT: ${{ needs.clients.outputs.mobile_result }}
399-
run: test "$RESULT" = success
420+
run: test "$SELECTION_RESULT" = success && test "$RESULT" = success
400421

401422
security:
402423
name: Security
403-
if: always() && needs.changes.result == 'success' && (github.event_name == 'push' || needs.changes.outputs.rust == 'true')
424+
if: always() && (needs.changes.result != 'success' || ((github.event_name == 'push' || needs.changes.outputs.rust == 'true')))
404425
needs: [changes, security-domain]
405426
runs-on: ubuntu-latest
406427
timeout-minutes: 5
407428
permissions: {}
408429
steps:
409430
- name: Check Security result
410431
env:
432+
SELECTION_RESULT: ${{ needs.changes.result }}
411433
RESULT: ${{ needs.security-domain.outputs.security_result }}
412-
run: test "$RESULT" = success
434+
run: test "$SELECTION_RESULT" = success && test "$RESULT" = success

‎desktop/tests/e2e/persistent-agent-audience.spec.ts‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -601,15 +601,23 @@ test("the mention button opens settings and can undo an address", async ({
601601
await expect(
602602
composer.getByRole("button", { name: "Mention someone" }),
603603
).toBeVisible();
604-
await input.fill("");
604+
await expect(
605+
composer.getByTestId(`composer-address-lock-${AGENT_A}`),
606+
).toHaveCount(0);
607+
// Clear through editor transactions before selecting a manual mention.
608+
await input.press("ControlOrMeta+A");
609+
await input.press("Backspace");
610+
await expect(input).toHaveText("");
605611

606612
await menu
607613
.getByRole("button", { name: "Mention Morgarita", exact: true })
608614
.click();
609615
await expect(input).toHaveText("@Morgarita ");
616+
// Explicitly opting out keeps subsequent mentions manual. The old avatar's
617+
// exit animation must not be mistaken for a newly pinned recipient.
610618
await expect(
611619
composer.getByTestId(`composer-address-lock-${AGENT_A}`),
612-
).toBeVisible();
620+
).toHaveCount(0);
613621

614622
await input.type("later");
615623
await input.press("Enter");

‎scripts/ci-required-gates.test.mjs‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
import assert from "node:assert/strict";
2+
import { spawnSync } from "node:child_process";
3+
import { readFileSync } from "node:fs";
4+
import { test } from "node:test";
5+
import { runInNewContext } from "node:vm";
6+
7+
const workflow = readFileSync(
8+
new URL("../.github/workflows/ci.yml", import.meta.url),
9+
"utf8",
10+
);
11+
const gates = [
12+
"rust-lint",
13+
"unit-tests",
14+
"windows-rust",
15+
"desktop",
16+
"desktop-build-macos",
17+
"desktop-e2e-relay",
18+
"desktop-e2e-integration",
19+
"backend-integration",
20+
"postgres-tests",
21+
"relay-e2e",
22+
"web",
23+
"mobile",
24+
"security",
25+
];
26+
for (const gate of gates) {
27+
const body = workflow.match(
28+
new RegExp(`^ ${gate}:\\n([\\s\\S]*?)(?=^ [\\w-]+:|$(?![\\s\\S]))`, "m"),
29+
)[1];
30+
const condition = body.match(/^ {4}if: (.+)$/m)[1];
31+
const command = body.match(/^ {8}run: (.+)$/m)[1];
32+
function shouldRun(
33+
selection,
34+
selected = false,
35+
event = "pull_request",
36+
artifacts = "skipped",
37+
) {
38+
// These workflow conditions use only booleans, string equality and grouping.
39+
// Evaluate the actual expression after substituting its GitHub context values.
40+
assert.match(
41+
condition,
42+
/\balways\(\)/,
43+
"Required wrapper must override GitHub implicit success()",
44+
);
45+
const expression = condition
46+
.replace(/always\(\)/g, "true")
47+
.replace(
48+
/github\.event_name|needs\.[\w-]+\.(?:result|outputs\.[\w-]+)/g,
49+
(key) => {
50+
if (key === "github.event_name") return JSON.stringify(event);
51+
if (key === "needs.changes.result") return JSON.stringify(selection);
52+
if (key === "needs.relay-artifacts-domain.result")
53+
return JSON.stringify(artifacts);
54+
assert.match(key, /^needs\.changes\.outputs\./);
55+
return JSON.stringify(selected ? "true" : "false");
56+
},
57+
);
58+
return runInNewContext(expression, {}, { timeout: 100 });
59+
}
60+
function check(selection, result) {
61+
assert.match(body, /SELECTION_RESULT: \$\{\{ needs.changes.result \}\}/);
62+
assert.match(body, /RESULT: \$\{\{ needs\.[\w-]+\.outputs\.[\w_]+ \}\}/);
63+
return spawnSync("bash", ["-c", command], {
64+
env: { ...process.env, SELECTION_RESULT: selection, RESULT: result },
65+
timeout: 1000,
66+
}).status;
67+
}
68+
test(`${gate}: selector failures run and fail the required check`, () => {
69+
for (const selection of ["failure", "cancelled", "skipped"]) {
70+
for (const artifacts of ["skipped", "success"]) {
71+
assert.equal(
72+
shouldRun(selection, false, "pull_request", artifacts),
73+
true,
74+
);
75+
}
76+
for (const result of ["", "skipped", "success"]) {
77+
assert.notEqual(check(selection, result), 0);
78+
}
79+
}
80+
});
81+
test(`${gate}: successful selection preserves path gating and suite results`, () => {
82+
assert.equal(shouldRun("success"), false);
83+
assert.equal(shouldRun("success", true, "pull_request", "success"), true);
84+
assert.equal(shouldRun("success", false, "push", "success"), true);
85+
assert.equal(check("success", "success"), 0);
86+
for (const result of ["", "failure", "cancelled", "skipped"]) {
87+
assert.notEqual(check("success", result), 0);
88+
}
89+
});
90+
}

0 commit comments

Comments
 (0)