You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit c90b1d9
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: .github/workflows/_test.yml
+25-9Lines changed: 25 additions & 9 deletions
Original file line number
Diff line number
Diff line change
@@ -25,9 +25,13 @@ on:
25
25
type: string
26
26
default: all
27
27
test_selection:
28
-
description: 'PR test selection (smart CI): the artifact the select-tests job uploaded; the Linux x86_64 test-unix leg runs only its tests, the arm64, macOS and Windows legs run every test. Empty = every test (dry run, release, and any PR the selection does not narrow)'
28
+
description: 'PR test selection (smart CI): the artifact the select-tests job uploaded; the Linux x86_64 test-unix leg runs only its tests. Empty = every test (dry run, release, and any PR the selection does not narrow)'
29
29
type: string
30
30
default: ''
31
+
test_selection_portable:
32
+
description: 'The test_selection artifact also holds portable/: the view of the legs the coverage is not recorded on (arm64, macOS, Windows), where every platform-specific suite runs whole; those legs run it. false = they run every test'
33
+
type: boolean
34
+
default: false
31
35
32
36
permissions:
33
37
contents: read
@@ -226,11 +230,14 @@ jobs:
226
230
ccache-test-${{ matrix.os }}-${{ matrix.cc }}-
227
231
228
232
- name: Fetch the test selection
229
-
# Only the Linux x86_64 leg narrows (user decision 2026-10-16): the
230
-
# coverage that justifies an omission comes from the Linux x86_64
231
-
# producer, so arm64, macOS and Windows code under #ifdef has static
Copy file name to clipboardExpand all lines: README.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -369,7 +369,7 @@ Use the graph to explain which tests a change reaches, and use that evidence to
369
369
370
370
**Narrowing currently supports C/C++ family code with this repository's test conventions or declared equivalents.** It is not general test selection for every language CBM indexes. Missing, stale, incompatible, or incomplete evidence broadens the selection or returns `run_all`; static reachability without admitted coverage never omits a suite. Begin with report-only shadow mode and validate your harness before enabling gating.
371
371
372
-
In this repository, test narrowing is wired to the Linux x86_64 test leg when `TEST_IMPACT_MODE=gate`; release and dry-run workflows retain full execution. See the [Smart CI guide](docs/SMART_CI.md) for supported conventions, coverage admission, setup, version requirements, and fallback behavior.
372
+
In this repository, test narrowing is on for every PR unless `TEST_IMPACT_MODE` says otherwise: the Linux x86_64 test leg runs the selection, and the arm64, macOS and Windows legs run it with every platform-specific suite whole; release and dry-run workflows retain full execution. See the [Smart CI guide](docs/SMART_CI.md) for supported conventions, coverage admission, setup, version requirements, and fallback behavior.
373
373
374
374
### Edge types (selected)
375
375
-`CALLS`: a callable is invoked at the source site
- `DIR/test-only.txt`: one token per line, either `suite` (run the whole
298
306
suite) or `suite:test`;
299
307
- `DIR/optional-suites.txt`: selected suites that are registered only under a
300
-
preprocessor condition. A build may lack them.
308
+
preprocessor condition. A build may lack them;
309
+
- `DIR/portable/` (with `portable=narrowed`): the same two files for the legs
310
+
the coverage is not recorded on, where every platform-specific suite runs
311
+
whole (see below).
301
312
302
313
Everything else means `all`: a `run_all` answer, an unknown schema or filter
303
314
encoding, a filter that disagrees with the lane's suite list, a crash, and a
@@ -338,25 +349,49 @@ spawner. A reached handler that no test names, a table entry the engine cannot
338
349
read, and a change reaching `main` any other way still run everything; the
339
350
engine note names the call chain into `main`.
340
351
341
-
**Only the legs the coverage speaks for should narrow.** Here the coverage
342
-
comes from one Linux x86_64 clang-21 producer. Only the Linux x86_64 test leg
343
-
downloads the selection (`runner.os == 'Linux' && runner.arch == 'X64'` in
344
-
`_test.yml`). The arm64, macOS and Windows legs always run everything,
345
-
because code under architecture or platform `#ifdef`s has static evidence
346
-
only there. That leg builds with gcc, not clang: code under a compiler `#if`
347
-
has static evidence only too, and `--platform` names the producer's platform,
348
-
so the engine's platform check admits the map by construction.
349
-
350
-
### (d) Flip the variable
351
-
352
-
Shadow is the default: the variable is unset, `select-tests` is skipped, and
353
-
every test runs. When the evidence is in, set the repository variable:
352
+
A role handler may start another role's child: `__cbm_rss_tree_root`starts
353
+
`__cbm_rss_tree_hold`. A handler that names another role's literal starts that
354
+
role, so a reached started role is lifted with its starter: the tests that
355
+
name the starter run, and the started role needs no test of its own.
356
+
357
+
**Platform code narrows only where the coverage speaks for it.** Here the
358
+
coverage comes from one Linux x86_64 clang-21 producer. The Linux x86_64 test
359
+
leg runs the selection as the engine answered it (`runner.os == 'Linux' &&
360
+
runner.arch == 'X64'` in `_test.yml`). The arm64, macOS and Windows legs run
361
+
its portable view (`DIR/portable/`): every platform-specific suite runs whole
362
+
there, selected or not, because code under architecture or platform `#ifdef`s
363
+
has static evidence only on those legs. A suite is platform-specific when its
364
+
test file holds a `SKIP_PLATFORM(` call or an `#if`/`#ifdef` on an OS or
365
+
architecture macro (the two markers `scripts/check-no-test-skips.sh` names for
366
+
a genuinely platform-specific test), at the base or at HEAD, or when the
367
+
runner registers it under such a conditional (`selection.py platform-suites`).
368
+
Platform-neutral suites narrow on every leg. The x86_64 leg builds with gcc,
369
+
not clang: code under a compiler `#if` has static evidence only too, and
370
+
`--platform`names the producer's platform, so the engine's platform check
371
+
admits the map by construction.
372
+
373
+
### (d) The mode switch
374
+
375
+
The expression `(vars.TEST_IMPACT_MODE || 'gate') == 'gate'` makes gate the
376
+
default: a pull request from a fork gets no repository variables, so a test of
377
+
`vars.TEST_IMPACT_MODE == 'gate'`alone never narrowed a fork PR. Never move
378
+
the job to `pull_request_target` to get the variables: that runs fork code
379
+
with a write token. Off switches:
354
380
355
381
```bash
356
-
gh variable set TEST_IMPACT_MODE --body gate # gate
357
-
gh variable delete TEST_IMPACT_MODE # back to shadow
382
+
gh variable set TEST_IMPACT_MODE --body shadow# same-repository PRs: shadow
383
+
gh variable delete TEST_IMPACT_MODE # back to the default (gate)
358
384
```
359
385
386
+
For fork PRs the off switch is the default in the expression: change `'gate'`
387
+
to `'shadow'` on main, and each PR picks it up with its next merge ref. A
388
+
repository that wants shadow until its evidence is in writes `'shadow'` there
389
+
from the start.
390
+
391
+
The selection runs the merge base's engine, `test-impact-select.sh` and
392
+
`selection.py`(`--repo` points them at the PR checkout), so a PR cannot
393
+
narrow its own tests by changing them.
394
+
360
395
Do not add `select-tests` to your required checks or to the aggregate gate.
361
396
When it breaks, PRs cost more minutes. They do not lose coverage.
362
397
@@ -439,7 +474,8 @@ map incompatible until the producer publishes a new one.
439
474
440
475
## Rollout and evidence
441
476
442
-
1. **Shadow.** Run (a) and (b). Leave `TEST_IMPACT_MODE` unset.
477
+
1. **Shadow.** Run (a) and (b) with `'shadow'` as the default in the
478
+
`select-tests`expression (d).
443
479
2. **Measure.** For each PR run, compare `selection-engine.json` with what
444
480
failed in that run's full test legs.
445
481
- A **miss** is a test that failed in the full run but was not selected.
@@ -448,8 +484,8 @@ map incompatible until the producer publishes a new one.
448
484
449
485
Nothing in this repository computes misses for Layer 2 yet. Layer 1's
450
486
history replay (`replay-selector.sh`) is the model to follow.
451
-
3. **Gate.** Set `TEST_IMPACT_MODE=gate`. Keep the shadow job running, so that
452
-
the evidence keeps coming.
487
+
3. **Gate.** Make `'gate'` the default (this repository's state). Keep the
488
+
shadow job running, so that the evidence keeps coming.
453
489
454
490
**Reading the step summary** (`### test-impact engine`):
455
491
@@ -469,7 +505,7 @@ Common run-all reasons:
469
505
| `RULE_RUN_ALL` | A `run_all` rule matched (see `run_all_rules`). |
470
506
| `UNMAPPED_FILE` | A changed file matched no rule and is not C-family source. |
471
507
| `RULE_TARGET_UNKNOWN` | A `referencing_tests` path that no test names (without `unreferenced: "ignore"`). |
472
-
| `RUNNER_REACHED` | The change reaches the test runner's `main` (or a child role none of whose literals a test names). |
508
+
| `RUNNER_REACHED` | The change reaches the test runner's `main` (or a child role none of whose literals a test names and that no role with known spawners starts). The engine note names the call chain. |
473
509
| `GIT_UNAVAILABLE` | No git, a shallow clone, or a diff that cannot be read in full. |
474
510
| `CONFIG_INVALID` / `POLICY_UNAVAILABLE` | The merge base's config or rules do not parse. |
0 commit comments