Skip to content

Commit 6dd5287

Browse files
authored
ci: refuse a workflow file on a gke- or gkp- label that the runner-group mirror does not list (#633)
Adds a `runner-allowlist` preflight stage: it refuses a workflow file whose jobs run on a gke- or gkp- label when the committed runner-group mirror `.github/runner-group-allowlist.txt` does not list that file (or when the mirror is malformed, unsorted, duplicated, for another repository or not on refs/heads/main). Moving jobs onto gke- labels without the org runner-group entry queued them for hours with no runner; this makes CI catch it. - `scripts/ci_preflight.py`: the new stage (fast and github-only tiers, parallel group), using the existing runs-on readers. - `scripts/test_ci_preflight.py`: its tests. - `.github/runner-group-allowlist.txt`: the mirror. - `docs/ci.md`: the stage list and one added sentence; the page keeps its line count. Proven by a lab preflight on the branch head (60 stages, the new stage run against the real workflows and mirror). Lands after the open 1.8.0 batch merges; main is merged in first. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
1 parent 62a0b70 commit 6dd5287

4 files changed

Lines changed: 658 additions & 14 deletions

File tree

‎.github/runner-group-allowlist.txt‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# Group 14 (gke-runners), live repository entries supplied by the supervisor, 2026-10-28 17:50.
2+
pineforge-4pass/pineforge-engine/.github/workflows/ci-post.yml@refs/heads/main
3+
pineforge-4pass/pineforge-engine/.github/workflows/ci.yml@refs/heads/main
4+
pineforge-4pass/pineforge-engine/.github/workflows/corpus-parity-subset.yml@refs/heads/main
5+
pineforge-4pass/pineforge-engine/.github/workflows/corpus-parity.yml@refs/heads/main
6+
pineforge-4pass/pineforge-engine/.github/workflows/docs-post.yml@refs/heads/main
7+
pineforge-4pass/pineforge-engine/.github/workflows/gke-pr.yml@refs/heads/main
8+
pineforge-4pass/pineforge-engine/.github/workflows/native-live.yml@refs/heads/main
9+
pineforge-4pass/pineforge-engine/.github/workflows/promote-baseline.yml@refs/heads/main
10+
pineforge-4pass/pineforge-engine/.github/workflows/release-ruleset-probe.yml@refs/heads/main

‎docs/ci.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ The classifier uses exact `.github/` and `docs/` prefixes, so lookalike
3838
directories and the classifier itself cannot create a skip.
3939

4040
`--tier github-only` runs `shellcheck-version`, `actionlint-runner-labels`,
41-
`workflow-lint`, `ci-workflow-contract`, `docs-workflow-contract` and
41+
`runner-allowlist`, `workflow-lint`, `ci-workflow-contract`, `docs-workflow-contract` and
4242
`docs-workflow-contract-tests`, plus the usual actionlint version check.
4343
It excludes source guards and the general preflight test suite. A failed
4444
classification selects the ordinary fast tier and fails the aggregate.
@@ -49,7 +49,7 @@ also avoids the Doxygen build.
4949
`unittest-shards-tests` and `run-json-harness-tests`; `preflight-slow` runs
5050
exactly those stages with `--tier slow`. Toolchain setup and the hash-pinned
5151
pytest install stay in the slow job. A plain local invocation below retains
52-
all stages and its existing event-aware verifier-class selection. `preflight-tests` runs every discovered method across up to eight shards, sized to available CPUs, with separate logs in `build-ci-preflight/preflight-tests-shards/`.
52+
all stages and its existing event-aware verifier-class selection. `runner-allowlist` independently scans workflow runner labels against the committed mirror in the fast tier's parallel group; `preflight-tests` runs every discovered method across up to eight shards, sized to available CPUs, with separate logs in `build-ci-preflight/preflight-tests-shards/`.
5353

5454
The smoke configures Release, builds `pineforge_pr_smoke` and directly runs
5555
`ctest --test-dir build -L '^fast$' --output-on-failure --no-tests=error`.
@@ -94,7 +94,7 @@ Python refuse a system-wide install (PEP 668), and `.venv/` is ignored. The
9494
and fails, never skips, without it.
9595

9696
Actionlint only accepts declared custom self-hosted runner labels; preflight
97-
checks the declarations in `.github/actionlint.yaml`.
97+
checks the declarations in `.github/actionlint.yaml`. Before a workflow file moves onto a gke- or gkp- label, add its line to `.github/runner-group-allowlist.txt` and ask for the org runner-group entry; the merge waits for both.
9898

9999
ORG-ACTIONPINS is enforced by preflight's `ci-workflow-contract`: every
100100
third-party `uses:` in `.github/workflows/*.yml` (also `.yaml`) and nested

‎scripts/ci_preflight.py‎

Lines changed: 238 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,10 @@
4141
from run_unittest_shards import default_shard_count
4242

4343
ACTIONLINT_VERSION = '1.7.12'
44+
# File-only policy needs an identity even in an exported tree with no git remote.
45+
# --repository overrides this canonical repository; the mirror test pins both.
46+
RUNNER_ALLOWLIST_REPOSITORY = 'pineforge-4pass/pineforge-engine'
47+
RUNNER_ALLOWLIST_FILE = '.github/runner-group-allowlist.txt'
4448
# Pin the exact ordered names extracted from the CMake slow-label list.
4549
# Verifier and command-after slices inherit slow labels; their covers do not.
4650
PR_SLOW_ROWS = (
@@ -503,7 +507,7 @@ def post_build_name(count: int) -> str:
503507
'verifier-tests', 'kernel-residuals-tests', 'unittest-shards-tests', 'run-json-harness-tests',
504508
))
505509
GITHUB_ONLY_PREFLIGHT_STAGES = frozenset((
506-
'shellcheck-version', 'actionlint-runner-labels', 'workflow-lint',
510+
'shellcheck-version', 'actionlint-runner-labels', 'runner-allowlist', 'workflow-lint',
507511
'ci-workflow-contract', 'docs-workflow-contract', 'docs-workflow-contract-tests',
508512
))
509513
ADMIT_RULE_STAGES = (
@@ -2223,8 +2227,10 @@ def _matrix_rows(body: str) -> list[dict[str, str]] | None:
22232227
if key in ('include', 'exclude'):
22242228
rows = []
22252229
for line in children:
2226-
item = re.fullmatch(r' - ([A-Za-z_]\w*): ([^{}\[\]]+)', line)
2227-
follow = re.fullmatch(r' ([A-Za-z_]\w*): ([^{}\[\]]+)', line)
2230+
# release.yml includes a literal runner label array. Keep its
2231+
# value intact for the same runner reader used for runs-on.
2232+
item = re.fullmatch(r' - ([A-Za-z_]\w*): ([^{}]+)', line)
2233+
follow = re.fullmatch(r' ([A-Za-z_]\w*): ([^{}]+)', line)
22282234
if item:
22292235
rows.append({item.group(1): item.group(2).strip().strip('\'"')})
22302236
elif follow and rows and follow.group(1) not in rows[-1]:
@@ -2251,7 +2257,9 @@ def _matrix_rows(body: str) -> list[dict[str, str]] | None:
22512257
elif not values or _indent(line) != 12:
22522258
return None
22532259
else:
2254-
values[-1] += ' ' + line.strip()
2260+
# Retain field boundaries for matrix.platform.runner and
2261+
# other dotted references read through _job_values.
2262+
values[-1] += '\n' + line.strip()
22552263
if not values or any('${{' in part for part in values):
22562264
return None
22572265
axes[key] = values
@@ -2274,6 +2282,196 @@ def _matrix_rows(body: str) -> list[dict[str, str]] | None:
22742282
return rows
22752283

22762284

2285+
def _resolved_runner_values(workflow: str, body: str) -> tuple[list[str], bool]:
2286+
"""Resolve static runner possibilities with the existing job/matrix readers.
2287+
2288+
This is deliberately not a general Actions expression evaluator. Unknown
2289+
output/vars/input references fail closed in the allowlist stage. Conditions
2290+
may select either literal branch; job ifs never excuse group membership.
2291+
"""
2292+
def key_values(text: str, key: str) -> list[str]:
2293+
block = _job_block(text, key)[1:]
2294+
if block and all(line.strip().startswith('- ') for line in block):
2295+
return [line.strip()[2:] for line in block]
2296+
return _job_values(text, key)
2297+
2298+
def combine(values: list[str], stack: tuple[str, ...]) -> list[str] | None:
2299+
resolved = [resolve(value, stack) for value in values]
2300+
return (None if any(value is None for value in resolved)
2301+
else [label for value in resolved for label in value])
2302+
2303+
def interpolate(value: str, stack: tuple[str, ...]) -> list[str] | None:
2304+
value = value.strip()
2305+
if _ACTION_YAML_QUOTED.fullmatch(value):
2306+
value = value[1:-1]
2307+
possibilities, end = [''], 0
2308+
for match in re.finditer(r'\$\{\{(.*?)\}\}', value):
2309+
choices = expression(match.group(1), stack)
2310+
if choices is None or len(possibilities) * len(choices) > 256:
2311+
return None
2312+
possibilities = [prefix + value[end:match.start()] + choice
2313+
for prefix in possibilities for choice in choices]
2314+
end = match.end()
2315+
return [prefix + value[end:] for prefix in possibilities]
2316+
2317+
def context_values(values: list[str], stack: tuple[str, ...]) -> list[str] | None:
2318+
# Preserve JSON strings until fromJSON explicitly decodes them.
2319+
expanded = [interpolate(value, stack) for value in values]
2320+
return (None if any(value is None for value in expanded)
2321+
else [item for value in expanded for item in value])
2322+
2323+
matrix: list[dict[str, str]] | None = None
2324+
matrix_read = False
2325+
2326+
def expression(value: str, stack: tuple[str, ...]) -> list[str] | None:
2327+
nonlocal matrix, matrix_read
2328+
value = value.strip()
2329+
if value in stack or len(stack) >= 16:
2330+
return None
2331+
stack = (*stack, value)
2332+
if _ACTION_YAML_QUOTED.fullmatch(value):
2333+
return [value[1:-1].replace("''", "'")]
2334+
context = re.fullmatch(r'(matrix|env)\.([A-Za-z_]\w*(?:\.[A-Za-z_]\w*)*)', value)
2335+
if context:
2336+
kind, key = context.groups()
2337+
if kind == 'matrix':
2338+
axis, *fields = key.split('.')
2339+
if not matrix_read:
2340+
matrix, matrix_read = _matrix_rows(body), True
2341+
if matrix is None or any(axis not in row for row in matrix):
2342+
return None
2343+
values = [row[axis] for row in matrix]
2344+
for field in fields:
2345+
nested = [_job_values(item, field) for item in values]
2346+
if any(len(items) != 1 for items in nested):
2347+
return None
2348+
values = [items[0] for items in nested]
2349+
return context_values(values, stack)
2350+
for owner in (body, workflow):
2351+
environment = '\n'.join(_job_block(owner, 'env')[1:])
2352+
values = _job_values(environment, key)
2353+
if values:
2354+
return context_values(values, stack) if len(values) == 1 else None
2355+
return None
2356+
decoded = re.fullmatch(r'fromJSON\((.*)\)', value, re.IGNORECASE)
2357+
if decoded:
2358+
encoded = expression(decoded.group(1), stack)
2359+
if encoded is None:
2360+
return None
2361+
values = []
2362+
for item in encoded:
2363+
try:
2364+
parsed = json.loads(item)
2365+
except ValueError:
2366+
return None
2367+
if isinstance(parsed, str):
2368+
values.append(parsed)
2369+
elif isinstance(parsed, list) and all(isinstance(label, str) for label in parsed):
2370+
values.extend(parsed)
2371+
else:
2372+
return None
2373+
return values
2374+
branches = re.fullmatch(r'[^&|]+\s*&&\s*(.+?)\s*\|\|\s*(.+)', value)
2375+
if branches:
2376+
left = expression(branches.group(1), stack)
2377+
right = expression(branches.group(2), stack)
2378+
return None if left is None or right is None else left + right
2379+
return None
2380+
2381+
def resolve(value: str, stack: tuple[str, ...]) -> list[str] | None:
2382+
value = value.strip()
2383+
if _ACTION_YAML_QUOTED.fullmatch(value):
2384+
value = value[1:-1]
2385+
if value.startswith('['):
2386+
parsed = _runner_value(value)
2387+
parts = parsed if isinstance(parsed, list) else _admit_flow_parts(value)
2388+
return None if parts is None else combine(parts, stack)
2389+
matches = list(re.finditer(r'\$\{\{(.*?)\}\}', value))
2390+
if matches:
2391+
possibilities = interpolate(value, stack)
2392+
return None if possibilities is None else combine(possibilities, stack)
2393+
return [value] if re.fullmatch(r'[A-Za-z0-9_.-]+', value) else None
2394+
2395+
runners = key_values(body, 'runs-on')
2396+
if not runners:
2397+
return [], bool(_job_values(body, 'uses'))
2398+
mapping = '\n'.join(_job_block(body, 'runs-on')[1:])
2399+
if mapping and _job_values(mapping, 'labels'):
2400+
runners = key_values(mapping, 'labels')
2401+
resolved = combine(runners, ())
2402+
# Keep visible labels in an unsupported expression in its missing-entry
2403+
# diagnostic too; the separate unresolved finding still refuses the job.
2404+
visible = [label for value in runners for label in _RUNNER_GROUP_LABEL.findall(value)]
2405+
return list(dict.fromkeys((resolved or []) + visible)), resolved is not None
2406+
2407+
2408+
_RUNNER_GROUP_LABEL = re.compile(
2409+
r'(?<![\w.-])(?:gke|gkp)-\d+cpu-\d+gb-[A-Za-z0-9_]+(?![\w.-])', re.IGNORECASE)
2410+
_REPOSITORY_ID = r'[A-Za-z0-9][A-Za-z0-9_.-]*/[A-Za-z0-9][A-Za-z0-9_.-]*'
2411+
_RUNNER_GROUP_REFERENCE = re.compile(
2412+
r'(' + _REPOSITORY_ID + r')/\.github/workflows/([A-Za-z0-9_][A-Za-z0-9_.-]*\.yml)@([^\s@]+)')
2413+
2414+
2415+
def runner_allowlist_findings(workflows: dict[str, str], mirror_text: str | None,
2416+
repository: str) -> list[str]:
2417+
"""Bind every GKE/GKP job's own workflow reference to the committed mirror."""
2418+
mirror = RUNNER_ALLOWLIST_FILE
2419+
if not re.fullmatch(_REPOSITORY_ID, repository):
2420+
return [f'{mirror}: cannot derive repository identity (expected owner/repo)']
2421+
if mirror_text is None:
2422+
return [f'{mirror}: mirror file is missing']
2423+
findings = []
2424+
lines = [line for line in mirror_text.splitlines()
2425+
if line.strip() and not line.lstrip().startswith('#')]
2426+
entries: dict[str, str] = {}
2427+
seen = set()
2428+
for line in lines:
2429+
if line in seen:
2430+
findings.append(f'{mirror}: duplicate mirror line "{line}"')
2431+
seen.add(line)
2432+
match = _RUNNER_GROUP_REFERENCE.fullmatch(line)
2433+
if not match:
2434+
findings.append(f'{mirror}: malformed mirror line "{line}"')
2435+
continue
2436+
owner_repo, name, ref = match.groups()
2437+
if ref != 'refs/heads/main':
2438+
findings.append(f'{mirror}: ref must be refs/heads/main: "{line}"')
2439+
continue
2440+
if owner_repo != repository:
2441+
findings.append(f'{mirror}: line is for another repository "{line}"')
2442+
continue
2443+
entries[line] = name
2444+
if lines != sorted(lines):
2445+
findings.append(f'{mirror}: mirror entries must be sorted')
2446+
labels_by_file: dict[str, set[str]] = {}
2447+
unresolved_files: set[str] = set()
2448+
for name, workflow in sorted(workflows.items()):
2449+
workflow = _universal_newlines(workflow)
2450+
labels: set[str] = set()
2451+
if _unparsed(workflow):
2452+
findings.append(f'{name}: cannot parse jobs for runner allowlist')
2453+
unresolved_files.add(name)
2454+
for job, body in _jobs(workflow).items():
2455+
values, resolved = _resolved_runner_values(workflow, body)
2456+
labels.update(label for value in values for label in _RUNNER_GROUP_LABEL.findall(value))
2457+
if not resolved:
2458+
unresolved_files.add(name)
2459+
findings.append(f'{name}: job {job} runs-on cannot be resolved from literal '
2460+
'labels, env or a static matrix')
2461+
labels_by_file[name] = labels
2462+
reference = f'{repository}/.github/workflows/{name}@refs/heads/main'
2463+
if reference not in entries:
2464+
findings.extend(f'{name}: label "{label}" requires mirror line "{reference}"; '
2465+
'the org runner group must contain it before merge'
2466+
for label in sorted(labels))
2467+
for reference, name in sorted(entries.items()):
2468+
if name not in workflows:
2469+
findings.append(f'advisory: {name}: mirror line "{reference}" names a missing workflow file')
2470+
elif not labels_by_file.get(name) and name not in unresolved_files:
2471+
findings.append(f'advisory: {name}: mirror line "{reference}" has no gke/gkp job')
2472+
return findings
2473+
2474+
22772475
def _input_condition_skips(body: str, inputs: dict[str, str]) -> bool:
22782476
"""Prune only a provably false literal input comparison in an AND condition."""
22792477
conditions = _job_values(body, 'if')
@@ -2394,7 +2592,8 @@ def runner_findings(workflows: dict[str, str]) -> list[str]:
23942592
for job, body in jobs.items():
23952593
runs_on, uses = _job_values(body, 'runs-on'), _job_values(body, 'uses')
23962594
gke_pin = GKE_JOB_RUNNERS.get(name, {}).get(job)
2397-
has_gke = bool(_GKE_ROUTE.search(' '.join(runs_on)))
2595+
resolved_runners, _ = _resolved_runner_values(workflow, body)
2596+
has_gke = bool(_GKE_ROUTE.search(' '.join(runs_on + resolved_runners)))
23982597
raw_runners = _job_values(body, 'runs-on', unquote=False)
23992598
if has_gke and (len(raw_runners) != 1
24002599
or not _GKE_SCALE_LABEL.fullmatch(raw_runners[0])):
@@ -3724,6 +3923,10 @@ def check_commands(source: Path, output: Path | None = None,
37243923
('actionlint-runner-labels',
37253924
[sys.executable, str(source / 'scripts/ci_preflight.py'),
37263925
'--check-actionlint-runner-labels']),
3926+
CheckCommand('runner-allowlist',
3927+
[sys.executable, str(source / 'scripts/ci_preflight.py'),
3928+
'--check-runner-allowlist'],
3929+
parallel=True),
37273930
# Require the config explicitly: absent/malformed config and undeclared
37283931
# literal self-hosted labels are actionlint findings, including in new
37293932
# workflows. A missing workflows directory must not lint another tree.
@@ -3999,6 +4202,9 @@ def main() -> int:
39994202
parser.add_argument('--self-test-docs-workflow', action='store_true')
40004203
parser.add_argument('--check-ci-workflow', action='store_true')
40014204
parser.add_argument('--check-actionlint-runner-labels', action='store_true')
4205+
parser.add_argument('--check-runner-allowlist', action='store_true')
4206+
parser.add_argument('--repository', default=RUNNER_ALLOWLIST_REPOSITORY,
4207+
help='owner/repo identity for the file-only runner-group mirror check')
40024208
parser.add_argument('--rules-only', action='store_true',
40034209
help='run the data-only admission rules over a fetched .github tree')
40044210
parser.add_argument('--workflows-dir', type=Path,
@@ -4021,6 +4227,33 @@ def main() -> int:
40214227
if not findings:
40224228
print('actionlint runner labels: all literal self-hosted labels declared')
40234229
return 1 if findings else 0
4230+
if args.check_runner_allowlist:
4231+
findings = []
4232+
try:
4233+
mirror_text = read_ci_text(ROOT / RUNNER_ALLOWLIST_FILE, findings)
4234+
except FileNotFoundError:
4235+
mirror_text = None
4236+
except OSError as error:
4237+
print(f'{RUNNER_ALLOWLIST_FILE}: cannot read mirror: {error}')
4238+
return 1
4239+
workflows = {}
4240+
try:
4241+
directory = ROOT / '.github/workflows'
4242+
if not directory.is_dir():
4243+
raise FileNotFoundError('workflow directory is missing')
4244+
for path in sorted(directory.iterdir()):
4245+
if path.suffix in ('.yml', '.yaml'):
4246+
workflows[path.name] = read_ci_text(path, findings)
4247+
except OSError as error:
4248+
print(f'.github/workflows: cannot read workflow files: {error}')
4249+
return 1
4250+
findings += runner_allowlist_findings(workflows, mirror_text, args.repository)
4251+
for finding in findings:
4252+
print(finding)
4253+
failed = any(not finding.startswith('advisory:') for finding in findings)
4254+
if not failed:
4255+
print('runner allowlist: all gke/gkp workflow references declared')
4256+
return 1 if failed else 0
40244257
if args.check_ci_workflow:
40254258
workflows = ROOT / '.github/workflows'
40264259
named = ('ci.yml', 'native-live.yml', 'promote-baseline.yml', 'corpus-parity.yml',

0 commit comments

Comments
 (0)