Skip to content

Commit 432e87e

Browse files
ci: gate the expensive jobs on lint, basic and unit (#22234)
* ci: gate the expensive jobs on lint, basic and unit A pull request that is not going to merge still spent the full CI budget on it. The seven heavy suites in test.yml were held behind `basic` alone, so a red `lint` or `unit` launched all 27 integration legs; the eight benchmark shards were held behind nothing at all, and on #22227 they ran for another 67 minutes after `basic` had already gone red — around 3.3 runner-hours of it. Hold the test.yml suites behind all three cheap jobs, which are the three a merge needs anyway. benchmarks.yml becomes callable and test.yml calls it behind the same three, so a pull request reaches the shards through `needs` rather than through a job that polls: nothing is spent while the cheap suites run, and a red one skips the shards outright. A push to main keeps benchmarks.yml's own trigger and is never gated, because CodSpeed draws every later comparison against that commit's baseline. Its concurrency group is named rather than derived from `github.workflow`, which reads as the caller's name once test.yml calls it. * ci: cancel superseded pull request runs Four workflows had no concurrency group, so a new commit left the previous commit's run going: CodeQL analyses both languages, pkg.pr.new publishes a preview of code that has been replaced, and dependency-review and the labeler each run again alongside the run they supersede. Cancel them on a pull request and leave the `main` runs alone, as code-size and deep-imports already do — code scanning reads a pull request's alerts against the default branch's last analysis, and every push to main publishes its own preview, so neither is superseded by anything. The labeler is keyed on the pull request number rather than on `github.ref`, which under `pull_request_target` is the base branch — the usual group would have every open pull request cancelling the others' labelling. pr-quality only fires on opened/reopened and release.yml only on a push to a tag, so a new commit supersedes nothing there. * test(css): compare the color corpus, and group the parser jobs by language css-parsing-tests has 8366 cases and webpack compared 13 of them — the whole color half was run only to confirm no crash, with upstream's expected values discarded. They are CSSOM serializations rather than minifier output, so they cannot be compared literally; reading both the input and that serialization through webpack's own folding and canonicalizing to `#rrggbb` does compare, wherever both land in sRGB. That is 796 cases, and webpack agrees on all of them — no exception list. Where it declines to fold (lab, lch, out-of-gamut) the two spellings are not claims about the same thing, so those stay out rather than becoming noise: comparing every case reports 5549 differences, essentially all of them that. The three jobs reading an upstream parser corpus become one matrix, named for the language rather than for whose repository the fixtures live in — which had already drifted, since `html5lib` reads WPT for the corpus html5lib-tests dropped. They differed only in submodule, suite and coverage flag. * docs: name syntax-equivalence (chrome) among the coverage uploaders Its matrix row sets `coverage: true` and the job's Codecov step runs on that flag, so the list of suites to wait for was one short — and the sentence after it said everything unlisted cannot move the number. A contributor following it could read a coverage result as final while that job was still running.
1 parent 107ebef commit 432e87e

8 files changed

Lines changed: 155 additions & 101 deletions

File tree

‎.github/workflows/benchmarks.yml‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,16 +3,21 @@ name: Benchmarks
33
on:
44
push:
55
branches: [main]
6-
pull_request:
7-
branches: [main]
6+
# A pull request reaches this through `test.yml`, which holds it behind the
7+
# three cheap jobs rather than spending four runner-hours on a tree that is
8+
# not going to merge. There is no `pull_request` trigger here, or every pull
9+
# request would run these shards twice.
10+
workflow_call:
811
# The only trigger that measures the `-on-schedule` cases; every other one
912
# filters them out as too expensive to run per pull request.
1013
schedule:
1114
- cron: "0 4 * * 1"
1215
workflow_dispatch:
1316

1417
concurrency:
15-
group: ${{ github.workflow }}-${{ github.ref }}
18+
# Named rather than `github.workflow`, which reads as the caller's name when
19+
# `test.yml` calls this — sharing that group would have the run cancel itself.
20+
group: Benchmarks-${{ github.ref }}
1621
# Every push to `main` shares one group, so cancelling leaves that commit with
1722
# no run for CodSpeed to compare against — it then falls back to an older
1823
# `main`, attributing the commits in between to the pull request. Only a

‎.github/workflows/codeql.yml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,13 @@ on:
99
schedule:
1010
- cron: "0 0 * * 1"
1111

12+
concurrency:
13+
group: ${{ github.workflow }}-${{ github.ref }}
14+
# A superseded pull request analysis is work on code nobody will merge. A
15+
# push to `main` is not superseded: code scanning reads a pull request's
16+
# alerts against the default branch's last analysis, so that one must land.
17+
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
18+
1219
permissions:
1320
contents: read
1421

‎.github/workflows/dependency-review.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,11 @@ name: Dependency Review
33
on:
44
pull_request:
55

6+
concurrency:
7+
group: ${{ github.workflow }}-${{ github.ref }}
8+
# Pull requests only, so every superseded run is one nobody is waiting on.
9+
cancel-in-progress: true
10+
611
permissions:
712
contents: read
813

‎.github/workflows/labeler.yml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,13 @@ on:
44
pull_request_target:
55
types: [opened, synchronize, reopened]
66

7+
concurrency:
8+
# Keyed on the pull request rather than on `github.ref`, which under
9+
# `pull_request_target` is the base branch — every open pull request would
10+
# share one group and cancel the others' labelling.
11+
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
12+
cancel-in-progress: true
13+
714
permissions:
815
contents: read
916

‎.github/workflows/publish-to-pkg-pr-new.yml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,12 @@ on:
77
branches: [main]
88
tags: ["!**"]
99

10+
concurrency:
11+
group: ${{ github.workflow }}-${{ github.ref }}
12+
# A preview of a commit that has been replaced is one nobody will install.
13+
# Every push to `main` publishes its own preview, so those are not dropped.
14+
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
15+
1016
permissions:
1117
issues: write
1218
pull-requests: write

‎.github/workflows/test.yml‎

Lines changed: 62 additions & 90 deletions
Original file line numberDiff line numberDiff line change
@@ -225,8 +225,11 @@ jobs:
225225
env:
226226
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
227227

228+
# Every suite from here down is held behind the three cheap ones, which are
229+
# also the three a merge needs: each costs tens of runner-minutes, and a tree
230+
# that fails `lint`, `basic` or `unit` is not going to land on what they say.
228231
test262:
229-
needs: basic
232+
needs: [lint, basic, unit]
230233
strategy:
231234
fail-fast: false
232235
matrix:
@@ -277,20 +280,45 @@ jobs:
277280
env:
278281
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
279282

280-
# Parses all of test262 with webpack's parser and with acorn, and compares the
281-
# trees. Coverage instrumentation makes the parser ~5x slower, so this one job
282-
# runs without it — every line it reaches is covered by the suites that upload.
283-
test262-parser:
284-
needs: basic
283+
# One job per language, the way `runtimes` is one per runtime: each reads that
284+
# language's upstream corpus, and they differ only in which submodule they
285+
# check out and which suite they run.
286+
parser:
287+
# Named for the language alone: a matrix row carries the suite and the
288+
# coverage flag too, and the generated name would otherwise print all three.
289+
name: parser (${{ matrix.language }})
290+
needs: [lint, basic, unit]
291+
strategy:
292+
fail-fast: false
293+
matrix:
294+
include:
295+
- language: css
296+
submodules: test/external/css-parsing-tests
297+
script: cover:css-parsing
298+
flags: css-parsing
299+
# WPT is 161k files, so only this leg checks it out, and only one
300+
# commit deep — it carries the tree-construction corpus html5lib-tests
301+
# dropped.
302+
- language: html
303+
submodules: test/external/html5lib-tests test/external/wpt
304+
script: cover:html5lib
305+
flags: html5lib
306+
# Parses all of test262 with webpack's parser and with acorn and
307+
# compares the trees. Coverage instrumentation makes the parser ~5x
308+
# slower, so this leg runs without it — every line it reaches is
309+
# covered by the suites that upload.
310+
- language: js
311+
submodules: test/external/test262-cases
312+
script: test:test262-parser
285313
runs-on: ubuntu-latest
286-
# Bounded: the suite allows ten minutes per area per mode, so a stall
314+
# Bounded: the js leg allows ten minutes per area per mode, so a stall
287315
# would otherwise hold a runner for the 6h default.
288316
timeout-minutes: 60
289317
steps:
290318
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
291319

292-
- name: Check out the test262 corpus
293-
run: node tooling/retry.js git submodule update --init --depth 1 test/external/test262-cases
320+
- name: Check out the ${{ matrix.language }} corpus
321+
run: node tooling/retry.js git submodule update --init --depth 1 ${{ matrix.submodules }}
294322

295323
- name: Use Node.js
296324
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
@@ -308,51 +336,18 @@ jobs:
308336
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
309337
with:
310338
path: .jest-cache
311-
key: jest-test262-parser-${{ runner.os }}-${{ hashFiles('**/yarn.lock', '**/jest.config.js') }}-${{ github.sha }}
339+
key: jest-parser-${{ matrix.language }}-${{ runner.os }}-${{ hashFiles('**/yarn.lock', '**/jest.config.js') }}-${{ github.sha }}
312340
restore-keys: |
313-
jest-test262-parser-${{ runner.os }}-${{ hashFiles('**/yarn.lock', '**/jest.config.js') }}-
314-
jest-test262-parser-${{ runner.os }}-
341+
jest-parser-${{ matrix.language }}-${{ runner.os }}-${{ hashFiles('**/yarn.lock', '**/jest.config.js') }}-
342+
jest-parser-${{ matrix.language }}-${{ runner.os }}-
315343
316-
- run: yarn test:test262-parser --ci --cacheDirectory .jest-cache
317-
318-
html5lib:
319-
needs: basic
320-
runs-on: ubuntu-latest
321-
steps:
322-
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
323-
324-
# WPT is 161k files, so only this job checks it out, and only one commit
325-
# deep — it carries the tree-construction corpus html5lib-tests dropped.
326-
- name: Check out the html5lib and WPT corpora
327-
run: node tooling/retry.js git submodule update --init --depth 1 test/external/html5lib-tests test/external/wpt
328-
329-
- name: Use Node.js
330-
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
331-
with:
332-
node-version: lts/*
333-
cache: yarn
334-
335-
- run: node tooling/retry.js yarn --frozen-lockfile
336-
337-
- run: yarn link --frozen-lockfile || true
338-
339-
- run: yarn link webpack --frozen-lockfile
340-
341-
- name: Cache jest result
342-
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
343-
with:
344-
path: .jest-cache
345-
key: jest-html5lib-${{ runner.os }}-${{ hashFiles('**/yarn.lock', '**/jest.config.js') }}-${{ github.sha }}
346-
restore-keys: |
347-
jest-html5lib-${{ runner.os }}-${{ hashFiles('**/yarn.lock', '**/jest.config.js') }}-
348-
jest-html5lib-${{ runner.os }}-
349-
350-
- run: yarn cover:html5lib --ci --cacheDirectory .jest-cache
344+
- run: yarn ${{ matrix.script }} --ci --cacheDirectory .jest-cache
351345

352346
- name: Codecov
347+
if: matrix.flags
353348
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
354349
with:
355-
flags: html5lib
350+
flags: ${{ matrix.flags }}
356351
env:
357352
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
358353

@@ -363,7 +358,7 @@ jobs:
363358
# Named for the engine alone: a matrix row carries the script and the
364359
# coverage flag too, and the generated name would otherwise print all three.
365360
name: syntax-equivalence (${{ matrix.browser }})
366-
needs: basic
361+
needs: [lint, basic, unit]
367362
strategy:
368363
fail-fast: false
369364
matrix:
@@ -463,48 +458,9 @@ jobs:
463458
env:
464459
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
465460

466-
css-parsing:
467-
needs: basic
468-
runs-on: ubuntu-latest
469-
steps:
470-
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
471-
472-
- name: Check out the css-parsing corpus
473-
run: node tooling/retry.js git submodule update --init --depth 1 test/external/css-parsing-tests
474-
475-
- name: Use Node.js
476-
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
477-
with:
478-
node-version: lts/*
479-
cache: yarn
480-
481-
- run: node tooling/retry.js yarn --frozen-lockfile
482-
483-
- run: yarn link --frozen-lockfile || true
484-
485-
- run: yarn link webpack --frozen-lockfile
486-
487-
- name: Cache jest result
488-
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
489-
with:
490-
path: .jest-cache
491-
key: jest-css-parsing-${{ runner.os }}-${{ hashFiles('**/yarn.lock', '**/jest.config.js') }}-${{ github.sha }}
492-
restore-keys: |
493-
jest-css-parsing-${{ runner.os }}-${{ hashFiles('**/yarn.lock', '**/jest.config.js') }}-
494-
jest-css-parsing-${{ runner.os }}-
495-
496-
- run: yarn cover:css-parsing --ci --cacheDirectory .jest-cache
497-
498-
- name: Codecov
499-
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
500-
with:
501-
flags: css-parsing
502-
env:
503-
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
504-
505461
# Run webpack's test suite under Deno and Bun (spec validation suites excluded).
506462
runtimes:
507-
needs: basic
463+
needs: [lint, basic, unit]
508464
strategy:
509465
fail-fast: false
510466
matrix:
@@ -560,7 +516,7 @@ jobs:
560516
- run: yarn test:${{ matrix.runtime }} --ci
561517

562518
integration:
563-
needs: basic
519+
needs: [lint, basic, unit]
564520
strategy:
565521
fail-fast: false
566522
matrix:
@@ -681,3 +637,19 @@ jobs:
681637
env:
682638
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
683639
if: matrix.node-version != '10.x' && matrix.node-version != '12.x' && matrix.node-version != '14.x' && matrix.node-version != '16.x' && matrix.node-version != '18.x' && matrix.node-version != '20.x'
640+
641+
# The eight benchmark shards are around four runner-hours, so on a pull
642+
# request they come through here, behind the same three cheap jobs. A push to
643+
# `main` runs `benchmarks.yml` on its own trigger instead and is never gated:
644+
# CodSpeed draws every later comparison against that commit's baseline.
645+
benchmarks:
646+
if: github.event_name == 'pull_request'
647+
needs: [lint, basic, unit]
648+
# At least what the called workflow declares, or the call is rejected.
649+
permissions:
650+
contents: read
651+
id-token: write
652+
issues: write
653+
pull-requests: write
654+
uses: ./.github/workflows/benchmarks.yml
655+
secrets: inherit

0 commit comments

Comments
 (0)