Repository navigation
Commit 556a35f
authored
test(benchmark): port the webpack/benchmark scenarios (#21935)
* test(benchmark): cover the loader pipeline, a cold cache and HMR
The benchmark suite measured webpack over synthetic graphs with no loader in
the pipeline, a filesystem cache only on its restore path, and no HMR at all —
the scenarios webpack/benchmark carried outside this repository.
Three cases close that: `loader-babel` runs babel-loader over generated JSX so
the loader-runner and the source round trip are measured; `cache-filesystem-cold`
drops the pack each iteration so the store path is measured beside
`cache-filesystem`'s restore; `hmr` builds with HotModuleReplacementPlugin,
which the rebuild scenario turns into an update round trip.
Two of them needed the harness to grow a hook. A config's `plugins` did not
survive `structuredClone` — it strips the prototype, and with it `apply` — so a
case declares them through `createPlugins(webpack)` instead, bound to the
baseline being measured; declaring them directly now throws rather than
silently building without them. A case's `options.mjs` may export `beforeEach`,
run before every iteration and outside the measured region, which is what lets
the cold case reset its pack.
`-on-schedule` cases ran nowhere: the workflow filtered them out and had no
schedule to run them on. It gets a weekly one, and an empty NEGATIVE_FILTER now
means unset instead of excluding every case. A failed task also reports the
rejection it failed with, not just its id.
* fix(benchmark): let the scheduled run actually lift the case filter
`event_name == 'schedule' && '' || 'on-schedule'` returns `on-schedule` on a
scheduled run too: an empty string is falsy in a GitHub Actions expression, so
the `||` branch always wins and the cases the schedule exists for stayed
excluded. Inverted so the truthy branch carries the non-empty value.
A case's per-iteration hook is renamed `beforeEachIteration`, matching the
parameter it feeds. Under the old name it read as tinybench's own `beforeEach`,
which is called with a mode string rather than the configuration.
Also shortens three comments to the two-line cap.
* ci: bound the benchmark jobs with a timeout
Neither job set timeout-minutes, so both inherited the 6-hour default, and the
harness has no deadline of its own: a watch bench awaits a rebuild callback that
nothing cancels. main's slowest shard runs ~55 minutes, so 120 leaves better
than twice the headroom and only trips on a genuine hang.
* test(benchmark): port the remaining webpack/benchmark scenarios
webpack/benchmark measured 21 addon scenarios; this repository covered eight of
them. Seven more land here as cases, leaving only the ones that need a toolchain
webpack does not depend on.
Four turn off an optimization webpack enables by default, so what each one costs
is measurable: no-minimize, no-concatenation, no-exports-analysis (which follows
the original in also dropping sideEffects, mangleExports and innerGraph) and, in
the other direction, module-unsafe-cache, resolve-unsafe-cache and unsafe-cache,
which widen caching past the node_modules-only default.
common-libs replaces the fixture of the same name, which pinned Material-UI 4,
Vue 2, moment and jQuery as of 2021. It builds a graph out of react, react-dom,
lodash-es and date-fns at the versions this repository already carries, so the
scenario survives without the dead packages.
Not ported: swc, thread-loader and babel preset-env each need a dependency
webpack does not have; Yarn PnP needs an install layout the harness cannot
produce; and no-cache is inexpressible here, since the watch scenario overrides
`cache` and every iteration builds a fresh compiler, so a memory cache never
carries over.
* chore: format the regenerated changelog
* fix(benchmark): fail the run when a case's setup does
The options.mjs load ignored every rejection as "benchmark has no
options.mjs", so a setup() that threw left the fixture unbuilt and the
case was still measured. Only a missing file is optional now.1 parent 7cb6dd4 commit 556a35f
33 files changed
Lines changed: 439 additions & 27 deletions
File tree
- .github/workflows
- test
- benchmarkCases
- cache-filesystem-cold
- common-libs
- hmr
- loader-babel
- module-unsafe-cache
- no-concatenation
- no-exports-analysis
- no-minimize
- resolve-unsafe-cache
- unsafe-cache
- harness/benchmark
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
8 | 12 | | |
9 | 13 | | |
10 | 14 | | |
| |||
26 | 30 | | |
27 | 31 | | |
28 | 32 | | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
29 | 36 | | |
30 | 37 | | |
31 | 38 | | |
| |||
60 | 67 | | |
61 | 68 | | |
62 | 69 | | |
63 | | - | |
| 70 | + | |
64 | 71 | | |
65 | 72 | | |
66 | 73 | | |
| |||
71 | 78 | | |
72 | 79 | | |
73 | 80 | | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
74 | 84 | | |
75 | 85 | | |
76 | 86 | | |
| |||
106 | 116 | | |
107 | 117 | | |
108 | 118 | | |
109 | | - | |
| 119 | + | |
110 | 120 | | |
111 | 121 | | |
112 | 122 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
757 | 757 | | |
758 | 758 | | |
759 | 759 | | |
760 | | - | |
761 | 760 | | |
762 | 761 | | |
763 | 762 | | |
| |||
900 | 899 | | |
901 | 900 | | |
902 | 901 | | |
903 | | - | |
904 | 902 | | |
905 | 903 | | |
906 | 904 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
294 | 294 | | |
295 | 295 | | |
296 | 296 | | |
297 | | - | |
298 | | - | |
299 | | - | |
300 | | - | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
301 | 302 | | |
302 | | - | |
303 | | - | |
304 | | - | |
305 | | - | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
306 | 306 | | |
307 | 307 | | |
308 | 308 | | |
| |||
406 | 406 | | |
407 | 407 | | |
408 | 408 | | |
409 | | - | |
410 | | - | |
411 | | - | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
412 | 415 | | |
413 | | - | |
414 | | - | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
415 | 423 | | |
416 | 424 | | |
417 | 425 | | |
| |||
463 | 471 | | |
464 | 472 | | |
465 | 473 | | |
| 474 | + | |
| 475 | + | |
466 | 476 | | |
467 | 477 | | |
468 | 478 | | |
469 | 479 | | |
470 | 480 | | |
471 | | - | |
| 481 | + | |
| 482 | + | |
| 483 | + | |
| 484 | + | |
| 485 | + | |
| 486 | + | |
| 487 | + | |
472 | 488 | | |
473 | 489 | | |
474 | 490 | | |
| |||
478 | 494 | | |
479 | 495 | | |
480 | 496 | | |
481 | | - | |
| 497 | + | |
| 498 | + | |
482 | 499 | | |
483 | 500 | | |
484 | 501 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
0 commit comments