Skip to content

Commit 5a07869

Browse files
alexander-akaitalexander-akait
authored
test: add perf cases
1 parent 9ad355b commit 5a07869

25 files changed

Lines changed: 2233 additions & 0 deletions

File tree

‎.github/workflows/benchmarks.yml‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
name: Benchmarks
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
branches: [main]
8+
workflow_dispatch:
9+
10+
concurrency:
11+
group: ${{ github.workflow }}-${{ github.ref }}
12+
cancel-in-progress: true
13+
14+
permissions:
15+
contents: read
16+
id-token: write # Required for OIDC authentication with CodSpeed
17+
18+
jobs:
19+
benchmark:
20+
runs-on: ubuntu-latest
21+
permissions:
22+
pull-requests: write
23+
steps:
24+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
25+
with:
26+
fetch-depth: 0
27+
28+
- name: Use Node.js
29+
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
30+
with:
31+
node-version: lts/*
32+
cache: npm
33+
34+
- run: npm ci
35+
36+
- name: Run benchmarks
37+
uses: CodSpeedHQ/action@db35df748deb45fdef0960669f57d627c1956c30 # v4.13.1
38+
with:
39+
run: npm run benchmark
40+
mode: "simulation"

‎benchmark/README.md‎

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
# Benchmarks
2+
3+
Performance benchmarks for `webpack-sources`, tracked over time via
4+
[CodSpeed](https://codspeed.io/).
5+
6+
Runner stack: [tinybench](https://github.com/tinylibs/tinybench) +
7+
[`@codspeed/core`](https://www.npmjs.com/package/@codspeed/core) with a local
8+
`withCodSpeed()` wrapper ported from webpack's
9+
`test/BenchmarkTestCases.benchmark.mjs` (also used by `enhanced-resolve`).
10+
Locally it falls back to plain tinybench wall-clock measurements, and under
11+
`CodSpeedHQ/action` in CI it automatically switches to CodSpeed's
12+
instrumentation mode.
13+
14+
## Running locally
15+
16+
```sh
17+
npm run benchmark
18+
```
19+
20+
Optional substring filter to run only matching cases:
21+
22+
```sh
23+
npm run benchmark -- replace-source
24+
BENCH_FILTER=source-map npm run benchmark
25+
```
26+
27+
Locally the runner uses tinybench's wall-clock measurements and prints a
28+
table of ops/s, mean, p99, and relative margin of error per task. Under CI,
29+
the wrapper detects the CodSpeed runner environment and switches to
30+
instruction-counting mode automatically.
31+
32+
The V8 flags in `package.json` (`--no-opt --predictable --hash-seed=1` etc.)
33+
are required by CodSpeed's instrumentation mode for deterministic results —
34+
do not drop them.
35+
36+
### Optional: running real instruction counts locally
37+
38+
If you want to reproduce CI's exact instrument-count numbers on your own
39+
machine (Linux only — the underlying Valgrind tooling has no macOS backend),
40+
install the standalone CodSpeed CLI and wrap `npm run benchmark` with it:
41+
42+
```sh
43+
curl -fsSL https://codspeed.io/install.sh | bash
44+
codspeed run npm run benchmark
45+
```
46+
47+
This is only useful if you want to debug an instruction-count regression
48+
outside CI. Day-to-day benchmark iteration should use `npm run benchmark`
49+
directly (wall-clock mode).
50+
51+
## Layout
52+
53+
```
54+
benchmark/
55+
├── run.mjs # entry point: discovers cases, runs bench
56+
├── with-codspeed.mjs # CodSpeed <-> tinybench bridge
57+
├── fixtures.mjs # shared fixture loaders
58+
└── cases/
59+
└── <case-name>/
60+
├── index.bench.mjs # default export: register(bench, ctx)
61+
└── fixture/ # optional: per-case input files
62+
```
63+
64+
Each case directory must contain `index.bench.mjs` exporting a default
65+
function with the signature:
66+
67+
```js
68+
export default function register(bench, { caseName, caseDir, fixtureDir }) {
69+
bench.add("my case: descriptive name", () => {
70+
// ... code to measure ...
71+
});
72+
}
73+
```
74+
75+
`fixtureDir` is the absolute path to the case's `fixture/` subdirectory
76+
(which may or may not exist). `caseDir` is the parent directory containing
77+
`index.bench.mjs`.
78+
79+
Each task body should loop over a small batch of operations rather than
80+
performing a single call — tinybench decides its own iteration count, so
81+
we want the measurement to reflect per-batch throughput, which is more
82+
stable than per-call timing for sub-microsecond work.
83+
84+
## Existing cases
85+
86+
### Per source class
87+
88+
| Case | What it measures |
89+
| ------------------- | -------------------------------------------------------------------------------- |
90+
| `raw-source` | `RawSource` constructor, string/buffer accessors, streamChunks, updateHash |
91+
| `original-source` | `OriginalSource` map/sourceAndMap/streamChunks across columns on/off combos |
92+
| `replace-source` | `ReplaceSource` source/map/streamChunks for no, few, and many replacements |
93+
| `concat-source` | `ConcatSource` _optimize, source/buffer/map, nested flattening, hash |
94+
| `prefix-source` | `PrefixSource` delegation + newline prefix rewriting |
95+
| `source-map-source` | `SourceMapSource` full + lines-only streamChunks, including combined inner maps |
96+
| `cached-source` | `CachedSource` cold vs warm, plus `getCachedData()` round-trip |
97+
| `compat-source` | `CompatSource` delegation vs `Source.prototype` fallback paths |
98+
| `size-only-source` | `SizeOnlySource` constructor, `size()`, and the throw paths for other accessors |
99+
100+
### Per helper module
101+
102+
| Case | What it measures |
103+
| ------------------------------------- | -------------------------------------------------------------------------- |
104+
| `helpers-split-into-lines` | `splitIntoLines` scanner on fixture / big / long-line / empty inputs |
105+
| `helpers-split-into-potential-tokens` | `splitIntoPotentialTokens` scanner used by column-aware OriginalSource |
106+
| `helpers-get-generated-source-info` | `getGeneratedSourceInfo` final-line/column probe on various input shapes |
107+
| `helpers-read-mappings` | VLQ decoder used by every source-map aware streamChunks path |
108+
| `helpers-create-mappings-serializer` | VLQ encoder (full + lines-only) fed a representative event stream |
109+
| `helpers-string-buffer-utils` | `internString` and enter/exitStringInterningRange |
110+
111+
### End-to-end
112+
113+
| Case | What it measures |
114+
| --------------------------------- | ----------------------------------------------------------------------------- |
115+
| `realistic-source-map-pipeline` | OriginalSource -> ReplaceSource -> ConcatSource -> CachedSource (cold + warm) |
116+
117+
## Adding a new case
118+
119+
1. Create `benchmark/cases/<case-name>/index.bench.mjs`.
120+
2. Export a default `register(bench, ctx)` function. Call `bench.add(name, fn)`
121+
for each task.
122+
3. If the case needs input files, place them under
123+
`benchmark/cases/<case-name>/fixture/` and read them from `ctx.fixtureDir`.
124+
4. Run `npm run benchmark -- <case-name>` to verify locally.
125+
126+
Each task name should start with the case directory name (e.g.
127+
`raw-source: source()`) so CodSpeed's report groups tasks by module.

‎benchmark/cases/cached-source/index.bench.mjs‎

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
/*
2+
* cached-source
3+
*
4+
* Two axes matter for CachedSource: cold vs warm (first call vs repeat call
5+
* on the same instance), and the cache-data round-trip that lets webpack
6+
* store cached state to disk via getCachedData() / re-hydrate.
7+
*/
8+
9+
import { createHash } from "crypto";
10+
import sources from "../../../lib/index.js";
11+
import { fixtureCode, fixtureMap, noop } from "../../fixtures.mjs";
12+
13+
/**
14+
* A CachedSource with all the common caches already populated. Reused
15+
* across tasks that explicitly measure the warm path.
16+
*/
17+
const warmed = (() => {
18+
const cached = new sources.CachedSource(
19+
new sources.SourceMapSource(fixtureCode, "fixture.js", fixtureMap),
20+
);
21+
cached.source();
22+
cached.map({});
23+
cached.sourceAndMap({});
24+
cached.buffer();
25+
cached.size();
26+
return cached;
27+
})();
28+
29+
/**
30+
* @param {import("tinybench").Bench} bench bench
31+
*/
32+
export default function register(bench) {
33+
bench.add("cached-source: new CachedSource()", () => {
34+
for (let i = 0; i < 100; i++) {
35+
new sources.CachedSource(new sources.RawSource(fixtureCode));
36+
}
37+
});
38+
39+
bench.add("cached-source: source() (cold)", () => {
40+
for (let i = 0; i < 50; i++) {
41+
new sources.CachedSource(new sources.RawSource(fixtureCode)).source();
42+
}
43+
});
44+
45+
bench.add("cached-source: source() (cached)", () => {
46+
for (let i = 0; i < 500; i++) warmed.source();
47+
});
48+
49+
bench.add("cached-source: buffer() (cached)", () => {
50+
for (let i = 0; i < 500; i++) warmed.buffer();
51+
});
52+
53+
bench.add("cached-source: size() (cached)", () => {
54+
for (let i = 0; i < 500; i++) warmed.size();
55+
});
56+
57+
bench.add("cached-source: map() (cold SourceMapSource)", () => {
58+
for (let i = 0; i < 10; i++) {
59+
new sources.CachedSource(
60+
new sources.SourceMapSource(fixtureCode, "fixture.js", fixtureMap),
61+
).map({});
62+
}
63+
});
64+
65+
bench.add("cached-source: map() (cached)", () => {
66+
for (let i = 0; i < 500; i++) warmed.map({});
67+
});
68+
69+
bench.add("cached-source: sourceAndMap() (cold)", () => {
70+
for (let i = 0; i < 10; i++) {
71+
new sources.CachedSource(
72+
new sources.OriginalSource(fixtureCode, "fixture.js"),
73+
).sourceAndMap({});
74+
}
75+
});
76+
77+
bench.add("cached-source: sourceAndMap() (cached)", () => {
78+
for (let i = 0; i < 500; i++) warmed.sourceAndMap({});
79+
});
80+
81+
bench.add("cached-source: streamChunks() (cold)", () => {
82+
for (let i = 0; i < 5; i++) {
83+
new sources.CachedSource(
84+
new sources.OriginalSource(fixtureCode, "fixture.js"),
85+
).streamChunks({}, noop, noop, noop);
86+
}
87+
});
88+
89+
bench.add("cached-source: streamChunks() (warm)", () => {
90+
for (let i = 0; i < 5; i++) {
91+
warmed.streamChunks({}, noop, noop, noop);
92+
}
93+
});
94+
95+
bench.add("cached-source: originalLazy()", () => {
96+
const lazy = new sources.CachedSource(
97+
() => new sources.RawSource(fixtureCode),
98+
);
99+
for (let i = 0; i < 500; i++) lazy.originalLazy();
100+
});
101+
102+
bench.add("cached-source: getCachedData() then restore", () => {
103+
for (let i = 0; i < 10; i++) {
104+
const a = new sources.CachedSource(
105+
new sources.OriginalSource(fixtureCode, "fixture.js"),
106+
);
107+
a.source();
108+
a.map({});
109+
const data = a.getCachedData();
110+
new sources.CachedSource(
111+
new sources.OriginalSource(fixtureCode, "fixture.js"),
112+
data,
113+
);
114+
}
115+
});
116+
117+
bench.add("cached-source: updateHash() (warm)", () => {
118+
for (let i = 0; i < 50; i++) warmed.updateHash(createHash("sha256"));
119+
});
120+
}

‎benchmark/cases/compat-source/index.bench.mjs‎

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
/*
2+
* compat-source
3+
*
4+
* CompatSource wraps a SourceLike. Interesting paths: the direct delegation
5+
* (when the wrapped object provides buffer/size/map/updateHash) vs the
6+
* Source.prototype fallback (when it doesn't).
7+
*/
8+
9+
import { createHash } from "crypto";
10+
import sources from "../../../lib/index.js";
11+
import { fixtureCode } from "../../fixtures.mjs";
12+
13+
const fixtureBuffer = Buffer.from(fixtureCode, "utf8");
14+
15+
const sourceLike = {
16+
source: () => fixtureCode,
17+
buffer: () => fixtureBuffer,
18+
};
19+
20+
const richSourceLike = {
21+
source: () => fixtureCode,
22+
buffer: () => fixtureBuffer,
23+
size: () => fixtureCode.length,
24+
map: () => null,
25+
sourceAndMap: () => ({ source: fixtureCode, map: null }),
26+
updateHash: (hash) => {
27+
hash.update(fixtureCode);
28+
},
29+
};
30+
31+
/**
32+
* @param {import("tinybench").Bench} bench bench
33+
*/
34+
export default function register(bench) {
35+
bench.add("compat-source: CompatSource.from(Source)", () => {
36+
const src = new sources.RawSource(fixtureCode);
37+
for (let i = 0; i < 100; i++) sources.CompatSource.from(src);
38+
});
39+
40+
bench.add("compat-source: CompatSource.from(SourceLike)", () => {
41+
for (let i = 0; i < 100; i++) sources.CompatSource.from(sourceLike);
42+
});
43+
44+
bench.add("compat-source: source() (wrapping SourceLike)", () => {
45+
const cs = new sources.CompatSource(sourceLike);
46+
for (let i = 0; i < 500; i++) cs.source();
47+
});
48+
49+
bench.add("compat-source: buffer() (fallback via super)", () => {
50+
const cs = new sources.CompatSource({ source: () => fixtureCode });
51+
for (let i = 0; i < 50; i++) cs.buffer();
52+
});
53+
54+
bench.add("compat-source: buffer() (delegated)", () => {
55+
const cs = new sources.CompatSource(sourceLike);
56+
for (let i = 0; i < 500; i++) cs.buffer();
57+
});
58+
59+
bench.add("compat-source: size() (fallback via super)", () => {
60+
const cs = new sources.CompatSource({ source: () => fixtureCode });
61+
for (let i = 0; i < 50; i++) cs.size();
62+
});
63+
64+
bench.add("compat-source: size() (delegated)", () => {
65+
const cs = new sources.CompatSource(richSourceLike);
66+
for (let i = 0; i < 500; i++) cs.size();
67+
});
68+
69+
bench.add("compat-source: map()", () => {
70+
const cs = new sources.CompatSource(richSourceLike);
71+
for (let i = 0; i < 500; i++) cs.map({});
72+
});
73+
74+
bench.add("compat-source: sourceAndMap()", () => {
75+
const cs = new sources.CompatSource(richSourceLike);
76+
for (let i = 0; i < 500; i++) cs.sourceAndMap({});
77+
});
78+
79+
bench.add("compat-source: updateHash() (delegated)", () => {
80+
const cs = new sources.CompatSource(richSourceLike);
81+
for (let i = 0; i < 20; i++) cs.updateHash(createHash("sha256"));
82+
});
83+
84+
bench.add("compat-source: updateHash() (fallback)", () => {
85+
const cs = new sources.CompatSource({ source: () => fixtureCode });
86+
for (let i = 0; i < 20; i++) cs.updateHash(createHash("sha256"));
87+
});
88+
89+
bench.add("compat-source: wraps OriginalSource", () => {
90+
for (let i = 0; i < 20; i++) {
91+
const cs = new sources.CompatSource(
92+
new sources.OriginalSource(fixtureCode, "fix.js"),
93+
);
94+
cs.source();
95+
cs.buffer();
96+
cs.size();
97+
cs.map({});
98+
}
99+
});
100+
}

0 commit comments

Comments
 (0)