Skip to content

Commit 3e73e45

Browse files
perf: validate options with a precompiled schema (#5746)
* perf: validate options with a precompiled schema `schema-utils`'s `validate()` compiles the options schema with ajv on its first call. That cost 117ms of every `new Server()` — measured over eight interleaved runs, 117.3 +/- 2.8ms, against 0.07ms for each call after it. It is a one-time price every user pays on every start. `lib/options.check.js` is that schema precompiled by `ajv`'s standalone codegen. The constructor asks it first and only falls back to `schema-utils` when it rejects, so the happy path never loads ajv and an invalid config still gets the same message. webpack solves its own schema the same way, and the generated validator agrees with `schema-utils` on all 183 cases of the `validate-options` corpus plus 62 hand-written ones. The constructor drops to 1.0ms, and time-to-listening for a hello-world falls from 565 +/- 14ms to 523 +/- 25ms. * chore: document the generator's callbacks * refactor: validate options through the compiler where webpack allows it `compiler.hooks.validate` and `compiler.validate`'s lazy-schema and precompiled-check parameters landed together in webpack 5.106, so the hook doubles as the feature probe for them. When it is there, validation goes through the compiler the way webpack's own plugins do, which also lets `validate: false` turn it off. The direct path stays as the fallback, with a TODO to drop it once the minimum supported webpack carries the hook. It is also what runs when no compiler was passed, which is how the server is constructed for `apply()`. * fix: validate against the compiler that owns the dev server `compiler.validate` honours that compiler's own `validate` option, so asking the first child of a `MultiCompiler` read a policy that was never about these options: with `validate: false` on child 0 and the dev server configured on child 1, invalid options reached normalization instead of throwing. `getCompilerOptions` already had the selection rule — the child naming `devServer`, else the one targeting the web, else the first. It moves to `findDevServerCompiler`, which both callers now use so the two cannot drift apart again. Reported by CodeRabbit on #5746. * chore: regenerate types for the validation helpers * test: cover the validation fallbacks codecov reported 6 uncovered lines in the diff, all of them fallbacks: `findDevServerCompiler` returning the first child when none names `devServer` or targets the web, and the direct `schema-utils` call taken when there is no compiler to ask, which is how the server is constructed for `apply()`. lib/Server.js's changed lines go from 70/76 to 76/76 covered. * test: close the compilers the routing tests create Each `webpack()` call leaves a compiler holding a file-system cache and its purge timer. Node's test runner runs files concurrently, so those stayed alive alongside the browser suites for the rest of the run. Windows Node 22 timed out in `test/e2e/overlay.test.js` on the previous commit, which was test-only; that suite already uses most of its 400s there, and this is the load these tests were adding to it. * fix: honour the owning compiler's validate option in plugin mode Used as a plugin the server is constructed without a compiler, so validating in the constructor had no `validate` policy to read and threw before `apply()` could bring the compiler the options are about. Validate in the constructor only when one is passed, and in `apply()` otherwise.
1 parent e44bdae commit 3e73e45

11 files changed

Lines changed: 486 additions & 42 deletions
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"webpack-dev-server": patch
3+
---
4+
5+
Validate options with a precompiled schema to cut ~115ms from startup.

‎.prettierignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,3 +4,4 @@ coverage
44
node_modules
55
CHANGELOG.md
66
examples/client/trusted-types-overlay/app.js
7+
/lib/options.check.js

‎eslint.config.mjs‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,12 @@ import config from "eslint-config-webpack";
33
import configs from "eslint-config-webpack/configs.js";
44

55
export default defineConfig([
6-
globalIgnores(["client/**/*", "dist/**/*", "examples/**/*"]),
6+
globalIgnores([
7+
"client/**/*",
8+
"dist/**/*",
9+
"examples/**/*",
10+
"lib/options.check.js",
11+
]),
712
{
813
extends: [config],
914
ignores: ["client-src/**/*", "!client-src/webpack.config.js"],

‎lib/Server.js‎

Lines changed: 81 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ import url from "node:url";
55
import fs from "graceful-fs";
66
import ipaddr from "ipaddr.js";
77
import { validate } from "schema-utils";
8+
import schemaCheck from "./options.check.js";
89
import schema from "./options.json" with { type: "json" };
910

1011
// Named `cjsRequire` (not `require`) so it doesn't shadow the implicit CommonJS
@@ -343,12 +344,9 @@ class Server {
343344
constructor(options, compiler) {
344345
options = options === undefined ? {} : options;
345346

346-
validate(/** @type {Schema} */ (schema), options, {
347-
name: "Dev Server",
348-
baseDataPath: "options",
349-
});
350-
351347
if (compiler) {
348+
Server.validateOptions(compiler, options);
349+
352350
this.compiler = compiler;
353351

354352
/**
@@ -578,6 +576,76 @@ class Server {
578576
return path.resolve(dir, `node_modules/.cache/${pluginName}`);
579577
}
580578

579+
/**
580+
* The compiler whose configuration the dev server belongs to. For a
581+
* `MultiCompiler` that is the child naming `devServer`, else the one
582+
* targeting the web, else the first — anything else reads another child's
583+
* settings.
584+
* @private
585+
* @param {Compiler | MultiCompiler} compiler compiler
586+
* @returns {Compiler} the compiler that owns the dev server
587+
*/
588+
static findDevServerCompiler(compiler) {
589+
const { compilers } = /** @type {MultiCompiler} */ (compiler);
590+
591+
if (typeof compilers === "undefined") {
592+
return /** @type {Compiler} */ (compiler);
593+
}
594+
595+
if (compilers.length === 1) {
596+
return compilers[0];
597+
}
598+
599+
return (
600+
compilers.find((child) => child.options.devServer) ||
601+
compilers.find((child) => Server.isWebTarget(child)) ||
602+
compilers[0]
603+
);
604+
}
605+
606+
/**
607+
* Throw unless the options match the schema.
608+
*
609+
* `compiler.hooks.validate` and `compiler.validate`'s lazy-schema and
610+
* precompiled-check parameters landed together in webpack 5.106, so the hook
611+
* doubles as the feature probe for them. Validating through the compiler that
612+
* owns the dev server matters: `compiler.validate` honours that compiler's
613+
* `validate` option, so asking any other child would read a policy that was
614+
* never about these options. Either way the schema is only read, and ajv only
615+
* compiled, once something is actually wrong — the precompiled validator
616+
* answers the common case in ~1ms, against the ~120ms `schema-utils` spends
617+
* compiling the schema on its first call.
618+
* @private
619+
* @param {Compiler | MultiCompiler} compiler compiler that owns the dev server
620+
* @param {EXPECTED_ANY} options options
621+
* @returns {void}
622+
*/
623+
static validateOptions(compiler, options) {
624+
const owner = Server.findDevServerCompiler(compiler);
625+
626+
if (owner.hooks.validate) {
627+
/** @type {EXPECTED_ANY} */
628+
(owner).validate(
629+
() => schema,
630+
options,
631+
{ name: "Dev Server", baseDataPath: "options" },
632+
schemaCheck,
633+
);
634+
635+
return;
636+
}
637+
638+
// TODO remove this fallback in the next major, once the minimum supported
639+
// webpack has `compiler.hooks.validate`, and validate through the compiler
640+
// alone.
641+
if (!schemaCheck(options)) {
642+
validate(/** @type {Schema} */ (schema), options, {
643+
name: "Dev Server",
644+
baseDataPath: "options",
645+
});
646+
}
647+
}
648+
581649
/**
582650
* @private
583651
* @param {Compiler} compiler compiler
@@ -792,40 +860,9 @@ class Server {
792860
* @returns {Compiler["options"]} compiler options
793861
*/
794862
getCompilerOptions() {
795-
if (
796-
typeof (/** @type {MultiCompiler} */ (this.compiler).compilers) !==
797-
"undefined"
798-
) {
799-
if (/** @type {MultiCompiler} */ (this.compiler).compilers.length === 1) {
800-
return (
801-
/** @type {MultiCompiler} */
802-
(this.compiler).compilers[0].options
803-
);
804-
}
805-
806-
// Configuration with the `devServer` options
807-
const compilerWithDevServer =
808-
/** @type {MultiCompiler} */
809-
(this.compiler).compilers.find((config) => config.options.devServer);
810-
811-
if (compilerWithDevServer) {
812-
return compilerWithDevServer.options;
813-
}
814-
815-
// Configuration with `web` preset
816-
const compilerWithWebPreset =
817-
/** @type {MultiCompiler} */
818-
(this.compiler).compilers.find((config) => Server.isWebTarget(config));
819-
820-
if (compilerWithWebPreset) {
821-
return compilerWithWebPreset.options;
822-
}
823-
824-
// Fallback
825-
return /** @type {MultiCompiler} */ (this.compiler).compilers[0].options;
826-
}
827-
828-
return /** @type {Compiler} */ (this.compiler).options;
863+
return Server.findDevServerCompiler(
864+
/** @type {Compiler | MultiCompiler} */ (this.compiler),
865+
).options;
829866
}
830867

831868
/**
@@ -3761,6 +3798,11 @@ class Server {
37613798
* @returns {void}
37623799
*/
37633800
apply(compiler) {
3801+
// As a plugin the server is constructed without a compiler, so validation
3802+
// waits for the one that will own it — otherwise that compiler's `validate`
3803+
// option would have no say over the options it is about.
3804+
Server.validateOptions(compiler, this.options);
3805+
37643806
this.compiler = compiler;
37653807
this.isPlugin = true;
37663808
this.logger = this.compiler.getInfrastructureLogger(pluginName);

‎lib/options.check.js‎

Lines changed: 5 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎package-lock.json‎

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎package.json‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@
4949
"lint": "npm-run-all -l -p \"lint:**\"",
5050
"fix:code": "npm run lint:code -- --fix",
5151
"fix:prettier": "npm run lint:prettier -- --write",
52-
"fix": "npm-run-all -l fix:code fix:prettier",
52+
"fix": "npm-run-all -l fix:code fix:schema-check fix:prettier",
5353
"validate:changeset": "node .changeset/changeset-validate.mjs",
5454
"build:cjs": "rimraf -g ./dist/* && babel lib --out-dir dist --env-name cjs --copy-files --no-copy-ignored && node ./scripts/finalize-cjs-build.mjs",
5555
"build:client": "rimraf -g ./client/* && babel client-src/ --out-dir client/ --ignore \"client-src/webpack.config.js\" --ignore \"client-src/modules\" && webpack --config client-src/webpack.config.js",
@@ -60,7 +60,9 @@
6060
"test:watch": "npm run test:only -- --watch",
6161
"test": "npm run test:coverage",
6262
"pretest": "npm run lint",
63-
"prepare": "husky && npm run build"
63+
"prepare": "husky && npm run build",
64+
"lint:schema-check": "node ./scripts/generate-schema-check.mjs --check",
65+
"fix:schema-check": "node ./scripts/generate-schema-check.mjs"
6466
},
6567
"dependencies": {
6668
"@types/bonjour": "^3.5.13",
@@ -102,6 +104,7 @@
102104
"@types/picomatch": "^4.0.2",
103105
"@types/trusted-types": "^2.0.7",
104106
"acorn": "^8.14.0",
107+
"ajv": "^8.20.0",
105108
"babel-loader": "^10.0.0",
106109
"babel-plugin-transform-import-meta": "^3.0.0",
107110
"connect": "^3.7.0",

‎scripts/generate-schema-check.mjs‎

Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
import { readFile, writeFile } from "node:fs/promises";
2+
import path from "node:path";
3+
import { fileURLToPath } from "node:url";
4+
import Ajv, { _ } from "ajv";
5+
import standaloneCode from "ajv/dist/standalone/index.js";
6+
7+
// Precompile `lib/options.json` into a standalone validator, `lib/options.check.js`.
8+
//
9+
// `schema-utils`'s `validate()` compiles the schema with ajv on its first call,
10+
// which cost ~120ms of every `new Server()` — a one-time startup price paid by
11+
// every user on every run. The generated validator answers the same question
12+
// with no compile step, so the happy path never loads ajv at all; `Server` only
13+
// falls back to `schema-utils` when this validator rejects, to build the
14+
// readable error message (the same trick webpack uses for its own schema).
15+
//
16+
// Run `npm run fix:schema-check` to regenerate, `npm run lint:schema-check` to
17+
// verify the committed output is current.
18+
19+
const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
20+
const SCHEMA_PATH = path.join(ROOT, "lib", "options.json");
21+
const OUTPUT_PATH = path.join(ROOT, "lib", "options.check.js");
22+
23+
const BANNER = `// This file was automatically generated.
24+
// DO NOT MODIFY BY HAND. Run \`npm run fix:schema-check\` to update.
25+
/* eslint-disable */
26+
// @ts-nocheck
27+
`;
28+
29+
/**
30+
* The constructors `"instanceof"` may name. ajv-keywords implements the keyword
31+
* with a closure, which cannot be serialized into standalone code, so the
32+
* keyword is re-implemented here as something ajv can emit inline.
33+
*/
34+
const CONSTRUCTORS = {
35+
Buffer: _`Buffer`,
36+
Function: _`Function`,
37+
};
38+
39+
/** @typedef {Record<string, unknown>} SchemaNode */
40+
41+
/**
42+
* Walk every schema node, depth first.
43+
* @param {unknown} node current node
44+
* @param {(node: SchemaNode, pointer: string) => void} visit called for each object node
45+
* @param {string} pointer JSON pointer to `node`
46+
* @returns {void}
47+
*/
48+
function walkSchema(node, visit, pointer = "#") {
49+
if (!node || typeof node !== "object") {
50+
return;
51+
}
52+
53+
if (!Array.isArray(node)) {
54+
visit(/** @type {SchemaNode} */ (node), pointer);
55+
}
56+
57+
for (const [key, value] of Object.entries(node)) {
58+
walkSchema(value, visit, `${pointer}/${key}`);
59+
}
60+
}
61+
62+
/**
63+
* Reject schema constructs the generated validator would silently mistranslate.
64+
* @param {SchemaNode} schema the options schema
65+
* @returns {void}
66+
*/
67+
function assertSupportedSchema(schema) {
68+
/**
69+
* @param {SchemaNode} node the schema node to check
70+
* @param {string} pointer JSON pointer to `node`
71+
* @returns {void}
72+
*/
73+
const assertNode = (node, pointer) => {
74+
// `unicode: false` below makes ajv measure string length in UTF-16 code
75+
// units rather than code points. The two agree only at a bound of 1, where
76+
// both mean "not empty".
77+
if (node.minLength !== undefined && node.minLength !== 1) {
78+
throw new Error(
79+
`"minLength" must be 1, but is ${node.minLength} at ${pointer}.`,
80+
);
81+
}
82+
83+
if (node.maxLength !== undefined) {
84+
throw new Error(`"maxLength" is not supported, found at ${pointer}.`);
85+
}
86+
87+
if (
88+
typeof node.instanceof === "string" &&
89+
!Object.hasOwn(CONSTRUCTORS, node.instanceof)
90+
) {
91+
throw new Error(
92+
`"instanceof": ${JSON.stringify(node.instanceof)} at ${pointer} is not supported. Add it to CONSTRUCTORS.`,
93+
);
94+
}
95+
};
96+
97+
walkSchema(schema, assertNode);
98+
}
99+
100+
/**
101+
* @param {SchemaNode} schema the options schema
102+
* @returns {string} source of the standalone validator
103+
*/
104+
function generate(schema) {
105+
assertSupportedSchema(schema);
106+
107+
const ajv = new Ajv({
108+
/* eslint-disable no-console -- a generator reports to the terminal */
109+
logger: {
110+
log: console.log,
111+
/**
112+
* `unicode` is deprecated but still honoured, and `assertSupportedSchema`
113+
* has already established that dropping it changes nothing here.
114+
* @param {...unknown} args ajv's warning arguments
115+
* @returns {void}
116+
*/
117+
warn: (...args) => {
118+
if (!String(args[0]).includes("option unicode")) {
119+
console.warn(...args);
120+
}
121+
},
122+
error: console.error,
123+
},
124+
/* eslint-enable no-console */
125+
strict: false,
126+
// The validator only reports whether the options are valid; `schema-utils`
127+
// produces the messages, so collecting every error here would be wasted work.
128+
allErrors: false,
129+
verbose: false,
130+
unicode: false,
131+
code: { source: true, esm: true },
132+
});
133+
134+
ajv.addKeyword({
135+
keyword: "instanceof",
136+
schemaType: "string",
137+
/**
138+
* @param {import("ajv").KeywordCxt} cxt keyword context
139+
* @returns {void}
140+
*/
141+
code(cxt) {
142+
cxt.fail(
143+
_`!(${cxt.data} instanceof ${CONSTRUCTORS[/** @type {keyof typeof CONSTRUCTORS} */ (cxt.schema)]})`,
144+
);
145+
},
146+
});
147+
148+
// Documentation-only keywords carried by the schema for the CLI and the docs.
149+
for (const keyword of ["cli", "link"]) {
150+
ajv.addKeyword({ keyword, schemaType: ["string", "object", "boolean"] });
151+
}
152+
153+
return BANNER + standaloneCode.default(ajv, ajv.compile(schema));
154+
}
155+
156+
const source = generate(JSON.parse(await readFile(SCHEMA_PATH, "utf8")));
157+
158+
if (process.argv.includes("--check")) {
159+
const current = await readFile(OUTPUT_PATH, "utf8").catch(() => undefined);
160+
161+
if (current !== source) {
162+
// eslint-disable-next-line no-console
163+
console.error(
164+
`${path.relative(ROOT, OUTPUT_PATH)} is out of date — run \`npm run fix:schema-check\`.`,
165+
);
166+
process.exitCode = 1;
167+
}
168+
} else {
169+
await writeFile(OUTPUT_PATH, source);
170+
}

0 commit comments

Comments
 (0)