Skip to content

Commit 3733511

Browse files
refactor: group loose lib/ sources into directories (#22118)
* refactor(externals): move externals module sources into lib/externals/ ExternalModule, ExternalModuleFactoryPlugin and ExternalsPlugin move out of the top of lib/ into their own directory, following lib/context/. Both of ExternalModule's serialization requests move with it, and the schema's tsType paths and the Architecture listing are updated. ExternalModuleFactoryPlugin declared ExternalItemFunctionPromise through `import("../lib/ExternalModuleFactoryPlugin")`, a path that resolved to the file itself and would not survive the move; it now names the local typedef directly, as the line above it already does. * refactor(optimize): move export-flagging plugins into lib/optimize/ The four Flag*Plugin files and OptimizationStages join the optimization plugins they belong with; eleven of OptimizationStages' fourteen callers already sat in that directory. ExportsInfo and ConcatenationScope stay where they are on purpose: ExportsInfo is module-graph state reached through ModuleGraph from dependencies, library and the css/json generators, and ConcatenationScope is the codegen contract every Generator receives, so neither is an optimization plugin. * fix(serialization): keep pre-move requests restorable from old caches Moving a serializable changes its request string, so a filesystem cache written before the move names a key nothing registers and the entry is rebuilt instead of restored. The existing generator aliases only load the module; the inversed lookup still misses, so restoring throws. ObjectMiddleware.registerLegacyRequest adds the historical request as an extra deserialization key for an already registered constructor. Writes keep using the current request, so only old caches reach the old key, and the generator derives the loader entry from the call. ContextModule, ExternalModule and ModuleExternalInitFragment register the requests they had before moving, each marked for removal in webpack 6. * chore: shorten the changeset summary to the 80-character limit * refactor(watch): move watch mode sources into lib/watch/ Watching, MultiWatching and WatchIgnorePlugin leave the top of lib/ for their own directory. No serializers are involved and the move is paths only; the "webpack.Watching" logger names stay as they are. * refactor(template): move template sources into lib/template/ Template, the three deprecated *Template shims, TemplatedPathPlugin, InitFragment, ConditionalInitFragment and DependencyTemplate(s) move into their own directory. Template alone is required from 185 files, so the specifiers were rewritten by resolving each one rather than by hand. InitFragment keeps its pre-move serialization request, the schema tsType paths for TemplatedPathPlugin follow the file, and codecov.yml's ignores for the deprecated shims are repointed so they stay out of coverage. * fix(serialization): restore the three historical requests, not just load them The generator's LEGACY_ALIASES table pointed three historical requests at the module that now owns them, which loaded that module but registered nothing under the old key, so restoring still threw and the entry was rebuilt. Each owner now calls registerLegacyRequest instead, and the generator derives the loader entry from that call, leaving internalSerializables byte-identical while the keys actually resolve. Also ignore a root dist/, which a watch test writes and a commit picked up. * refactor(devtool): move source map sources into lib/devtool/ The three devtool plugins, SourceMapDevToolModuleOptionsPlugin and ModuleFilenameHelpers move into their own directory. No serializers are involved; the schema tsType paths for ModuleFilenameHelpers follow the file across WebpackOptions, BannerPlugin and SourceMapDevToolPlugin. * refactor(runtime): move the runtime module base into lib/runtime/ RuntimeGlobals, RuntimeModule and RuntimePlugin join the 39 runtime modules already there: RuntimeModule is the class each extends, RuntimeGlobals the symbols each declares, and RuntimePlugin what injects them for the requirements a build collects. RuntimeTemplate stays put — it is the codegen printing helper the generators receive, and none of its referrers are in lib/runtime/. * refactor(template): move RuntimeTemplate into lib/template/ Despite the name it is not a runtime-module concern: none of its referrers sit in lib/runtime/, and what it does is print — module ids, import statements, returning functions — for the generators and dependency templates handed it. TemplatedPathPlugin becomes a sibling, so prettier collapses the memoize call that reaches for it onto one line. * refactor(graph): move the module and chunk graphs into lib/graph/ ModuleGraph, ModuleGraphConnection, ChunkGraph, buildChunkGraph and ExportsInfo form a unit: eight of the references between them point at another member, and nothing else in lib/ sits between them. ExportsInfo registers RestoreProvidedData under ModuleGraph's request rather than its own, so that request moves with it and keeps a registerLegacyRequest for packs written before. The generator no longer needs its hand-written REQUIRE_OVERRIDES entry: that existed to point the ModuleGraph key at a file other than the one registering it, and the derived path now names ExportsInfo directly. ConcatenationScope was considered and left alone — it holds no reference to any of these, being the codegen contract every Generator receives. * refactor(prefetch): move the build-time prefetch plugins into lib/prefetch/ PrefetchPlugin and AutomaticPrefetchPlugin were the only two files left whose subject the directory already names. Nothing outside lib/index.js reached for either. They are prefetch in a different sense from what was already there: the runtime modules emit a link rel=prefetch for a chunk, while these two resolve a module eagerly at build time through PrefetchDependency. The Architecture bullet now says so rather than letting the name blur them. * refactor(errors): move ErrorHelpers into lib/errors/ Four of its referrers are the error classes themselves, and deriveStackFromNestedError.js already shows a helper belongs beside them rather than at the root. Compilation reads toError off it through a destructure that sat between two requires and blocked eslint's import sort, so that line now follows the require block instead of splitting it. * refactor(cache): move Cache and CacheFacade into lib/cache/ Cache has its referrer plurality there and CacheFacade is the handle every cache/ plugin hands out, so both join the strategies rather than sitting a directory above them. The changeset folds in the ErrorHelpers move, which rides the same branch. * test: pin the two lib paths derived from __dirname Moving ErrorHelpers into lib/errors/ changed what its WEBPACK_SOURCE_PATH named, so frames in webpack's own source kept a position that moves with a release. Nothing failed until a cache case noticed. Both derivations are now pinned. The ErrorHelpers one walks lib/ and asserts every file loses its position, so a file at any depth is covered and a failure names the offenders. The defaults.js one asserts cache.buildDependencies.defaultWebpack is lib/, which decides whether a pack invalidates when webpack itself changes. * refactor(entry): move the entry option's plugins into lib/entry/ EntryPlugin, EntryOptionPlugin and DynamicEntryPlugin reference each other four times over and nothing else sits between them. Two files that look like they belong were left where they are. SingleEntryPlugin is the webpack 4 name and re-exports EntryPlugin, so it stays at the root for require("webpack/lib/SingleEntryPlugin"). Entrypoint extends ChunkGroup and is reached from ten directories, only two of them these plugins, so it stays beside ChunkGroup. The schema's two DynamicEntryPlugin tsType paths follow the file. * build: report which webpack/lib paths the ecosystem imports directly A file moved out of lib/ root breaks any published package that named its old path, and nothing in the repo could say which paths those are. This ranks the most downloaded webpack-plugin and webpack-loader packages, reads what each one requires, and exits non-zero while this checkout fails to resolve something they name. The registry's search endpoint carries weekly downloads with each hit, so it answers discovery and ranking in one set of calls and api.npmjs.org is never needed. Each package is read from its tarball rather than installed, so no lifecycle script of a stranger's package runs. * fix: re-export the three moved paths published plugins import yarn find-deep-imports over the 100 most downloaded webpack-plugin and webpack-loader packages found three paths this stack moved that shipped packages still require by their old name: ModuleFilenameHelpers, at 521k weekly downloads across six of them, plus InitFragment and DependencyTemplate. Each keeps a re-export at the old path, as WebpackError and ModuleNotFoundError already did. DependencyTemplate is not on compiler.webpack, so its note names the new deep path instead of a public API that does not exist. A test asserts every shim resolves to the moved file, so the next move fails rather than silently breaking them again. * build: record what the ecosystem imports deeply, and check it offline The scan always exited non-zero, because four of the paths it finds are webpack 4 API that webpack 5 dropped on purpose, so it could gate nothing. What it collects now lands in tooling/deep-webpack-imports.json, whose removed map names those four with the reason each is deliberate. That splits the tool in two. --write refreshes the record from npm and is the half needing the network. --check reads only the record and fails when a recorded path stopped resolving, naming the packages and the downloads behind it, and a unit test asserts the same off the same file, so a move out of lib/ root is caught by a test rather than by a report from a user. The record is written through prettier, so a refresh cannot leave fmt:check red, and cspell ignores it because it is full of package names no dictionary should carry. * chore: fold the lib grouping changesets into one Eleven entries for what lands as one pull request; the guide asks for a single one naming them together. * fix(tooling): keep a package name from naming a path outside the cache CodeQL flagged the single-occurrence replace building the packument url. The same replace named the cache directory, where it mattered more: a name holding a second separator resolved outside CACHE_ROOT, and the name comes from a registry response rather than from us. Both now replace globally, and the directory keeps only characters a name may safely contribute. Checked: "@a/b/../../../x" escaped before and is contained after. * fix: restore the relative-path literals the move rewrote Moving RuntimeTemplate into lib/template/ ran its string literals through the same relative-path resolver as its requires, so three "./" literals became ".." and the public path gained a second "./": analyzable asset urls read "././asset.txt". Audited every moved file for the same class of damage by comparing each against main with module specifiers blanked out; only this one was hit. Also drops the JSDoc block the REQUIRE_OVERRIDES removal left orphaned. * build(tooling): widen the deep-import scan and match only real imports The scan took the top 100 `webpack-plugin` and `webpack-loader` packages. Adding the broad `webpack` keyword and raising the default to 200 widens it to 570 packages; one two keywords share is scanned once. It also matched any quoted `webpack/lib/…` string, so a package that bundled webpack reported webpack's own `makeSerializable` requests as imports it does not make. Matching import positions only drops those 51. What is left is two paths this pull request broke: ExternalsPlugin, which worker-loader requires at load on webpack 5 at 568k weekly downloads, and EntryPlugin, which @stylable/webpack-extensions requires. Both keep a re-export at their old path. The ten still unresolved are recorded under `removed`: nine reach webpack only through hard-source-webpack-plugin, which is webpack 4 only, and @sector-labs/lingui-loader probes JavascriptGenerator inside a `try` to detect webpack 4, so re-exporting it would pick the wrong branch. * ci: report the deep imports a pull request breaks The record already said which `webpack/lib/…` paths published plugins and loaders import, but only someone running the check by hand learned that a move broke one. `--check` now writes the finding as markdown, and a job posts it on the pull request: which path stopped resolving, which packages import it and how many weekly downloads sit behind them. It comments only once there is something to say and edits that comment away when the paths resolve again, so a green pull request stays quiet. The check reads the record and the checkout alone, so the job needs no install and no network — `prettier` moves into the one function that writes the record. It runs in under a second, and fails after commenting so the thread names the break rather than only the red check.
1 parent db24532 commit 3733511

386 files changed

Lines changed: 3759 additions & 2086 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"webpack": patch
3+
---
4+
5+
Group loose `lib/` sources into directories, keeping old deep paths working.

‎.github/workflows/deep-imports.yml‎

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
name: Deep Imports
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
branches: [main]
8+
workflow_dispatch:
9+
10+
concurrency:
11+
group: ${{ github.workflow }}-${{ github.ref }}
12+
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
13+
14+
permissions:
15+
contents: read
16+
# Posting the report as a pull request comment.
17+
pull-requests: write
18+
19+
jobs:
20+
deep-imports:
21+
runs-on: ubuntu-latest
22+
steps:
23+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
24+
25+
- name: Use Node.js
26+
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
27+
with:
28+
node-version: lts/*
29+
30+
# No install: `--check` reads the recorded scan and this checkout, and
31+
# nothing else. It takes no network and runs in well under a second.
32+
- name: Check the paths the ecosystem imports directly
33+
id: check
34+
run: |
35+
if node tooling/find-deep-webpack-imports.js \
36+
--check --summary deep-imports-report.md; then
37+
echo "broken=false" >> "$GITHUB_OUTPUT"
38+
else
39+
echo "broken=true" >> "$GITHUB_OUTPUT"
40+
fi
41+
cat deep-imports-report.md >> "$GITHUB_STEP_SUMMARY"
42+
43+
# Only once something is broken, and afterwards to retract it — a comment
44+
# on every green pull request is noise nobody reads. Skipped for forks,
45+
# whose token cannot comment; the job summary still carries the report.
46+
- name: Comment the report on the pull request
47+
if: >
48+
github.event_name == 'pull_request' &&
49+
github.event.pull_request.head.repo.full_name == github.repository
50+
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
51+
env:
52+
BROKEN: ${{ steps.check.outputs.broken }}
53+
with:
54+
script: |
55+
const body = require("fs").readFileSync("deep-imports-report.md", "utf8");
56+
const marker = "<!-- deep-imports-report -->";
57+
const comments = await github.paginate(
58+
github.rest.issues.listComments,
59+
{ ...context.repo, issue_number: context.issue.number, per_page: 100 }
60+
);
61+
// Only a comment this workflow itself wrote: anyone can post one
62+
// carrying the marker, and overwriting it would let them edit the
63+
// report afterwards and pass their own findings off as CI's.
64+
const previous = comments.find(
65+
(comment) =>
66+
comment.user.type === "Bot" &&
67+
comment.user.login === "github-actions[bot]" &&
68+
comment.body.startsWith(marker)
69+
);
70+
if (previous) {
71+
await github.rest.issues.updateComment({
72+
...context.repo,
73+
comment_id: previous.id,
74+
body
75+
});
76+
} else if (process.env.BROKEN === "true") {
77+
await github.rest.issues.createComment({
78+
...context.repo,
79+
issue_number: context.issue.number,
80+
body
81+
});
82+
}
83+
84+
# After the comment, so the thread names what failed rather than only the
85+
# red check.
86+
- name: Fail on a path that stopped resolving
87+
if: steps.check.outputs.broken == 'true'
88+
run: exit 1

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@
2222
.cspellcache
2323
package-lock.json
2424
/events.json
25+
/dist
2526

2627
# Generated from schemas/**/*.json by tooling/generate-types.js
2728
/declarations/

‎AGENTS.md‎

Lines changed: 20 additions & 4 deletions
Large diffs are not rendered by default.

‎codecov.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,6 @@ ignore:
3636
- "schemas/**/*"
3737
- "examples/**/*"
3838
# TODO webpack 6 remove
39-
- "lib/ChunkTemplate.js"
40-
- "lib/MainTemplate.js"
41-
- "lib/ModuleTemplate.js"
39+
- "lib/template/ChunkTemplate.js"
40+
- "lib/template/MainTemplate.js"
41+
- "lib/template/ModuleTemplate.js"

‎cspell.json‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -309,6 +309,7 @@
309309
"Noto",
310310
"novalidate",
311311
"nowrap",
312+
"npmjs",
312313
"nwjs",
313314
"oklab",
314315
"oklch",
@@ -326,6 +327,7 @@
326327
"overridable",
327328
"overridables",
328329
"ovflowd",
330+
"packument",
329331
"parallelism",
330332
"parseargs",
331333
"passthrough",
@@ -594,6 +596,7 @@
594596
"**/webpack.lock.data/**",
595597
"package.json",
596598
"yarn.lock",
599+
"tooling/deep-webpack-imports.json",
597600
"types.d.ts",
598601
"**/**/*.snap",
599602
"test/cases/json/weird-properties/globals.json",

‎lib/APIPlugin.js‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,19 +5,18 @@
55

66
"use strict";
77

8-
const {
9-
getExternalModuleNodeCommonjsInitFragment
10-
} = require("./ExternalModule");
118
const {
129
JAVASCRIPT_MODULE_TYPE_AUTO,
1310
JAVASCRIPT_MODULE_TYPE_DYNAMIC,
1411
JAVASCRIPT_MODULE_TYPE_ESM
1512
} = require("./ModuleTypeConstants");
16-
const RuntimeGlobals = require("./RuntimeGlobals");
1713
const ConstDependency = require("./dependencies/ConstDependency");
1814
const ModuleInitFragmentDependency = require("./dependencies/ModuleInitFragmentDependency");
1915
const RuntimeRequirementsDependency = require("./dependencies/RuntimeRequirementsDependency");
2016
const WebpackError = require("./errors/WebpackError");
17+
const {
18+
getExternalModuleNodeCommonjsInitFragment
19+
} = require("./externals/ExternalModule");
2120
const BasicEvaluatedExpression = require("./javascript/BasicEvaluatedExpression");
2221
const JavascriptModulesPlugin = require("./javascript/JavascriptModulesPlugin");
2322
const {
@@ -26,6 +25,7 @@ const {
2625
} = require("./javascript/JavascriptParserHelpers");
2726
const ChunkNameRuntimeModule = require("./runtime/ChunkNameRuntimeModule");
2827
const GetFullHashRuntimeModule = require("./runtime/GetFullHashRuntimeModule");
28+
const RuntimeGlobals = require("./runtime/RuntimeGlobals");
2929
const memoize = require("./util/memoize");
3030
const { forEachRuntime } = require("./util/runtime");
3131

@@ -41,7 +41,7 @@ const getConcatenatedModule = memoize(() =>
4141
* } from "./javascript/JavascriptModule"
4242
*/
4343
/** @import Compilation from "./Compilation" */
44-
/** @import ChunkGraph from "./ChunkGraph" */
44+
/** @import ChunkGraph from "./graph/ChunkGraph" */
4545
/** @import { RuntimeSpec } from "./util/runtime" */
4646
/** @import JavascriptParser, { Range } from "./javascript/JavascriptParser" */
4747

‎lib/BannerPlugin.js‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,8 @@
77

88
const { ConcatSource } = require("webpack-sources");
99
const Compilation = require("./Compilation");
10-
const ModuleFilenameHelpers = require("./ModuleFilenameHelpers");
11-
const Template = require("./Template");
10+
const ModuleFilenameHelpers = require("./devtool/ModuleFilenameHelpers");
11+
const Template = require("./template/Template");
1212

1313
/** @import { Source } from "webpack-sources" */
1414
/**
@@ -124,7 +124,7 @@ class BannerPlugin {
124124
const data = { chunk, filename: file };
125125

126126
const comment = compilation.getPath(
127-
/** @type {string | import("./TemplatedPathPlugin").TemplatePathFn<PathDataChunk>} */
127+
/** @type {string | import("./template/TemplatedPathPlugin").TemplatePathFn<PathDataChunk>} */
128128
(banner),
129129
data
130130
);

‎lib/Chunk.js‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,8 @@
55

66
"use strict";
77

8-
const ChunkGraph = require("./ChunkGraph");
98
const Entrypoint = require("./Entrypoint");
9+
const ChunkGraph = require("./graph/ChunkGraph");
1010
const { intersect } = require("./util/SetHelpers");
1111
const SortableSet = require("./util/SortableSet");
1212
const StringXor = require("./util/StringXor");
@@ -24,13 +24,13 @@ const { mergeRuntime } = require("./util/runtime");
2424
* ChunkSizeOptions,
2525
* ModuleFilterPredicate,
2626
* ModuleId
27-
* } from "./ChunkGraph"
27+
* } from "./graph/ChunkGraph"
2828
*/
2929
/** @import ChunkGroup from "./ChunkGroup" */
3030
/** @import { EntryOptions } from "./Entrypoint" */
3131
/** @import Module from "./Module" */
3232
/** @import { PathDataChunk } from "./Compilation" */
33-
/** @typedef {import("./TemplatedPathPlugin").TemplatePathFn<PathDataChunk>} ChunkFilenameTemplateFn */
33+
/** @typedef {import("./template/TemplatedPathPlugin").TemplatePathFn<PathDataChunk>} ChunkFilenameTemplateFn */
3434
/** @typedef {string | ChunkFilenameTemplateFn} ChunkFilenameTemplate */
3535
/** @import Hash from "./util/Hash" */
3636
/** @import { RuntimeSpec } from "./util/runtime" */

‎lib/ChunkGroup.js‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,11 +15,11 @@ const {
1515

1616
/** @import AsyncDependenciesBlock from "./AsyncDependenciesBlock" */
1717
/** @import Chunk from "./Chunk" */
18-
/** @import ChunkGraph from "./ChunkGraph" */
18+
/** @import ChunkGraph from "./graph/ChunkGraph" */
1919
/** @import { DependencyLocation } from "./Dependency" */
2020
/** @import Entrypoint from "./Entrypoint" */
2121
/** @import Module from "./Module" */
22-
/** @import ModuleGraph from "./ModuleGraph" */
22+
/** @import ModuleGraph from "./graph/ModuleGraph" */
2323

2424
/** @typedef {{ module: Module | null, loc: DependencyLocation, request: string }} OriginRecord */
2525

0 commit comments

Comments
 (0)