Skip to content

Commit 43d7fc0

Browse files
alexander-akaitalexander-akait
authored
feat(output): add the output.copy option (#21938)
* feat(output): add the output.copy option Copies files and directories into the output directory without a plugin. A pattern is a `from` plus an optional `to`, `filename` and `transform`, glob matching reuses webpack's own, and each copied file is cached behind a file system snapshot so a rebuild re-reads only what changed. * fix(output): stop the copy walk where the glob cannot reach Only `**` matches across depths, so a glob without one names how deep a match can sit and the walk no longer descends past it. `from: "*.txt"` at a project root drops from 239 stats and 7 readdirs to 29 and 1 over a 200-file `node_modules`. Also states 5.111.0 as the release the option ships in, and gives the zero-match case a directory to match nothing in. * test(output): give the copy glob a non-matching file git keeps `assets/skip.log` was never in the repository — `.gitignore` drops `*.log` — so the assertion that the glob leaves it behind passed on an absent file. A `.md` fixture is tracked, and the case now also states what the glob does copy. * fix(output): read the copy base under the name the watcher purges `extractGlobBaseDir` returns a forward-slashed path with a trailing separator, so the walk read `D:/…/static/` while the context dependency registered for it was `D:\…\static`. The input file system caches readdir under that exact string, so on Windows nothing ever invalidated the listing and a file added to a watched directory was never copied — `watchCases/copy/copy` step 2 failed on all three Windows shards. POSIX only differed by the trailing separator, which the prefix purge forgave. The base is now normalized once, and the walk and the watched directory are that one path. * test(output): cover the copy plugin's file-system fallbacks `copy/unreadable` hands the plugin a file system which denies a file and a directory and offers no `realpath`, so the error paths around `stat`, `readdir` and the pattern-level catch are exercised and an unreadable source is shown to surface as a build error rather than a crash. `copy/plugin` uses the exported `webpack.CopyPlugin` directly. * fix(output): anchor a copy glob on the base the walk resolves to `_resolveGlob` returns an absolute pattern as written, so `..` survived in the glob while the walk resolved it away — `/repo/src/../assets/*.txt` matched nothing and warned that it copied no file. The glob is now re-anchored on the normalized base, so both sides name the same directory. * perf(output): read the copy walk's entry types with the directory `readdir` hands back the type of every entry alongside its name, so the walk no longer stats each one to ask. Only a symlink still pays for a stat, and it alone can lead back to a directory already walked, so the realpath guard moved there too — one per pattern instead of one per directory. Over 1000 files where `from: "assets/*.png"` matches 20, the pattern costs 49 file system operations instead of 1029; copying a 350-file tree whole costs 749 instead of 829, the rest being the per-file snapshots. A watch rebuild reads each directory once more than before, because a listing carrying types is not served from the input file system's cache — which is also why an added file can no longer be missed. This fixes a directory reachable both directly and through a symlink: the realpath guard applied to real directories too, so whichever of the two the concurrent walk reached first won and the other was silently dropped. * fix(output): read a copy pattern's windows separators as separators `from: "C:\\*.png"` left the walk a relative base of `.`, which failed the build with "is neither a posix nor a windows path". An absolute pattern is a path, so its backslashes now separate rather than escape it, as copy-webpack-plugin does; a relative one keeps them, since a posix name may hold one. Resolving the base against the context guarantees it is absolute, which is also what `trimTrailingSeparator` was hand-rolling. Covers the `from` shapes in configCases/copy/glob, and pins the two dependency sets a watch rebuild of a changed copied file goes through. * docs(output): say where a copy pattern lands and what its glob takes `to` defaults to `output.path` itself, and a glob separates with `/`, matches dot files and takes `*`, `**`, `?`, `[]` and `{}` — none of which the option said. A TODO records why a changed copied file does not reload a dev server: it is emitted after the compilation hash, which therefore does not move. * feat(output): let a copy pattern set case sensitivity and symlink following `caseSensitive: false` makes the glob ignore the case of a file name, which `globUtils` already took and the plugin never passed on. `followSymlinks: false` leaves a symbolic link alone rather than walking into it and copying what it names, and skips the stat and the realpath a link otherwise costs. Both default to true, so nothing changes for a pattern that sets neither. * feat(output): gather a copy pattern's glob settings under globOptions `caseSensitive` and `followSymlinks` move there, where the other copy plugins keep them, and `dot` joins them: false leaves a file or a directory whose name starts with a dot to a pattern which names it, as `globUtils` already knew how to and the plugin never asked for. All three default to true, so a pattern without `globOptions` copies what it did before. * feat(output): widen the output.copy option surface Closes the gaps against copy-webpack-plugin and the other bundlers' copy plugins, all inside the pattern that already exists: - `from` also takes a list, copied in order; a file two of them reach is copied once, where the first puts it. - `context` roots how a relative `from` resolves and where what it copies lands, so several patterns can share one structure. - `globOptions.ignore` leaves files out, and skips a directory it names whole rather than listing what it would drop. - `globOptions.deep` caps how far below the base the walk reads. - `info` adds asset info, as an object or per file. - `transform` also takes `{ transformer, cache }`, so a transform can opt out of the cache or name what else it depends on. - `output.copy` also takes `{ patterns, concurrency }`, where `concurrency` bounds how many files are read at once. File mode, symbolic links kept as links, and timestamps stay out: they need per-asset support in the emit path, not another pattern option. * feat(output): let a copy pattern pick 'to' per file `to` also takes a function now, and it reads the file from the same object `info` does, so a pattern has one callback argument rather than two unrelated ones. That object's `filename` is the path relative to `to`, which is what both callbacks can know. `filename` keeps webpack's `(pathData, assetInfo)`: it is a filename template like every other one in the configuration, and that is what lets it interpolate `[contenthash]`. * docs(examples): show output.copy, and transformAll as a plugin The example copies a directory and two roots into one destination, and registers a ~30-line plugin that merges copied assets into one — what `copy-webpack-plugin`'s `transformAll` does. `output.copy` keeps one source file to one asset, so merging is a second pass over what it emitted: pick the assets carrying `info.copied`, key a cache item on their contents, emit the merged asset and delete the parts. * feat(output): copy a symbolic link as a symbolic link `globOptions.followSymlinks: false` used to skip a link; it now copies the link itself, pointing where it already points. That is what `dereference: false` means to the other bundlers' copy plugins, and it answers copy-webpack-plugin#381: a relative link keeps resolving once the link and its target are both copied. Emitting a link needs the emit path to create one rather than write bytes, so an asset can now carry `info.symlink` ({ target, isDirectory }) and `Compiler` creates the link, replacing whatever is at that path and leaving an existing link that already points there alone. The asset's source is the target, so nothing of the file is read and stats, watch and `output.clean` still see it — `CleanPlugin` lstats, so a copied link is unlinked rather than walked into. Resolving stays the default: a link in the output directory is only valid where its target travels with it, which is why vite-plugin-static-copy defaults `dereference` to true as well. Also records a TODO that the copy stage is fixed, so a copied file is minified and compressed like any other asset. * feat(output): let output.copy pick the stage it copies at `output.copy.stage` is the `processAssets` stage the copy runs at, defaulting to `PROCESS_ASSETS_STAGE_ADDITIONAL` as before. A later one puts a copied file past whatever would otherwise act on it, so it lands on disk as it is rather than minimized and compressed like every other asset — the analogue of the `hook` option the other bundlers' copy plugins expose. It belongs to the whole option rather than to one pattern: patterns are copied in order and a later one replaces an earlier one, which a per-pattern stage would silently reorder. `output.clean` is unaffected, tapping `emit` after every stage. * feat(output): add 'preserveTimestamps' to an 'output.copy' pattern A copied file is stamped with the time it was written, which loses what the file it came from carried. An asset now carries its times in 'info.timestamps' and the compiler sets them on what it wrote. * fix: stamp an asset's times onto a file the emit did not rewrite A copied file whose content did not change is compared and left alone, so a build which turned 'preserveTimestamps' on, or one whose source was touched, never set the times it carried. * feat(output): add 'preservePermissions' to an 'output.copy' pattern A copied file is written with the permissions a new file gets, so an executable one loses its bit. An asset now carries 'info.mode' and the compiler chmods what it wrote, or what it left alone when the mode differs. * chore: say what the copy reload TODO needs * refactor(output): copy with webpack's own async helpers 'main' replaced neo-async with lib/util/async.js and dropped the dependency, which left the copy plugin requiring a package webpack no longer has. Adds the 'mapLimit' it needs next to 'eachLimit'. * fix(test): build the copy symlink fixture on the node baseline 'fs.rmSync' arrived in node 14.14, and the harness runs a fixture on every version from 10.13, so the node 10 and 12 jobs failed before the case ran. * fix(test): keep the copy fixtures on LF '* text=auto' checks a fixture out with CRLF on windows, so a case comparing what it copied read one byte more than it wrote. * fix(output): stop a copy replacing an asset the compilation emits A pattern reaching a file named like a chunk silently put itself in the bundle's place. A later pattern still replaces an earlier one. * feat(output): take only patterns in 'output.copy' 'concurrency' and 'stage' stay on the plugin, which is what a build needing them reaches for. Annotates three helpers with the normalized output options they are actually handed, which the narrower option type now catches. * docs(examples): show 'output.copy' used as a plugin Copies a prebuilt vendor bundle through 'new CopyPlugin({...})', which is where 'concurrency' and 'stage' live. Says what 'stage' does not do: the minimizer taps with 'additionalAssets', so it re-runs for an asset added at any later stage and only 'info.minimized' leaves a built file alone. * fix(output): keep a copy inside 'output.path' and off stale asset info An explicit 'context' below what 'from' names made the copied path climb out of it, which 'emitAssets' then joined onto 'output.path' and wrote outside the build's own directory. 'updateAsset' merges an info object, so a copy landing on an earlier one kept a mode or a symlink it never asked for. The kept-file check also compared times a 'Date' had already truncated, redoing 'utimes' and 'chmod' on every rebuild. * fix(stats): give 'copied' a place in the asset info order It was the one 'asset.info' flag with a printer but no entry, so its position came from the fallback rather than the list. Sits next to 'sourceFilename', which the same copy sets, and no output moves.
1 parent d0572de commit 43d7fc0

142 files changed

Lines changed: 4188 additions & 33 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.changeset/005-output-copy.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"webpack": minor
3+
---
4+
5+
Add `output.copy` to copy files and directories into the output directory.

‎.gitattributes‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ test/configCases/asset-modules/query-suffixes/raw.txt eol=lf
55
test/configCases/asset-modules/minify-embedded-source/note.txt eol=lf
66
test/configCases/asset-modules/minimize-embedded-asset-source/note.txt eol=lf
77
test/configCases/css/public-path/*.js eol=lf
8+
test/configCases/copy/** eol=lf
89
test/configCases/defer-import/** eol=lf
910
test/configCases/html/sources/** eol=lf
1011
test/configCases/html/link/** eol=lf

‎declarations/WebpackOptions.d.ts‎

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -569,6 +569,15 @@ export type Clean = boolean | CleanOptions;
569569
* Check if to be emitted file already exists and have the same content before writing to output filesystem.
570570
*/
571571
export type CompareBeforeEmit = boolean;
572+
/**
573+
* Copy files and directories to the output directory.
574+
* @since 5.111.0
575+
*/
576+
export type Copy = CopyPattern[] | string;
577+
/**
578+
* A glob or a path of files which are copied to the output directory.
579+
*/
580+
export type CopyPattern = string | CopyObjectPattern;
572581
/**
573582
* This option enables cross-origin loading of chunks.
574583
*/
@@ -878,6 +887,10 @@ export type AssetModuleOutputPath =
878887
*/
879888
export type AssetParserDataUrlFunction =
880889
import("../lib/asset/AssetParser").AssetParserDataUrlFunction;
890+
/**
891+
* Patterns of files which are copied to the output directory.
892+
*/
893+
export type CopyPatterns = CopyPattern[];
881894
/**
882895
* Enable/disable renaming of `@keyframes`.
883896
*/
@@ -2962,6 +2975,11 @@ export interface Output {
29622975
* Check if to be emitted file already exists and have the same content before writing to output filesystem.
29632976
*/
29642977
compareBeforeEmit?: CompareBeforeEmit;
2978+
/**
2979+
* Copy files and directories to the output directory.
2980+
* @since 5.111.0
2981+
*/
2982+
copy?: Copy;
29652983
/**
29662984
* This option enables cross-origin loading of chunks.
29672985
*/
@@ -3177,6 +3195,94 @@ export interface CleanOptions {
31773195
*/
31783196
keep?: RegExp | string | import("../lib/CleanPlugin").KeepFn;
31793197
}
3198+
/**
3199+
* A pattern of files which are copied to the output directory.
3200+
*/
3201+
export interface CopyObjectPattern {
3202+
/**
3203+
* Directory 'from' is resolved from and the copied paths are relative to. Defaults to the compiler context, and to what 'from' names when it is not a glob.
3204+
*/
3205+
context?: string;
3206+
/**
3207+
* Filename template of a copied file inside 'to'. Defaults to '[path][base]', which keeps the name and the directory structure below 'from'.
3208+
*/
3209+
filename?: string | import("../lib/CopyPlugin").CopyFilenameFunction;
3210+
/**
3211+
* Glob or path from where the files are copied.
3212+
*/
3213+
from: string[] | string;
3214+
/**
3215+
* Options of the glob in 'from'.
3216+
*/
3217+
globOptions?: CopyGlobOptions;
3218+
/**
3219+
* Asset info of a copied file.
3220+
*/
3221+
info?:
3222+
| import("../lib/Compilation").AssetInfo
3223+
| import("../lib/CopyPlugin").CopyInfoFunction;
3224+
/**
3225+
* Whether a copied file keeps the permissions of the file it was copied from. Defaults to 'false', which gives it the ones a new file gets. Has no effect on Windows.
3226+
*/
3227+
preservePermissions?: boolean;
3228+
/**
3229+
* Whether a copied file keeps the access and modification times of the file it was copied from. Defaults to 'false', which stamps it with the time it was written.
3230+
*/
3231+
preserveTimestamps?: boolean;
3232+
/**
3233+
* Directory the files are copied to, relative to 'output.path', which is where they land by default.
3234+
*/
3235+
to?: string | import("../lib/CopyPlugin").CopyToFunction;
3236+
/**
3237+
* Modifies the content of a copied file.
3238+
*/
3239+
transform?:
3240+
| {
3241+
/**
3242+
* Whether the result of the transform is cached, and what it is cached under. Defaults to 'true'.
3243+
*/
3244+
cache?:
3245+
| boolean
3246+
| {
3247+
/**
3248+
* Everything beside the content of the file the transform depends on, serialized into the cache key as JSON.
3249+
*/
3250+
keys?:
3251+
| import("../lib/CopyPlugin").CopyTransformCacheKeys
3252+
| import("../lib/CopyPlugin").CopyTransformCacheKeysFunction;
3253+
};
3254+
/**
3255+
* Modifies the content of a copied file.
3256+
*/
3257+
transformer: import("../lib/CopyPlugin").CopyTransform;
3258+
}
3259+
| import("../lib/CopyPlugin").CopyTransform;
3260+
}
3261+
/**
3262+
* Options of the glob in 'from'.
3263+
*/
3264+
export interface CopyGlobOptions {
3265+
/**
3266+
* Whether the glob matches the case of a file name. Defaults to 'true'.
3267+
*/
3268+
caseSensitive?: boolean;
3269+
/**
3270+
* How many directory levels below the base of the glob are read, where '1' reads only the base itself. Defaults to no limit.
3271+
*/
3272+
deep?: number;
3273+
/**
3274+
* Whether the glob reaches a file or a directory whose name starts with a dot without naming it. Defaults to 'true'.
3275+
*/
3276+
dot?: boolean;
3277+
/**
3278+
* Whether a symbolic link is resolved and copied as what it points at. Defaults to 'true'; 'false' copies the link itself, pointing where it already points.
3279+
*/
3280+
followSymlinks?: boolean;
3281+
/**
3282+
* Globs of the files which are not copied, resolved like 'from'. A directory one of them matches is skipped whole.
3283+
*/
3284+
ignore?: string[];
3285+
}
31803286
/**
31813287
* The abilities of the environment where the webpack generated code should run.
31823288
*/
@@ -4136,6 +4242,23 @@ export interface AssetResourceGeneratorOptions {
41364242
*/
41374243
publicPath?: RawPublicPath;
41384244
}
4245+
/**
4246+
* Patterns of files which are copied to the output directory, and the options of the copying itself.
4247+
*/
4248+
export interface CopyOptions {
4249+
/**
4250+
* Maximum number of files which are read at the same time. Defaults to '100'.
4251+
*/
4252+
concurrency?: number;
4253+
/**
4254+
* Patterns of files which are copied to the output directory.
4255+
*/
4256+
patterns: CopyPatterns;
4257+
/**
4258+
* Stage of 'processAssets' the files are copied at. Defaults to 'Compilation.PROCESS_ASSETS_STAGE_ADDITIONAL', where a copied file is still minimized and compressed like every other asset; a later stage leaves it as it is on disk.
4259+
*/
4260+
stage?: number;
4261+
}
41394262
/**
41404263
* Parser options for css/auto and css/module modules.
41414264
*/
@@ -5028,6 +5151,10 @@ export interface OutputNormalized {
50285151
* Check if to be emitted file already exists and have the same content before writing to output filesystem.
50295152
*/
50305153
compareBeforeEmit?: CompareBeforeEmit;
5154+
/**
5155+
* Patterns of files which are copied to the output directory, and the options of the copying itself.
5156+
*/
5157+
copy?: CopyOptions;
50315158
/**
50325159
* This option enables cross-origin loading of chunks.
50335160
*/

0 commit comments

Comments
 (0)