Skip to content

feat: incremental make for watch rebuilds - #21556

Open
bjohansebas wants to merge 67 commits into
mainfrom
feat/incremental-make
Open

bjohansebas wants to merge 67 commits into
mainfrom
feat/incremental-make

Conversation

@bjohansebas

@bjohansebas bjohansebas commented Jul 30, 2026 •

Copy link
Copy Markdown
Member

Summary

Makes watch rebuilds incremental: the previous compilation's module graph and module set are reused, changed modules are detected via their build/resolve snapshots and only those are re-factorized and rebuilt — the O(modules) entry re-walk is skipped (on the original branch this measured make 285ms→78ms for a 1-module change on a react-5k project; numbers for this port to be re-measured). No config option: it is automatic in watch mode; compilations that ran build-time execution (importModule) disable reuse for the rest of the session since execution mutates the durable graph. Fourth PR of the incremental-rebuilds series (stacked on #21555, extracted from perf/incremental-rebuild).

What kind of change does this PR introduce?

feat

Did you add tests for your changes?

Yes, test/watchCases/incremental/*; and since the behavior is always on in watch mode, the whole WatchTestCases suite now exercises it.

Does this PR introduce a breaking change?

No; watch rebuilds keep producing identical output, and single (non-watch) builds are untouched.

If relevant, what needs to be documented once your changes are merged or what have you already documented?

n/a

Use of AI

AI (Claude Code) was used to extract this change from my working branch, adapt it onto current main, and debug the build-time-execution interaction; I reviewed and validated everything locally.


Note

High Risk
Touches the core make/seal pipeline and durable module graph semantics; incorrect reuse could cause stale bundles or missed rebuilds across many edge cases (entries, lazy barrels, concat, resolve).

Overview
Watch mode now performs incremental make: when the previous compilation was sealed with a reusable base snapshot and mutation journal, the next watch compilation keeps the module graph, entries, and end-of-make module set, then runs the make hook mainly for entry deduplication before _incrementalMake scans needBuild (plus resolve snapshots) and rebuilds only changed modules. Unchanged modules skip dependency re-walks; changed loader/scheme modules can be re-factorized through the factory so resolution changes are picked up, with pruning of unreachable modules and reversal of seal-time graph mutations (e.g. concatenation) via ModuleGraph journal APIs.

Compiler retains _lastCompilation across watch cycles (without clearing module graph state), gates reuse on _incrementalReady and disables it permanently after executeModule / importModule. Seal snapshots base module maps, starts the mutation journal, and cleans up parsers; finish paths reset export/usage flags and extend mem-cache reference keys with connection active state so side-effect optimizations still invalidate correctly.

Adds a minor changeset and broad test/watchCases/incremental/* coverage (entries, loaders, concat, side effects, errors, dynamic entries).

Reviewed by Cursor Bugbot for commit 3e0aa2e. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by CodeRabbit

  • New Features

    • Incremental watch rebuilds now reuse prior compilation state and rebuild only affected modules, improving rebuild speed.
    • Rebuilds better handle changed dependencies, dynamic entries, removed modules, loaders, side effects, custom entry contexts, and failed resolutions.
    • Unchanged cached output remains stable across rebuilds.
  • Bug Fixes

    • Improved dependency tracking, cleanup, and recovery during incremental rebuilds.
  • Tests

    • Added comprehensive coverage for incremental rebuild scenarios and watch-mode behavior.
  • Documentation

    • Documented incremental watch rebuild support.

@changeset-bot

changeset-bot Bot commented Jul 30, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 3b8f1f7

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
webpack Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@alexander-akait
alexander-akait force-pushed the feat/incremental-make branch from 04cfd4b to 9992e43 Compare July 30, 2026 17:16
@codecov

codecov Bot commented Jul 30, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.66667% with 84 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.91%. Comparing base (7908fa7) to head (3b8f1f7).
⚠️ Report is 7 commits behind head on main.

Files with missing lines Patch % Lines
lib/watch/Incremental.js 89.27% 80 Missing ⚠️
lib/Compilation.js 97.54% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #21556      +/-   ##
==========================================
- Coverage   96.92%   96.91%   -0.01%     
==========================================
  Files         802      803       +1     
  Lines      136245   137460    +1215     
  Branches    46422    46953     +531     
==========================================
+ Hits       132055   133222    +1167     
- Misses       4190     4238      +48     
Flag Coverage Δ
integration 81.00% <91.66%> (+0.25%) ⬆️
spec 47.50% <20.09%> (-0.17%) ⬇️
syntax-equivalence 78.57% <ø> (ø)
test262 54.19% <21.19%> (-0.29%) ⬇️
unit 76.90% <69.21%> (-0.06%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codspeed

codspeed Bot commented Jul 30, 2026 •

Copy link
Copy Markdown

Merging this PR will regress 4 benchmarks

2 warnings, 1 note

⚠️ 117 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚠️ 22 benchmarks with a significant change ran on different hardware

Their base and head results were measured on machines with a different CPU, core count or memory, so the changes might not come from the code. Run base and head on the same runner type.

Learn more about Macro Runners

See affected benchmarks

ℹ️ 39 benchmarks have no flamegraph

The flamegraph wasn't generated: most of the time was spent in runtime internals, which would leave nothing useful to analyze.

See affected benchmarks

⚡ 22 improved benchmarks
❌ 4 regressed benchmarks
✅ 325 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ Memory benchmark "asset-modules-bytes", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 219.9 KB 459.8 KB -52.18%
❌ CPU Simulation benchmark "three-long", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 613.9 ms 782.6 ms -21.56%
❌ CPU Simulation benchmark "cache-filesystem", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 611.4 ms 778.3 ms -21.44%
❌ CPU Simulation benchmark "common-libs", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 1.2 s 1.6 s -20.57%
⚡ Memory benchmark "concatenate-modules", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 489.5 KB 298.3 KB +64.1%
⚡ Memory benchmark "css-modules", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 351.5 KB 222 KB +58.31%
⚡ CPU Simulation benchmark "import-meta-glob", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 574.6 ms 363.2 ms +58.2%
⚡ Memory benchmark "devtool-eval", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 220.9 KB 152.3 KB +45.04%
⚡ Memory benchmark "context-esm", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 245.8 KB 171 KB +43.75%
⚡ CPU Simulation benchmark "many-modules-commonjs", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 359.9 ms 256.7 ms +40.22%
⚡ Memory benchmark "future-defaults", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 120.9 KB 87.9 KB +37.6%
⚡ CPU Simulation benchmark "many-modules-esm", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 353 ms 258.4 ms +36.59%
⚡ Memory benchmark "asset-modules-source", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 216.8 KB 160.1 KB +35.45%
⚡ CPU Simulation benchmark "future-defaults", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 162.5 ms 126.9 ms +28.01%
⚡ CPU Simulation benchmark "no-exports-analysis", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 170.4 ms 133.1 ms +28%
⚡ CPU Simulation benchmark "cache-filesystem-cold", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 173.8 ms 136.3 ms +27.54%
⚡ Memory benchmark "wasm-modules-async", scenario '{"name":"mode-production","mode":"production"}' 399.2 KB 317.2 KB +25.85%
⚡ CPU Simulation benchmark "concatenate-modules", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 247.3 ms 197.2 ms +25.42%
⚡ CPU Simulation benchmark "no-concatenation", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 170.7 ms 136.2 ms +25.39%
⚡ CPU Simulation benchmark "no-minimize", scenario '{"name":"mode-development-rebuild","mode":"development","watch":true}' 170.7 ms 136.3 ms +25.26%
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing feat/incremental-make (3b8f1f7) with main (d3b2ce8)

Open in CodSpeed

@bjohansebas
bjohansebas force-pushed the feat/incremental-make branch from 9992e43 to def7c1a Compare July 30, 2026 19:41
@bjohansebas
bjohansebas force-pushed the feat/incremental-make branch 2 times, most recently from ff0c2fb to 520b839 Compare July 30, 2026 20:51
@bjohansebas
bjohansebas marked this pull request as ready for review July 30, 2026 21:27
Comment thread lib/Compilation.js Resolved
Comment thread lib/Compilation.js Resolved
Base automatically changed from refactor/module-parser-lifecycle to main July 31, 2026 09:30
@alexander-akait

Copy link
Copy Markdown
Member

Let’s fix tests and we can merge

@bjohansebas
bjohansebas force-pushed the feat/incremental-make branch from e21d62c to 272a333 Compare July 31, 2026 15:06
Comment thread lib/Compilation.js Resolved
Comment thread lib/Compilation.js Resolved
@bjohansebas
bjohansebas force-pushed the feat/incremental-make branch from 272a333 to 816a091 Compare July 31, 2026 15:24
Comment thread lib/Compilation.js Resolved
Comment thread lib/Compilation.js Resolved
Comment thread lib/graph/ModuleGraph.js Resolved
Comment thread lib/Compilation.js Resolved
Comment thread lib/Compilation.js Resolved
@bjohansebas
bjohansebas marked this pull request as draft July 31, 2026 16:46
@bjohansebas
bjohansebas marked this pull request as ready for review July 31, 2026 18:09
Comment thread lib/Compilation.js Resolved
Comment thread lib/Compilation.js Resolved
Comment thread lib/graph/ModuleGraph.js Resolved
Comment thread lib/Compiler.js Resolved
Comment thread lib/Compilation.js Resolved

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 3618e21. Configure here.

Comment thread lib/Compilation.js Resolved
@bjohansebas
bjohansebas force-pushed the feat/incremental-make branch from 5ca1c5e to 3e0aa2e Compare August 1, 2026 01:04
Revert the partial traversal from 65b1bf4 so changed dependencies can start before retained subtrees finish. Keep invalidation tracking and update the resolution counts.
Exercise an extra invalidation without editing files. Require an entry rebuild when its source changes, while preserving hook and reuse assertions.
Keep the stable request in an unchanged subtree so the option test still distinguishes graph reuse from regular resolution.
Give each compiler a separate filename and execute its own bundle. Keep one compiler from cleaning outputs written by the others.
Keep existing file, context and missing dependency sets when the factory adds no entries. Preserve missing dependency propagation to importers.

A common-libs watch build creates 1612 replacement collections instead of 5718, with identical output.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: config Options, defaults, validation (lib/config, schemas/) area: optimization Tree-shaking, splitChunks, concatenation, ids (lib/optimize, lib/ids) area: parser JavaScript parsing and dependency analysis (lib/javascript, lib/dependencies) area: tooling Repo tooling, codegen, CLI entry (tooling/, bin/) area: types types.d.ts, JSDoc annotations, hand-maintained declarations

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants