Skip to content

perf(build): deduplicate plugin-sdk chunks to fix ~2x memory regression#45426

Merged
steipete merged 2 commits intoopenclaw:mainfrom
TarasShyn:perf/deduplicate-plugin-sdk-chunks
Mar 13, 2026
Merged

perf(build): deduplicate plugin-sdk chunks to fix ~2x memory regression#45426
steipete merged 2 commits intoopenclaw:mainfrom
TarasShyn:perf/deduplicate-plugin-sdk-chunks

Conversation

@TarasShyn
Copy link
Copy Markdown
Contributor

@TarasShyn TarasShyn commented Mar 13, 2026

Summary

  • Bundle all 38 plugin-sdk entries in a single tsdown build pass instead of separate builds
  • The separate builds prevented the bundler from sharing common chunks, causing massive duplication (e.g. 20 copies of query-expansion, 14 copies of fetch, 11 copies of logger)
  • This was the root cause of the ~2x memory regression reported in 3.12

Measured impact

Metric Before After Improvement
dist/ size 190MB 64MB -66%
plugin-sdk/ size 142MB 16MB -89%
JS files 1,395 789 -43%
5MB+ files 27 7 -74%
Plugin-SDK heap cost +1,309MB +63MB -95%
Total heap (all chunks loaded) 1,926MB 711MB -63%

Test plan

  • pnpm build succeeds with no INEFFECTIVE_DYNAMIC_IMPORT warnings
  • All 25 plugin-sdk entry points import correctly (smoke test)
  • pnpm test:fast passes (956/958 pass; 2 pre-existing failures on main)
  • Verify gateway starts and runs normally
  • Verify plugins load correctly at runtime

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 13, 2026

Greptile Summary

This PR fixes a ~2x memory regression introduced in 3.12 by consolidating all plugin-sdk entries into a single tsdown build pass, enabling Rollup's chunk-sharing mechanism. Previously, each of the 44 entries was built independently, causing massive duplication of shared modules (e.g. 20 copies of query-expansion). The fix is a minimal, targeted change: replacing a .map() over pluginSdkEntrypoints that produced 44 separate configs with a single config whose entry is an Object.fromEntries map. The output paths (dist/plugin-sdk/<name>.js) are preserved exactly. Empirical measurements in the PR description confirm the optimization is working (89% reduction in plugin-sdk/ size, 95% reduction in heap cost).

Key changes:

  • tsdown.config.ts: Replaces 44 individual nodeBuildConfig calls for plugin-sdk entries with one multi-entry config, allowing the bundler to deduplicate shared chunks.

Minor observation:

  • The PR description refers to "38 plugin-sdk entries" and the test plan to "25 plugin-sdk entry points", but the pluginSdkEntrypoints array contains 44 entries. This is a documentation inaccuracy and does not affect build correctness.

Unchecked test-plan items:

  • "Verify gateway starts and runs normally" and "Verify plugins load correctly at runtime" are still open — worth completing before merging to production.

Confidence Score: 4/5

  • Safe to merge pending runtime verification (gateway start and plugin loading)
  • The code change is mechanically correct and well-supported by empirical build metrics. The transformation from 44 separate configs to one multi-entry config preserves output paths and is a standard Rollup pattern. Unit and fast tests pass. The main remaining uncertainty is the two unchecked test-plan items: runtime gateway startup and plugin-loading validation, which could surface issues that build-time and unit tests don't cover.
  • No files require special attention — the change is confined to a single config line in tsdown.config.ts.

Last reviewed commit: f62ec20

@TarasShyn TarasShyn force-pushed the perf/deduplicate-plugin-sdk-chunks branch 2 times, most recently from 5587e7e to 310f8f0 Compare March 13, 2026 20:49
@steipete steipete self-requested a review March 13, 2026 21:41
TarasShyn and others added 2 commits March 13, 2026 21:54
Bundle all plugin-sdk entries in a single tsdown build pass instead of
38 separate builds. The separate builds prevented the bundler from
sharing common chunks, causing massive duplication (e.g. 20 copies of
query-expansion, 14 copies of fetch, 11 copies of logger).

Measured impact:
- dist/ size: 190MB → 64MB (-66%)
- plugin-sdk/ size: 142MB → 16MB (-89%)
- JS files: 1,395 → 789 (-43%)
- 5MB+ files: 27 → 7 (-74%)
- Plugin-SDK heap cost: +1,309MB → +63MB (-95%)
- Total heap (all chunks loaded): 1,926MB → 711MB (-63%)
@steipete steipete force-pushed the perf/deduplicate-plugin-sdk-chunks branch from 310f8f0 to 98acb17 Compare March 13, 2026 21:57
@steipete steipete merged commit b7ff825 into openclaw:main Mar 13, 2026
9 checks passed
@steipete
Copy link
Copy Markdown
Contributor

Landed via temp rebase onto main.

  • Gate: pnpm test src/plugin-sdk/index.test.ts passed. Full repo gate was blocked by pre-existing base-branch failures (pnpm check hit an existing conflict marker in src/daemon/schtasks.startup-fallback.test.ts; PR CI was already red on that plus unrelated undici audit + OOM in a full test shard).
  • Land commit: 98acb17
  • Merge commit: b7ff825

Thanks @TarasShyn!

frankekn pushed a commit to xinhuagu/openclaw that referenced this pull request Mar 14, 2026
ecochran76 pushed a commit to ecochran76/openclaw that referenced this pull request Mar 14, 2026
Interstellar-code pushed a commit to Interstellar-code/operator1 that referenced this pull request Mar 16, 2026
sbezludny pushed a commit to sbezludny/openclaw that referenced this pull request Mar 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants