Skip to content

refactor: introducing the faster code-splitting algorithm#8823

Merged
JSerFeng merged 5 commits intoweb-infra-dev:mainfrom
JSerFeng:refactor/code-splitter
Jan 20, 2025
Merged

refactor: introducing the faster code-splitting algorithm#8823
JSerFeng merged 5 commits intoweb-infra-dev:mainfrom
JSerFeng:refactor/code-splitter

Conversation

@JSerFeng
Copy link
Copy Markdown
Contributor

@JSerFeng JSerFeng commented Dec 23, 2024

Summary

Change the algorithm of code-splitting.

Making it more maintainable, concurrent and easy to working with incoming incremental arch.

Because of the multi-threads, now the code-splitting is more performant than before even using single thread. There can be a huge improvement on large repo.

There is a internal test project which contains about 1.5 million chunk modules, using many dynamic imports, costs about 1200 ms to complete a whole code-splitting phase, using the new algorithm on my 15 core MacBook Pro M1, only takse 600 ms to finish !

The detailed benchmark data will be present soon

Checklist

  • Tests updated (or not required).
  • Documentation updated (or not required).

@netlify
Copy link
Copy Markdown

netlify Bot commented Dec 23, 2024

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented Dec 23, 2024

CodSpeed Performance Report

Merging #8823 will not alter performance

Comparing JSerFeng:refactor/code-splitter (e76cafc) with main (8d6406d)

Summary

✅ 3 untouched benchmarks

@JSerFeng JSerFeng force-pushed the refactor/code-splitter branch from dab14a8 to d4e7793 Compare January 4, 2025 19:27
@JSerFeng JSerFeng marked this pull request as ready for review January 4, 2025 19:34
@JSerFeng JSerFeng force-pushed the refactor/code-splitter branch from d4e7793 to 159fb7d Compare January 4, 2025 20:42
@JSerFeng JSerFeng enabled auto-merge (squash) January 4, 2025 20:42
@JSerFeng JSerFeng force-pushed the refactor/code-splitter branch 2 times, most recently from ac5893a to 4526881 Compare January 5, 2025 20:37
@hardfist
Copy link
Copy Markdown
Contributor

hardfist commented Jan 6, 2025

maybe we should add a benchmark case with lots of dynamic import module first and to see the bench metrics avoid future regression

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jan 6, 2025

📝 Benchmark detail: Open

Name Base (2025-01-06 f81f1c7) Current Change
10000_big_production-mode_disable-minimize + exec 38 s ± 659 ms 38 s ± 421 ms +0.02 %
10000_development-mode + exec 1.91 s ± 39 ms 1.88 s ± 32 ms -1.57 %
10000_development-mode_hmr + exec 688 ms ± 25 ms 676 ms ± 27 ms -1.73 %
10000_production-mode + exec 2.51 s ± 29 ms 2.54 s ± 88 ms +1.12 %
arco-pro_development-mode + exec 1.78 s ± 139 ms 1.75 s ± 62 ms -1.71 %
arco-pro_development-mode_hmr + exec 378 ms ± 4.1 ms 378 ms ± 1.7 ms -0.05 %
arco-pro_production-mode + exec 3.61 s ± 65 ms 3.53 s ± 99 ms -2.27 %
arco-pro_production-mode_generate-package-json-webpack-plugin + exec 3.68 s ± 100 ms 3.67 s ± 84 ms -0.27 %
arco-pro_production-mode_traverse-chunk-modules + exec 3.64 s ± 80 ms 3.59 s ± 109 ms -1.29 %
threejs_development-mode_10x + exec 1.5 s ± 15 ms 1.5 s ± 12 ms -0.25 %
threejs_development-mode_10x_hmr + exec 771 ms ± 14 ms 765 ms ± 21 ms -0.75 %
threejs_production-mode_10x + exec 5.36 s ± 68 ms 5.38 s ± 157 ms +0.45 %
10000_big_production-mode_disable-minimize + rss memory 9499 MiB ± 279 MiB 9571 MiB ± 141 MiB +0.76 %
10000_development-mode + rss memory 666 MiB ± 20.7 MiB 653 MiB ± 22.6 MiB -1.97 %
10000_development-mode_hmr + rss memory 1433 MiB ± 386 MiB 1392 MiB ± 191 MiB -2.87 %
10000_production-mode + rss memory 623 MiB ± 22.8 MiB 628 MiB ± 18.4 MiB +0.92 %
arco-pro_development-mode + rss memory 567 MiB ± 29.5 MiB 559 MiB ± 35.5 MiB -1.25 %
arco-pro_development-mode_hmr + rss memory 625 MiB ± 64.7 MiB 598 MiB ± 56.6 MiB -4.36 %
arco-pro_production-mode + rss memory 722 MiB ± 54.8 MiB 702 MiB ± 65.9 MiB -2.73 %
arco-pro_production-mode_generate-package-json-webpack-plugin + rss memory 712 MiB ± 54.5 MiB 713 MiB ± 60 MiB +0.08 %
arco-pro_production-mode_traverse-chunk-modules + rss memory 725 MiB ± 62.4 MiB 713 MiB ± 51.2 MiB -1.61 %
threejs_development-mode_10x + rss memory 574 MiB ± 11.9 MiB 569 MiB ± 18.1 MiB -0.91 %
threejs_development-mode_10x_hmr + rss memory 1141 MiB ± 70.6 MiB 1118 MiB ± 198 MiB -2.02 %
threejs_production-mode_10x + rss memory 842 MiB ± 37.9 MiB 837 MiB ± 39.1 MiB -0.52 %

@JSerFeng JSerFeng force-pushed the refactor/code-splitter branch 11 times, most recently from bca01bd to 706ae15 Compare January 10, 2025 06:15
ahabhgk
ahabhgk previously approved these changes Jan 10, 2025
@JSerFeng JSerFeng force-pushed the refactor/code-splitter branch 6 times, most recently from 4a23903 to 4ea7c45 Compare January 13, 2025 10:41
@JSerFeng JSerFeng force-pushed the refactor/code-splitter branch from 4ea7c45 to 91c3ff2 Compare January 14, 2025 04:02
@chenjiahan chenjiahan changed the title refactor: code splitter refactor: introducing the faster code-splitting algorithm Jan 15, 2025
@JSerFeng JSerFeng force-pushed the refactor/code-splitter branch 2 times, most recently from 25c0c61 to 391f835 Compare January 16, 2025 08:21
@JSerFeng JSerFeng force-pushed the refactor/code-splitter branch from 391f835 to d308f3e Compare January 17, 2025 05:59
@JSerFeng JSerFeng force-pushed the refactor/code-splitter branch from d308f3e to 425c331 Compare January 17, 2025 07:49
@JSerFeng JSerFeng merged commit 58a6af6 into web-infra-dev:main Jan 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants