Skip to content

Comments

perf: reduce temporary memory usage by avoiding output chunk clone until needed#5058

Merged
shulaoda merged 1 commit intomainfrom
06-25-perf_reduce_temporary_memory_usage_by_avoiding_output_chunk_cloning_until_needed
Jun 25, 2025
Merged

perf: reduce temporary memory usage by avoiding output chunk clone until needed#5058
shulaoda merged 1 commit intomainfrom
06-25-perf_reduce_temporary_memory_usage_by_avoiding_output_chunk_cloning_until_needed

Conversation

@sapphi-red
Copy link
Member

@sapphi-red sapphi-red commented Jun 25, 2025

These two lines clones the whole bundle output. This is because args.bundle is Vec<Output> where Output is a enum that contains a Box.


While this .clone is inevitable to allow referencing the bundle parameter of generateBundle hook after the hook has end, the content of bundle parameter does not need to have a separate value, because it is readonly (the modification is done via ChangedOutputs instead). This PR changes Box to Arc so that we can share the underlying value.

This PR will have a bigger impact if the output is large. For example, astro build with https://github.com/withastro/astro.build used ~8GB of memory previously (if the GC does not run frequently enough). With this PR, it now uses ~3GB.

refs #1082

Copy link
Member Author

sapphi-red commented Jun 25, 2025

This stack of pull requests is managed by Graphite. Learn more about stacking.

@sapphi-red sapphi-red changed the base branch from 06-25-perf_skip_empty_write_bundle_hook_with_hook_usage_meta to graphite-base/5058 June 25, 2025 04:44
@sapphi-red sapphi-red force-pushed the 06-25-perf_reduce_temporary_memory_usage_by_avoiding_output_chunk_cloning_until_needed branch from f3ddb46 to c9c48c2 Compare June 25, 2025 04:44
@sapphi-red sapphi-red force-pushed the graphite-base/5058 branch from 302e90e to 35d8e15 Compare June 25, 2025 04:44
@sapphi-red sapphi-red changed the base branch from graphite-base/5058 to main June 25, 2025 04:44
@sapphi-red sapphi-red force-pushed the 06-25-perf_reduce_temporary_memory_usage_by_avoiding_output_chunk_cloning_until_needed branch from c9c48c2 to a3d4959 Compare June 25, 2025 04:54
@netlify
Copy link

netlify bot commented Jun 25, 2025

Deploy Preview for rolldown-rs canceled.

Name Link
🔨 Latest commit a3d4959
🔍 Latest deploy log https://app.netlify.com/projects/rolldown-rs/deploys/685b811540a85c000811aa7c

@sapphi-red sapphi-red marked this pull request as ready for review June 25, 2025 04:55
@sapphi-red sapphi-red changed the title perf: reduce temporary memory usage by avoiding output chunk cloning until needed perf: reduce temporary memory usage by avoiding output chunk clone until needed Jun 25, 2025
@github-actions
Copy link
Contributor

Benchmarks Rust

  • target: main(35d8e15)
  • pr: 06-25-perf_reduce_temporary_memory_usage_by_avoiding_output_chunk_cloning_until_needed(a3d4959)
group                                                        pr                                     target
-----                                                        --                                     ------
bundle/bundle@multi-duplicated-top-level-symbol              1.00     75.8±2.57ms        ? ?/sec    1.01     76.2±1.56ms        ? ?/sec
bundle/bundle@multi-duplicated-top-level-symbol-sourcemap    1.00     86.1±1.22ms        ? ?/sec    1.02     87.4±1.93ms        ? ?/sec
bundle/bundle@rome_ts                                        1.02    115.8±3.10ms        ? ?/sec    1.00    113.4±1.65ms        ? ?/sec
bundle/bundle@rome_ts-sourcemap                              1.00    127.5±1.51ms        ? ?/sec    1.01    128.2±1.32ms        ? ?/sec
bundle/bundle@threejs                                        1.00     40.4±1.58ms        ? ?/sec    1.00     40.5±0.43ms        ? ?/sec
bundle/bundle@threejs-sourcemap                              1.00     46.8±0.41ms        ? ?/sec    1.01     47.3±0.89ms        ? ?/sec
bundle/bundle@threejs10x                                     1.00    436.3±8.22ms        ? ?/sec    1.00    435.1±3.48ms        ? ?/sec
bundle/bundle@threejs10x-sourcemap                           1.00    506.6±2.77ms        ? ?/sec    1.00    507.9±6.87ms        ? ?/sec
scan/scan@rome_ts                                            1.00     88.4±1.63ms        ? ?/sec    1.00     88.1±1.15ms        ? ?/sec
scan/scan@threejs                                            1.00     30.8±0.44ms        ? ?/sec    1.00     30.8±0.52ms        ? ?/sec
scan/scan@threejs10x                                         1.00    321.9±4.40ms        ? ?/sec    1.00    321.9±8.11ms        ? ?/sec

@sapphi-red
Copy link
Member Author

sapphi-red commented Jun 25, 2025

@shulaoda shulaoda requested review from IWANABETHATGUY and hyf0 June 25, 2025 05:35
Copy link
Member

@IWANABETHATGUY IWANABETHATGUY left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice

@shulaoda shulaoda added this pull request to the merge queue Jun 25, 2025
Merged via the queue into main with commit 299e8c0 Jun 25, 2025
26 checks passed
@shulaoda shulaoda deleted the 06-25-perf_reduce_temporary_memory_usage_by_avoiding_output_chunk_cloning_until_needed branch June 25, 2025 06:55
Boshen added a commit that referenced this pull request Jun 25, 2025
## [1.0.0-beta.20] - 2025-06-25

### 🚀 Features

- rolldown: oxc v0.75.0 (#5069) by @Boshen
- rolldown: oxc 0.74.0 (#5047) by @Boshen

### 🐛 Bug Fixes

- vite-tests: avoid crash when `packageJson.pnpm` is undefined (#5066)
by @shulaoda
- JSX preset options not working correctly (#5060) by @shulaoda
- remove improper jsdoc of builtin define plugin (#5046) by
@AliceLanniste

### 🚜 Refactor

- passing vec length directly (#5050) by @ShroXd

### 📚 Documentation

- explain that a chunk may be bigger than `maxSize` (#5014) by
@sapphi-red
- update example about `withFilter` (#5040) by @IWANABETHATGUY

### ⚡ Performance

- preallocate smaller vec for hash placeholder finder (#5064) by
@sapphi-red
- reduce temporary memory usage by avoiding output chunk clone until
needed (#5058) by @sapphi-red
- skip empty write bundle hook with `hook_usage` meta (#5057) by
@sapphi-red

### ⚙️ Miscellaneous Tasks

- deps: update dependency rolldown-plugin-dts to v0.13.12 (#5065) by
@renovate[bot]
- rolldown: upgrade NAPI-RS to 3.0.0-beta.10 (#5063) by @Brooooooklyn
- bump oxc-resolver to v11.2.1 (#5061) by @Boshen
- deps: update dependency tsdown to v0.12.9 (#5056) by @renovate[bot]
- deps: update dependency rolldown-plugin-dts to v0.13.12 (#5054) by
@renovate[bot]
- remove long-commented-out code (#5048) by @shulaoda
- deps: lock file maintenance npm packages (#5042) by @renovate[bot]
- deps: lock file maintenance rust crates (#5043) by @renovate[bot]
- deps: update taiki-e/install-action action to v2.54.0 (#5045) by
@renovate[bot]
- deps: update taiki-e/install-action action to v2.53.2 (#5041) by
@renovate[bot]

Co-authored-by: Boshen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants