Skip to content

feat: add buffers() method returning Buffer[]#204

Merged
alexander-akait merged 3 commits intomainfrom
claude/fix-issue-157-f18u4
Apr 22, 2026
Merged

feat: add buffers() method returning Buffer[]#204
alexander-akait merged 3 commits intomainfrom
claude/fix-issue-157-f18u4

Conversation

@alexander-akait
Copy link
Copy Markdown
Member

Adds a new Source.prototype.buffers() method that returns the represented
source code as an array of Buffers, avoiding the intermediate Buffer.concat
allocation that buffer() performs on composite sources such as
ConcatSource. Consumers that can accept Buffer[] (e.g. fs.createWriteStream
backed by writev) can now keep data as Buffer[] across nested
CachedSource/ConcatSource layers and write it out in one shot, which avoids
repeatedly copying the same bytes.

  • Source: default buffers() returns [this.buffer()].
  • ConcatSource: flattens child buffers() into a single Buffer[] without
    concatenating, and buffer() is now implemented via Buffer.concat(buffers()).
  • CachedSource: caches the Buffer[] separately from the Buffer; buffer()
    concatenates lazily when requested, so repeated buffers() calls do not
    trigger copies.
  • CompatSource: forwards to sourceLike.buffers() when available, otherwise
    falls back to the default.
  • SizeOnlySource: throws like buffer().

Closes #157

claude added 2 commits April 22, 2026 14:00
Adds a new Source.prototype.buffers() method that returns the represented
source code as an array of Buffers, avoiding the intermediate Buffer.concat
allocation that buffer() performs on composite sources such as
ConcatSource. Consumers that can accept Buffer[] (e.g. fs.createWriteStream
backed by writev) can now keep data as Buffer[] across nested
CachedSource/ConcatSource layers and write it out in one shot, which avoids
repeatedly copying the same bytes.

- Source: default buffers() returns [this.buffer()].
- ConcatSource: flattens child buffers() into a single Buffer[] without
  concatenating, and buffer() is now implemented via Buffer.concat(buffers()).
- CachedSource: caches the Buffer[] separately from the Buffer; buffer()
  concatenates lazily when requested, so repeated buffers() calls do not
  trigger copies.
- CompatSource: forwards to sourceLike.buffers() when available, otherwise
  falls back to the default.
- SizeOnlySource: throws like buffer().

Closes #157
Covers the new Source.prototype.buffers() method across the cases where it
matters:

- concat-source: flat (10 raw) and nested (4x10 raw) — isolates the
  Buffer.concat avoided by buffers().
- cached-source: cold and warm paths against a CachedSource wrapping a
  ConcatSource, so both "first call" and "hit the cache" are observable.
- compat-source: delegated vs super-fallback to mirror the buffer()
  benchmarks.
- realistic-source-map-pipeline: reproduces the
  CachedSource -> ConcatSource -> CachedSource -> ConcatSource layering
  from issue #157 and measures cold + warm buffer() vs buffers().

Local wall-clock sample (not CodSpeed) shows ~3x on the flat ConcatSource
and ~2.4x on the nested ConcatSource; warm paths are unchanged (CachedSource
returns the cached array/buffer either way), which is the expected shape.
@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented Apr 22, 2026

🦋 Changeset detected

Latest commit: c0d8294

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
webpack-sources Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@linux-foundation-easycla
Copy link
Copy Markdown

linux-foundation-easycla Bot commented Apr 22, 2026

CLA Not Signed

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 22, 2026

Codecov Report

❌ Patch coverage is 92.30769% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 97.79%. Comparing base (2081a31) to head (c0d8294).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
lib/CachedSource.js 81.81% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #204      +/-   ##
==========================================
- Coverage   98.07%   97.79%   -0.28%     
==========================================
  Files          25       25              
  Lines        1866     1908      +42     
  Branches      596      601       +5     
==========================================
+ Hits         1830     1866      +36     
- Misses         34       40       +6     
  Partials        2        2              
Flag Coverage Δ
integration 97.79% <92.30%> (-0.28%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented Apr 22, 2026

Merging this PR will degrade performance by 96.42%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 7 improved benchmarks
❌ 74 regressed benchmarks
✅ 66 untouched benchmarks
🆕 14 new benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
concat-source: getChildren() 40.7 µs 119.5 µs -65.99%
🆕 cached-source: buffer() (warm, wraps ConcatSource x10) N/A 21 µs N/A
concat-source: new ConcatSource() (10 raw) 170.2 µs 261.2 µs -34.84%
🆕 cached-source: buffer() (cold, wraps ConcatSource x10) N/A 9 ms N/A
concat-source: size() 117.7 µs 197.3 µs -40.33%
concat-source: map() 19.6 ms 22 ms -11%
concat-source: source() (10 raw) 71.3 µs 132.8 µs -46.34%
concat-source: nested flattening 310.5 µs 725.8 µs -57.22%
cached-source: getCachedData() then restore 18.3 ms 20.6 ms -11.09%
compat-source: sourceAndMap() 186.2 µs 236.9 µs -21.41%
concat-source: source() (mixed) 48.6 µs 132.3 µs -63.29%
compat-source: source() (wrapping SourceLike) 134.2 µs 153.9 µs -12.79%
concat-source: sourceAndMap() 19.6 ms 21.9 ms -10.62%
🆕 cached-source: buffers() (cached) N/A 20.7 µs N/A
cached-source: buffer() (cached) 19.4 µs 27.1 µs -28.6%
🆕 cached-source: buffers() (cold, wraps ConcatSource x10) N/A 5.1 ms N/A
cached-source: map() (cold SourceMapSource) 54.2 µs 172.5 µs -68.59%
compat-source: buffer() (delegated) 152.2 µs 170.4 µs -10.68%
cached-source: source() (cached) 19.8 µs 552.8 µs -96.42%
cached-source: sourceAndMap() (cold) 17 ms 19.4 ms -12.02%
... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.


Comparing claude/fix-issue-157-f18u4 (c0d8294) with main (58420d1)

Open in CodSpeed

The previous CodSpeed run on #204 showed ±10-16% drift on tasks that
weren't touched by the buffers() work (e.g. cached-source: size() (cached),
cached-source: source() (cold), concat-source: getChildren()). Root cause
is module-level shared state (warmed, warmedConcat, sink, *SourceLike)
that grows whenever a new task is added to a case file. That state
perturbs V8's hidden-class cache and GC heap layout for every task in the
file, so adding a new task shifts pre-existing tasks' measurements.

Two related fixes:

1. Teach the CodSpeed bridge about tinybench's beforeAll/beforeEach/
   afterEach/afterAll hooks. The walltime path already honored them via
   tinybench itself, but the simulation path was just calling task.fn()
   raw, so any hook-based fixture setup was ignored. The bridge now runs
   beforeAll before warmup, beforeEach around every iteration (warmup and
   instrumented), afterEach after each, and afterAll after the
   instrumented pass. global.gc() still runs right before the
   instrumented call, after beforeEach.

2. Move heavy fixtures out of module scope into per-task beforeAll
   closures. Each case file's warmed/warmedConcat/warmLayeredChunk/
   sourceLike/richSourceLike/sink now lives inside register() and is
   assigned in beforeAll and nulled in afterAll, so the set of tasks in
   a file only retains memory for the task currently running. Adding a
   future task to any of these files should no longer shift the
   pre-existing tasks' measurements.

Also bumped warmupIterations from 2 to 10 in run.mjs so V8 hidden-class
caches and the GC heap settle before the measured iteration.

Side-effect visible in wall-clock numbers: new ConcatSource buffer/
buffers tasks now measure just the method call (construction moved to
beforeAll), so the ratio grew from ~3x to ~10x on the flat 10-raw case
and from ~2.4x to ~12x on nested 4x10. That's the honest comparison —
the previous number was diluted by per-task fixture construction.
@alexander-akait alexander-akait merged commit 9788e55 into main Apr 22, 2026
27 of 31 checks passed
@alexander-akait alexander-akait deleted the claude/fix-issue-157-f18u4 branch April 22, 2026 15:06
alexander-akait pushed a commit that referenced this pull request Apr 22, 2026
Follow-up to the _cachedSize deferral: PR #204 added a new
`this._cachedBuffers = undefined` preallocation to the CachedSource
constructor, which re-introduces the same per-construction cost we
previously removed from RawSource/OriginalSource/SourceMapSource.

`_cachedBuffers` is only read from `buffer()` and `buffers()`, and both
call sites already guard with `!== undefined`, which treats missing
properties identically to an undefined slot. Drop the eager assignment
so every CachedSource construction skips one write and one hidden-class
transition, and instances that never call `buffers()` stay on a
tighter shape.

https://claude.ai/code/session_01LZbaaPrnDTu6y7s4nK4cJz
alexander-akait pushed a commit that referenced this pull request Apr 22, 2026
Follow-up to the _cachedSize deferral: PR #204 added a new
`this._cachedBuffers = undefined` preallocation to the CachedSource
constructor, which re-introduces the same per-construction cost we
previously removed from RawSource/OriginalSource/SourceMapSource.

`_cachedBuffers` is only read from `buffer()` and `buffers()`, and both
call sites already guard with `!== undefined`, which treats missing
properties identically to an undefined slot. Drop the eager assignment
so every CachedSource construction skips one write and one hidden-class
transition, and instances that never call `buffers()` stay on a
tighter shape.

https://claude.ai/code/session_01LZbaaPrnDTu6y7s4nK4cJz
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve performance by offering a buffers() method that returns Buffer[]

2 participants