feat: add buffers() method returning Buffer[]#204
Conversation
Adds a new Source.prototype.buffers() method that returns the represented source code as an array of Buffers, avoiding the intermediate Buffer.concat allocation that buffer() performs on composite sources such as ConcatSource. Consumers that can accept Buffer[] (e.g. fs.createWriteStream backed by writev) can now keep data as Buffer[] across nested CachedSource/ConcatSource layers and write it out in one shot, which avoids repeatedly copying the same bytes. - Source: default buffers() returns [this.buffer()]. - ConcatSource: flattens child buffers() into a single Buffer[] without concatenating, and buffer() is now implemented via Buffer.concat(buffers()). - CachedSource: caches the Buffer[] separately from the Buffer; buffer() concatenates lazily when requested, so repeated buffers() calls do not trigger copies. - CompatSource: forwards to sourceLike.buffers() when available, otherwise falls back to the default. - SizeOnlySource: throws like buffer(). Closes #157
Covers the new Source.prototype.buffers() method across the cases where it matters: - concat-source: flat (10 raw) and nested (4x10 raw) — isolates the Buffer.concat avoided by buffers(). - cached-source: cold and warm paths against a CachedSource wrapping a ConcatSource, so both "first call" and "hit the cache" are observable. - compat-source: delegated vs super-fallback to mirror the buffer() benchmarks. - realistic-source-map-pipeline: reproduces the CachedSource -> ConcatSource -> CachedSource -> ConcatSource layering from issue #157 and measures cold + warm buffer() vs buffers(). Local wall-clock sample (not CodSpeed) shows ~3x on the flat ConcatSource and ~2.4x on the nested ConcatSource; warm paths are unchanged (CachedSource returns the cached array/buffer either way), which is the expected shape.
🦋 Changeset detectedLatest commit: c0d8294 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #204 +/- ##
==========================================
- Coverage 98.07% 97.79% -0.28%
==========================================
Files 25 25
Lines 1866 1908 +42
Branches 596 601 +5
==========================================
+ Hits 1830 1866 +36
- Misses 34 40 +6
Partials 2 2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Merging this PR will degrade performance by 96.42%
|
The previous CodSpeed run on #204 showed ±10-16% drift on tasks that weren't touched by the buffers() work (e.g. cached-source: size() (cached), cached-source: source() (cold), concat-source: getChildren()). Root cause is module-level shared state (warmed, warmedConcat, sink, *SourceLike) that grows whenever a new task is added to a case file. That state perturbs V8's hidden-class cache and GC heap layout for every task in the file, so adding a new task shifts pre-existing tasks' measurements. Two related fixes: 1. Teach the CodSpeed bridge about tinybench's beforeAll/beforeEach/ afterEach/afterAll hooks. The walltime path already honored them via tinybench itself, but the simulation path was just calling task.fn() raw, so any hook-based fixture setup was ignored. The bridge now runs beforeAll before warmup, beforeEach around every iteration (warmup and instrumented), afterEach after each, and afterAll after the instrumented pass. global.gc() still runs right before the instrumented call, after beforeEach. 2. Move heavy fixtures out of module scope into per-task beforeAll closures. Each case file's warmed/warmedConcat/warmLayeredChunk/ sourceLike/richSourceLike/sink now lives inside register() and is assigned in beforeAll and nulled in afterAll, so the set of tasks in a file only retains memory for the task currently running. Adding a future task to any of these files should no longer shift the pre-existing tasks' measurements. Also bumped warmupIterations from 2 to 10 in run.mjs so V8 hidden-class caches and the GC heap settle before the measured iteration. Side-effect visible in wall-clock numbers: new ConcatSource buffer/ buffers tasks now measure just the method call (construction moved to beforeAll), so the ratio grew from ~3x to ~10x on the flat 10-raw case and from ~2.4x to ~12x on nested 4x10. That's the honest comparison — the previous number was diluted by per-task fixture construction.
Follow-up to the _cachedSize deferral: PR #204 added a new `this._cachedBuffers = undefined` preallocation to the CachedSource constructor, which re-introduces the same per-construction cost we previously removed from RawSource/OriginalSource/SourceMapSource. `_cachedBuffers` is only read from `buffer()` and `buffers()`, and both call sites already guard with `!== undefined`, which treats missing properties identically to an undefined slot. Drop the eager assignment so every CachedSource construction skips one write and one hidden-class transition, and instances that never call `buffers()` stay on a tighter shape. https://claude.ai/code/session_01LZbaaPrnDTu6y7s4nK4cJz
Follow-up to the _cachedSize deferral: PR #204 added a new `this._cachedBuffers = undefined` preallocation to the CachedSource constructor, which re-introduces the same per-construction cost we previously removed from RawSource/OriginalSource/SourceMapSource. `_cachedBuffers` is only read from `buffer()` and `buffers()`, and both call sites already guard with `!== undefined`, which treats missing properties identically to an undefined slot. Drop the eager assignment so every CachedSource construction skips one write and one hidden-class transition, and instances that never call `buffers()` stay on a tighter shape. https://claude.ai/code/session_01LZbaaPrnDTu6y7s4nK4cJz
Adds a new Source.prototype.buffers() method that returns the represented
source code as an array of Buffers, avoiding the intermediate Buffer.concat
allocation that buffer() performs on composite sources such as
ConcatSource. Consumers that can accept Buffer[] (e.g. fs.createWriteStream
backed by writev) can now keep data as Buffer[] across nested
CachedSource/ConcatSource layers and write it out in one shot, which avoids
repeatedly copying the same bytes.
concatenating, and buffer() is now implemented via Buffer.concat(buffers()).
concatenates lazily when requested, so repeated buffers() calls do not
trigger copies.
falls back to the default.
Closes #157