Avoid Node's promise pipeline by piping streams directly #2633
Conversation
|
Warning Rate limit exceeded@ardatan has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 14 minutes and 29 seconds before requesting another review. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📒 Files selected for processing (5)
📝 WalkthroughSummary by CodeRabbit
""" WalkthroughThis change refactors internal stream handling in the Changes
Sequence Diagram(s)sequenceDiagram
participant IncomingMessage
participant PassThrough
participant pipeThrough
participant Consumer
IncomingMessage->>PassThrough: Data stream
pipeThrough->>PassThrough: Pipe IncomingMessage into PassThrough (with abort/error handling)
PassThrough->>Consumer: Provides readable stream
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~15 minutes Suggested reviewers
Poem
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
✅
|
✅
|
✅
|
✅
|
✅
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
packages/node-fetch/src/utils.ts (1)
52-120: Successful refactor that achieves PR objectives.The transformation from
wrapIncomingMessageWithPassthroughtopipeThroughsuccessfully eliminates the dependency on Node's promise-based pipeline while maintaining all necessary functionality. The more generic interface increases reusability across the codebase.Consider adding JSDoc comments to document the function's behavior, especially the error handling semantics and the performance characteristics that motivated this refactor.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
packages/node-fetch/src/utils.ts(3 hunks)
🧰 Additional context used
🧠 Learnings (1)
📓 Common learnings
Learnt from: ardatan
PR: ardatan/whatwg-node#0
File: :0-0
Timestamp: 2025-01-29T19:45:47.290Z
Learning: The `handleNodeRequestAndResponse` method in `@whatwg-node/server` can be used to integrate with Fastify by passing both the request and reply objects in the route handler, along with the context containing `req` and `reply`.
Learnt from: ardatan
PR: ardatan/whatwg-node#0
File: :0-0
Timestamp: 2025-01-29T19:45:36.028Z
Learning: The node-fetch package tries different fetch implementations in the following order: 1) node-libcurl 2) undici 3) built-in undici in Node 4) node:http as fallback
🧬 Code Graph Analysis (1)
packages/node-fetch/src/utils.ts (3)
packages/node-fetch/src/ReadableStream.ts (1)
pipeThrough(246-262)packages/server/src/utils.ts (1)
signal(663-668)packages/node-fetch/src/Request.ts (1)
signal(113-116)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (13)
- GitHub Check: unit / node 24
- GitHub Check: unit / deno
- GitHub Check: e2e / azure-function
- GitHub Check: e2e / aws-lambda
- GitHub Check: unit / node 20
- GitHub Check: unit / node 18
- GitHub Check: unit / bun
- GitHub Check: alpha / snapshot
- GitHub Check: server (undici)
- GitHub Check: server (ponyfill)
- GitHub Check: server (native)
- GitHub Check: node-fetch (noConsumeBody)
- GitHub Check: node-fetch (consumeBody)
🔇 Additional comments (5)
packages/node-fetch/src/utils.ts (5)
2-2: Import changes align well with the refactor.The transition from specific stream types (
IncomingMessage,PassThrough) to generic stream interfaces (Readable,Writable) makes the utility more flexible and reusable.
70-76: Correct error handling for pipe operations.The manual destruction of the destination stream when the source errors is the correct approach, as Node.js doesn't automatically close the writable stream in error scenarios during pipe operations.
78-97: Robust abort signal handling with proper cleanup.The implementation correctly uses
WeakRefto prevent memory leaks and provides comprehensive cleanup by listening to multiple stream events. The abort handling properly destroys the stream with the customAbortError.
114-120: Clean implementation of custom AbortError class.The
AbortErrorclass follows standard patterns for custom error classes and aligns with Node.js internal implementations. The default message and name property are appropriate.
52-100: ApprovepipeThroughimplementation; please benchmark against Node’s built-insThe
pipeThroughrefactoring is sound: it gives you explicit error- and abort-signal control, and the use ofWeakReffor cleanup is a solid memory-leak safeguard. We’ve confirmed that Node.js exports bothaddAbortSignal(fromnode:stream) andfinished(fromnode:stream/promises) as functions, so your manual implementations are valid fallbacks.To substantiate the “faster than Node’s built-in utilities” claim, please add a brief benchmark comparing:
- manual abort handling (current
onAbort/cleanup)addAbortSignal(signal, src)+finished(src, cleanup)This will ensure the performance assertion is backed by data before merging.
This reverts commit 6afd027.
Alternative to and closes #2632
This repo does not have proper benchmarking set up to have detected this regression, but other repo's have - like feTS:
~9% increase in perf benchmarking Hive Gateway, memory consumption feels lower too
~20% increase in perf benchmarking feTS