Skip to content

feat: add WebWritableStream for Web Streams API support#2376

Merged
fb55 merged 1 commit intofb55:masterfrom
vimzh:feat/web-streams-api
Mar 19, 2026
Merged

feat: add WebWritableStream for Web Streams API support#2376
fb55 merged 1 commit intofb55:masterfrom
vimzh:feat/web-streams-api

Conversation

@vimzh
Copy link
Copy Markdown
Contributor

@vimzh vimzh commented Mar 14, 2026

Summary

Adds a WebWritableStream class that wraps the Parser using the Web Streams API's WritableStream, enabling direct piping from fetch() response bodies into the parser.

Closes #1862

Motivation

The existing WritableStream wraps Node.js stream.Writable, which is incompatible with the Web Streams API. Users in browsers, Deno, Bun, or modern Node.js cannot pipe a fetch() response body into the parser without manual glue code.

Design

  • New file: src/WritableStreamWeb.ts — a WebWritableStream class extending globalThis.WritableStream<string | Uint8Array>
  • Subpath export: htmlparser2/WebWritableStream (mirrors the existing htmlparser2/WritableStream pattern)
  • Constructor: (cbs: Partial<Handler>, options?: ParserOptions) — same signature as the Node.js WritableStream
  • Chunk handling: accepts string (passed directly) and Uint8Array (decoded via TextDecoder in streaming mode for correct multi-byte UTF-8 handling)
  • Lifecycle: write() sink calls parser.write(), close() sink flushes the decoder then calls parser.end()

Usage

import { WebWritableStream } from "htmlparser2/WebWritableStream";

const stream = new WebWritableStream({
    onopentag(name, attribs) {
        console.log("Opened:", name);
    },
    ontext(text) {
        console.log("Text:", text);
    },
});

const response = await fetch("https://example.com");
await response.body.pipeTo(stream);

Test plan

  • Fragmented UTF-8 decoding via split Uint8Array chunks
  • String chunk handling
  • Empty stream (verifies onend fires)
  • Abort signal (verifies neither ontext nor onend fires)
  • ReadableStream.pipeTo() integration with HTML parsing assertions
  • Snapshot tests for all 6 fixture documents (Basic.html, Attributes.html, Svg.html, RSS, Atom, RDF)
  • Single-pass vs streamed consistency verification for each fixture
  • All existing 103 tests continue to pass (114 total with the 11 new tests)
  • ESLint, TypeScript (tsc --noEmit), and Biome all clean

The existing WritableStream wraps Node.js stream.Writable, which
isn't compatible with the Web Streams API. This makes it impossible
to pipe fetch() response bodies directly into the parser.

Add a new WebWritableStream class exposed via the subpath export
"htmlparser2/WebWritableStream" that wraps the Parser using the
standard Web Streams API WritableStream. It accepts both string
and Uint8Array chunks, using TextDecoder in streaming mode for
proper handling of fragmented multi-byte UTF-8 sequences.

Closes fb55#1862
@fb55 fb55 force-pushed the feat/web-streams-api branch from 5480151 to 4339aab Compare March 19, 2026 10:57
@fb55 fb55 merged commit 9008dfd into fb55:master Mar 19, 2026
6 checks passed
@fb55
Copy link
Copy Markdown
Owner

fb55 commented Mar 19, 2026

Thanks @vimzh! Rebased & moved the Web to a prefix, will be out with the next release

@vimzh vimzh deleted the feat/web-streams-api branch March 19, 2026 12:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support for the web streams API

2 participants