Session Replay: introduce `ScheduledStreamCapture` (streaming leading chunks + precise end-of-window export) to survive multiple rrweb checkouts by matux · Pull Request #1371 · rollbar/rollbar.js

matux · 2025-10-13T17:57:23Z

Note

PR Title/Description mostly generated by AI, guided by me and diff

Description of the change

This PR replaces the single-shot leading capture with a streaming capture strategy that exports and sends leading replay data in small, sequential chunks during the postDuration window—eliminating event loss across multiple rrweb checkouts.

Why

With long postDuration, rrweb’s periodic checkout can overwrite early leading events before a single-shot export runs.
Streaming chunking (< checkout interval) ensures no event ages out.
We keep strict sequential sending (abort on first failure) to avoid rrweb gaps.

What changed

New controller: ScheduledStreamCapture
Exports every chunkMs, queues payloads, and opportunistically sends if trailing is SENT and nothing is in flight.
Adds a one-shot endTimer at postDuration to do a final boundary export (captures the [tail] from last cursor up to end-of-window), then flush & cleanup.
Enforces at-most-one POST in flight per replay.
Abort on first leading POST failure (prevents gaps); no abort for “no events” exports.
Robust cleanup paths:
Immediate finalize if finished && queue empty && not sending
Finalize at end of sendIfReady() when finished === true
discard(replayId) stops timers, clears queue, and completes.
ReplayManager integration: uses ScheduledStreamCapture instead of ScheduledCapture; keeps existing gating:
Gating only on trailing send (headers/quota).
Telemetry per chunk (trailing + each leading).
No new headers/attrs; server orders by span timestamps.

Behavior summary

Chunk cadence: currently 1:1 with checkoutEveryNms() (design supports clamping later).
Final export at exact window end via endTimer (prevents “6s” stray export).
Sequential sends: sendIfReady() drains queue FIFO; any POST failure ⇒ abort stream.
Multiple triggers: independent controllers; no merge/pause.
No page-unload special handling (unchanged).

Files

src/browser/replay/scheduledStreamCapture.js — new streaming controller
src/browser/replay/replayManager.js — wires in streaming controller; unchanged trailing semantics
src/browser/replay/recorder.js — unchanged (source of truth for checkout interval, cursor, collection)

Tests

Updated/added integration tests to verify invariants (not implementation-specific call counts):

replayManager.bufferIndex.test.js
replayManager.bufferIndex.checkoutResilience.test.js
Added: “sends exactly ceil(postDuration / checkoutInterval) leading chunks covering the window” (deterministic with fake timers + emitEveryNms).
Adjusted brittle “exactly twice” assertions to focus on:
first leading export starts at the captured cursor
≥1 leading POST when expected; all share same X-Rollbar-Replay-Id
sequential sends (no overlap)
cleanup after success/failure/discard
monotonic cursor advancement across chunks

Type of change

Bug fix (non-breaking change that fixes an issue)
New feature (non-breaking change that adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Maintenance
New release

Related issues

CAT-485/extended-post-trigger-replay-duration

some renames wat rename abort to discard dont remove import

Copilot

Pull Request Overview

This PR introduces a streaming capture strategy for session replay to replace single-shot leading capture. The new approach exports and sends replay data in sequential chunks during the postDuration window to prevent event loss when rrweb performs periodic checkouts that could overwrite early events.

Implements ScheduledStreamCapture class for streaming chunk captures at safe intervals
Replaces ScheduledCapture with ScheduledStreamCapture in the replay manager
Updates tests to accommodate the new streaming behavior with multiple chunk exports instead of single exports

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
`src/browser/replay/scheduledStreamCapture.js`	New streaming controller that exports chunks periodically and manages sequential sending
`src/browser/replay/replayManager.js`	Integrates the new streaming capture controller and adjusts error logging
`test/replay/integration/replayManager.bufferIndex.test.js`	Adds assertion to verify trigger existence
`test/replay/integration/replayManager.bufferIndex.checkoutResilience.test.js`	Updates tests to handle multiple chunk exports and adds new deterministic test case

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

matux · 2025-10-13T18:25:14Z

+  schedule(replayId, occurrenceUuid, postDuration) {
+    const startTime = Date.now();
+    const endAt = startTime + postDuration * 1000;
+    const chunkMs = this._recorder.checkoutEveryNms();


For now, I'm running this at exactly checkoutEveryNms and it seems to work fine, I did some testing with a multiplier k ≈ 0.6...0.8, but it resulted in more chunks, ergo more post calls.

Streaming leading chunks + precise end-of-window export

* Add _pendingContextIfReady and rename cursor in context (#1364) * Remove index.js from tests (#1366) * Init logger on replay integration tests to log errors (#1367) * Rethrow span export errors to prevent unnecessary sendIfReady calls (#1368) * Call _onComplete on all leading replay discard paths (#1369) * `ScheduledStreamCapture` to survive multiple rrweb checkouts (#1371)

matux added 10 commits October 13, 2025 14:52

Initial implementation

078f37b

some renames wat rename abort to discard dont remove import

rename

7ed620c

fix

2a69fcb

logic logic logic

557d8d9

export before end-gate check

b4d6127

rename exportChunk to export and type

d73a0d7

pending context if ready

3cf7329

opportunistic send

a984332

overzealous cleanup

d45d0cd

cleanup

4bb3557

matux requested review from Copilot and waltjones October 13, 2025 17:57

matux self-assigned this Oct 13, 2025

Copilot AI reviewed Oct 13, 2025

View reviewed changes

Comment thread src/browser/replay/scheduledStreamCapture.js

Comment thread src/browser/replay/replayManager.js

silly

40e842c

matux commented Oct 13, 2025

View reviewed changes

matux marked this pull request as ready for review October 13, 2025 18:55

waltjones approved these changes Oct 13, 2025

View reviewed changes

matux merged commit 71f6f6b into feature/matux/streaming-capture Oct 13, 2025
6 checks passed

matux deleted the matux/stream-capture branch October 13, 2025 21:51

matux added a commit that referenced this pull request Oct 13, 2025

ScheduledStreamCapture to survive multiple rrweb checkouts (#1371)

483f201

Streaming leading chunks + precise end-of-window export

matux mentioned this pull request Oct 13, 2025

Add ScheduledStreamCapture for extended leading replay support #1372

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Session Replay: introduce `ScheduledStreamCapture` (streaming leading chunks + precise end-of-window export) to survive multiple rrweb checkouts#1371

Session Replay: introduce `ScheduledStreamCapture` (streaming leading chunks + precise end-of-window export) to survive multiple rrweb checkouts#1371
matux merged 11 commits into
feature/matux/streaming-capturefrom
matux/stream-capture

matux commented Oct 13, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

matux Oct 13, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

matux commented Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of the change

Why

What changed

Behavior summary

Files

Tests

Type of change

Related issues

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

matux Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

matux commented Oct 13, 2025 •

edited

Loading