Skip to content

Commit 6944a1c

Browse files
committed
feat(engine): surface protocolTimeout env + flag in Puppeteer timeout errors
Field signal ts=1784047847 (darwin/arm64, 8GB M1, 9 videos + 22 images): reporter hit Runtime.callFunctionOn timeout and switched to FFmpeg because the error didn't surface HyperFrames' existing knobs (PRODUCER_PUPPETEER_PROTOCOL_TIMEOUT_MS env, --protocol-timeout CLI). Wraps main-render Puppeteer errors matching /Runtime\.callFunctionOn timed out|Target closed|protocolTimeout/i with an augmented message that names the effective timeout, the env var, the CLI flag, and the field-signal shape. Non-matching errors pass through unchanged (returned as the same instance). Original error preserved via err.cause. Also adds a dedicated --protocol-timeout row to the CLI docs Flags table so PRODUCER_PUPPETEER_PROTOCOL_TIMEOUT_MS is discoverable via search. Signed-off-by: Via <[email protected]>
1 parent 35e623b commit 6944a1c

5 files changed

Lines changed: 145 additions & 1 deletion

File tree

‎docs/packages/cli.mdx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -718,6 +718,7 @@ Word-level transcripts (whisper output) are grouped into readable caption cues o
718718
| `--variables-file` | path | — | Path to a JSON file with variable overrides (alternative to `--variables`) |
719719
| `--strict-variables` | — | off | Fail render if any `--variables` key is undeclared or has a wrong type vs the composition's `data-composition-variables`. Without this flag, mismatches print as warnings and the render continues. |
720720
| `--browser-timeout` | seconds (0.001–86400) | 60 | Puppeteer page-navigation timeout for the entry HTML. Increase when heavy compositions (many videos, fonts, or asset requests) cannot reach `domcontentloaded` within the default 60 s. The flag takes **seconds**; the env fallback `PRODUCER_PAGE_NAVIGATION_TIMEOUT_MS` takes **milliseconds**. This controls `page.goto` only — very heavy compositions may also need `PRODUCER_PUPPETEER_PROTOCOL_TIMEOUT_MS` and/or `PRODUCER_PLAYER_READY_TIMEOUT_MS` bumped (post-navigation `window.__hf` readiness has its own 45 s budget). |
721+
| `--protocol-timeout` | milliseconds (≥ 1000) | 300000 (5 min) | Puppeteer CDP protocol timeout — the per-call budget for `Runtime.callFunctionOn` seek/paint, `Page.captureScreenshot`, and other CDP round-trips. Raise on RAM-pressured hosts (≤ 8 GB), heavy-asset compositions (many videos + images), or when the render fails with `Runtime.callFunctionOn timed out` / `Target closed`. The default is auto-scaled per composition by output pixel area (a 4K comp bumps the ceiling proportionally, capped at 30 min); an explicit override sets the floor and disables scaling below it. Env fallback `PRODUCER_PUPPETEER_PROTOCOL_TIMEOUT_MS` (also **milliseconds**). |
721722

722723
CRF and target bitrate default to the `--quality` preset. Use `--crf` or `--video-bitrate` for fine-grained overrides; `RenderConfig.crf` and `RenderConfig.videoBitrate` accept the same overrides programmatically. Use `--video-frame-format png` when source videos are UI recordings, screen captures, or other color-sensitive clips that should avoid JPEG frame extraction.
723724

‎packages/engine/src/index.ts‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,10 @@ export {
7979
type CaptureMode,
8080
type AcquiredBrowser,
8181
} from "./services/browserManager.js";
82+
export {
83+
augmentProtocolTimeoutError,
84+
isProtocolTimeoutError,
85+
} from "./services/protocolTimeoutErrorHint.js";
8286

8387
// ── Frame capture pipeline ──────────────────────────────────────────────────────
8488
export {
Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
import { describe, expect, it } from "vitest";
2+
import { augmentProtocolTimeoutError, isProtocolTimeoutError } from "./protocolTimeoutErrorHint.js";
3+
4+
describe("augmentProtocolTimeoutError", () => {
5+
it("passes non-timeout errors through unchanged (same instance)", () => {
6+
const original = new Error("V8 heap exhausted");
7+
const result = augmentProtocolTimeoutError(original, 300_000);
8+
expect(result).toBe(original);
9+
expect(result.message).toBe("V8 heap exhausted");
10+
});
11+
12+
it("augments Runtime.callFunctionOn timed out with the effective timeout", () => {
13+
const original = new Error(
14+
"Runtime.callFunctionOn timed out. Increase the 'protocolTimeout' setting.",
15+
);
16+
const result = augmentProtocolTimeoutError(original, 600_000);
17+
expect(result).not.toBe(original);
18+
expect(result.message).toContain(original.message);
19+
expect(result.message).toContain("HyperFrames effective protocolTimeout: 600000 ms");
20+
});
21+
22+
it("includes both env and CLI hints", () => {
23+
const original = new Error("Runtime.callFunctionOn timed out");
24+
const result = augmentProtocolTimeoutError(original, 300_000);
25+
expect(result.message).toContain("PRODUCER_PUPPETEER_PROTOCOL_TIMEOUT_MS");
26+
expect(result.message).toContain("--protocol-timeout");
27+
});
28+
29+
it("preserves err.cause on the augmented error", () => {
30+
const original = new Error("Runtime.callFunctionOn timed out");
31+
const result = augmentProtocolTimeoutError(original, 300_000);
32+
expect((result as Error & { cause?: unknown }).cause).toBe(original);
33+
});
34+
35+
it("augments Target closed errors", () => {
36+
const original = new Error("Protocol error (Runtime.callFunctionOn): Target closed.");
37+
const result = augmentProtocolTimeoutError(original, 300_000);
38+
expect(result).not.toBe(original);
39+
expect(result.message).toContain("HyperFrames effective protocolTimeout");
40+
});
41+
42+
it("matches the protocolTimeout keyword case-insensitively", () => {
43+
const original = new Error("some upstream saying PROTOCOLTIMEOUT was hit");
44+
const result = augmentProtocolTimeoutError(original, 300_000);
45+
expect(result).not.toBe(original);
46+
});
47+
48+
it("coerces non-Error thrown values into Error without augmenting", () => {
49+
const result = augmentProtocolTimeoutError("plain string failure", 300_000);
50+
expect(result).toBeInstanceOf(Error);
51+
expect(result.message).toBe("plain string failure");
52+
// Not augmented: coerced string doesn't match the protocol-timeout regex.
53+
expect(result.message).not.toContain("HyperFrames effective protocolTimeout");
54+
});
55+
56+
it("mentions the field-signal shape reporters hit", () => {
57+
const original = new Error("Runtime.callFunctionOn timed out");
58+
const result = augmentProtocolTimeoutError(original, 300_000);
59+
expect(result.message).toContain("ts=1784047847");
60+
expect(result.message).toContain("FFmpeg-only encoding");
61+
});
62+
});
63+
64+
describe("isProtocolTimeoutError", () => {
65+
it("returns true for matching messages", () => {
66+
expect(isProtocolTimeoutError(new Error("Runtime.callFunctionOn timed out"))).toBe(true);
67+
expect(isProtocolTimeoutError(new Error("Target closed"))).toBe(true);
68+
expect(isProtocolTimeoutError("protocolTimeout exceeded")).toBe(true);
69+
});
70+
71+
it("returns false for non-matching messages", () => {
72+
expect(isProtocolTimeoutError(new Error("V8 heap exhausted"))).toBe(false);
73+
expect(isProtocolTimeoutError(new Error("Navigation timeout of 60000 ms exceeded"))).toBe(
74+
false,
75+
);
76+
expect(isProtocolTimeoutError(null)).toBe(false);
77+
expect(isProtocolTimeoutError(undefined)).toBe(false);
78+
expect(isProtocolTimeoutError(42)).toBe(false);
79+
});
80+
});
Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
/**
2+
* Augment Puppeteer CDP protocol-timeout errors with actionable guidance that
3+
* points at the HyperFrames-specific knobs. Puppeteer's stock error text
4+
* ("Runtime.callFunctionOn timed out. Increase the 'protocolTimeout' setting")
5+
* doesn't tell the user which env var / CLI flag raises this timeout in
6+
* HyperFrames, or what the currently-applied effective value is — so field
7+
* reporters have hit this class of failure, given up, and switched to
8+
* FFmpeg-only encoding rather than raise a knob they didn't know existed
9+
* (field signal ts=1784047847).
10+
*
11+
* The helper is deliberately conservative:
12+
* - Only augments errors whose message matches known protocol-timeout
13+
* strings (`Runtime.callFunctionOn timed out`, `Target closed`,
14+
* `protocolTimeout`).
15+
* - Non-matching errors are returned unchanged (same instance).
16+
* - Non-Error inputs are coerced with `new Error(String(err))` so callers
17+
* get a well-typed `Error` back regardless of what was thrown.
18+
* - The original error is preserved via `err.cause`, so stack introspection
19+
* and downstream logging still see the raw Puppeteer message.
20+
*/
21+
22+
const PROTOCOL_TIMEOUT_MATCHER = /Runtime\.callFunctionOn timed out|Target closed|protocolTimeout/i;
23+
24+
export function augmentProtocolTimeoutError(err: unknown, effectiveTimeoutMs: number): Error {
25+
if (!(err instanceof Error)) return new Error(String(err));
26+
if (!PROTOCOL_TIMEOUT_MATCHER.test(err.message)) return err;
27+
const augmented = new Error(
28+
`${err.message}\n\n` +
29+
`HyperFrames effective protocolTimeout: ${effectiveTimeoutMs} ms.\n\n` +
30+
`To raise the timeout:\n` +
31+
` Env: PRODUCER_PUPPETEER_PROTOCOL_TIMEOUT_MS=<higher-ms>\n` +
32+
` CLI: --protocol-timeout <higher-ms>\n\n` +
33+
`Field signal ts=1784047847: this class of failure appears on RAM-pressured hosts with heavy-asset compositions (9+ videos + 20+ images). If raising the timeout doesn't help, consider FFmpeg-only encoding.`,
34+
);
35+
(augmented as Error & { cause?: unknown }).cause = err;
36+
return augmented;
37+
}
38+
39+
/**
40+
* Predicate variant: exposed for callers that only need to classify an error
41+
* (e.g. observability, tests) without materialising an augmented Error. Uses
42+
* the same matcher as the augmentation path so the two never drift.
43+
*/
44+
export function isProtocolTimeoutError(err: unknown): boolean {
45+
const message = err instanceof Error ? err.message : typeof err === "string" ? err : "";
46+
return PROTOCOL_TIMEOUT_MATCHER.test(message);
47+
}

‎packages/producer/src/services/renderOrchestrator.ts‎

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,7 @@ import {
7878
isTransientBrowserError,
7979
isDrawElementVerificationError,
8080
getDrawElementVerificationDetails,
81+
augmentProtocolTimeoutError,
8182
} from "@hyperframes/engine";
8283
import { join, dirname, resolve } from "path";
8384
import { totalmem } from "node:os";
@@ -3143,7 +3144,18 @@ export async function executeRenderJob(
31433144
// Retry burn on a render that STILL failed — the actionable signal for tuning
31443145
// MAX_TRANSIENT_CAPTURE_RETRIES (mirrors the success-path record above).
31453146
recordTransientRetryObservability();
3146-
const errorMessage = memoryGuidance ?? normalizeErrorMessage(error);
3147+
// Surface HyperFrames' PRODUCER_PUPPETEER_PROTOCOL_TIMEOUT_MS env +
3148+
// --protocol-timeout CLI in Puppeteer CDP protocol-timeout errors. Puppeteer's
3149+
// stock "Runtime.callFunctionOn timed out. Increase the 'protocolTimeout'
3150+
// setting" text doesn't name the HyperFrames knob and doesn't state the
3151+
// effective timeout that was already applied (300000 ms base + auto-scaling
3152+
// via `scaleProtocolTimeoutForComposition`). Field signal ts=1784047847
3153+
// reporter gave up on HF and switched to FFmpeg because the error didn't
3154+
// point them at the lever. `augmentProtocolTimeoutError` returns the input
3155+
// unchanged when the message doesn't match, so non-timeout failures (memory
3156+
// exhaustion, other CDP errors) flow through with no change.
3157+
const protocolTimeoutError = augmentProtocolTimeoutError(error, cfg.protocolTimeout);
3158+
const errorMessage = memoryGuidance ?? normalizeErrorMessage(protocolTimeoutError);
31473159
const carriedBrowserConsole = getCaptureStageBrowserConsole(error);
31483160
if (carriedBrowserConsole.length > 0) {
31493161
lastBrowserConsole = [...lastBrowserConsole, ...carriedBrowserConsole].slice(-200);

0 commit comments

Comments
 (0)