Skip to content

Commit ec06f4b

Browse files
vanceingallsclaude
andauthored
feat(engine,producer): drawElement fast-capture default-on with runtime self-verification safety net (#1998)
* feat(engine,producer): drawElement fast-capture default-on with runtime self-verification safety net Flip useDrawElement + worker-encode defaults on (HF_DE_BATCH default 4), clamped in resolveConfig to hosts where drawElement can engage (macOS + hardware-GPU browser) so page-side shader compositing is untouched everywhere else; explicit env opt-in keeps attempt-and-gate semantics. Safety net makes default-on safe: the compile/init gates catch predictable incompatibility; this catches the intermittent residue no static analysis can see (stale paints, dropped background images, transient blank frames). - engine: captureDeVerificationFrames — K=4 (HF_DE_VERIFY) ground-truth screenshots at init, after gates + armStaticDedup, BEFORE canvas injection (post-injection screenshots show the canvas bitmap, not the DOM). Runs the video-injection hook per sample; double-captures so rAF-driven text counters settle (a single immediate screenshot captures stale text and false-positives). Skips png, <10 frames, implausible __hf.duration (infinite-repeat GSAP sentinel). - producer: guardFrame on both worker-encode drains — rolling-median blank guard with retry-once at drain (byte-identical retry ⇒ deterministic dark frame, accepted; retry save/restores the static-dedup anchor) + ffmpeg PSNR self-verify vs ground truth (HF_DE_VERIFY_MIN_DB, default 32dB; natural agreement ≥45dB, damage ≤25dB). Breach dumps the frame pair to tmpdir and throws DrawElementVerificationError. - orchestrator: one-shot retry — on verification error the whole render re-runs with forceScreenshot (slower, never wrong); telemetry flag deSelfVerifyFallback. - tooling: de-canary-suite.sh (7-comp release gate with expected verdicts), de-gatecheck.sh (init-only corpus routing classifier), we-render.mjs. Validated: canary suite 7/7; 611-comp routing sample 54% drawelement / 37.5% gated / 8.3% comp-defect; 12/12 risk-band renders clean on bare defaults (48/48 verify samples); engine suite 888 passed; caught two real intermittent damage classes in the wild (background-image drop, root-props offset) that previously shipped silently. Kill switches: PRODUCER_EXPERIMENTAL_FAST_CAPTURE=false, HF_DE_WORKER_ENCODE=false, HF_DE_BATCH=0, HF_DE_VERIFY=0. Co-Authored-By: Claude Fable 5 <[email protected]> * fix(engine,producer): harden the drawElement self-verification net (max code-review findings) 15 confirmed findings from the adversarial review of the default-on flip; the load-bearing five: - Ground-truth capture no longer scrubs GSAP state: seek(0) + forced frame FIRST (lazy .from()/overlap tweens record start values on first seek — mid-timeline scrubs corrupted them for the whole render, and since DE frames and truth shared the corruption, PSNR passed on damaged output), then ascending even-spread fractions, page left at frame 0. - Default-on drawElement is confined to the verified path: resolveConfig requires worker-encode (the drain that runs the net), the orchestrator disengages the default when the render takes the disk path or parallel capture (no drain verification there), and closes a drawElement-initialized probe session rather than letting the unverified path reuse it. Explicit PRODUCER_EXPERIMENTAL_FAST_CAPTURE=true keeps old attempt-and-gate behavior. - Blank-frame retry can no longer splice wrong-frame pixels: recapture goes through recaptureDrawElementFrameForVerify — no static-dedup shortcut (lastEncodeResult runs ahead of the drain) and no "No cached paint record" screenshot fallback (post-injection that captures the canvas = the LAST drawn frame); any recapture failure falls back the whole render. - Verify indices derive from the producer-resolved duration (CaptureOptions.compositionDurationSeconds) instead of raw __hf.duration, so samples always land inside the drained range. - The platform clamp accepts "auto" GPU mode — the stock CLI resolves auto, and the literal-"hardware" clamp made default-on a no-op for the primary audience (masked in validation by explicitly-set env). Also: NaN-safe env parses (HF_DE_VERIFY / HF_DE_VERIFY_MIN_DB / HF_DE_BATCH); video comps skip verification when the session has no frame injector (probe sessions — black-video truth false-positived); psnr infrastructure failures skip the sample instead of failing the render; boundary-saturated sample indices are skipped; shader-transition comps prefer page-side compositing over default drawElement and compile-gated comps get page-side compositing restored; observability.clearFailure un-brands the recovered first streaming attempt; canary suite exempts known-marginal "any" comps from the cross-path PSNR gate; dead we-render options removed; clamp tests pin their env. Validated: canary suite 7/7; auto-GPU bare render engages the full stack; disk-path and worker-encode-off renders disengage default drawElement; malformed HF_DE_VERIFY_MIN_DB still verifies at the default threshold; engine suite 890 passed. Co-Authored-By: Claude Fable 5 <[email protected]> * fix(engine,producer): review fixes — PSC intent, verify-threshold clamp, fail-closed canaries Addresses miguel-heygen's review on #1998: - Page-side compositing restore preserves explicit caller intent (blocker): resolveConfig now records pageSideCompositingAutoDisabled only when IT turned page-side compositing off because drawElement was on; the compile-time drawElement gates restore page-side compositing only when that flag is set. An explicit enablePageSideCompositing:false from the programmatic API or HF_PAGE_SIDE_COMPOSITING=false stays off. Pinned by two config tests. - HF_DE_VERIFY_MIN_DB clamped to [10, 60] with a warning on out-of-range values: below ~10dB the check passes severe damage; above ~60dB natural encoder differences force a screenshot fallback on every verified render. - de-canary-suite.sh + de-gatecheck.sh run under set -euo pipefail with explicit `|| true` on expected-nonzero commands (render exits handled by the suite's own checks, grep no-match, kill/pkill/wait races) and a hard FAIL when the PSNR compare produces no value — release canaries fail closed. Full suite re-run green (7/7) under the new flags. Co-Authored-By: Claude Fable 5 <[email protected]> --------- Co-authored-by: Claude Fable 5 <[email protected]>
1 parent d5ecb01 commit ec06f4b

17 files changed

Lines changed: 764 additions & 88 deletions

File tree

‎.fallowrc.jsonc‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414
"packages/producer/src/runtime-conformance.ts",
1515
"packages/producer/src/benchmark.ts",
1616
"packages/producer/scripts/generate-font-data.ts",
17+
"packages/producer/we-render.mjs",
1718
"packages/producer/scripts/validate-fast-video.ts",
1819
"packages/cli/scripts/generate-font-data.ts",
1920
"packages/engine/scripts/test-fitTextFontSize-browser.ts",

‎packages/cli/src/commands/render.ts‎

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -361,11 +361,12 @@ export default defineCommand({
361361
"experimental-fast-capture": {
362362
type: "boolean",
363363
description:
364-
"EXPERIMENTAL. Capture frames via Chrome's drawElementImage API " +
365-
"instead of Page.captureScreenshot — reads DOM paint records directly, " +
366-
"~46% faster on GPU. Transparent (PNG) renders on SwiftShader (Docker) " +
367-
"auto-fall back to screenshot capture. Incompatible with page-side " +
368-
"shader compositing. Default: false. Env: PRODUCER_EXPERIMENTAL_FAST_CAPTURE.",
364+
"Capture frames via Chrome's drawElementImage API instead of " +
365+
"Page.captureScreenshot — reads DOM paint records directly, ~2x faster. " +
366+
"Default: on where it can engage (macOS + hardware-GPU browser); " +
367+
"incompatible compositions and self-verification failures fall back to " +
368+
"screenshot capture automatically. Pass =false to disable. " +
369+
"Env: PRODUCER_EXPERIMENTAL_FAST_CAPTURE.",
369370
// No `default` — an omitted flag must stay `undefined` so the `!= null`
370371
// guard below leaves PRODUCER_EXPERIMENTAL_FAST_CAPTURE untouched and the
371372
// env fallback survives (matches the --low-memory-mode idiom).

‎packages/engine/src/config.test.ts‎

Lines changed: 47 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -222,17 +222,53 @@ describe("resolveConfig", () => {
222222
});
223223

224224
describe("useDrawElement (PRODUCER_EXPERIMENTAL_FAST_CAPTURE)", () => {
225-
it("defaults to false", () => {
225+
it("default is clamped off on software-GPU hosts (page-side compositing preserved)", () => {
226+
setEnv("PRODUCER_BROWSER_GPU_MODE", "software");
227+
unsetEnv("PRODUCER_EXPERIMENTAL_FAST_CAPTURE");
228+
unsetEnv("HF_DE_WORKER_ENCODE");
229+
const config = resolveConfig();
230+
expect(config.useDrawElement).toBe(false);
231+
expect(config.enablePageSideCompositing).toBe(true);
232+
});
233+
234+
it("default engages on macOS with a hardware-GPU browser", () => {
235+
setEnv("PRODUCER_BROWSER_GPU_MODE", "hardware");
236+
unsetEnv("PRODUCER_EXPERIMENTAL_FAST_CAPTURE");
237+
unsetEnv("HF_DE_WORKER_ENCODE");
238+
const config = resolveConfig();
239+
expect(config.useDrawElement).toBe(process.platform === "darwin");
240+
});
241+
242+
it("default engages on macOS with auto GPU mode (the stock CLI path)", () => {
243+
setEnv("PRODUCER_BROWSER_GPU_MODE", "auto");
244+
unsetEnv("PRODUCER_EXPERIMENTAL_FAST_CAPTURE");
245+
unsetEnv("HF_DE_WORKER_ENCODE");
246+
const config = resolveConfig();
247+
expect(config.useDrawElement).toBe(process.platform === "darwin");
248+
});
249+
250+
it("default requires worker-encode (the verified drain)", () => {
251+
setEnv("PRODUCER_BROWSER_GPU_MODE", "hardware");
252+
setEnv("HF_DE_WORKER_ENCODE", "false");
253+
unsetEnv("PRODUCER_EXPERIMENTAL_FAST_CAPTURE");
226254
const config = resolveConfig();
227255
expect(config.useDrawElement).toBe(false);
228256
});
229257

230-
it("enabled when PRODUCER_EXPERIMENTAL_FAST_CAPTURE=true", () => {
258+
it("explicit env opt-in skips the platform clamp", () => {
259+
setEnv("PRODUCER_BROWSER_GPU_MODE", "software");
231260
setEnv("PRODUCER_EXPERIMENTAL_FAST_CAPTURE", "true");
232261
const config = resolveConfig();
233262
expect(config.useDrawElement).toBe(true);
234263
});
235264

265+
it("env kill switch wins over the default", () => {
266+
setEnv("PRODUCER_BROWSER_GPU_MODE", "hardware");
267+
setEnv("PRODUCER_EXPERIMENTAL_FAST_CAPTURE", "false");
268+
const config = resolveConfig();
269+
expect(config.useDrawElement).toBe(false);
270+
});
271+
236272
it("explicit override wins over the env var", () => {
237273
setEnv("PRODUCER_EXPERIMENTAL_FAST_CAPTURE", "true");
238274
const config = resolveConfig({ useDrawElement: false });
@@ -243,6 +279,15 @@ describe("resolveConfig", () => {
243279
const config = resolveConfig({ useDrawElement: true, enablePageSideCompositing: true });
244280
expect(config.useDrawElement).toBe(true);
245281
expect(config.enablePageSideCompositing).toBe(false);
282+
// The auto-disable is recorded so compile-time gates can restore it.
283+
expect(config.pageSideCompositingAutoDisabled).toBe(true);
284+
});
285+
286+
it("does NOT mark auto-disabled when the caller explicitly opted out of page-side compositing", () => {
287+
const config = resolveConfig({ useDrawElement: true, enablePageSideCompositing: false });
288+
expect(config.enablePageSideCompositing).toBe(false);
289+
// Explicit caller intent — a compile-time drawElement gate must not restore it.
290+
expect(config.pageSideCompositingAutoDisabled).not.toBe(true);
246291
});
247292

248293
it("leaves page-side compositing on when fast capture is off", () => {

‎packages/engine/src/config.ts‎

Lines changed: 56 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -64,20 +64,30 @@ export interface EngineConfig {
6464
*/
6565
staticFrameDedup: boolean;
6666
/**
67-
* EXPERIMENTAL. Use drawElementImage for frame capture (requires the
68-
* CanvasDrawElement Chrome flag, added globally in buildChromeArgs).
69-
* Surfaced via the CLI `--experimental-fast-capture` flag.
70-
* Env fallback: `PRODUCER_EXPERIMENTAL_FAST_CAPTURE`.
67+
* Use drawElementImage for frame capture (requires the CanvasDrawElement
68+
* Chrome flag, added globally in buildChromeArgs). Default ON, clamped in
69+
* `resolveConfig` to hosts where it can actually engage (macOS + hardware-GPU
70+
* browser); compile/init gates and the runtime self-verification net route
71+
* incompatible or damaged renders back to screenshot capture.
72+
* Kill switch: `PRODUCER_EXPERIMENTAL_FAST_CAPTURE=false` (or the CLI
73+
* `--experimental-fast-capture=false`).
7174
*/
7275
useDrawElement: boolean;
7376
/**
74-
* EXPERIMENTAL. Pipeline JPEG encode into an in-page OffscreenCanvas Worker
75-
* for the drawElement fast-capture path (macOS hardware GPU only). The worker
76-
* encodes frame N while the main thread seeks+paints frame N+1, targeting
77-
* ~1.65–1.96× wall-time speedup. No-op unless `useDrawElement` is also true.
78-
* Default: off. Env: `HF_DE_WORKER_ENCODE=true`.
77+
* Pipeline JPEG encode into an in-page OffscreenCanvas Worker for the
78+
* drawElement fast-capture path (macOS hardware GPU only). The worker
79+
* encodes frame N while the main thread seeks+paints frame N+1
80+
* (~1.65–1.96× wall-time speedup). No-op unless `useDrawElement` is also
81+
* true. Kill switch: `HF_DE_WORKER_ENCODE=false`.
7982
*/
8083
enableDrawElementWorkerEncode: boolean;
84+
/**
85+
* INTERNAL. Set by resolveConfig when it disabled enablePageSideCompositing
86+
* solely because drawElement was on. Lets the producer's compile-time gates
87+
* restore page-side compositing without overriding an explicit caller/env
88+
* opt-out. Not intended to be set by callers.
89+
*/
90+
pageSideCompositingAutoDisabled?: boolean;
8191
/**
8292
* Low-memory render profile. When `true`, the orchestrator collapses the
8393
* pipeline to its cheapest shape on memory-constrained hosts: it skips the
@@ -250,8 +260,8 @@ export const DEFAULT_CONFIG: EngineConfig = {
250260
protocolTimeout: 300_000,
251261
forceScreenshot: false,
252262
staticFrameDedup: true,
253-
useDrawElement: false,
254-
enableDrawElementWorkerEncode: false,
263+
useDrawElement: true,
264+
enableDrawElementWorkerEncode: true,
255265
// Auto-detected per host in `resolveConfig`; defaults off for the raw
256266
// DEFAULT_CONFIG (used directly by tests and worker-sizing fallbacks).
257267
lowMemoryMode: false,
@@ -516,15 +526,48 @@ export function resolveConfig(overrides?: Partial<EngineConfig>): EngineConfig {
516526
...overrides,
517527
};
518528

529+
// Default-on drawElement is clamped to hosts where it can actually engage
530+
// (macOS with a non-software-GPU browser; SwiftShader drops transparent
531+
// sub-layers — crbug 521434899). "auto" passes the clamp: the stock CLI
532+
// resolves GPU mode to auto, which probes to hardware on real Macs — and if
533+
// it resolves to software after all, the SwiftShader init-time gate still
534+
// routes the session to the screenshot baseline. Without the clamp, the
535+
// default would needlessly disable page-side shader compositing (below) on
536+
// Linux/Docker hosts where DE never runs. An EXPLICIT opt-in (env or caller override)
537+
// skips the clamp and keeps the old semantics — attempt DE, let the
538+
// init-time gates route away — which debugging relies on.
539+
const explicitDrawElementOptIn =
540+
env("PRODUCER_EXPERIMENTAL_FAST_CAPTURE") === "true" || overrides?.useDrawElement === true;
541+
if (
542+
merged.useDrawElement &&
543+
!explicitDrawElementOptIn &&
544+
!(process.platform === "darwin" && merged.browserGpuMode !== "software")
545+
) {
546+
merged.useDrawElement = false;
547+
}
548+
// The runtime self-verification net lives in the worker-encode drain — the
549+
// serial drawElement path has only the blank guard. Default-on drawElement
550+
// therefore requires worker-encode; disabling HF_DE_WORKER_ENCODE without an
551+
// explicit drawElement opt-in falls back to the screenshot baseline rather
552+
// than shipping unverified drawElement frames.
553+
if (merged.useDrawElement && !explicitDrawElementOptIn && !merged.enableDrawElementWorkerEncode) {
554+
merged.useDrawElement = false;
555+
}
556+
519557
// drawElement capture and page-side shader compositing are mutually
520558
// incompatible capture strategies (drawElement reads paint records directly
521559
// and bypasses the page-side prepare→composite→resolve protocol). When
522-
// experimental fast capture is on, force page-side compositing off so shader
560+
// fast capture is on, force page-side compositing off so shader
523561
// transitions fall back to the Node-side layered blend rather than silently
524562
// dropping. This keeps the flag self-consistent and avoids a per-session
525563
// incompatibility warning on every fast-capture render.
526-
if (merged.useDrawElement) {
564+
if (merged.useDrawElement && merged.enablePageSideCompositing) {
527565
merged.enablePageSideCompositing = false;
566+
// Record that THIS resolution (not the caller) turned page-side
567+
// compositing off, so a later compile-time drawElement gate can restore
568+
// it without clobbering an explicit enablePageSideCompositing:false from
569+
// the programmatic API or HF_PAGE_SIDE_COMPOSITING=false.
570+
merged.pageSideCompositingAutoDisabled = true;
528571
}
529572

530573
return {

‎packages/engine/src/index.ts‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,9 @@ export {
8484
captureFrameToBuffer,
8585
captureFrameToBufferPipelined,
8686
captureFramesBatchPipelined,
87+
DrawElementVerificationError,
88+
isDrawElementVerificationError,
89+
recaptureDrawElementFrameForVerify,
8790
writeCapturedFrame,
8891
discardWarmupCapture,
8992
getCompositionDuration,

0 commit comments

Comments
 (0)