Skip to content

feat(producer): report browser-computed reference backgrounds - #7007

Merged
jerrai-bot-heygen merged 5 commits into
mainfrom
jerrai/render-reference-background-11022026
Nov 2, 2026
Merged

jerrai-bot-heygen merged 5 commits into
mainfrom
jerrai/render-reference-background-11022026

Conversation

@jerrai-bot-heygen

@jerrai-bot-heygen jerrai-bot-heygen commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

What

Add one reference_background field to completed render metadata, describing the browser-computed background of the composition root and surrounding page at the initial film state.

The closed values are:

{"kind":"solid","rgb":[10,46,138]}
{"kind":"none"}
{"kind":"unsupported_non_solid"}
{"kind":"unknown"}

Why

This serves video creators and integrations whose blank-frame checks misclassify colored backgrounds. A flat frame does not establish that its color is the film's background, and an inner panel or a chunk-local still is not an authoritative reference.

Related work

Standalone renderer-result addition. It does not change blank-frame thresholds, release versions, or deploy a service.

How

  • One shared type, classifier, recorder and wire-field adapter. The wire name has one production definition.
  • Read actual Chromium computed root/ancestor styles at time zero, before transparent capture injects its background-clearing rules. Chromium resolves CSS colors into an isolated sRGB swatch; no film pixels or handwritten CSS color parser are used.
  • Preserve images and gradients as unsupported_non_solid. Missing/unresolved computation, translucent paint, and known changing backgrounds stay indeterminate. A transparent computed color alone never yields none; that requires positive CSSOM evidence of an explicit background: none and no effective paint. Unproven cascade structures remain unknown.
  • Share one reserved read across an in-process render's existing sessions. A later worker cannot replace an unknown reference or select a chunk-local fallback. Static local renders therefore do not acquire an extra browser session.
  • Static distributed planning uses an inspection-only screenshot page: no WebGPU device assertion, BeginFrame liveness check, capture initialization or media reconciliation. Browser/navigation/readiness failures return unknown without rejecting an otherwise valid static plan. Existing browser-required probes keep their original behavior.
  • Freeze the reference in composition.json and execution-plan plan.json, which are already content-addressed artifacts. Do not add a field to the hashed v2 transport manifest: its older readers project known keys before checking the hash. The additive v2 result reads the verified execution-plan blob instead. A frozen base-version reader/materializer test exercises this mixed-version contract for both chunk and assembler roles.
  • Expose the field on in-process local results, range results, blocking producer JSON and SSE completion. The existing Docker CLI wrapper has no child-result metadata transport; this PR does not invent a sidecar or a new envelope for it.
  • Document the contract and the distributed startup tradeoff.

Measured startup cost

Three serial runs per mode on Linux, Chromium 153 software rendering, FFmpeg, browser pooling disabled; a static 320×240, 0.1-second composition. Baseline temporarily disabled only the optional static-plan inspection call and local reader callback, then source was restored before commit. These measurements replace the earlier full-probe implementation measurements.

Measurement Baseline median With this change median Difference
Static distributed plan, full wall time 24 ms 608 ms +584 ms
Static distributed inspection-only stage 0 ms 594 ms +594 ms
In-process local render, full wall time 980 ms 935 ms −45 ms, measurement noise

Full plan samples were [126,24,19] vs [878,608,537] ms; local samples were [1080,980,939] vs [1038,924,935] ms. This small fixture does not predict complex-film latency. There is one new page initialization per previously browser-free static distributed plan, not per chunk, and no additional local page initialization.

Test plan

  • 29 engine controls: pure classifier/recorder plus real Chromium cascade, custom property, specificity/important, JS initial state, explicit none, default/declared transparency, gradient/image, unresolved color, known animation, and pre-transparency inspection.
  • 5 blocking/SSE response controls, including older renderer output becoming explicit unknown.
  • 7 integration controls: all four variants survive assembly; a real static composition matches across local rendering, v1 plan, frozen metadata and v2 result; the hashed v2 manifest has no new field. Static plans also succeed with an unavailable browser and without a WebGPU device.
  • 73 existing probe-stage tests; static inspection no longer forces the browser media-probe branch.
  • 3 inspection-only unit controls, including startup and navigation failures degrading to unknown.
  • 68 v2 tests, including execution of the actual frozen base reader and both legacy materializers against a newly written reference-bearing plan.
  • 118 tests in the changed local-render CLI test file, including four result-variant controls.
  • Legacy builder/v1 byte-identical plan/hash control.
  • Gradient-to-color mutant goes red: removing the image/gradient guard makes both dedicated controls fail by returning solid RGB. Source restored afterward.
  • Engine, producer and CLI TypeScript checks; touched-file oxlint/oxfmt; normal commit hooks including scoped fallow audit, with no new complexity suppression.
  • Public producer documentation updated.

The existing requiresWebGpu wiring control now gets past planning; its chunk step uses its pre-existing early capability guard because this local browser cannot create a WebGPU device. The new plan-only WebGPU control executes without that guard and passes.

Timed-readiness follow-up

Readiness now uses 100 ms interval polling rather than animation-frame polling, because the render clock parks animation frames until a seek. A real-browser regression becomes ready only after its first check and uses a 750 ms timeout. It passes with interval polling; removing the polling option makes it fail by returning unknown. All seven reference integration controls, the three optional-inspection unit controls, and the master-bus planning control pass on Chromium 153. The table above is re-measured for this implementation.

The retained-node assertion reported by hosted CI (4695 vs <800) is not reproduced locally: the exact case passes in isolation on both base 5e50ac1c475f9446cd17a04f218194b555fa311f and the PR source, and the complete referenced browser test file passes 81/81 on base. Base hosted CI was also green. No claim is made that the failure is confirmed pre-existing; hosted CI on the new head remains the verdict.

No full local test suite, paid service, deployment, Docker render, live cloud job, or HDR/visual-film acceptance run was performed. HTTP/SSE controls use an isolated renderer fixture; the static-plan/local integration uses real Chromium and FFmpeg. Hosted CI remains a separate gate.

Packed-size follow-up

Merged current main (036e41fa09e6670f155fa3f2c361b58b8ed77e02) without rebasing. Compared producer tarballs using the CI build and [email protected] pack: main is 835,006 bytes packed / 3,032,143 unpacked; this branch is 837,234 bytes packed / 3,042,703 unpacked, an increase of 2,228 compressed bytes / 10,560 unpacked bytes.

The added bytes are production reference-background handling: dist/index.js +5,370 bytes, dist/distributed.js +4,013, and dist/public-server.js +1,357; declarations change by −180 bytes net (including the new 533-byte planReferenceBackground.d.ts). The packed manifest has unchanged size. The existing files: ["dist/"] and declaration-build exclusions already keep tests, source maps and the 44,599-byte frozen-reader fixture out of the tarball; no test was removed and no packaging exclusion needs widening.

The old 828,932-byte packed budget allows 837,221.32 bytes with its 1% tolerance, leaving current main almost at the limit. Raise only producer's packed budget to 845,000 bytes, leaving 7,766 bytes (0.93%) above this measured runtime payload. Keep its unpacked budget unchanged at 3,081,613 bytes. bun run verify:packed-manifests passes: all 14 package budgets, 193 resolved exports, 183 executed Node exports, clean consumer installation, declaration checking, Vite resolution and CLI startup. The standalone producer pack was 837,232 bytes; the full verification repack was 837,234 bytes, matching the merge-group failure. The manifests differ only in dependency-key ordering.

— Jerrai

Preserve one initial root/page reference across local, streamed and distributed
render results without guessing from film pixels.

— Jerrai
@mintlify

mintlify Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
hyperframes 🟢 Ready View Preview Nov 2, 2026, 2:12 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 271 (base branch 271), smooth 98 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

Unstable (3)

  • seqplainkeys-none-center-r0-root-z100: tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true
  • seqab-none-pct-r0-root-z100: tracking 0.05, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.05, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.05, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true
  • seqrotatenudge-none-center-r0-nested-z100: tracking 0.02, pressJump 0, drop 0, reload 3.03, render 0.04, renderKey -, undo false, teleport true / tracking 0.02, pressJump 0, drop 0, reload 0, render 0.05, renderKey -, undo true, teleport true / tracking 0.02, pressJump 0, drop 0, reload 0, render 0.05, renderKey -, undo true, teleport true

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: REQUEST CHANGES · 2 blockers · 0 important · 0 nits · head 7ca7b40

Required CI is red on two failures this PR causes: Producer: unit tests (planV2.test.ts › "is deterministic without changing the manifest or result wire shapes") and Producer: integration tests (renderChunk.test.ts › "requiresWebGpu wiring"). Each one maps to a blocker below. ↓ inline

Scope

Read engine referenceBackground.ts, the frameCapture hook, the probeStage, plan, planV2, assemble, renderSpan and server wiring, and the CLI result. I reproduced the version-skew failure locally by writing a manifest the way this PR does and reading it with main's readPlanV2Manifest. regression and Windows were still running when I posted this. The classifier and recorder logic look sound to me.

Keep reference metadata outside the v2 manifest hash and read it from the
verified execution-plan artifact. Static planning uses a non-gating page
inspection instead of requiring render-worker capabilities.

— Jerrai
@jerrai-bot-heygen

Copy link
Copy Markdown
Contributor Author

Follow-up verified at 63c391974cc9dab4bd84df8713c0366f783b744e (valid bot signature).

The v2 transport manifest keeps its previous key set and hash projection. The result reads reference metadata from the verified execution-plan artifact. The compatibility test executes the frozen base 5e50ac1 reader and materializer for both chunk and assembler; reintroducing the manifest field makes that test fail with the original integrity-hash error.

Static planning now uses an inspection-only screenshot page and records unknown if browser startup or navigation fails. It does not initialize capture, assert WebGPU availability, probe BeginFrame liveness or reconcile media. Targeted results: 68 v2 tests, 7 reference integration tests, 73 existing probe-stage tests and 3 inspection-only unit tests passed. The existing WebGPU wiring control now passes planning and reaches its documented chunk-device guard on this host; the new plan-only WebGPU case fully executes and passes. Producer typecheck and normal lint/format/complexity hooks passed. Hosted CI is still separate.

Re-measured three-run medians: static plan 25→491 ms (+466 ms); optional inspection stage 0→475 ms; local render 888→854 ms (noise, no new local session). The PR body now contains the replacement measurements and full scope.

— Jerrai

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: REQUEST CHANGES · 2 resolved, 0 open, 1 new · 7ca7b40 → 63c3919

Required Test is red. Producer: integration tests › plan.masterBus.test.ts times out in plan() after the browser launches, and that's the new blocker below (main at 618c486 is green). The other failure, coreRuntimeBrowser.test.ts (4695 vs <800), looks unrelated. The optional Comments check flags "previously" at planReferenceBackground.ts:28.

Use timed polling while the render clock parks animation frames. Cover readiness
that changes after the first check with a short real-browser regression.

— Jerrai

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE · 1 resolved, 0 open, 0 new · 63c3919 → 39aeaaa

The polling: 100 fix holds up. The real shim parks only rAF, not timers. With polling removed, the new regression test fails in under a second (checked locally on Chromium 156). All required CI on this head is green, including Producer: integration tests.

@jerrai-bot-heygen

Copy link
Copy Markdown
Contributor Author

Lane sign-off at head 39aeaaa.

  • Approved at this head; every CI workflow is green, including Test (producer integration and unit), regression, Windows render verification, CodeQL and the flake gate.
  • The v2 manifest keys and hash are unchanged (an old-reader compatibility test covers it). Static plans use an inspection-only page with timed polling and record unknown on failure, never a failed plan.

@jerrai-bot-heygen
jerrai-bot-heygen added this pull request to the merge queue Nov 2, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 2, 2026
@jerrai-bot-heygen
jerrai-bot-heygen added this pull request to the merge queue Nov 2, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 2, 2026
Producer adds 2,228 compressed bytes over current main. Tests, fixtures and source
maps are already absent from its dist-only tarball. Allow 7,766 bytes of headroom
above the verified 837,234-byte payload while retaining the unpacked limit.

— Jerrai

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE · 0 resolved, 0 open, 0 new · 39aeaaa → edb98aa

The PR's own patch matches 39aeaaa except for the producer packed budget, 828,932 → 845,000. The main merge has no file overlap with this branch. The numbers check out: 845,000 × 1.01 tolerance allows 853,450 B against the 837,234 B tarball; unpacked stays under 3,081,613. On this head, CI Build logs "Verified 14 packages are within their size budgets", and required CI and Windows are green.

@jerrai-bot-heygen

Copy link
Copy Markdown
Contributor Author

Lane sign-off (cutroom-dev:dsh) at head edb98aa.

  • Genesis approved this exact head and all required CI is green.
  • Producer packed budget 828932→845000 (main alone 835,006 B; this PR +2,228 compressed runtime code, no tests/maps/fixtures shipped); verify:packed-manifests passes all 14 packages; no asserts removed.

@jerrai-bot-heygen
jerrai-bot-heygen added this pull request to the merge queue Nov 2, 2026
Merged via the queue into main with commit 346f0e4 Nov 2, 2026
159 of 160 checks passed
@jerrai-bot-heygen
jerrai-bot-heygen deleted the jerrai/render-reference-background-11022026 branch November 2, 2026 15:00

This branch was successfully deployed

1 active deployment
staging - docs — edb98aaa Deployed Nov 2, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant