Repository navigation
Commit 84e36fc
feat(runtime): data-vfx-chain backdrop capture and two-source kernels (luma-matte, noise) (#4330)
* feat(runtime): data-vfx-chain backdrop capture and ref second sources
Interface v1.1's two remaining capture shapes, in the runtime.
`backdrop`: a host resolves its source from a sibling
`<canvas layoutsubtree class="hf-vfx-src" data-vfx-for="<host id>">`
before its own child wrapper, so the same kernel reads either its own
pixels or every layer below it — which element the wrapper holds is the
exporter's placement, not a def's declaration. `VfxEntry.capture` now
reports what was resolved rather than what the defs asked for.
The trap that shape creates: a `backdrop` wrapper lives OUTSIDE the host,
and children of a `layoutsubtree` canvas are never painted by the page
compositor. Skipping a hidden host the way a `self` host is skipped would
therefore delete every layer below the adjustment instead of passing it
through. A hidden backdrop host now still captures and KEEPS the bitmap,
painting no kernel.
`ref` params: a node's second source, captured into `u_src2` from the
wrapper the exporter emitted around the referenced element. Per node, not
per entry, and shared when two nodes name the same element. Every source
is drawn at the host's device size so `u_src`/`u_src2`/`.hf-vfx-out`
share one coordinate space, and each source's canvas gets its own
`paint` wait — paint records are per canvas. A source the clip runtime
has hidden captures EMPTY rather than waiting out a paint that cannot
come and then throwing "No cached paint record" every frame; an empty
matte is also what After Effects shows outside a matte layer's window.
Also: a `.hf-vfx-in` that measures 0×0 is now reported once instead of
capturing nothing in silence, and one that is not the canvas's immediate
child is refused at registration instead of throwing every frame. Both
are measured Chrome rules (vault `layoutsubtree-capture-rules`).
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
* feat(runtime): luma-matte and noise two-source vfx kernels
`luma-matte` is the first def with a `ref` param: `out = src · coverage(u_src2)`
with Alpha / Alpha Inverted / Luma / Luma Inverted coverage. It is not one
After Effects effect but the runtime form of all of them — unsupported
track-matte sources, stencils, silhouettes, Set Matte — which the exporter
distinguishes by choosing what the host wraps and what `matte` names. `src`
is premultiplied, so one scalar multiply covers all four channels, and Luma
reads the matte over black, which is what a premultiplied texel already holds.
`noise` ports `ADBE Noise` from `AEFX_Noise.metallib`: Bob Jenkins' 1997
integer mix over `(x, y, seed32)`, the two glibc-style LCG rounds, and the
`0x7F80`/`0xFF` byte the IR XORs out of them — no table, no state, so it is
seek-safe by construction. Reading the metallib's struct metadata rather
than the deep dive's prose settles two of its open questions:
`ADBE Noise-0002` is `inUseColorNoise` (a checkbox), not a Uniform/Gaussian
popup, and the mystery seed floats are `inRandomSeed_High`/`_Low`, the two
halves of one 32-bit seed. What the CPU puts in those halves per frame is
still unprobed (plan Task 4.3), so the kernel seeds the low half with the
frame index; the tests assert distribution and seek determinism, not parity.
The shader declares `precision highp int` — the whole port rests on 32-bit
wraparound and a fragment shader's integers default to mediump, which ESSL
guarantees only 16 bits of.
Both defs declare `capture: "self"`, which says a texture is needed and
nothing about whose: `noise` on an adjustment layer is the same kernel, and
the runtime reads the `backdrop` placement off the DOM.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
* test(vfx): browser checks for backdrop capture, luma-matte and noise
Task 2.6 Step 5 and Task 2.7's owed browser checks, folded into the
existing puppeteer harness: an identity backdrop kernel composites two
stacked blocks captured through a data-vfx-for sibling wrapper back
over themselves (PSNR >= 40dB vs a plain-DOM control panel); a backdrop
host outside its own clip window still keeps those blocks on the
wrapper canvas's own bitmap via capturePassThrough; luma-matte's four
modes are checked against alpha- and luma-varying matte strips at four
sample points each; noise is checked for amount=0 identity, same-time
determinism, and a changed pattern when its only seed (seek time)
changes.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
* feat(runtime): external displacement maps and visible ref sources
Two gaps the exporter side found in the `ref` plumbing.
1. `displacement-map` had no `map` ref param, so the exporter's
`params.map = "<element id>"` was dropped by `normalizeVfxParams` (it
iterates the def) and the node silently self-displaced. `map` is now a
ref param, bound to `u_src2` and selected by a new `u_hasSrc2` uniform
the runtime sets from whether the ref resolved. Refs gain
`optional: true` — empty `map` is the self-referential form every
corpus instance uses, while an empty `matte` stays a loud failure — and
a ref naming its own host resolves to self rather than capturing the
same pixels twice (on a backdrop host it would otherwise have read the
layers below as the map).
2. Interface v1.1 assumed a ref source is invisible in After Effects.
retro-wave `Logo Anim` layer 5 is layer 4's displacement map AND an
enabled, opaque layer, and the runtime cleared its wrapper's bitmap
after the upload, deleting it from the frame. A wrapper marked
`data-vfx-ref-visible` now keeps its bitmap and is captured at the
REF's own box instead of the host's, so it paints at its own size and
resolution. The three capture shapes are one `{ upload, keepBitmap }`
mode: texture only, bitmap only (a backdrop pass-through), or both.
Sampling is unaffected by the box choice, and the browser test proves it
rather than assuming it: `drawElementImage` scales the element into the
whole canvas, so a 320px-wide matte's four strips still land as four
40px bands in a 160px host. The v1 rule — a ref is scaled into the host's
box, not placed in composition space — is unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
* fix(runtime): capture a visible ref while its host is hidden; review nits
Addresses PR #4330 review feedback (jrusso1020, CHANGES_REQUESTED).
Blocking: both the engine-mode and preview-mode capture paths treated
"host not paintable" as binary — a backdrop entry got a pass-through
capture, everything else was skipped outright, including a ref source
that lives on a different, independently-visible element (e.g.
retro-wave's Logo Anim layer, which is simultaneously layer 4's
displacement map and its own fully opaque visible layer). Adds
visibleRefSources()/captureVisibleRefsOnly() and a unified
FrameCaptureMode discriminator so a visible ref still gets captured
even while its own host is hidden, on both paths.
Non-blocking, addressed:
- luma-matte's mandatory ref that names its own host now resolves to
an explicit RefResolution {kind:"self"} and binds u_src2 to unit 0,
instead of relying on an unbound sampler's implicit default.
displacement-map's optional self-ref is unaffected (still folds into
{kind:"none"}, using its existing u_hasSrc2 shader branch).
- noise shader now enforces rgb<=a (valid premultiplied alpha) after
adding noise, independent of u_clipping — documented why u_clipping
is currently a permanent no-op on this runtime's 8-bit render
targets.
- added a hash-pinning test for hfJenkinsMix; documented, with a
brute-force proof, which one of the reviewer's suggested mutations
is unobservable within this fixture's reachable seed range.
- fixed two misplaced doc comments.
fallow flagged resolveVfxCapture/resolveRefSource as newly over the
complexity threshold after this diff; extracted resolveEmptyRefParam()
and captureHiddenEntry() to bring both back under.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
* fix(runtime): capture a visible ref while a hidden BACKDROP host passes through
Re-review of #4330 at b6d51bd: the visible-ref-while-hidden fix only
covered non-backdrop hosts. captureHiddenEntry returned
capturePassThrough(entry) for a backdrop and never reached
captureVisibleRefsOnly; frameCaptureMode picked "passThrough" before
considering refs and sourcesForFrame("passThrough") was [entry.src]
alone. An adjustment layer whose displacement map is itself a visible
layer still lost that layer outside the adjustment's window.
- captureHiddenEntry runs the pass-through AND the visible-ref capture,
OR-ing the results.
- sourcesForFrame("passThrough") is the backdrop wrapper plus any
visible refs, and capturePaintedHost's pass-through branch also calls
captureVisibleRefsOnly.
- Tests for the backdrop+visible-ref shape in both engine and preview
modes (the reviewer's probe).
Also closes the non-blocking display:none item: isPaintableHost(src.inner)
missed a ref whose OWNING element is display:none, since display does not
inherit — the preview path waited out the 2 s ceiling, then failed with a
misleading 0×0 report. New isPaintableSource walks inner's ancestors for
display:none (visibility inherits, so one read suffices there); used at
both source-side sites. checkVisibility()/offsetParent are absent or
always-null in the unit harness's DOM. Test added; verified it fails on
the old inner-only check.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
---------
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>1 parent d1048cf commit 84e36fc
11 files changed
Lines changed: 2522 additions & 127 deletions
File tree
- packages
- core/src
- runtime
- vfx
- refs
- producer/src/services
Large diffs are not rendered by default.
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
| 13 | + | |
| 14 | + | |
13 | 15 | | |
14 | 16 | | |
15 | 17 | | |
| |||
67 | 69 | | |
68 | 70 | | |
69 | 71 | | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
70 | 80 | | |
71 | 81 | | |
72 | 82 | | |
| |||
323 | 333 | | |
324 | 334 | | |
325 | 335 | | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
326 | 344 | | |
327 | 345 | | |
328 | 346 | | |
| |||
385 | 403 | | |
386 | 404 | | |
387 | 405 | | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
| 423 | + | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
| 429 | + | |
| 430 | + | |
| 431 | + | |
| 432 | + | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
| 436 | + | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
| 441 | + | |
| 442 | + | |
| 443 | + | |
| 444 | + | |
| 445 | + | |
| 446 | + | |
| 447 | + | |
| 448 | + | |
| 449 | + | |
| 450 | + | |
| 451 | + | |
| 452 | + | |
| 453 | + | |
| 454 | + | |
| 455 | + | |
| 456 | + | |
| 457 | + | |
| 458 | + | |
| 459 | + | |
| 460 | + | |
| 461 | + | |
| 462 | + | |
| 463 | + | |
| 464 | + | |
| 465 | + | |
| 466 | + | |
388 | 467 | | |
389 | 468 | | |
390 | 469 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
5 | | - | |
6 | | - | |
7 | | - | |
8 | | - | |
9 | | - | |
10 | | - | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
11 | 14 | | |
12 | 15 | | |
13 | 16 | | |
| |||
51 | 54 | | |
52 | 55 | | |
53 | 56 | | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
54 | 61 | | |
55 | 62 | | |
56 | 63 | | |
| |||
78 | 85 | | |
79 | 86 | | |
80 | 87 | | |
81 | | - | |
| 88 | + | |
82 | 89 | | |
83 | 90 | | |
84 | 91 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
0 commit comments