Skip to content

Commit 5562c28

Browse files
fix(core): stop small previews dropping layers on heavy compositions (#5268)
* fix(player): render the preview at its shown size so heavy compositions stop dropping layers * test(player): check a small preview rasters its will-change layers at the shown size * fix(player): accept device-pixel snapping in the zoom probe * fix(player): probe the initial frame and preserve editing coordinates * fix(player): defer every initial source reload until setup * fix(studio): convert measured frame coordinates once * test(studio): bank reduced preview opening work * fix(core): keep centered layers exact when the preview is zoomed GSAP infers a -50% CSS translate from the rendered pixel matrix. At a fractional preview zoom that matrix is snapped, so a centered layer lost its percentage and shifted by half its height after reload, render and redo. The runtime now restores the authored percentage before GSAP's first transform parse. Also restores the original thumbnail ceilings. * test(studio): bank the two resize cases that now pass edit accuracy resize-scale-px-r30-{root,nested}-z50-after measured 0.53 px tracking on main and 0.49 px here. Also drops a comment line the runtime name covers. * fix(core): keep stylesheet-centered layers exact under preview zoom The centering repair now also reads percentages from a stylesheet transform (the common translate(-50%, -50%) pattern), runs for every property that makes GSAP parse a transform (transformOrigin first, too), resolves against the border box, and leaves elements without a layout box alone instead of throwing. The browser check now runs zoomed. * fix(core): restore only centering, the one percentage GSAP infers GSAP folds every other CSS percentage translate into pixels, and compositions rely on that (a translateX(-100%) title tweened to x: 0 slides in). The repair now gives back only a -50% that zoom snapping lost, and treats a size that is not in px (a hidden element sized in percent) as no box. * fix(core): restore centering only where GSAP would infer it A pixel part in the translate means GSAP never infers -50%, so the repair leaves it alone, and a layer without a layout box is left to GSAP whatever size its computed style reports (that differs by Chrome build for hidden percent-sized layers). * fix(player): fit the preview before the composition starts loading The zoom was applied on the first resize callback, after the source was assigned, so a fast-loading composition could run its scripts and lay out unzoomed and then reflow. Fitting on connect, before any source, makes the first layout use the preview zoom. * test(player): pin the first fit before the composition source * revert(player): drop the zoom fit so the preview keeps the export's layout CSS zoom shaped text at the zoomed size, so paragraph heights in the preview differed from the render by up to a line. The transform fit stays; the centering adapter and zoom-only bench changes go with it. * fix(core): raster a small preview at the size it is shown Chromium never rasters a will-change: transform layer below device scale, so a preview shown at a quarter size rastered every hinted layer at full size and could run out of tile memory, dropping layers. While the player shows the composition below 1x, the runtime swaps the hint for an identity filter, which keeps its stacking context and containing block; preserve-3d containers keep the hint. * fix(core): keep a small preview's raster swap out of filters edits can save The identity filter that replaced will-change: transform showed up in computed filter, so Studio's blur slider saved it, GSAP filter tweens and clearProps picked it up, and it cut backdrop-filter off from what is behind it. A far perspective keeps the same stacking context and containing block, is never composited for flat content, and nothing reads or writes it. Also: the runtime guard lint (realm-safe element checks), a class change that drops the hint undoes the swap, and the browser check measures raster scale with the swap off and on at one shown size and compares frosted glass and 3D pixels. * fix(core): drop preview raster hints only where nothing depends on them The perspective that stood in for will-change: transform could still reach a saved project: Studio records perspective keyframes from GSAP's computed value, and copying a generated element serialised its inline styles. It also went stale when a parent class or a later stylesheet changed the element. Now the runtime writes no inline style. It marks a hinted layer with a preview attribute and one injected rule sets its will-change to auto. Only layers whose hint changes nothing are marked: positioned, with no fixed descendants, and no z-indexed or blending descendants unless the layer has its own z-index. Every class or tree change re-checks the whole page. Studio strips the mark from copied markup, as it does the other preview marks. * fix(core): keep raster hints that pseudo-elements, other hints or inline styles need A layer's ::before or ::after can sit behind it, blend with it or be fixed to it, so those count like children. A layer that also hints opacity or filter keeps every hint, since those make their own boundaries. Inline style changes, which scripts make every frame, are watched cheaply: they only give the hint back to the layers holding an element that now depends on it, or to a layer that turned static or changed its z-index. Text changes no longer trigger a full pass; a new stylesheet still does. * fix(core): re-read a marked layer's own hints and follow any attribute or late sheet A marked layer now has its mark lifted while its hints are read, so a class or inline value that adds opacity or filter gives every hint back. Any attribute change re-checks its subtree, so attribute selectors are followed, and a linked stylesheet re-runs the pass once it loads. * perf(core): only lift a raster mark on the element that changed Lifting a mark to read a layer's own hints repaints it, and a change on a scene container lifted every marked layer under it (about 6 ms and a re-raster per event). Now only the changed element is lifted, and a full pass clears all marks at once so the page restyles a single time. Tests cover leaving marked layers alone, ignoring our own marks, and the linked-sheet load pass. * fix(core): decide raster marks before writing them in a full pass A full pass re-marked layers one at a time, so each read restyled the page again. Every mark is now decided first and written after. The browser check holds off the player's own scale messages while it turns the swap off and on, since a late resize tick could turn it back on. * test(player): hold the player's own scale while the raster check turns the swap off Turning the swap off restyles the page, and the player answers by resending its real scale, which turned the swap back on within milliseconds, so the off half compared the swapped page with itself. The check now holds the player's sends on the host page, asserts the swap is off before the native shots, and removes the layers that keep their hint before either trace. * test(player): give the raster check's 3D probe a solid fill Stripes a few pixels wide alias differently at each raster scale, so the probe measured texture resampling rather than the 3D shape it is there for. A solid fill keeps the shape and position. * fix(studio): keep the preview raster mark out of stacking and saved text edits Studio's stacking model read a marked layer's will-change as auto, so a timeline drag could write a z-index that does nothing in the render; a marked element now still counts as a stacking context. A text edit copied every live attribute but style, so it could save the mark; the preview-only attributes are now one list, used by both the copy and the text-edit paths. * fix(core): keep a marked video through a scene swap The runtime keeps a video across a scene swap only when its live attributes match the fresh import, and the preview raster mark made them differ, so a hinted video in a small preview was rebuilt on every edit. The comparison now ignores every preview-only attribute, which also fixes the same rebuild for data-hf-upcoming. The shared list is renamed PREVIEW_ONLY_ATTRS, and the lazy-image strip in copied markup gets a test.
1 parent 6a83959 commit 5562c28

15 files changed

Lines changed: 903 additions & 9 deletions

File tree

‎.github/workflows/player-perf.yml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -124,6 +124,13 @@ jobs:
124124
PUPPETEER_EXECUTABLE_PATH: ${{ steps.setup-chrome.outputs.chrome-path }}
125125
run: bun run test:browser-security
126126

127+
- name: Verify the preview rasters at its shown size (load shard only)
128+
if: matrix.shard == 'load'
129+
working-directory: packages/player
130+
env:
131+
PUPPETEER_EXECUTABLE_PATH: ${{ steps.setup-chrome.outputs.chrome-path }}
132+
run: bun run test:browser-raster
133+
127134
- name: Run player perf — ${{ matrix.shard }} (measure mode)
128135
working-directory: packages/player
129136
env:

‎packages/core/src/runtime/init.swapScenes.test.ts‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -691,6 +691,13 @@ describe("__hfSwapScenes", () => {
691691
},
692692
],
693693
["the opacity stamp", '<video src="clip.mp4"></video>', () => {}, "data-hf-authored-opacity"],
694+
[
695+
"the preview raster mark",
696+
'<video src="clip.mp4"></video>',
697+
() => {},
698+
"data-hf-preview-raster",
699+
],
700+
["the upcoming mark", '<video src="clip.mp4"></video>', () => {}, "data-hf-upcoming"],
694701
])("keeps a video on which the runtime wrote %s", async (_, video, arrange, stamp?: string) => {
695702
quietMedia();
696703
arrange();

‎packages/core/src/runtime/init.ts‎

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,17 @@
11
// fallow-ignore-file code-duplication complexity
2+
import { setPreviewRasterScale } from "./previewRasterHints";
23
import { refreshSvgSelectorAliases } from "../compiler/svgSelectorAliases";
34
import { RUNTIME_FILLER } from "./protocol";
45
import { preloadMedia, releaseMedia, lengthIsAuthored, stopMediaDownload } from "./preloadMedia";
56
import { installRuntimeControlBridge, postRuntimeMessage, setRuntimeProtocolFps } from "./bridge";
67
import { instantTolerance } from "../clipFacts";
78
import { isInClipWindow } from "./clipWindow";
89
import { revealTimedClipsAfterFirstPass, SKIPPED_CLIP, skipsHiddenImages } from "./timedClipHide";
9-
import { STUDIO_PREVIEW_LAZY_ATTR, STUDIO_PREVIEW_UPCOMING_ATTR } from "../studioPreviewMark";
10+
import {
11+
PREVIEW_ONLY_ATTRS,
12+
STUDIO_PREVIEW_LAZY_ATTR,
13+
STUDIO_PREVIEW_UPCOMING_ATTR,
14+
} from "../studioPreviewMark";
1015
import { initRuntimeAnalytics, emitAnalyticsEvent } from "./analytics";
1116
import { injectCompositionCssVariables } from "./getVariables";
1217
import { createCssAdapter } from "./adapters/css";
@@ -280,13 +285,14 @@ const authoredShape = (el: Element): string =>
280285
]);
281286

282287
// What the runtime writes on a video or audio: style (reset on a kept one), preload, the opacity stamp, a moved
283-
// element's original translate and its own __hf- classes. A proxied src is compared as written; a bound one is not.
288+
// element's original translate, preview marks and its own __hf- classes. A proxied src is compared as written.
284289
const RUNTIME_MEDIA_ATTRS = new Set([
285290
"style",
286291
"preload",
287292
"src",
288293
COLOR_GRADING_AUTHORED_OPACITY_ATTR,
289294
EDIT_ORIGINAL_TRANSLATE_ATTR,
295+
...PREVIEW_ONLY_ATTRS,
290296
]);
291297
const ownClasses = (value: string) =>
292298
value
@@ -5024,7 +5030,10 @@ export function initSandboxRuntimeModular(): void {
50245030
if (state.transportClock) state.transportClock.setRate(state.playbackRate);
50255031
applyWebAudioRate();
50265032
},
5027-
onSetDisplayScale: setProxyDisplayScale,
5033+
onSetDisplayScale: (scale) => {
5034+
setProxyDisplayScale(scale);
5035+
setPreviewRasterScale(scale);
5036+
},
50285037
onSetIdleHeartbeat: (slow) => {
50295038
slowIdleHeartbeat = slow;
50305039
wakeTransport();
Lines changed: 178 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,178 @@
1+
import { afterEach, describe, expect, it, vi } from "vitest";
2+
import { PREVIEW_RASTER_ATTR } from "../studioPreviewMark";
3+
import { setPreviewRasterScale } from "./previewRasterHints";
4+
5+
function layer(style: string, parent: Element = document.body): HTMLElement {
6+
const element = document.createElement("div");
7+
element.setAttribute("style", style);
8+
parent.append(element);
9+
return element;
10+
}
11+
12+
const POSITIONED = "position: absolute; will-change: transform";
13+
const marked = (element: Element) => element.hasAttribute(PREVIEW_RASTER_ATTR);
14+
15+
afterEach(() => {
16+
setPreviewRasterScale(1);
17+
document.body.innerHTML = "";
18+
});
19+
20+
describe("setPreviewRasterScale", () => {
21+
it("touches nothing until the host reports a scale", () => {
22+
const words = layer(POSITIONED);
23+
expect(document.querySelectorAll(`style, [${PREVIEW_RASTER_ATTR}]`)).toHaveLength(0);
24+
expect(getComputedStyle(words).willChange).toBe("transform");
25+
});
26+
27+
it("drops a positioned layer's transform hint while the preview is shown small", () => {
28+
const words = layer(POSITIONED);
29+
const word = layer("position: absolute", words);
30+
const mixed = layer("position: relative; will-change: transform, opacity");
31+
setPreviewRasterScale(0.275);
32+
expect(marked(words)).toBe(true);
33+
expect(marked(word)).toBe(false);
34+
expect(marked(mixed)).toBe(false);
35+
expect(getComputedStyle(words).willChange).toBe("auto");
36+
expect(words.getAttribute("style")).toBe(POSITIONED);
37+
});
38+
39+
it("keeps the hint where the stacking context or containing block matters", () => {
40+
const unpositioned = layer("will-change: transform");
41+
const holdsFixed = layer(POSITIONED);
42+
layer("position: fixed", holdsFixed);
43+
const holdsZIndex = layer(POSITIONED);
44+
layer("position: relative; z-index: 2", holdsZIndex);
45+
const holdsBlend = layer(POSITIONED);
46+
layer("mix-blend-mode: multiply", holdsBlend);
47+
const stacked = layer(`${POSITIONED}; z-index: 1`);
48+
layer("position: relative; z-index: 2", stacked);
49+
setPreviewRasterScale(0.5);
50+
expect([unpositioned, holdsFixed, holdsZIndex, holdsBlend].map(marked)).toEqual([
51+
false,
52+
false,
53+
false,
54+
false,
55+
]);
56+
expect(marked(stacked)).toBe(true);
57+
});
58+
59+
it("restores the document once shown at full size", () => {
60+
const words = layer(POSITIONED);
61+
const before = document.documentElement.outerHTML;
62+
setPreviewRasterScale(0.5);
63+
expect(document.documentElement.outerHTML).not.toBe(before);
64+
setPreviewRasterScale(1);
65+
expect(document.documentElement.outerHTML).toBe(before);
66+
expect(getComputedStyle(words).willChange).toBe("transform");
67+
});
68+
69+
it("follows layers added and classes changed while the preview is small", async () => {
70+
const sheet = document.createElement("style");
71+
sheet.textContent = ".pinned { position: fixed }";
72+
document.head.append(sheet);
73+
setPreviewRasterScale(0.5);
74+
const late = layer(POSITIONED);
75+
const child = layer("", late);
76+
await Promise.resolve();
77+
expect(marked(late)).toBe(true);
78+
child.className = "pinned";
79+
await Promise.resolve();
80+
expect(marked(late)).toBe(false);
81+
sheet.remove();
82+
});
83+
84+
it("gives the hint back when an inline style makes a child depend on it", async () => {
85+
const words = layer(POSITIONED);
86+
const word = layer("position: absolute", words);
87+
setPreviewRasterScale(0.5);
88+
expect(marked(words)).toBe(true);
89+
word.style.zIndex = "5";
90+
await Promise.resolve();
91+
expect(marked(words)).toBe(false);
92+
});
93+
94+
it("re-reads a marked layer's own hints and position when they change", async () => {
95+
const sheet = document.createElement("style");
96+
sheet.textContent =
97+
".hinted { will-change: transform } .hinted.fading { will-change: transform, opacity }";
98+
document.head.append(sheet);
99+
const classed = layer("position: absolute");
100+
classed.className = "hinted";
101+
const inline = layer(POSITIONED);
102+
const inlineHint = layer(POSITIONED);
103+
setPreviewRasterScale(0.5);
104+
expect([classed, inline, inlineHint].map(marked)).toEqual([true, true, true]);
105+
classed.className = "hinted fading";
106+
inline.style.position = "static";
107+
inlineHint.style.willChange = "transform, opacity";
108+
await Promise.resolve();
109+
expect([classed, inline, inlineHint].map(marked)).toEqual([false, false, false]);
110+
sheet.remove();
111+
});
112+
113+
it("follows stylesheets and attribute selectors added while the preview is small", async () => {
114+
const byStylesheet = layer(POSITIONED);
115+
layer("", byStylesheet).className = "late-pinned";
116+
const byAttribute = layer(POSITIONED);
117+
const scene = layer("", byAttribute);
118+
setPreviewRasterScale(0.5);
119+
expect([byStylesheet, byAttribute].map(marked)).toEqual([true, true]);
120+
const sheet = document.createElement("style");
121+
sheet.textContent = '.late-pinned, [data-scene="pinned"] { position: fixed }';
122+
document.head.append(sheet);
123+
await Promise.resolve();
124+
expect(marked(byStylesheet)).toBe(false);
125+
expect(marked(byAttribute)).toBe(true);
126+
scene.dataset.scene = "pinned";
127+
await Promise.resolve();
128+
expect(marked(byAttribute)).toBe(false);
129+
sheet.remove();
130+
});
131+
132+
it("leaves marked layers alone when only something around them changes", async () => {
133+
const scene = layer("");
134+
const words = [layer(POSITIONED, scene), layer(POSITIONED, scene)];
135+
setPreviewRasterScale(0.5);
136+
const markChanges: MutationRecord[] = [];
137+
const watch = new MutationObserver((records) => markChanges.push(...records));
138+
watch.observe(document.body, { subtree: true, attributeFilter: [PREVIEW_RASTER_ATTR] });
139+
scene.dataset.upcoming = "";
140+
await Promise.resolve();
141+
await Promise.resolve();
142+
watch.disconnect();
143+
expect(words.map(marked)).toEqual([true, true]);
144+
expect(markChanges).toEqual([]);
145+
});
146+
147+
it("does not react to its own marks", async () => {
148+
const words = layer(POSITIONED);
149+
setPreviewRasterScale(0.5);
150+
words.dataset.scene = "intro";
151+
await Promise.resolve();
152+
await Promise.resolve();
153+
const reads = vi.spyOn(window, "getComputedStyle");
154+
await Promise.resolve();
155+
await Promise.resolve();
156+
expect(reads).not.toHaveBeenCalled();
157+
reads.mockRestore();
158+
expect(marked(words)).toBe(true);
159+
});
160+
161+
it("re-checks the page when a linked stylesheet loads", async () => {
162+
const sheet = document.createElement("style");
163+
document.head.append(sheet);
164+
const words = layer(POSITIONED);
165+
layer("", words).className = "loaded-pinned";
166+
setPreviewRasterScale(0.5);
167+
const link = document.createElement("link");
168+
document.head.append(link);
169+
await Promise.resolve();
170+
expect(marked(words)).toBe(true);
171+
// Stands in for the linked rules arriving: stylesheet object edits are not observed.
172+
sheet.sheet!.insertRule(".loaded-pinned { position: fixed }");
173+
link.dispatchEvent(new Event("load"));
174+
expect(marked(words)).toBe(false);
175+
sheet.remove();
176+
link.remove();
177+
});
178+
});

0 commit comments

Comments
 (0)