Skip to content

Commit 0c158be

Browse files
feat(studio): edit a clip's speed ramp with a rate lane and presets (#4153)
* feat(studio): edit a clip's speed ramp with a rate lane and presets * fix(studio): keep the live preview attribute in sync with what gets persisted * fix(cli): snapshot picks the source frame through a rate lane and up to 10x hyperframes snapshot clamped the constant playback rate at 5 and ignored a rate lane in data-automation, so a sped-up or ramped clip showed the wrong frame. It now reads the rate through the runtime's own reader and maps time with the shared source-time function. * docs(skills): teach speed ramps as a rate lane in one reference page Adds the speed-ramps reference, corrects the 0.1 to 5 rate range to 0.1 to 10 and the 'speed ramps are not supported' lines in the skills, and regenerates the skills manifest. * docs(skills): say what fixes a ramped clip's length and how the render approximates pitch * fix(studio): hide the speed preset until the clip has a known duration A preset on a clip with no data-duration was stretched over a made-up one second. The panel now offers presets only when the duration is known. * docs(skills): fix the last stale ramp line in the editing recipes and the slice-cap wording
1 parent fa699b7 commit 0c158be

19 files changed

Lines changed: 371 additions & 62 deletions

‎docs/docs.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -916,7 +916,8 @@
916916
"pages": [
917917
"reference/html-schema",
918918
"reference/color-grading",
919-
"reference/audio-effects"
919+
"reference/audio-effects",
920+
"reference/speed-ramps"
920921
]
921922
},
922923
{

‎docs/reference/html-schema.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -124,7 +124,7 @@ Audio has no visual lifecycle.
124124
| Attribute | Applies to | Meaning |
125125
| --- | --- | --- |
126126
| `data-media-start` / `data-playback-start` | Video, audio, nested composition | Offset into the source file, used by trim and split. Two groups of readers disagree, so the right name depends on the element. **Read only `data-media-start`:** the timing compiler, the HTML parser, `hyperframes validate` (which only inspects `<audio>`), and the engine's audio mixer (which feeds ffmpeg `-ss`). **Read `data-playback-start` first, falling back to `data-media-start`:** the runtime player, Studio (which also writes it), and `hyperframes snapshot`. Because the audio mixer reads only `data-media-start`, a `<video>` authored with just `data-playback-start` renders a trimmed picture over untrimmed audio. Set the name by kind: **`<video>` / `<audio>` → `data-media-start`**; **nested composition → `data-playback-start`** — composition hosts are inspected only by the playback-start-first readers (the media-start-only ones are all `<video>`/`<audio>`-scoped). `data-media-start` still works there as a fallback, but `data-playback-start` is the canonical name Studio writes and normalises to for new composition hosts, so the other name works until an edit rewrites it (it is the [child-timeline offset](/concepts/compositions)). |
127-
| `data-playback-rate` | Video, audio, nested composition | Playback multiplier from `0.1` to `5` |
127+
| `data-playback-rate` | Video, audio, nested composition | Playback multiplier from `0.1` to `10`. A `rate` lane in `data-automation` overrides it; see [Speed ramps](/reference/speed-ramps) |
128128
| `data-volume` | Video and audio | Static gain. `1` is 0 dB, `0` is silence, and values above `1` boost up to `3.98` (+12 dB) |
129129
| `data-has-audio="true"` | Video | Declares that the video contributes audio |
130130

‎docs/reference/speed-ramps.mdx‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
---
2+
title: "Speed ramps"
3+
sidebarTitle: "Speed ramps"
4+
description: "Change a clip's playback speed over time with a rate lane in data-automation. One curve drives the preview, the render and Studio."
5+
---
6+
7+
A speed ramp changes how fast a `<video>` or `<audio>` clip plays while it
8+
plays: slow into a beat, fast out of it. It is a `rate` lane in
9+
[`data-automation`](/reference/audio-effects#the-four-attributes), the same
10+
attribute that carries volume envelopes.
11+
12+
```html
13+
<video
14+
src="clip.mp4"
15+
data-start="0"
16+
data-duration="4"
17+
data-track-index="0"
18+
data-automation='{"version":1,"lanes":[{"target":"rate","points":[{"t":0,"v":1},{"t":2,"v":4},{"t":4,"v":0.5}]}]}'
19+
></video>
20+
```
21+
22+
## The lane
23+
24+
| Field | Meaning |
25+
| --- | --- |
26+
| `target` | `"rate"` |
27+
| `points[].t` | Seconds from the start of the clip, not from the start of the composition |
28+
| `points[].v` | Speed multiplier. `1` is normal, `2` is twice as fast. Clamped to `0.1` to `10` |
29+
| `points[].curve` | Optional, `-1` to `1`. Bends the segment leaving this point |
30+
31+
Speed moves between points on a logarithmic scale, so going from `0.5` to `2`
32+
passes through `1` at the midpoint. A lane wins over the constant
33+
`data-playback-rate` on the same element. Without a lane, the constant applies.
34+
35+
## What stays the same everywhere
36+
37+
The source frame shown at each moment comes from one function, `sourceTimeAt`,
38+
which the preview and the renderer both use, so the render shows the frames the
39+
preview showed. Set `data-duration` on the clip to fix how long it holds the
40+
timeline; without it the length follows from the source and the speed.
41+
42+
Pitch is kept. The preview leaves the browser's default pitch correction on.
43+
The render time-stretches audio in short constant-speed slices (a quarter
44+
second, coarser on clips longer than a minute), which approximates the curve
45+
while keeping pitch, so the sound follows the ramp closely rather than exactly.
46+
47+
## Presets
48+
49+
Studio's **Speed preset** menu draws a ready curve over the clip: Montage,
50+
Hero, Bullet, Jump cut, Flash in and Flash out. An agent can write the same
51+
points by hand.
52+
53+
## Editing in Studio
54+
55+
Select a clip and open its Media section. The **Speed** slider sets the
56+
constant rate. The **A** toggle next to it automates speed, which shows a rate
57+
lane on the clip in the timeline that you edit like the volume lane.
58+
59+
## Limits
60+
61+
- Only `<video>` and `<audio>` clips carry a rate lane.
62+
- Speeds outside `0.1` to `10` are clamped.
63+
- The viewer's own transport speed is a separate control that multiplies with
64+
the clip's speed, and it is narrower: the Studio speed menu offers `0.25` to
65+
`2`, the player's menu offers up to `4`, and the runtime clamps a transport
66+
request to `0.1` to `5`.

‎packages/cli/src/commands/snapshot.test.ts‎

Lines changed: 35 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ import { describe, expect, it, vi } from "vitest";
22
import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
33
import { dirname, join } from "node:path";
44
import { tmpdir } from "node:os";
5+
import { sourceTimeAt } from "@hyperframes/core";
56

67
const snapshotState = vi.hoisted(() => ({
78
openSettledPage: vi.fn(async () => {
@@ -29,7 +30,7 @@ import snapshotCommand, {
2930
requireSnapshotFfmpeg,
3031
resolveSnapshotVideoClipStart,
3132
resolveSnapshotVideoFrameTime,
32-
resolveSnapshotVideoPlaybackRate,
33+
resolveSnapshotVideoRateSpec,
3334
tailFrameTime,
3435
} from "./snapshot.js";
3536

@@ -274,9 +275,40 @@ describe("resolveSnapshotVideoClipStart", () => {
274275
});
275276
});
276277

277-
describe("resolveSnapshotVideoPlaybackRate", () => {
278+
describe("resolveSnapshotVideoRateSpec", () => {
278279
it("prefers the authored data-playback-rate over the browser default", () => {
279-
expect(resolveSnapshotVideoPlaybackRate({ authoredRate: "1.8", defaultRate: 1 })).toBe(1.8);
280+
expect(resolveSnapshotVideoRateSpec({ authoredRate: "1.8", defaultRate: 1 })).toBe(1.8);
281+
});
282+
283+
it("falls back to the browser default when the authored rate is invalid", () => {
284+
expect(resolveSnapshotVideoRateSpec({ authoredRate: "abc", defaultRate: 2 })).toBe(2);
285+
expect(resolveSnapshotVideoRateSpec({ authoredRate: "0", defaultRate: 2 })).toBe(2);
286+
});
287+
288+
it("allows rates up to the shared 10x bound", () => {
289+
expect(resolveSnapshotVideoRateSpec({ authoredRate: "8", defaultRate: 1 })).toBe(8);
290+
});
291+
292+
it("maps a frame through a rate lane instead of the constant", () => {
293+
const lane = JSON.stringify({
294+
version: 1,
295+
lanes: [
296+
{
297+
target: "rate",
298+
points: [
299+
{ t: 0, v: 1 },
300+
{ t: 2, v: 3 },
301+
],
302+
},
303+
],
304+
});
305+
const spec = resolveSnapshotVideoRateSpec({
306+
authoredRate: "1",
307+
authoredAutomation: lane,
308+
defaultRate: 1,
309+
});
310+
expect(typeof spec).toBe("object");
311+
expect(sourceTimeAt(spec, 2)).toBeCloseTo(3.641, 2);
280312
});
281313
});
282314

‎packages/cli/src/commands/snapshot.ts‎

Lines changed: 26 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,12 @@ import {
1414
seekCompositionTimeline,
1515
type ZoomTarget,
1616
} from "../capture/captureCompositionFrame.js";
17+
import {
18+
readElementRateSpec,
19+
sourceTimeAt,
20+
timeAtSourceTime,
21+
type RateSpec,
22+
} from "@hyperframes/core";
1723
import { resolveProject } from "../utils/project.js";
1824
import {
1925
definitiveEntryMismatchComposition,
@@ -110,16 +116,22 @@ export function resolveSnapshotVideoClipStart(input: {
110116
return input.runtimeResolvedStart ?? input.authoredStart;
111117
}
112118

113-
/** Match runtime/render timing: authored data-playback-rate wins over the
114-
* browser default, then the effective rate is clamped to the supported range. */
115-
export function resolveSnapshotVideoPlaybackRate(input: {
119+
/** Match runtime/render timing: a `rate` lane in data-automation wins, then the authored
120+
* data-playback-rate, then the browser default, all through the runtime's own reader. */
121+
export function resolveSnapshotVideoRateSpec(input: {
116122
authoredRate: string | undefined;
123+
authoredAutomation?: string | undefined;
117124
defaultRate: number;
118-
}): number {
125+
}): RateSpec {
119126
const authoredRate = Number.parseFloat(input.authoredRate ?? "");
120-
const rawRate =
121-
Number.isFinite(authoredRate) && authoredRate > 0 ? authoredRate : input.defaultRate;
122-
return Number.isFinite(rawRate) && rawRate > 0 ? Math.max(0.1, Math.min(5, rawRate)) : 1;
127+
const attrs: Record<string, string | undefined> = {
128+
"data-playback-rate":
129+
Number.isFinite(authoredRate) && authoredRate > 0
130+
? input.authoredRate
131+
: String(input.defaultRate),
132+
"data-automation": input.authoredAutomation,
133+
};
134+
return readElementRateSpec({ getAttribute: (name) => attrs[name] ?? null });
123135
}
124136

125137
export function requireSnapshotFfmpeg(ffmpegPath: string | undefined): string {
@@ -454,6 +466,7 @@ async function captureSnapshots(
454466
src: v.currentSrc || v.src,
455467
authoredStart,
456468
authoredRate: v.dataset.playbackRate,
469+
authoredAutomation: v.dataset.automation,
457470
defaultRate: v.defaultPlaybackRate,
458471
runtimeResolvedStart:
459472
runtimeResolvedStart !== undefined && Number.isFinite(runtimeResolvedStart)
@@ -469,13 +482,16 @@ async function captureSnapshots(
469482
});
470483
const active = candidates.flatMap((candidate) => {
471484
const start = resolveSnapshotVideoClipStart(candidate);
472-
const playbackRate = resolveSnapshotVideoPlaybackRate(candidate);
485+
const playbackRate = resolveSnapshotVideoRateSpec(candidate);
473486
const duration =
474487
candidate.authoredDuration ??
475488
(candidate.srcDuration > 0
476-
? Math.max(0, (candidate.srcDuration - candidate.mediaStart) / playbackRate)
489+
? Math.max(
490+
0,
491+
timeAtSourceTime(playbackRate, candidate.srcDuration - candidate.mediaStart),
492+
)
477493
: Number.POSITIVE_INFINITY);
478-
let relTime = (time - start) * playbackRate + candidate.mediaStart;
494+
let relTime = sourceTimeAt(playbackRate, time - start) + candidate.mediaStart;
479495
if (
480496
candidate.loop &&
481497
candidate.srcDuration > candidate.mediaStart &&

‎packages/core/package-subpaths.json‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -188,6 +188,12 @@
188188
"types": "./dist/audioGain.d.ts",
189189
"environments": ["browser", "bun", "node"]
190190
},
191+
"./speed-ramp": {
192+
"source": "./src/speedRamp.ts",
193+
"runtime": "./dist/speedRamp.js",
194+
"types": "./dist/speedRamp.d.ts",
195+
"environments": ["browser", "bun", "node"]
196+
},
191197
"./color-grading": {
192198
"source": "./src/colorGrading.ts",
193199
"runtime": "./dist/colorGrading.js",

‎packages/core/package.json‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -202,6 +202,12 @@
202202
"import": "./src/audioGain.ts",
203203
"types": "./src/audioGain.ts"
204204
},
205+
"./speed-ramp": {
206+
"bun": "./src/speedRamp.ts",
207+
"node": "./dist/speedRamp.js",
208+
"import": "./src/speedRamp.ts",
209+
"types": "./src/speedRamp.ts"
210+
},
205211
"./color-grading": {
206212
"bun": "./src/colorGrading.ts",
207213
"node": "./dist/colorGrading.js",
@@ -540,6 +546,10 @@
540546
"import": "./dist/audioGain.js",
541547
"types": "./dist/audioGain.d.ts"
542548
},
549+
"./speed-ramp": {
550+
"import": "./dist/speedRamp.js",
551+
"types": "./dist/speedRamp.d.ts"
552+
},
543553
"./color-grading": {
544554
"import": "./dist/colorGrading.js",
545555
"types": "./dist/colorGrading.d.ts"

‎packages/studio/src/components/editor/propertyPanelFlatMediaSection.test.tsx‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -250,8 +250,8 @@ describe("FlatMediaSection — volume/rate/media-start", () => {
250250
rateTrack.dispatchEvent(new MouseEvent("pointerdown", { bubbles: true, clientX: 100 }));
251251
rateTrack.dispatchEvent(new MouseEvent("pointerup", { bubbles: true, clientX: 100 }));
252252
});
253-
// min=25, max=300, ratio=1.0 -> raw=300 -> commit(300) -> 300/100=3 -> "3"
254-
expect(onSetAttribute).toHaveBeenCalledWith("playback-rate", "3");
253+
// the speed slider is log-scaled 0.1x..10x, so the far end of the track is 10x
254+
expect(onSetAttribute).toHaveBeenCalledWith("playback-rate", "10");
255255
act(() => root.unmount());
256256
});
257257

‎packages/studio/src/components/editor/propertyPanelFlatMediaSection.tsx‎

Lines changed: 46 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,10 @@ import {
1313
import { FlatSelectRow, FlatSlider } from "./propertyPanelFlatPrimitives";
1414
import { FlatToggle } from "./propertyPanelFlatToggle";
1515
import { AutomationToggle } from "./propertyPanelFxControls";
16+
import { RATE_RANGE } from "@hyperframes/core/audio-automation";
17+
import { type SpeedPresetId } from "@hyperframes/core/speed-ramp";
18+
import { SPEED_PRESET_OPTIONS, type RateBinding } from "./useVolumeAutomation";
19+
import { fromUnit, toUnit } from "../../player/components/automationLaneGeometry";
1620
import {
1721
AUDIO_GAIN_FADER_MAX,
1822
AUDIO_GAIN_FADER_MIN,
@@ -36,6 +40,7 @@ export function FlatMediaSection({
3640
onRemoveVolumeAutomation,
3741
onCommitVolumeAt,
3842
automatedVolumeValue,
43+
rate,
3944
}: {
4045
projectDir: string | null;
4146
element: DomEditSelection;
@@ -49,6 +54,8 @@ export function FlatMediaSection({
4954
onRemoveVolumeAutomation?: () => void;
5055
onCommitVolumeAt?: (v: number) => void;
5156
automatedVolumeValue?: number;
57+
/** Speed lane binding and presets; absent outside the Studio panel. */
58+
rate?: RateBinding;
5259
onRemoveBackground?: (
5360
inputPath: string,
5461
options: {
@@ -77,7 +84,9 @@ export function FlatMediaSection({
7784
Number.parseFloat(
7885
element.dataAttributes["media-start"] ?? element.dataAttributes["playback-start"] ?? "0",
7986
) || 0;
80-
const playbackRate = Number.parseFloat(element.dataAttributes["playback-rate"] ?? "1") || 1;
87+
const constantRate = Number.parseFloat(element.dataAttributes["playback-rate"] ?? "1") || 1;
88+
const playbackRate =
89+
rate?.automated && rate.automatedValue !== undefined ? rate.automatedValue : constantRate;
8190
const sourceDuration =
8291
Number.parseFloat(element.dataAttributes["source-duration"] ?? "") ||
8392
(el as HTMLMediaElement).duration ||
@@ -259,17 +268,42 @@ export function FlatMediaSection({
259268
}
260269
/>
261270
</div>
262-
<FlatSlider
263-
label="Rate"
264-
value={playbackRate * 100}
265-
min={25}
266-
max={300}
267-
tier={playbackRate === 1 ? "default" : "explicitCustom"}
268-
displayValue={`${formatNumericValue(playbackRate)}x`}
269-
onCommit={(next) =>
270-
void onSetAttribute("playback-rate", formatNumericValue(next / 100))
271-
}
272-
/>
271+
<div className="flex items-center gap-1">
272+
<div className="min-w-0 flex-1">
273+
<FlatSlider
274+
label="Speed"
275+
value={Math.round(toUnit(RATE_RANGE, playbackRate) * 1000)}
276+
min={0}
277+
max={1000}
278+
tier={playbackRate === 1 ? "default" : "explicitCustom"}
279+
displayValue={`${formatNumericValue(playbackRate)}x`}
280+
onCommit={(next) => {
281+
const speed = fromUnit(RATE_RANGE, next / 1000);
282+
if (rate?.automated) {
283+
rate.onCommitAt(speed);
284+
} else {
285+
void onSetAttribute("playback-rate", formatNumericValue(speed));
286+
}
287+
}}
288+
/>
289+
</div>
290+
<AutomationToggle
291+
paramKey="rate"
292+
label="Speed"
293+
automated={Boolean(rate?.automated)}
294+
onAutomate={rate ? () => rate.onAutomate() : undefined}
295+
onRemoveAutomation={rate ? () => rate.onRemoveAutomation() : undefined}
296+
/>
297+
</div>
298+
{rate?.canApplyPreset && (
299+
<FlatSelectRow
300+
label="Speed preset"
301+
value=""
302+
options={[{ value: "", label: "Choose…" }, ...SPEED_PRESET_OPTIONS]}
303+
tier="default"
304+
onChange={(id) => id && rate.onApplyPreset(id as SpeedPresetId)}
305+
/>
306+
)}
273307
<FlatSlider
274308
label="Media start"
275309
value={Math.round(mediaStart * 100)}

0 commit comments

Comments
 (0)