Skip to content

Commit f7d2260

Browse files
authored
feat(engine): stamp rendered files with hidden renderer provenance (#3264)
* feat(engine): stamp rendered files with hidden renderer provenance * fix(engine,producer): re-assert provenance at every container writer Review found that a no-audio MOV render still shipped untagged. The concat step is the last container write on that path (mux is skipped without audio, and applyFaststart only copies mov/webm), and the concat demuxer does not carry the chunks' container metadata through. The same hole applies to no-audio WebM, and to the in-process chunked encode in chunkEncoder, not just the distributed assemble path. mp4 was masked throughout because applyFaststart re-runs ffmpeg for that format and re-tagged the output. Tags the four remaining writers: the chunked-encode concat, and assemble's single-chunk remux, concat and cfr re-encode. Also corrects the trust claim. These are unsigned, freely writable keys, so a present tag means the file claims to be HyperFrames output, not that HyperFrames wrote it. Documented as an unauthenticated diagnostic hint rather than an authenticity or attribution boundary. Tests assert on the assembled file through the real assemble() path for both mov and webm; both fail without the concat fix. * test(engine): pin provenance through the in-process chunked concat Review noted the distributed writers are mutation-pinned but the encodeFramesChunkedConcat fix had no real-file regression of its own. Encodes 70 frames at a 30-frame chunk size so the concat step actually runs, then asserts the tags on the resulting no-audio mov. Fails without the concat fix, passes with it.
1 parent 9ba5289 commit f7d2260

11 files changed

Lines changed: 561 additions & 20 deletions

File tree

‎docs/guides/rendering.mdx‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -147,6 +147,40 @@ For several variable-driven versions, use batch rendering. For remote infrastruc
147147

148148
Those workflows involve output naming, credentials, concurrency, and infrastructure choices. Start in the [CLI guide](/developers/cli) and use the complete [CLI reference](/packages/cli) when you need every flag.
149149

150+
## Render provenance
151+
152+
Rendered video carries two container metadata tags that say which tool wrote the file:
153+
154+
```bash
155+
ffprobe -v error -show_entries format_tags -of json out.mp4
156+
```
157+
158+
```json
159+
{ "hyperframes_renderer": "hyperframes", "hyperframes_version": "0.7.107" }
160+
```
161+
162+
That is the whole of it. The tags name the renderer and its version, and nothing else: no file
163+
paths, usernames, machine names, project names, or anything about the composition. They are
164+
container metadata, not a visible watermark, so no pixel of your video changes. Matroska
165+
uppercases tag names on read, so a `.webm` reports `HYPERFRAMES_RENDERER`.
166+
167+
Strip them whenever you like:
168+
169+
```bash
170+
ffmpeg -i out.mp4 -map_metadata -1 -c copy clean.mp4
171+
```
172+
173+
<Note>
174+
These tags are an unauthenticated diagnostic hint, not proof of origin. They are ordinary unsigned
175+
container keys, so anything can write the same two values with a single `ffmpeg -metadata`
176+
command: a tag that is present means the file *claims* to be HyperFrames output, not that
177+
HyperFrames wrote it. A tag that is absent means just as little, because re-encoding, remuxing, or
178+
any tool that drops unknown keys strips it, and files rendered by older versions never carried it.
179+
Treat it as a "what probably produced this file?" hint for support and debugging, never as an
180+
authenticity, attribution, or licensing check. Verifiable provenance needs signed claims such as
181+
C2PA.
182+
</Note>
183+
150184
## If rendering fails
151185

152186
Run:

‎packages/cli/src/background-removal/pipeline.ts‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ import { extname } from "node:path";
1818
import { findFFmpeg, findFFprobe, getFFmpegInstallHint } from "../browser/ffmpeg.js";
1919
import { createSession, type Session } from "./inference.js";
2020
import { type Device, type ModelId } from "./manager.js";
21-
import { DEFAULT_VP9_CPU_USED } from "@hyperframes/engine";
21+
import { DEFAULT_VP9_CPU_USED, renderProvenanceArgs } from "@hyperframes/engine";
2222

2323
export type OutputFormat = "webm" | "mov" | "png";
2424

@@ -182,6 +182,7 @@ export function buildEncoderArgs(
182182
"-metadata:s:v:0",
183183
"alpha_mode=1",
184184
"-an",
185+
...renderProvenanceArgs(outputPath),
185186
outputPath,
186187
];
187188
}
@@ -197,6 +198,7 @@ export function buildEncoderArgs(
197198
"-pix_fmt",
198199
"yuva444p10le",
199200
"-an",
201+
...renderProvenanceArgs(outputPath),
200202
outputPath,
201203
];
202204
}

‎packages/engine/src/index.ts‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -393,3 +393,13 @@ export {
393393
type HdrMasteringMetadata,
394394
} from "./utils/hdr.js";
395395
export type { VideoColorSpace } from "./utils/ffprobe.js";
396+
export {
397+
renderProvenanceArgs,
398+
appendRenderProvenanceArgs,
399+
readRenderProvenance,
400+
PROVENANCE_RENDERER_TAG,
401+
PROVENANCE_VERSION_TAG,
402+
PROVENANCE_RENDERER_NAME,
403+
PROVENANCE_VERSION,
404+
type RenderProvenance,
405+
} from "./utils/renderProvenance.js";

‎packages/engine/src/services/chunkEncoder.test.ts‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ import { tmpdir } from "node:os";
44
import { join } from "node:path";
55
import { afterEach, describe, it, expect, vi } from "vitest";
66
import { ENCODER_PRESETS, getEncoderPreset, buildEncoderArgs } from "./chunkEncoder.js";
7+
import { renderProvenanceArgs } from "../utils/renderProvenance.js";
78

89
const TINY_PNG = Buffer.from(
910
"iVBORw0KGgoAAAANSUhEUgAAAAIAAAACCAIAAAD91JpzAAAACXBIWXMAAAABAAAAAQBPJcTWAAAAEElEQVR4nGP8wwACLGCSAQANBAECv1AVswAAAABJRU5ErkJggg==",
@@ -403,13 +404,18 @@ describe("muxVideoWithAudio audio codec handling", () => {
403404
"+faststart",
404405
"-avoid_negative_ts",
405406
"make_zero",
407+
...renderProvenanceArgs("/tmp/output.mp4"),
406408
"-r",
407409
"30",
408410
"-y",
409411
"/tmp/output.mp4",
410412
]);
411413
expect(calls[0]!.args).not.toContain("-shortest");
412414
expect(calls[0]!.args).not.toContain("-use_editlist");
415+
// The faststart flag set above must survive the provenance flag: ffmpeg
416+
// takes the last -movflags occurrence, and a non-additive one would drop it.
417+
expect(calls[0]!.args.filter((a) => a === "-movflags")).toHaveLength(2);
418+
expect(calls[0]!.args).toContain("+faststart");
413419

414420
emitClose(calls[0]!.proc, 0);
415421
await expect(muxPromise).resolves.toMatchObject({

‎packages/engine/src/services/chunkEncoder.ts‎

Lines changed: 17 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ import { extractAudioMetadata } from "../utils/ffprobe.js";
2222
import { type Fps, fpsToFfmpegArg } from "@hyperframes/core";
2323
import type { EncoderOptions, EncodeResult, MuxResult } from "./chunkEncoder.types.js";
2424
import { appendVp9CpuUsedArg } from "./vp9Options.js";
25+
import { appendRenderProvenanceArgs } from "../utils/renderProvenance.js";
2526

2627
export type { EncoderOptions, EncodeResult, MuxResult } from "./chunkEncoder.types.js";
2728

@@ -361,6 +362,7 @@ export function buildEncoderArgs(
361362
} else if (codec === "prores") {
362363
args.push("-c:v", "prores_ks", "-profile:v", preset, "-vendor", "apl0");
363364
args.push("-pix_fmt", pixelFormat);
365+
appendRenderProvenanceArgs(args, outputPath);
364366
return [...args, "-y", outputPath];
365367
}
366368

@@ -445,6 +447,8 @@ export function buildEncoderArgs(
445447

446448
args.push("-avoid_negative_ts", "make_zero");
447449

450+
appendRenderProvenanceArgs(args, outputPath);
451+
448452
args.push("-y", outputPath);
449453
return args;
450454
}
@@ -611,18 +615,14 @@ export async function encodeFramesChunkedConcat(
611615
const concatInput = chunkPaths.map((path) => `file '${path.replace(/'/g, "'\\''")}'`).join("\n");
612616
writeFileSync(concatListPath, concatInput, "utf-8");
613617

614-
const concatArgs = [
615-
"-f",
616-
"concat",
617-
"-safe",
618-
"0",
619-
"-i",
620-
concatListPath,
621-
"-c",
622-
"copy",
623-
"-y",
624-
outputPath,
625-
];
618+
const concatArgs = ["-f", "concat", "-safe", "0", "-i", concatListPath, "-c", "copy"];
619+
// The concat demuxer does not carry per-chunk container metadata into the
620+
// output, so the chunks' provenance is dropped here even though every chunk
621+
// carries it. Re-assert on the concatenated file: for a no-audio mov/webm
622+
// this is the last container write, since mux is skipped and applyFaststart
623+
// only copies those two formats.
624+
appendRenderProvenanceArgs(concatArgs, outputPath);
625+
concatArgs.push("-y", outputPath);
626626
const encodeTimeout = config?.ffmpegEncodeTimeout ?? DEFAULT_CONFIG.ffmpegEncodeTimeout;
627627
const concatProcessResult = await runFfmpeg(concatArgs, { signal, timeout: encodeTimeout });
628628
const concatResult = {
@@ -701,6 +701,10 @@ export async function muxVideoWithAudio(
701701
// AAC priming packet. `make_zero` discards that edit and shifts copied video
702702
// forward by one AAC frame (~21ms), creating a visible first-frame offset.
703703
if (!copiesContainerizedAac) args.push("-avoid_negative_ts", "make_zero");
704+
// Re-assert provenance here: this stage re-muxes into the delivered
705+
// container, and the mp4 muxer drops the encode stage's tags without the
706+
// use_metadata_tags flag that appendRenderProvenanceArgs adds.
707+
appendRenderProvenanceArgs(args, outputPath);
704708
if (fps !== undefined) {
705709
// Set the exact output framerate so the muxer doesn't PTS-average a
706710
// fractional rational like `360000/12001` instead of `30/1` into the
@@ -742,6 +746,7 @@ export async function applyFaststart(
742746
return { success: true, outputPath, durationMs: 0 };
743747
}
744748
const args = ["-i", inputPath, "-c", "copy", "-movflags", "+faststart"];
749+
appendRenderProvenanceArgs(args, outputPath);
745750
if (fps !== undefined) {
746751
// Set the exact output framerate so the final remux doesn't PTS-average
747752
// a fractional rational like `360000/12001` instead of `30/1` into the

‎packages/engine/src/services/streamingEncoder.ts‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ import { withEvenDimensionPad } from "../utils/evenDimensions.js";
3636
import { DEFAULT_CONFIG, type EngineConfig } from "../config.js";
3737
import { fpsToFfmpegArg, type Fps } from "@hyperframes/core";
3838
import { appendVp9CpuUsedArg } from "./vp9Options.js";
39+
import { appendRenderProvenanceArgs } from "../utils/renderProvenance.js";
3940

4041
// Re-export EncoderOptions so callers can reference the type via this module.
4142
export type { EncoderOptions } from "./chunkEncoder.types.js";
@@ -350,6 +351,7 @@ export function buildStreamingArgs(
350351
} else if (codec === "prores") {
351352
args.push("-c:v", "prores_ks", "-profile:v", preset, "-vendor", "apl0");
352353
args.push("-pix_fmt", pixelFormat);
354+
appendRenderProvenanceArgs(args, outputPath);
353355
return [...args, "-y", outputPath];
354356
}
355357

@@ -428,6 +430,8 @@ export function buildStreamingArgs(
428430
// for the full explanation; same playback compatibility class.
429431
args.push("-avoid_negative_ts", "make_zero");
430432

433+
appendRenderProvenanceArgs(args, outputPath);
434+
431435
args.push("-y", outputPath);
432436
return args;
433437
}

‎packages/engine/src/utils/ffprobe.ts‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -608,7 +608,10 @@ function extractStillImageMetadata(filePath: string): StillImageMetadata | null
608608
* in newer ones; HDR tags vary similarly. Use this for any sidecar tag where
609609
* you want to be resilient across muxer versions.
610610
*/
611-
function readTagCI(tags: Record<string, string | undefined> | undefined, name: string): string {
611+
export function readTagCI(
612+
tags: Record<string, string | undefined> | undefined,
613+
name: string,
614+
): string {
612615
if (!tags) return "";
613616
const target = name.toLowerCase();
614617
for (const [key, value] of Object.entries(tags)) {

0 commit comments

Comments
 (0)