Repository navigation
fix(cli): snapshot --proxy exits once its frames are written, without waiting for copies - #5895
Conversation
… waiting for copies Closing the page server now stops the media proxy copies it started (pre-warms and the page's own asks). A copy whose last caller leaves is now stopped even after its ffmpeg started; callers without a signal still always get their copy.
somanshreddy
left a comment
There was a problem hiding this comment.
Reviewed at 5926aa4a. No blocker, so I'm approving.
What I checked
- Who can be cut off: the only caller that passes a signal is the new one in
staticProjectServer. Studio'spreview.ts(resolveProxyandinjectMediaCodecMap),play.ts, andresolveProxies(check and publish) pass none. Their callers never leave, so a copy any of them shares with a snapshot pre-warm still finishes.checkresolves its copies throughresolveProxiesbefore closing its server, so closing it doesn't abort anythingcheckstill waits on. - Job lifecycle after an abort:
leavedrops a queued job as before. For a started job it callsforgetand then aborts.forgetchecks identity (inFlight.get(...)?.job === job), so neither it nor the later.finallycan evict a newer job for the same key.- The killed child's
errorevent is ignored once the signal has aborted, and thecloserejection isn't remembered as a failure, so the next ask spawns a fresh copy instead of getting a cached 502. - The temp file is still unlinked in
finally, and the slot is released when the killed child exits. - Aborting after the copy has already been renamed into place does nothing.
- Tests: studio-server
proxyTranscoderandmediaProxyPreviewpass 47 of 47. CLIstaticProjectServer,snapshot,snapshot.proxyandplaypass 124 of 124. - Mutation checks:
- Removing
forget(job); job.abandoned.abort()fromleavefails 2 tests: "stops a started copy…" and "kills the running ffmpeg…". - Removing
copies.abort()from bothclosepaths fails "stops the copies it started, pre-warmed or asked for, when it closes".
- Removing
Nit: a caller with a signal that joins a job a signal-less caller started counts toward callers as usual, so the copy only stops once both have left. That's correct. It just means the "kept while a caller without a signal still waits" rule depends on signal-less callers never calling leave. A one-line comment in joinJob saying so would protect it from a future refactor.
I reproduced neither the 118 s nor the 19 s timing. CI was 23 pass, 19 pending and 0 failing when I approved.
— Somu
Edit accuracy: accurate 271 (base branch 271), smooth 187 of thoseThe gate passes. Quarantined, measured but not gated (0) Unstable (1)
|
Conflicts, all from #5895 (a copy whose last caller leaves is stopped) except one line in init.ts: - proxyTranscoder.ts: each job keeps both main's `abandoned` signal (handed to spawn, so the last caller leaving kills the encoder) and this branch's slot handoff and one-copy-per-source replacement. A copy that was replaced or abandoned is not remembered as failed. - staticProjectServer.ts: the page's own `?hf-proxy=` asks keep main's close signal. The pre-warm call drops it with the pre-warm itself, which this branch removed; passing main's `true` there would now have turned on seek copies for the static server. - mediaProxyPreview.ts and its test: keep this branch (no pre-warm, so no pre-warm signal). - staticProjectServer.test.ts: main's close test now covers the page's own ask only. - proxyTranscoder.test.ts: both import sets. - init.ts: keep both the held-timeline post and the seek-copy playback note. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Main's #5895 adds the abandoned-copy signal to the transcoder this branch already grows. Packed with pnpm 10.17.1 on Linux x86_64, two runs each, identical: this branch 141,802 bytes, main (e45832f) 140,139. The budget moves to 148,893 (141,802 times the script's 1.05 headroom). Unpacked (556,247) stays inside its budget. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Four conflicts over 1024 upstream files. - audioPadTrim.ts: upstream moved the true-peak correction behind an encode(input, output, duration, filter) callback (heygen-com#5956, heygen-com#5981) and kept whole-mix attenuation. The fork's limiter now runs through that callback as alimiter=...:level=false:latency=true,apad,atrim=0:<dur>. encodeAac filters the whole input before each segment's trim, so the limiter stays continuous across segment joins. - audioPadTrim.test.ts: fork limiter tests move onto encodeAac; upstream's two attenuation-only tests are dropped. - staticProjectServer.test.ts: kept both sides' new tests. - bun.lock: upstream's, regenerated; the diff is the fork's gsap dependency. Two fixes the merge needed beyond the conflicts: - staticProjectServer.ts: the TAB-805 close dropped every socket at once, which destroyed the 500 that upstream's close now writes to an aborted proxy request (heygen-com#5895). The drop now waits a 250 ms grace. - audioTruePeak.fakeFfmpeg.test.ts: the fork reports the true-peak drop (1.3 dB), not upstream's applied attenuation (1.2 dB). All 23 other fork-patched files that upstream also changed keep their patch hunks unchanged. Co-Authored-By: Claude Opus 5.5 <[email protected]>
What
hyperframes snapshot --proxynow exits as soon as its frames are written. Before, it stayed running until every media copy its page server had started was finished: on ten one-minute 1080x1920 HEVC clips, 118 s instead of 19 s, plus 555 MB of copies nobody used.Why
The static page server pre-warms an H.264 copy of every browser-hostile clip when it serves the page, and serves
?hf-proxy=requests by transcoding. Nothing stopped those copies when the server closed, so each spawned ffmpeg kept the CLI's process alive until it finished. A snapshot takes its frames from the original files, so once it is done the copies are pure cost.How
proxyTranscoder.ts: each copy job gets anabandonedsignal. When its last caller with a signal leaves, a queued copy is dropped (as before) and a started one now has its ffmpeg killed and its entry forgotten, so a new ask starts afresh and the kill is not remembered as a failure. A caller without a signal never leaves, so Studio's preview,play, publish andcheck, which pass none, always get their copy as before. This reverses one rule from fix(studio-server): the preview's proxy copy jumps queued thumbnails, and abandoned copies drop #5340 ("a started copy is never stopped for a caller that leaves"): it now holds only while someone still waits for the copy.mediaProxyPreview.ts:injectMediaCodecMapIntoHtmltakes an optional signal for its pre-warms.staticProjectServer.ts: the server owns oneAbortController, passes its signal to every pre-warm and every?hf-proxy=request it serves, and aborts it in bothclosepaths (listening and in-process).Before / after
Devbox (Linux, 4 CPUs), ten 60 s 1080x1920 HEVC clips,
media.autoProxy: true, cold copy cache, one run each, both trees fully built:snapshot --proxy --at 1,10,20checkcheckdoes not change: it resolves the copies it judges before it starts, so it waits for them by design. No ffmpeg was left running after any of the four runs.Tests
proxyTranscoder.test.ts: the last caller leaving a started copy aborts its ffmpeg's signal, and the next ask spawns a fresh copy; a copy is kept while a caller without a signal still waits; with a real child process standing in for ffmpeg, leaving kills it (SIGKILL). The first and last fail on main; the old "a caller leaving a started copy does not stop it" test is replaced.staticProjectServer.test.ts: closing the server aborts the signal handed to its pre-warms and to a held?hf-proxy=request, and the held request answers. Fails on main.proxyTranscoder+mediaProxyPreview(47) and CLIstaticProjectServer,snapshot,snapshot.proxy,play(122): 3 runs in a row green on the commit before this rebase onto main (no conflicts; the files this touches did not change upstream); typecheck, oxlint, the comment ratchet and fallow clean.