Repository navigation
fix(cli): stop a caught post-render throw reporting a valid render as failed - #3409
Merged
Merged
Conversation
… failed A render that produced and validated its artifact still exited 1. Reported again from the field on 0.8.7: the MP4 was on disk and an independent ffprobe and full decode both passed, and the CLI exited 1 immediately after logging `artifact validated`. `render-success-state.ts` exists for exactly this and documents three earlier cases, so the sentinel was already there. Its gap is which paths read it: the uncaughtException and unhandledRejection handlers both consult `isRenderSucceeded()`, but a post-render throw that the command wrapper CATCHES never reaches either. It becomes an ordinary non-zero CommandResult, and `finalizeCli` wrote that straight to `process.exitCode`. The result was a run that disagreed with itself: `commandSucceededForTelemetry()` already lets a validated render override a failure, so telemetry recorded success while the shell saw exit 1. Sanitize once in `finalizeCli`, where every command result funnels through, rather than wrapping the individual steps. Which step threw does not matter; that the artifact is committed does. The throw is still printed, so it stays visible for diagnosis without being fatal. Reproduced first as a failing test (`expected 1 to be +0`) on macOS, so this is not Windows-specific — the field reports are one instance of it. A second test pins the other side: a command that throws with no validated render still exits non-zero, so the sanitizer cannot swallow a genuine failure.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A render that produced and validated its artifact could still exit
1.finalizeClinowsanitizes the exit code once a render has validated, so a caught teardown throw cannot
report a good render as a failure.
Why
From the field on 0.8.7: the MP4 was on disk, an independent
ffprobeand a full decodeboth passed, and the CLI exited
1immediately after loggingartifact validated.This shape is already known —
render-success-state.tsexists solely for it and documentsthree earlier cases — so the sentinel was in place. The gap is which paths read it:
isRenderSucceeded()uncaughtExceptionhandlerunhandledRejectionhandlerA caught post-render throw never reaches either handler. It becomes an ordinary non-zero
CommandResult, andfinalizeCliwrote that straight toprocess.exitCode.That left a run disagreeing with itself.
commandSucceededForTelemetry()already lets avalidated render override a failure, so telemetry recorded the run as a success while the
shell saw exit
1.How
One line of policy in
finalizeCli, where every command result funnels through:used for
commandFailed, the telemetry record, andprocess.exitCode, so the three cannotdisagree.
Deliberately not wrapping the individual teardown steps. Which step threw does not
matter; that the artifact is committed does. Wrapping N call sites would leave the N+1th
unguarded, which is how this reached the field in the first place. The throw is still
printed, so it stays visible for diagnosis without being fatal.
Test plan
Reproduced as a failing test before fixing, and on macOS — this is not Windows-specific,
the field reports are one instance of it.
Before:
After: 8 passed in
cli.lifecycle.test.ts, and the full CLI package is green at2812 passed / 3 skipped across 193 files.
Two tests, one per side of the invariant:
0, and reportssuccess: true, exitCode: 0to telemetry;success: false— the guard that stops the sanitizer swallowing genuine failures.Not covered
chrome-headless-shelltrees from earlier interrupted runs.That is a separate cleanup problem and is untouched here; this PR only stops a delivered
render being scored as failed.
deliberately independent of that — it holds for any of them — but it does mean the
underlying throw is still occurring and will now surface as a printed warning rather than
a failed exit.