Repository navigation
fix(process): preserve UTF-8 at diagnostic truncation boundaries - #8907
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
[Low impact] The PR appears safe to merge; both previous findings are fixed and no new actionable issues were found. SummaryPreserves whole UTF-8 characters around the diagnostic omission marker while keeping malformed-byte evidence visible.
Reviews (2) · Last reviewed commit: "fix(process): preserve malformed UTF-8 e..." · Reviewed by Greptile |
|
Both CI and author readiness are now passing on current head |
Problem
Byte-bounded command diagnostics can introduce replacement characters into valid UTF-8 output when the retained head or tail cuts through a multibyte character. For example, an eight-byte excerpt of
A🙂...🙂Zshows broken emoji bytes around the omission marker. A one-shotStringDecoder.write()also hides some malformed bytes at the end of the retained head, such as0xc0.Changes
Decode the retained head after excluding only a suffix that could still become a valid UTF-8 code point. Preserve invalid lead bytes and already-invalid prefixes, including overlong encodings, surrogates, and values above U+10FFFF. Skip only the possible continuation-byte fragment at the cut edge of the tail. Raw retention stays within the existing byte budget.
Complete output is decoded after joining the head and tail. The non-truncated regression now uses
A🙂, whose five-byte buffer is split inside the emoji by the three-byte head limit; every two-chunk split and bytewise append must reconstruct the exact original text.The semantic owner is
BoundedOutputExcerpt, shared by the bounded command runner's stdout/stderr diagnostics. This corrects an existing process-output boundary without adding an execution capability or changing byte limits, exit codes, or the tail policy.Current validation
Head:
30aaca9bffbfad607bb008083f720120cf6024fc.node scripts/run-development-tests.mjs local src/process/BoundedOutputExcerpt.test.ts src/process/ExecFileBoundedOutput.test.ts) passed 29 tests / 0 failures on Node 24.22.1 / Vitest 5.0.3: 25 module cases plus the existing four subprocess tests.ea26bd29891b3466297e86654e63978fb7de8a72) reported 9 failed / 20 passed, exposing the malformed-byte loss.A� … [output omitted] … BZin both stdout and stderr after the fix, with exit code 7 and truncation unchanged. The prior source lost the replacement character.--max-warnings 0, passed for both changed files. Targeted strict TypeScript 5.9.3 checking of the two modules and their tests, plusgit diff --check, also passed.The full project checks and real Hopper setup were not run locally. The hosted CI run 41866035849 has now completed successfully on this exact head,
30aaca9bffbfad607bb008083f720120cf6024fc.Author readiness
Human owner: @Afloat16