Skip to content

fix(cli): keep UTF-8 log tails valid#101029

Merged
steipete merged 2 commits into
openclaw:mainfrom
ly85206559:codex/windows-android-small-prs-2
Jul 6, 2026
Merged

fix(cli): keep UTF-8 log tails valid#101029
steipete merged 2 commits into
openclaw:mainfrom
ly85206559:codex/windows-android-small-prs-2

Conversation

@ly85206559

@ly85206559 ly85206559 commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

Fixes an issue where bounded CLI log tails could show a UTF-8 replacement character
when trimming kept the newest bytes starting in the middle of a multibyte character.

Why This Change Was Made

execFileUtf8Tail now keeps stdout and stderr in a shared byte-tail representation
and decodes truncated tails from the next valid UTF-8 boundary. The existing result
contract is unchanged: truncated still reports stdout truncation for the journal
caller, while stderr remains independently bounded for diagnostics.

User Impact

Users reading bounded CLI journal output get valid UTF-8 text when the byte limit cuts
through a character. Failure diagnostics receive the same boundary-safe decoding.

Evidence

  • Direct baseline reproduction: retaining the last three bytes of UTF-8 €z produces
    bytes [130,172,122], which current main decodes as "��z".
  • node scripts/run-vitest.mjs src/cli/logs-cli.runtime.test.ts src/cli/logs-cli.test.ts
    • 45 tests passed.
  • git diff --check origin/main...HEAD
  • python3 .agents/skills/autoreview/scripts/autoreview --mode branch --base origin/main
    • No accepted or actionable findings.

@openclaw-barnacle openclaw-barnacle Bot added cli CLI command changes size: XS labels Jul 6, 2026
@ly85206559
ly85206559 marked this pull request as ready for review July 6, 2026 17:14
@clawsweeper

clawsweeper Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed July 6, 2026, 1:54 PM ET / 17:54 UTC.

Summary
The branch adds UTF-8-boundary decoding for truncated execFileUtf8Tail stdout and a regression test for a retained tail that starts inside a multibyte character.

PR surface: Source +12, Tests +22. Total +34 across 2 files.

Reproducibility: yes. from source inspection: current main can retain stdout beginning with a UTF-8 continuation byte and then decode that suffix with Buffer.toString("utf8"). I did not run tests in this read-only review, so this is source-reproducible rather than locally reproduced.

Review metrics: none identified.

Merge readiness
Overall: 🦪 silver shellfish
Proof: 🦪 silver shellfish
Patch quality: 🐚 platinum hermit
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Add redacted real terminal/log output showing the after-fix truncated non-ASCII tail behavior without private paths, tokens, IPs, phone numbers, or endpoints.

Proof guidance:

  • [P1] Needs real behavior proof before merge: The PR body shows tests/static checks but no after-fix real CLI behavior. Before merge, add a redacted terminal transcript, screenshot, log, or linked artifact from openclaw logs or an equivalent real helper run showing the UTF-8 tail result; updating the PR body should trigger re-review, or a maintainer can comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

  • [P1] The only material merge blocker found is proof: the contributor has not shown a redacted real CLI/log-tail run demonstrating the after-fix UTF-8 boundary behavior.

Maintainer options:

  1. Decide the mitigation before merge
    Land the narrow helper fix after the contributor adds redacted terminal output, logs, a screenshot, or a linked artifact showing a truncated non-ASCII log tail no longer emits a replacement character.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • [P1] Human-only follow-up is needed because the contributor must provide real behavior proof from a run of the changed CLI/log-tail behavior; ClawSweeper should not generate repair markers for this proof gap.

Security
Cleared: The diff only changes bounded child-process stdout decoding and a Vitest regression test; it does not touch dependencies, workflows, credentials, permissions, or package resolution.

Review details

Best possible solution:

Land the narrow helper fix after the contributor adds redacted terminal output, logs, a screenshot, or a linked artifact showing a truncated non-ASCII log tail no longer emits a replacement character.

Do we have a high-confidence way to reproduce the issue?

Yes from source inspection: current main can retain stdout beginning with a UTF-8 continuation byte and then decode that suffix with Buffer.toString("utf8"). I did not run tests in this read-only review, so this is source-reproducible rather than locally reproduced.

Is this the best way to solve the issue?

Yes, this appears to be the narrowest maintainable fix: normalize only the truncated stdout tail before decoding, leaving stderr and untruncated output behavior alone. The remaining issue is proof, not the code shape.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against bb5979064602.

Label changes

Label changes:

  • add P2: This is a normal-priority CLI diagnostics correctness fix with limited blast radius and no evidence of data loss, security bypass, or runtime availability impact.
  • add rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🐚 platinum hermit.
  • add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body shows tests/static checks but no after-fix real CLI behavior. Before merge, add a redacted terminal transcript, screenshot, log, or linked artifact from openclaw logs or an equivalent real helper run showing the UTF-8 tail result; updating the PR body should trigger re-review, or a maintainer can comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Label justifications:

  • P2: This is a normal-priority CLI diagnostics correctness fix with limited blast radius and no evidence of data loss, security bypass, or runtime availability impact.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🐚 platinum hermit.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body shows tests/static checks but no after-fix real CLI behavior. Before merge, add a redacted terminal transcript, screenshot, log, or linked artifact from openclaw logs or an equivalent real helper run showing the UTF-8 tail result; updating the PR body should trigger re-review, or a maintainer can comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed

PR surface:

Source +12, Tests +22. Total +34 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 14 2 +12
Tests 1 22 0 +22
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 36 2 +34

What I checked:

  • Current main still decodes a byte-trimmed stdout suffix directly: execFileUtf8Tail on current main trims stdout by bytes and resolves with Buffer.concat(stdoutChunks).toString("utf8"), so a retained suffix beginning with a UTF-8 continuation byte can decode to a replacement character. (src/cli/logs-cli.runtime.ts:31, bb5979064602)
  • Runtime entry point is the systemd journal fallback for openclaw logs --follow: readSystemdJournalFallback invokes runtime.execFileUtf8Tail("journalctl", args, { env: process.env, maxBytes }), so the helper owns bounded journal stdout before parsing and redaction. (src/cli/logs-cli.ts:314, bb5979064602)
  • PR patch addresses only truncated stdout decoding: The diff adds decodeUtf8Tail, skips leading continuation bytes only when stdout was truncated, and keeps stderr/untruncated decoding behavior unchanged. (src/cli/logs-cli.runtime.ts:10, ce6876e72762)
  • Regression coverage targets the claimed byte-boundary failure: The added test emits Buffer.from("é", "utf8") with maxBytes: 1 and expects truncated stdout to be empty rather than a replacement character. (src/cli/logs-cli.runtime.test.ts:58, ce6876e72762)
  • Sibling log-tail path was checked: The configured file-log tail already drops a partial first line when byte-bounded reads start mid-file, so the PR's subprocess stdout fix is not obviously missing the same invariant in the file fallback. (src/logging/log-tail.ts:120, bb5979064602)
  • Related history for the helper was inspected: The current helper file was introduced on main by de1bac6bc0636199df0d40e2bcc812efd9f01e5b; a prior adjacent PR, fix(cli): handle stdout/stderr stream errors in execFileUtf8Tail #100850, fixed stream-error handling in the same helper but not this UTF-8 boundary issue. (src/cli/logs-cli.runtime.ts:10, de1bac6bc063)

Likely related people:

  • sunlit-deng: Git history shows the current logs-cli.runtime.ts and logs-cli.ts files entered main through de1bac6bc0636199df0d40e2bcc812efd9f01e5b, which introduced the helper and journal fallback now being patched. (role: introduced current helper surface; confidence: high; commits: de1bac6bc063; files: src/cli/logs-cli.runtime.ts, src/cli/logs-cli.ts)
  • cxbAsDev: Merged fix(cli): handle stdout/stderr stream errors in execFileUtf8Tail #100850 authored adjacent execFileUtf8Tail stream-error handling and tests shortly before this PR. (role: adjacent helper contributor; confidence: medium; commits: d898ff5582b7, 79dcc46a6165, f23ed142e5a2; files: src/cli/logs-cli.runtime.ts, src/cli/logs-cli.runtime.test.ts)
  • vincentkoc: GitHub PR metadata for fix(cli): handle stdout/stderr stream errors in execFileUtf8Tail #100850 shows vincentkoc merged the adjacent helper fix and authored the final repair commit on that PR. (role: adjacent merger and repair author; confidence: medium; commits: 6e68bb958cf6, 9606fe43231b; files: src/cli/logs-cli.runtime.ts, src/cli/logs-cli.runtime.test.ts)
  • steipete: GitHub metadata for fix(discord): bound gateway websocket payloads #99998 shows steipete merged the PR whose squashed commit introduced the current CLI logs runtime files on main. (role: recent merger and co-author on current CLI log surface; confidence: medium; commits: de1bac6bc063; files: src/cli/logs-cli.runtime.ts, src/cli/logs-cli.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. labels Jul 6, 2026
@steipete steipete self-assigned this Jul 6, 2026
@vincentkoc vincentkoc self-assigned this Jul 6, 2026
@steipete

steipete commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Land-ready review completed on exact head 365d54c9e4f0bba4a7c921443add9437773e7ef4.

I rewrote the initial stdout-only decoder patch into one bounded byte-tail implementation shared by stdout and stderr. It now tracks truncation independently, trims only discarded leading UTF-8 continuation bytes, preserves complete 2/3/4-byte code points, and keeps the existing 64 KiB stderr cap. The regression suite covers every partial-width boundary, an exact boundary, and stderr truncation.

Proof:

  • sanitized AWS Crabbox run_a272a0485e98: pnpm test src/cli/logs-cli.runtime.test.ts src/cli/logs-cli.test.ts — 45/45 passed
  • sanitized AWS Crabbox run_9067cbda3a81: real child process split 😀z at a 4-byte tail and returned exactly z, with truncated: true
  • oxfmt check and git diff --check passed
  • fresh GPT-5.5 high-thought autoreview: no actionable findings
  • hosted exact-head CI run 28814405255: all required jobs passed
  • native OPENCLAW_TESTBOX=1 scripts/pr prepare-run 101029: passed

No known proof gaps. Thanks @ly85206559 for finding and fixing the invalid UTF-8 tail behavior.

@steipete
steipete merged commit afcc93b into openclaw:main Jul 6, 2026
109 of 111 checks passed
@steipete

steipete commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Merged via squash.

@ly85206559
ly85206559 deleted the codex/windows-android-small-prs-2 branch July 7, 2026 05:34
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jul 7, 2026
* fix(cli): keep UTF-8 log tails valid

* fix(cli): share UTF-8-safe byte tails

---------

Co-authored-by: Peter Steinberger <[email protected]>
giodl73-repo pushed a commit to giodl73-repo/openclaw that referenced this pull request Jul 8, 2026
* fix(cli): keep UTF-8 log tails valid

* fix(cli): share UTF-8-safe byte tails

---------

Co-authored-by: Peter Steinberger <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cli CLI command changes P2 Normal backlog priority with limited blast radius. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. size: S status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants