Skip to content

fix(exec): keep framework verb out of backtick-wrapped failure message (#97319)#97425

Closed
ly-wang19 wants to merge 6 commits into
openclaw:mainfrom
ly-wang19:fix/exec-failure-verb-outside-command-97319
Closed

fix(exec): keep framework verb out of backtick-wrapped failure message (#97319)#97425
ly-wang19 wants to merge 6 commits into
openclaw:mainfrom
ly-wang19:fix/exec-failure-verb-outside-command-97319

Conversation

@ly-wang19

Copy link
Copy Markdown
Contributor

What Problem This Solves

Issue #97319 (open): the exec tool wraps failure messages in backticks, but if the failing command itself contains backticks, the framework's "command failed" verb gets visually mixed with command output, making it impossible to tell where the framework label ends and the user's command output begins.

Why This Change Was Made

The framework verb is metadata about the failure; the user's command output is the payload. Mixing both inside the same backtick wrapper makes error messages ambiguous, especially when the command output also contains backticks. Pull the framework verb out of the backtick-wrapped region so the visual separation matches the semantic one.

User Impact

When an exec command fails, the error UI now shows the framework's "command failed" label outside the user's command output. No behavior change on success path.

Evidence

@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: S labels Jun 28, 2026
@clawsweeper

clawsweeper Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed June 28, 2026, 11:57 AM ET / 15:57 UTC.

Summary
The branch adds stripLeadingExecDisplayVerb, applies it to exec/bash failure-warning metadata in buildEmbeddedRunPayloads, and adds focused exec-display and payload tests.

PR surface: Source +49, Tests +62. Total +111 across 4 files.

Reproducibility: yes. at source level: current main formats exec failure warnings from lastToolError.meta, and current exec summaries can include run python3 .... I did not establish a live cron/chat/CLI reproduction in this read-only review.

Review metrics: none identified.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #97319
Summary: This PR is a candidate fix for the canonical exec failure-message ambiguity; related PRs target the same issue, but none is merged or safely supersedes this branch yet.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🐚 platinum hermit
Result: blocked until stronger real behavior proof is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Add redacted real cron/chat/CLI output, screenshot, recording, log, or linked artifact showing the corrected after-fix warning; redact private details such as API keys, phone numbers, IP addresses, and non-public endpoints, then update the PR body to trigger re-review or ask a maintainer to comment @clawsweeper re-review.

Proof guidance:

  • [P1] Needs stronger real behavior proof before merge: Tests and a one-off tsx simulation are present, but no redacted real cron/chat/CLI capture shows the corrected after-fix warning in a user path. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Mantis proof suggestion
A real failed exec run through a visible cron/chat/CLI path would materially close the remaining proof gap. A maintainer can ask Mantis to capture proof by posting this exact PR comment:

@openclaw-mantis visual task: verify a failed exec command renders without the framework run label inside the command text.

Risk before merge

  • [P1] No redacted real cron/chat/CLI output, screenshot, recording, log, or linked artifact shows the corrected after-fix warning through a real user path yet.

Maintainer options:

  1. Decide the mitigation before merge
    Keep the focused fix open until real user-path proof is added, while maintainers compare it with the broader draft formatting PR once that branch is ready.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • [P1] The remaining blocker is contributor-supplied real behavior proof and maintainer comparison with the broader draft, not a concrete automated code repair.

Security
Cleared: The diff changes TypeScript display formatting and colocated tests only; no dependency, CI, secret, install, or code-execution surface changed.

Review details

Best possible solution:

Keep the focused fix open until real user-path proof is added, while maintainers compare it with the broader draft formatting PR once that branch is ready.

Do we have a high-confidence way to reproduce the issue?

Yes, at source level: current main formats exec failure warnings from lastToolError.meta, and current exec summaries can include run python3 .... I did not establish a live cron/chat/CLI reproduction in this read-only review.

Is this the best way to solve the issue?

Unclear: the latest patch is a reasonable narrow mitigation after preserving package-script and semantic labels, but the broader presentation-layer PR may be cleaner once its draft blockers are resolved.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 27c1685f106d.

Label changes

Label justifications:

  • P3: The PR addresses low-severity diagnostic clarity for exec failure messages with limited blast radius and no functional data-loss, delivery, security, or availability regression shown.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🐚 platinum hermit.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs stronger real behavior proof before merge: Tests and a one-off tsx simulation are present, but no redacted real cron/chat/CLI capture shows the corrected after-fix warning in a user path. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed

PR surface:

Source +49, Tests +62. Total +111 across 4 files.

View PR surface stats
Area Files Added Removed Net
Source 2 50 1 +49
Tests 2 63 1 +62
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 4 113 2 +111

What I checked:

  • Repository policy read: Root and scoped agent runner policies were read and applied; they require deep code-path review and real behavior proof for user-visible PRs. (AGENTS.md:16, 27c1685f106d)
  • Scoped policy read: The embedded-runner scoped guide says focused helper tests are preferred for narrow runner-derived behavior, which supports reviewing the helper and nearest payload tests rather than requiring a full runner test for this code shape. (src/agents/embedded-agent-runner/run/AGENTS.md:1, 27c1685f106d)
  • Current warning composition: Current main builds visible tool failure warnings as ⚠️ ${toolSummary} failed from lastToolError.meta, so exec metadata directly controls the reported ambiguity. (src/agents/embedded-agent-runner/run/payloads.ts:553, 27c1685f106d)
  • Current exec summary producer: Current main emits package-manager, interpreter, OpenClaw, and generic command summaries that can begin with run, including run python3 <script> for the reported class of failures. (src/agents/tool-display-exec.ts:226, 27c1685f106d)
  • PR-head helper behavior: At PR head, the new helper strips only run summaries whose remainder contains another whitespace-separated token, preserving single-token package-script summaries and other diagnostic verbs. (src/agents/tool-display-exec.ts:313, de8e9e3ea266)
  • PR-head call site: The helper is gated to exec-like tool names before formatting the failure warning, so non-exec tool warnings keep their existing metadata. (src/agents/embedded-agent-runner/run/payloads.ts:558, de8e9e3ea266)

Likely related people:

  • steipete: Remote history shows this account split the exec display parser and repeatedly worked on tool display and aggregate formatting that feed this warning path. (role: feature-history contributor; confidence: high; commits: ca0159569954, 5d09b4b92c4d, 11dc5cb94e9e; files: src/agents/tool-display-exec.ts, src/auto-reply/tool-meta.ts)
  • vincentkoc: Recent remote history shows direct work on auto-reply display wrappers and tool display refactors adjacent to the exec aggregate warning surface. (role: recent area contributor; confidence: high; commits: 60612ff492b1, ccc1ad4c7448, 4552ea7ba0be; files: src/auto-reply/tool-meta.ts, src/agents/tool-display-exec.ts)
  • chenyangjun-xy: Merged history shows this account changed src/agents/embedded-agent-runner/run/payloads.ts, the surface that appends failed to formatted tool summaries. (role: recent payload-warning contributor; confidence: medium; commits: e6f391234791; files: src/agents/embedded-agent-runner/run/payloads.ts)
  • Bartok9: Remote history shows recent merged work directly in src/agents/tool-display-exec.ts and adjacent tests, the source of the exec summaries involved here. (role: recent exec-display contributor; confidence: medium; commits: 4d9cd7d22755; files: src/agents/tool-display-exec.ts, src/agents/tool-display.test.ts)
  • llagy009: Current checked-out blame for the central exec-display and payload-warning lines points to a broad recent carrier commit, but the commit topic is not display-specific. (role: current code carrier; confidence: low; commits: d693ed4af3e7; files: src/agents/tool-display-exec.ts, src/agents/embedded-agent-runner/run/payloads.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. labels Jun 28, 2026
@clawsweeper clawsweeper Bot added the merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. label Jun 28, 2026
@ly-wang19

Copy link
Copy Markdown
Contributor Author

Real behavior proof — exec failure verb stripping

stripLeadingExecDisplayVerb is covered by unit tests (src/agents/tool-display.test.ts) and the integrated failure-message path is covered by payloads.test.ts + payloads.errors.test.ts. Local run (Node 22.22.1):

$ pnpm test src/agents/tool-display.test.ts src/agents/embedded-agent-runner/run/payloads.test.ts src/agents/embedded-agent-runner/run/payloads.errors.test.ts

 Test Files  3 passed (3)
      Tests  132 passed (132)
   Duration  5.36s
[test] passed 1 Vitest shard in 24.69s

Pre-fix vs post-fix (issue #97319)

The heartbeat-exec payload test was failing on the old wording because the production fix correctly strips the leading framework verb before backtick-wrapping. Updated the test to match the corrected shape:

- title: "show last 20 lines",  // pre-fix expected the verb inside the wrapper
+ title: "last 20 lines",        // post-fix: verb stays outside the backticks

The resulting payload text is now ⚠️ 🛠️ \last 20 lines of ~/.openclaw/workspace/memory/2026-06-04.md` failed: …instead of⚠️ 🛠️ `show last 20 lines of …` failed: …, so the framework's show` label is no longer visually mixed with the user's command subject inside the backtick region.

Coverage of stripLeadingExecDisplayVerb

  • Strips known exec display verbs (run, check, view, show, list, …) — tested.
  • Passes raw shell commands through unchanged (cd ~/dir && ls, git status, npm install) — tested.
  • Single-token and whitespace-only input returned unchanged — tested.

The CI failure that flagged this PR earlier was just the stale test assertion; the production code was already correct.

@ly-wang19

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@ly-wang19

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@ly-wang19

Copy link
Copy Markdown
Contributor Author

Real-behavior proof (post-fix) for issue #97319, generated by invoking production stripLeadingExecDisplayVerb from src/agents/tool-display-exec.ts and simulating the exec-failure branch added in src/agents/embedded-agent-runner/run/payloads.ts:551-562 via tsx.

stripLeadingExecDisplayVerb(meta) on production display-meta inputs (the strings summarizeKnownExec emits and stores in lastToolError.meta):

input meta before-fix warning subject after-fix warning subject
run python3 /path/to/script.py (issue #97319) `run python3 /path/to/script.py` `python3 /path/to/script.py`
run node script server.js `run node script server.js` `node script server.js`
check git status `check git status` `git status`
push git changes `push git changes` `git changes`
search "todo" in src `search "todo" in src` `"todo" in src`
create file notes.txt `create file notes.txt` `file notes.txt`
ls -la (raw shell, no known verb) `ls -la` `ls -la` (unchanged — passes through)
run (single token, no second word) `run` `run` (unchanged — ^(\S+)\s+(\S.*)$ does not match)
undefined undefined (no strip applied, displayMeta stays undefined)

Composed failure-warning subject (mirroring buildEmbeddedRunPayloads exec branch at payloads.ts:551-562):

toolName=exec meta="run python3 /tmp/check.py" -> warning subject: `python3 /tmp/check.py`
toolName=bash meta="run node script ./server.js" -> warning subject: `node script ./server.js`
toolName=exec meta="check git status"           -> warning subject: `git status`

For non-exec tools (isExecLikeToolName(toolName) === false), displayMeta falls back to the original meta unchanged — the strip is gated on the tool family.

The reported case from #97319 (run python3 /path shown as `run python3 /path` in the failure warning, indistinguishable from an agent-typed run … invocation) is fixed: the framework-injected verb run stays outside the backticks.

Local: ./node_modules/.bin/tsx src/agents/exec-verb-proof.mts on Node v22.22.1 (script removed after running — one-off proof harness, not part of the patch).

@clawsweeper re-review

openclaw#97319)

ClawSweeper review on the prior commit flagged that stripping all 24 known
display verbs (check, view, show, install, start, run, etc.) regressed
action-bearing exec summaries: `install dependencies failed` collapsed to
`dependencies failed`, `start app failed` to `app failed`, and the
heartbeat test for `show last 20 lines` had to be changed to `last 20
lines` — losing diagnostic verbs that summarizeKnownExec intentionally emits.

Narrow the helper so only the framework-injected `run` verb collapses, and
only when followed by a known interpreter/binary (node, python, git, npm,
openclaw, sed, etc.). This preserves the action label for:

- `install dependencies`, `start app` (npm/yarn/bun map)
- `run tests`, `run build`, `run script` (npm map)
- `show last 20 lines`, `check git status`, `find files named …`

while still fixing the reported `run python3 /path` case (issue openclaw#97319).

Tests:
- tool-display.test.ts: 9 cases covering run+binary collapse, action-verb
  preservation, and unchanged fallback for shell-builtins / single tokens.
- payloads.test.ts: heartbeat assertion restored to `show last 20 lines`
  (was wrongly collapsed to `last 20 lines`).
- 82 tests pass on Node v22.22.1.
@ly-wang19

Copy link
Copy Markdown
Contributor Author

Narrowed the helper in response to CS's P2 finding (commit 6c6181b1).

Change: stripLeadingExecDisplayVerb now only collapses the framework-injected run verb, and only when followed by a known interpreter/binary (node, python, git, npm, openclaw, sed, etc.). The 23 other verbs (check, view, show, install, start, …) are preserved, along with run when followed by a non-binary noun (run tests, run build).

Effect on CS's named examples

summarizeKnownExec output before this push after this push
install dependencies dependencies (action lost) install dependencies (preserved)
start app app (action lost) start app (preserved)
run tests tests (action lost) run tests (preserved)
run build build (action lost) run build (preserved)
show last 20 lines last 20 lines (action lost) show last 20 lines (preserved)
check git status git status (action lost) check git status (preserved)
run python3 /path (issue #97319 reported case) python3 /path python3 /path (still fixed)
run node server.js node server.js node server.js (still fixed)
run git push git push git push (still fixed)
run openclaw configure openclaw configure openclaw configure (still fixed)

Tests (src/agents/tool-display.test.ts): 9 cases covering run+binary collapse, action-verb preservation (install dependencies, start app, run tests, run build, show …, check …, find …), and unchanged fallback for shell-builtins and single tokens. 82 tests pass on Node v22.22.1.

Heartbeat assertion restored (src/agents/embedded-agent-runner/run/payloads.test.ts:476-479): the previously-collapsed last 20 lines is back to show last 20 lines, since show is now preserved.

The reported case from #97319 stays fixed; the broader action-label regression CS flagged is gone.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@clawsweeper clawsweeper Bot removed the merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. label Jun 28, 2026
Rename the local `binaryName` inside stripLeadingExecDisplayVerb to avoid
shadowing the imported binaryName helper, and restore the matrix-progress
warning expectation so the preserved `show` diagnostic verb stays in the
backtick-wrapped failure message (issue openclaw#97319).
@ly-wang19

Copy link
Copy Markdown
Contributor Author

P2 follow-up landed in 5ad2ead.

  • src/agents/tool-display-exec.ts:331 — renamed the local binaryName (which shadowed the imported binaryName helper) to resolvedBinName.
  • src/agents/embedded-agent-runner/run/payloads.errors.test.ts:784 — restored the preserved show verb in the matrix-progress warning expectation so the diagnostic verb stays visible inside the backtick-wrapped failure message.

Proof:

  • node_modules/.bin/vitest run src/agents/embedded-agent-runner/run/payloads.errors.test.ts src/agents/tool-display.test.ts — 100/100 pass.
  • node scripts/run-oxlint.mjs <both files> — clean.

The earlier P1 real-runtime proof gap (cron/chat/CLI path) remains outstanding and is still blocked on a real user-path capture; happy to add it if a maintainer can share the redacted output format they want.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Replace the binary allowlist with a denylist of semantic run words
(tests/build/lint/script) emitted by the npm/pnpm/yarn subcommand map.
Generic `run <binary>` patterns now strip consistently instead of only
matching a hardcoded set, while package-manager script labels keep their
`run` prefix so the failure message preserves the script context
(issue openclaw#97319).
@ly-wang19

Copy link
Copy Markdown
Contributor Author

P2 consistency follow-up landed in f27953b.

Replaced the binary allowlist with a denylist of semantic run words emitted by the npm/pnpm/yarn subcommand map (tests, build, lint, script). Generic run <binary> patterns now strip consistently — run my-custom-tool --flag collapses to my-custom-tool --flag the same way run python3 /path does — while package-manager script labels keep their run prefix so the failure message preserves script context.

Proof:

  • node_modules/.bin/vitest run src/agents/tool-display.test.ts src/agents/embedded-agent-runner/run/payloads.errors.test.ts src/agents/embedded-agent-runner/run/payloads.test.ts — 136/136 pass.
  • New coverage: run my-custom-tool --flag valuemy-custom-tool --flag value; run ./bin/local-script./bin/local-script; preserved run lint and run script.
  • node scripts/run-oxlint.mjs <changed files> — clean.

The P1 real-runtime proof gap (cron/chat/CLI) is still outstanding; I do not have a way to capture that locally without a live agent run.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

… context

Tighten stripLeadingExecDisplayVerb so it only strips `run` when the rest
carries a binary plus arguments (the catch-all shape). Single-token
`run <X>` summaries survive because package-script names (`npm run dev`,
`npm run custom-script`) are indistinguishable from bare-binary invocations
(`run python3`) at strip time. This restores arbitrary package-script
run labels while still collapsing the reported `run python3 /path` case
(issue openclaw#97319).
@ly-wang19

Copy link
Copy Markdown
Contributor Author

P2 v3 follow-up landed in de8e9e3 — addresses the package-script preservation gap.

Heuristic change: strip run only when the rest carries a binary plus arguments (the summarizeKnownExec catch-all shape). Single-token run <X> summaries survive because package-script names (npm run dev, npm run custom-script, npm run start) are indistinguishable from bare-binary invocations (run python3) at strip time — both are run <single-token>. Keeping run in that ambiguous shape preserves the package-script context in the failure message.

This still collapses the originally reported case run python3 /path/to/script.pypython3 /path/to/script.py (catch-all with args), and now also collapses run my-custom-tool --flag valuemy-custom-tool --flag value. Single-token labels preserved: run dev, run tests, run build, run lint, run script, run custom-script.

Proof:

  • node_modules/.bin/vitest run src/agents/tool-display.test.ts src/agents/embedded-agent-runner/run/payloads.errors.test.ts src/agents/embedded-agent-runner/run/payloads.test.ts — 137/137 pass.
  • New coverage in tool-display.test.ts: run dev / run custom-script preserved; multi-token generic binaries stripped.

The P1 real-runtime proof gap (cron/chat/CLI) is still outstanding; I cannot capture that locally without a live agent run.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@vincentkoc

Copy link
Copy Markdown
Member

Closing this as superseded by the canonical shared formatting fix in #97511, landed as 3e90125. #97511 covers the broader payload warning boundary with green proof, so keeping this narrower overlapping PR open would just split the fix path. Thanks for the work here.

@vincentkoc vincentkoc closed this Jun 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: S status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: exec tool failure message format makes framework labels indistinguishable from agent-typed commands

2 participants