Skip to content

fix(agent): clarify edit mismatch tool warning#67276

Closed
soloclz wants to merge 2 commits into
openclaw:mainfrom
soloclz:fix/clarify-edit-mismatch-tool-warning-upstream
Closed

fix(agent): clarify edit mismatch tool warning#67276
soloclz wants to merge 2 commits into
openclaw:mainfrom
soloclz:fix/clarify-edit-mismatch-tool-warning-upstream

Conversation

@soloclz

@soloclz soloclz commented Apr 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Describe the problem and fix in 2–5 bullets:

  • Problem: when the edit tool fails because oldText does not exactly match the file contents, verbose-off user-facing output collapses into a vague failure warning.
  • Why it matters: users do not get actionable guidance for a common whitespace/newline mismatch case, even though the underlying tool error already knows what went wrong.
  • What changed: the embedded runner payload builder now recognizes the edit exact-match mismatch case and surfaces a concise sanitized summary that tells the user oldText must match exactly, including whitespace/newlines.
  • What did NOT change (scope boundary): this does not change edit tool semantics, generic tool-error policy, replay/recovery policy, or expose raw file paths / file contents in verbose-off UI.

AI-assisted: Yes (Sona). Testing level: targeted local regression tests.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor required for the fix
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

Root Cause (if applicable)

  • Root cause: the useful mismatch detail already existed in the underlying edit error, but the final payload/UI warning path flattened it into a generic tool failure when verbose mode was off.
  • Missing detection / guardrail: there was no targeted regression test asserting that edit exact-match mismatches should surface actionable, sanitized guidance in verbose-off mode.
  • Contributing context (if known): the raw tool error may include file paths or file contents, so the UI layer had previously erred on the side of hiding too much instead of summarizing this well-known mismatch case safely.

Regression Test Plan (if applicable)

  • Coverage level that should have caught this:
    • Unit test
    • Seam / integration test
    • End-to-end test
    • Existing coverage already sufficient
  • Target test or file:
    • src/agents/pi-embedded-runner/run/payloads.test.ts
  • Scenario the test should lock in: an edit exact-match mismatch should still show actionable guidance in verbose-off mode, mention exact-text / whitespace-newline mismatch, and avoid leaking raw paths.
  • Why this is the smallest reliable guardrail: the behavior change lives in payload summarization, so the smallest reliable coverage is a focused unit test at the payload builder layer.
  • Existing test that already covers this (if any): src/agents/pi-embedded-runner/run/payloads.errors.test.ts covers adjacent tool-error surfacing behavior, but not this exact sanitized mismatch summary.
  • If no new test is added, why not: N/A

User-visible / Behavior Changes

  • Users now see actionable guidance for edit exact-match mismatches even when verbose mode is off.
  • The warning remains sanitized and does not expose raw file paths from the underlying error.

Diagram (if applicable)

Before:
[edit exact-match mismatch] -> [generic tool failed warning]

After:
[edit exact-match mismatch] -> [sanitized actionable summary] -> [user knows to match whitespace/newlines exactly]

Security Impact (required)

  • New permissions/capabilities? (No)
  • Secrets/tokens handling changed? (No)
  • New/changed network calls? (No)
  • Command/tool execution surface changed? (No)
  • Data access scope changed? (No)
  • If any Yes, explain risk + mitigation:

Repro + Verification

Environment

  • OS: Linux
  • Runtime/container: Node 22 via npx -y node@22
  • Model/provider: N/A (payload/tool error surfacing path)
  • Integration/channel (if any): embedded runner payload generation
  • Relevant config (redacted): verbose mode off

Steps

  1. Trigger an edit tool failure where oldText does not exactly match the current file contents because of whitespace/newline mismatch.
  2. Observe the user-facing warning generated from the embedded runner payload path with verbose mode off.
  3. Compare the warning text before and after this change.

Expected

  • The user-facing warning should say the exact text was not found and clarify that oldText must match exactly, including whitespace/newlines.
  • The warning should remain sanitized and not expose raw file paths.

Actual

  • Before this change, verbose-off output could collapse into a vague Edit ... failed warning without actionable guidance.

Evidence

Attach at least one:

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)

Human Verification (required)

What you personally verified (not just CI), and how:

  • Verified scenarios:
    • targeted tests passed locally for payloads.errors.test.ts and payloads.test.ts
    • the new mismatch summary includes actionable guidance
    • the summary does not include raw path details such as /tmp/demo.txt
  • Edge cases checked:
    • non-edit tool errors still follow existing behavior
    • only the exact-match mismatch case gets the special sanitized summary
  • What you did not verify:
    • live end-to-end messaging-channel behavior outside the targeted payload tests
    • broader tool-error policy changes beyond this specific mismatch case

Review Conversations

  • I replied to or resolved every bot review conversation I addressed in this PR.
  • I left unresolved only the conversations that still need reviewer or maintainer judgment.

If a bot review conversation is addressed by this PR, resolve that conversation yourself. Do not leave bot review conversation cleanup for maintainers.

Compatibility / Migration

  • Backward compatible? (Yes)
  • Config/env changes? (No)
  • Migration needed? (No)
  • If yes, exact upgrade steps:

Risks and Mitigations

  • Risk: a sanitized summary could accidentally become too broad and affect unrelated tool failures.
    • Mitigation: the special-case path is restricted to toolName === "edit" plus the known exact-match mismatch prefix, and regression coverage locks that in.
  • Risk: surfacing more detail could leak raw file paths or file contents.
    • Mitigation: the UI summary is hand-sanitized and tests assert that raw path details are not included.

@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: XS labels Apr 15, 2026
@greptile-apps

greptile-apps Bot commented Apr 15, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes a UX bug where edit exact-match mismatch failures collapsed into a vague warning in verbose-off mode. A new summarizeToolErrorForUi helper detects the known mismatch prefix and substitutes a sanitized, actionable summary that avoids leaking file paths.

The fix is well-scoped and the regression test correctly asserts both the guidance content and path-leakage guard. One open question: summarizedDetail takes priority over warningPolicy.includeDetails unconditionally, so verbose-on users also receive the sanitized message instead of the raw error (which would include the file path). Whether that's intentional is worth confirming, and a verbose-on test case would lock in the intended behavior either way.

Confidence Score: 5/5

Safe to merge; the change is narrowly scoped to a single error-summarization path with no P0/P1 issues.

All findings are P2 style/coverage suggestions. The core logic is correct, the fix is backward compatible, and the regression test covers the primary scenario. The only open item is whether verbose-on should also use the sanitized summary — a style question that does not block correctness.

src/agents/pi-embedded-runner/run/payloads.ts — the verbose-on interaction with summarizedDetail is worth a second look.

Prompt To Fix All With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-runner/run/payloads.ts
Line: 341-346

Comment:
**Sanitized summary overrides verbose-on detail too**

`summarizedDetail` takes priority unconditionally, so when `verboseLevel` is `"on"` or `"full"`, verbose users also receive the sanitized summary rather than the raw error (which would include the file path via `warningPolicy.includeDetails`). The PR description scopes the fix to verbose-off mode, but the implementation silently changes verbose-on behavior as well. A matching test for the verbose-on case would lock in whichever behavior is intended; if verbose mode should still expose the raw error, the guard should be conditioned on `!warningPolicy.includeDetails`.

```suggestion
      const summarizedDetail =
        !warningPolicy.includeDetails ? summarizeToolErrorForUi(params.lastToolError) : undefined;
      const errorSuffix = summarizedDetail
        ? `: ${summarizedDetail}`
        : warningPolicy.includeDetails && params.lastToolError.error
          ? `: ${params.lastToolError.error}`
          : "";
```

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "fix(agent): clarify edit mismatch tool w..." | Re-trigger Greptile

Comment thread src/agents/pi-embedded-runner/run/payloads.ts Outdated
@clawsweeper

clawsweeper Bot commented Apr 30, 2026

Copy link
Copy Markdown
Contributor

Thanks for the context here. I swept through the related work, and this is now duplicate or superseded.

Keep open. The PR is in the right payload-layer area, but it only recognizes the single-edit exact-match error while the current edit tool emits a different oldText mismatch string for multi-edit calls, so that path would still produce the vague verbose-off warning. It also still needs after-fix real behavior proof beyond local unit-test claims.

Canonical path: Close this stale PR. The latest review rated it F, the branch still lacks merge-ready proof, and there has been no human follow-up after the durable review.

So I’m closing this here because the remaining work is already tracked in the canonical issue.

Review details

Best possible solution:

Close this stale PR. The latest review rated it F, the branch still lacks merge-ready proof, and there has been no human follow-up after the durable review.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection shows current main emits edit not-found errors with actionable text, then buildEmbeddedRunPayloads hides raw details in verbose-off warnings; I did not run a live repro because this review is read-only.

Is this the best way to solve the issue?

No. The payload boundary is appropriate, but the implementation only recognizes the single-edit mismatch format and needs to cover the multi-edit oldText mismatch path too.

Security review:

Security review cleared: The diff only changes local warning text summarization and tests; it does not touch permissions, dependencies, workflows, network calls, secrets, or code execution surfaces.

AGENTS.md: found and applied where relevant.

What I checked:

  • stale F-rated PR: PR was opened 2026-04-15T15:55:17Z, is older than 30 days, and the latest review rated it F.
  • proof blocker: real behavior proof is mock_only and proof tier is F, so this branch is not merge-ready without contributor follow-up.
  • no human follow-up: live comments and timeline hydrated by apply contain no non-automation activity after the ClawSweeper review.

Likely related people:

  • steipete: GitHub path history for the embedded payload and agent runtime surfaces shows recent agent-runtime documentation/refactor work, including the large internalize-agent-runtime change that carried these files forward. (role: recent area contributor; confidence: medium; commits: bb46b79d3c14, e95e51a24f48; files: src/agents/embedded-agent-runner/run/payloads.ts, src/agents/sessions/tools/edit-diff.ts, src/agents/embedded-agent-subscribe.handlers.tools.ts)
  • joshavant: Local blame in the current checkout attributes the payload warning suffix and edit-diff mismatch strings to commit 33cb1c1, which imported or carried the current lines in this shallow history. (role: current-line provenance in checkout; confidence: medium; commits: 33cb1c18aca1; files: src/agents/embedded-agent-runner/run/payloads.ts, src/agents/sessions/tools/edit-diff.ts)

Codex review notes: model gpt-5.5, reasoning high; reviewed against 4ee50ce18e35.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. labels May 19, 2026
@openclaw-barnacle openclaw-barnacle Bot added the triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. label May 19, 2026
@clawsweeper

clawsweeper Bot commented May 20, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper PR egg

🎁 Pass real behavior proof to wake the egg and unlock a hatchable treat.

Where did the egg go?
  • The egg game starts only after the PR passes the real-behavior proof check.
  • Before that, no creature or rarity is rolled. The treat waits for real proof.
  • This is still just collectible flavor: proof affects review readiness, not creature quality.

@openclaw-barnacle

Copy link
Copy Markdown

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

@openclaw-barnacle openclaw-barnacle Bot added the stale Marked as stale due to inactivity label Jun 6, 2026
@clawsweeper clawsweeper Bot added rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jun 6, 2026
@soloclz
soloclz force-pushed the fix/clarify-edit-mismatch-tool-warning-upstream branch from 8f7bd1d to fde9954 Compare June 6, 2026 13:39
@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. labels Jun 6, 2026
@clawsweeper

clawsweeper Bot commented Jun 6, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper applied the proposed close for this PR.

@clawsweeper clawsweeper Bot closed this Jun 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling P2 Normal backlog priority with limited blast radius. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: S stale Marked as stale due to inactivity status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: edit tool: confusing error on whitespace mismatch, needs better surfacing

1 participant