Skip to content

fix(opencode): edit tool reports fuzzy matches as exact success - #45491

Closed
yaoyi1222 wants to merge 1 commit into
anomalyco:devfrom
yaoyi1222:fix/edit-fuzzy-match-strategy
Closed

yaoyi1222 wants to merge 1 commit into
anomalyco:devfrom
yaoyi1222:fix/edit-fuzzy-match-strategy

Conversation

@yaoyi1222

@yaoyi1222 yaoyi1222 commented Aug 27, 2026 •

Copy link
Copy Markdown

Issue for this PR

Closes #34424

This covers points 2 and 3 of that issue. Points 1, 4 and 5 are out of scope here and can be tracked separately.

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

edit.txt tells the model the tool "Performs exact string replacements in files", that it must "preserve the exact indentation (tabs/spaces)", and that "the edit will FAIL if oldString is not found in the file". The not-found error says the same thing: "It must match exactly, including whitespace, indentation, and line endings."

That is not what happens. replace() tries SimpleReplacer (byte-exact) first, then eight tolerant fallbacks. When a fallback wins, the tool returns the same plain "Edit applied successfully." as an exact match. Given the contract above, the only thing the model can conclude from that is "my oldString matched exactly, indentation included" — and it is wrong. A wrong oldString, e.g. 4-space indentation where the file has 3, is silently absorbed and the line gets rewritten with the wrong indentation.

The strategy that won was already known inside the loop; it was just being dropped at return.

  • replace() returns { content, strategy } instead of a bare string, and the chain is a list of [name, replacer] pairs so the winning matcher comes back with the result.
  • On a non-exact hit the tool appends a note naming the strategy and echoes the diff, so the caller sees the actual change instead of being told to go look for it.
  • matchStrategy is added to the tool metadata.

Three things I deliberately did not do:

  • Reword edit.txt instead. That is the other way to resolve a docs-vs-behaviour mismatch, but a static "sometimes it matches loosely" cannot tell the model whether this particular call was loose. The per-call signal is the part that is missing.
  • Tighten the fuzzy chain. It exists because LLMs drift on whitespace. Making it strict would trade silent corruption for a pile of retry failures. Tolerance is unchanged here; only the reporting changes.
  • Detect it at the tool layer with contentOld.includes(oldString). Looks cheaper, but it reports exact incorrectly when SimpleReplacer matches in more than one place: the loop continues past it (index !== lastIndex), a fuzzy replacer wins, and includes is still true. Changing the signature costs the same and does not lie — replace() has exactly one caller in the repo (edit.ts:133), and the tests go through EditTool, not replace directly.

Related: #44996 is the same mismatch on the failure path (the not-found error text). Left alone to keep this focused.

How did you verify your code works?

  • Added a test using the indentation case from the issue: file has beta_line (3 spaces), oldString has 4. Asserts matchStrategy === "line-trimmed", that the note names the strategy, and that the diff is echoed.
  • Extended the existing exact-match test to assert matchStrategy === "exact" and that no note is emitted, so the exact path stays byte-identical to before.
  • bun test test/tool/edit.test.ts — 30/30 pass. (rejects empty oldString on existing files intermittently hits the 5s timeout on my machine, but it does that on a clean dev checkout too, with my changes stashed.)
  • bun test test/acp/tool.test.ts — 9/9 pass. These assert the exact "Edit applied successfully." string, so they confirm the exact path is untouched.
  • bun run typecheck in packages/opencode — clean.

To see the original behaviour on dev: create a file with a 3-space-indented line, call edit with a 4-space-indented oldString, and note the output is indistinguishable from an exact match.

Screenshots / recordings

n/a — not a UI change.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

@yaoyi1222
yaoyi1222 force-pushed the fix/edit-fuzzy-match-strategy branch from aeb843f to e3c04a0 Compare August 27, 2026 06:10
@yaoyi1222 yaoyi1222 changed the title feat(opencode): surface fuzzy match strategy in edit tool output fix(opencode): edit tool reports fuzzy matches as exact success Aug 27, 2026
edit.txt tells the model the tool "Performs exact string replacements in
files", that it must "preserve the exact indentation (tabs/spaces)", and that
the edit "will FAIL if oldString is not found in the file". The not-found
error repeats it: "It must match exactly, including whitespace, indentation,
and line endings."

That is not what happens. replace() tries SimpleReplacer (byte-exact) first,
then eight tolerant fallbacks. When a fallback wins, the tool returns the same
plain "Edit applied successfully." as an exact match, so the only thing the
caller can conclude from the documented contract is that oldString matched
exactly, indentation included - and that is wrong. A wrong oldString, e.g.
4-space indentation where the file has 3, is silently absorbed and the line is
rewritten with the wrong indentation.

The winning strategy was already known inside the loop; it was just dropped at
return. replace() now returns { content, strategy } and the chain is a list of
[name, replacer] pairs. On a non-exact hit the tool appends a note naming the
strategy and echoes the diff. matchStrategy is added to the tool metadata.

Fuzzy tolerance itself is unchanged - only the reporting.
@yaoyi1222
yaoyi1222 force-pushed the fix/edit-fuzzy-match-strategy branch from e3c04a0 to f7956a0 Compare August 27, 2026 06:18
@yaoyi1222
yaoyi1222 marked this pull request as ready for review August 27, 2026 06:20
@github-actions

Copy link
Copy Markdown
Contributor

Automated PR Cleanup

Thank you for contributing to opencode.

Due to the high volume of PRs from users and AI agents, we periodically close older PRs using automated criteria so maintainers can focus review time on the most active and community-supported contributions.

This PR was closed because it matched the following cleanup criteria:

  • The PR was created more than 1 month ago
  • The PR had fewer than 2 positive reactions
  • Positive reactions are counted as thumbs-up, heart, celebration, or rocket reactions on the PR

PRs created within the last month are not affected by this cleanup.

If you believe this PR was closed incorrectly, or if you are still actively working on it, please leave a comment explaining why it should be reopened. A maintainer can review and reopen it if appropriate.

Thanks again for taking the time to contribute.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE]: Make fuzzy edit feedback stricter to improve SWE-bench reliability

1 participant