Skip to content

Bug: a fuzzy edit silently normalizes and rewrites unrelated lines across the whole file #89994

Description

@yetval

Summary

Whenever any edit in an edit tool call falls back to fuzzy matching, the tool normalizes the entire file and writes that normalized text to disk, silently mutating lines that the edit never touched. normalizeForFuzzyMatch runs NFKC, strips trailing whitespace on every line, and rewrites smart quotes, Unicode dashes, and special spaces to ASCII file-wide. Lines unrelated to the edit are changed on disk, and the diff/patch returned to the model is computed against the already-normalized base, so the collateral damage is invisible to the model and the TUI preview.

A single fuzzy edit to one line can therefore rewrite string literals, regex character classes, and golden/snapshot fixtures elsewhere in the file (em dash to hyphen, smart quotes to straight, trailing whitespace removed, any NFKC-decomposable character recomposed) with no signal to the model that anything else changed.

  • Impact: silent on-disk content corruption on the core coding tool. Every fuzzy edit rewrites unrelated lines; the model is never told.
  • Trigger: the normal, intended fuzzy case. The model copies oldText from rendered output where a normal space was actually a non-breaking space (U+00A0 / U+202F), or where a smart quote or Unicode dash was involved, so exact match fails and fuzzy match succeeds.
  • Affected: any agent turn that uses the edit tool and hits the fuzzy path.
  • Version audited: main @ a0717ef6

Root cause

When any edit's initial match used fuzzy matching, baseContent is set to the whole file run through normalizeForFuzzyMatch, and both the on-disk content and the model-facing diff are derived from that normalized base.

src/agents/sessions/tools/edit-diff.ts:223-224 (selection of the base):

// src/agents/sessions/tools/edit-diff.ts:223-224
const baseContent = initialMatches.some((match) => match.usedFuzzyMatch)
  ? normalizeForFuzzyMatch(normalizedContent)   // the ENTIRE file, normalized
  : normalizedContent;

newContent is spliced out of that same normalized base, so every untouched line keeps its normalized form (src/agents/sessions/tools/edit-diff.ts:259-272):

// src/agents/sessions/tools/edit-diff.ts:259-272
let newContent = baseContent;
for (let i = matchedEdits.length - 1; i >= 0; i--) {
  const edit = matchedEdits[i];
  newContent =
    newContent.slice(0, edit.matchIndex) +
    edit.newText +
    newContent.slice(edit.matchIndex + edit.matchLength);
}
...
return { baseContent, newContent };

normalizeForFuzzyMatch is destructive across the whole string, not just the matched region (src/agents/sessions/tools/edit-diff.ts:38-55):

// src/agents/sessions/tools/edit-diff.ts:38-55
export function normalizeForFuzzyMatch(text: string): string {
  return (
    text
      .normalize("NFKC")
      .split("\n")
      .map((line) => line.trimEnd())          // strips trailing whitespace on EVERY line
      .join("\n")
      .replace(/[]/g, "'")   // smart single quotes
      .replace(/[]/g, '"')   // smart double quotes
      .replace(/[]/g, "-")  // dashes
      // ... special spaces -> regular space
  );
}

The shipped edit tool (createEditToolDefinition, name: "edit") then writes newContent to disk and shows the model a diff of baseContent vs newContent, both already normalized (src/agents/sessions/tools/edit.ts:407-419):

// src/agents/sessions/tools/edit.ts:407-419
const { baseContent, newContent } = applyEditsToNormalizedContent(
  normalizedContent,
  edits,
  path,
);
const finalContent = bom + restoreLineEndings(newContent, originalEnding);
await ops.writeFile(absolutePath, finalContent);          // normalized bytes to disk
...
const diffResult = generateDiffString(baseContent, newContent);   // diff vs the NORMALIZED base
const patch = generateUnifiedPatch(path, baseContent, newContent);

Because the diff is baseContent vs newContent (both normalized), the lines that were silently changed appear as unchanged context, so neither the model nor the TUI preview can see the collateral mutation.

Reproduction

Standalone vitest against pristine main (place at src/bedit.test.ts). All special characters are explicit escapes so there are no invisible bytes:

import { describe, expect, it } from "vitest";
import {
  applyEditsToNormalizedContent,
  generateDiffString,
} from "./agents/sessions/tools/edit-diff.js";

// Two lines that NO edit targets:
//   line 1: an em dash (—) inside a string literal
//   line 2: trailing whitespace
const original =
  'const label = "a — b";\n' +
  "let x = 1;   \n" +
  "\n" +
  "function bar() {\n" +
  "  return 2;\n" + // the only line the edit targets
  "}\n";

// oldText indents the return line with two NON-BREAKING spaces ( ) where
// the file uses normal spaces, so exact match fails and the shipped fuzzy path
// takes over (the case fuzzy matching exists for).
const edits = [{ oldText: "  return 2;", newText: "  return 3;" }];

describe("edit tool: a fuzzy edit silently rewrites the WHOLE file", () => {
  it("mutates unrelated lines, invisibly to the diff shown to the model", () => {
    const { baseContent, newContent } = applyEditsToNormalizedContent(
      original,
      edits,
      "/x.ts",
    );
    const { diff } = generateDiffString(baseContent, newContent);
    console.log("ON_DISK", JSON.stringify(newContent));
    console.log("DIFF\n" + diff);
    // Only the return line was edited; the em dash and trailing whitespace must survive.
    expect(newContent.includes("—")).toBe(true);          // FAILS
    expect(newContent.includes("let x = 1;   ")).toBe(true);   // FAILS
  });
});

Run:

export PATH=/opt/node/bin:$PATH
node scripts/run-vitest.mjs src/bedit.test.ts

Observed (main @ a0717ef6)

ON_DISK "const label = \"a - b\";\nlet x = 1;\n\nfunction bar() {\n  return 3;\n}\n"

DIFF
 1 const label = "a - b";
 2 let x = 1;
 3
 4 function bar() {
-5   return 2;
+5   return 3;
 6 }

AssertionError: expected false to be true   // newContent.includes("—")

The edit only changed return 2; to return 3;, yet on disk the em dash became - and the three trailing spaces on let x = 1; were stripped. The diff shown to the model marks only line 5; lines 1 and 2 appear as unchanged context already in their corrupted form, so the mutation is invisible.

Expected

A fuzzy edit changes only the matched region. Lines unrelated to any edit are written back byte-for-byte identical, and the diff shown to the model reflects exactly what changed on disk.

Suggested direction

  • Keep baseContent equal to the original normalizedContent. Use the fuzzy-normalized text only to locate match offsets, then splice newText into the original at those offsets (or normalize only the matched substring), leaving untouched bytes identical.
  • That also makes the model-facing diff (baseContent vs newContent) honest, since the base is no longer pre-normalized.

This is the silent-corruption side effect of the whitespace/normalization fuzzy matching requested in #62863; the fix preserves that fuzzy matching while scoping the rewrite to the matched region.

Happy to send a PR with the fix plus the regression test above.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P0Emergency: data loss, security bypass, crash loop, or unusable core runtime.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.impact:data-lossCan lose, corrupt, or silently drop user/session/config data.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions