fix(edit): preserve unrelated lines during fuzzy text matching by rizwan-saddal · Pull Request #96636 · openclaw/openclaw

rizwan-saddal · 2026-06-25T02:39:02Z

What Problem This Solves

When the edit tool uses fuzzy matching (smart quotes, dashes, trailing whitespace), it normalizes the entire file content before applying replacements. This silently rewrites every unrelated line that contains Unicode characters, smart quotes, em dashes, or trailing whitespace. Builds on #90060 (by @SebTardif) with a critical NFKC fast-path correction.

Why This Change Was Made

The fuzzy match path normalizes the whole file to find the target text, but then applies the replacement against the original offsets. Non-NFKC characters (Arabic, CJK, smart quotes, em-dashes) get corrupted because normalization shifts their byte positions. The fix uses an offset mapper built from the normalization diff to translate match positions back to the original file, preserving every unrelated line. A fast-path using string identity skips the mapper entirely when normalization is a no-op.

User Impact

Users editing files containing non-ASCII content (Arabic, CJK, accented characters, smart quotes, em-dashes) will no longer see silent corruption of unrelated lines during fuzzy edits. The matched region is changed; everything else stays byte-identical. Previously, any fuzzy edit on such files would corrupt Unicode lines outside the edit target.

Evidence

Fix deployed to running OpenClaw gateway (aa69b12, zai/glm-5-turbo). Fuzzy edit on a file containing Arabic text, smart quotes, and em-dashes.

Before edit:

# الخلاصة
This is a line with smart quotes: "Hello" and 'World'
Another line with em-dash — and en-dash –
 سعر: ١٠٠٠ د.إ

Fuzzy edit command (smart quotes to straight quotes):

oldText: This is a line with smart quotes: "Hello" and 'World'
newText: This line was fuzzy-edited: 'Hello' and 'World'

After edit, Arabic, em-dash, and en-dash lines preserved unchanged:

# الخلاصة
This line was fuzzy-edited: 'Hello' and 'World'
Another line with em-dash — and en-dash –
 سعر: ١٠٠٠ د.إ

Without the fix, normalizeForFuzzyMatch would corrupt every line containing Unicode or compatibility characters. The offset mapper ensures only the matched region is affected.

NFKC fast-path correction: original PR checked length equality which can preserve length while shifting positions. Fixed to use string identity.

Fixes #89994. Test plan: pnpm test src/agents/sessions/tools/edit-diff.test.ts

clawsweeper · 2026-06-25T02:42:39Z

Codex review: needs changes before merge. Reviewed June 25, 2026, 9:53 PM ET / 01:53 UTC.

Summary
The branch keeps original LF-normalized content as the edit write/diff base, maps fuzzy-normalized match offsets back to original ranges, and adds edit-diff Unicode regression tests.

PR surface: Source +136, Tests +210. Total +346 across 2 files.

Reproducibility: yes. Current main still has a source-level reproduction through whole-file fuzzy normalization, and a no-write probe of the PR mapper shows a concrete wrong splice after decomposed Hangul Jamo.

Review metrics: none identified.

Stored data model
Persistent data-model change detected: serialized state: src/agents/sessions/tools/edit-diff.ts, unknown-data-model-change: src/agents/sessions/tools/edit-diff.test.ts, unknown-data-model-change: src/agents/sessions/tools/edit-diff.ts. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #89994
Summary: This PR is a candidate fix for the canonical fuzzy-edit whole-file normalization corruption bug; one older open PR shares the root cause, and one closed PR is already superseded by this branch.

Members:

canonical: Bug: a fuzzy edit silently normalizes and rewrites unrelated lines across the whole file #89994 - The PR body uses closing syntax for this open report and changes the implicated edit-diff helper.
same_root_cause: fix(edit): preserve unrelated lines during fuzzy text matching #90060 - The older open PR changes the same helper for the same fuzzy-edit corruption report and supplied related proof.
superseded: fix: scope fuzzy-edit normalization to matched region only (#89994) #96814 - This closed unmerged PR targeted the same bug and its thread identifies this PR as the superseding candidate.
adjacent_distinct: Suggest closest edit match when oldText is missing #88531 - That open PR changes closest-match hints for missing oldText, not unrelated-line preservation during fuzzy replacement.
partial_overlap: edit 工具稳定性改进：4 重防护机制 #62863 - The older closed issue requested broader edit-tool whitespace and fuzzy matching; this PR addresses a corruption side effect in that area.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🦞 diamond lobster
Patch quality: 🧂 unranked krab
Result: blocked by patch quality or review findings.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

[P2] Fix the NFKC composition mapper gap and add the Hangul Jamo regression.
Rerun node scripts/run-vitest.mjs src/agents/sessions/tools/edit-diff.test.ts, node scripts/run-vitest.mjs src/agents/sessions/tools/edit.test.ts, and git diff --check.

Risk before merge

[P1] Merging as-is can still splice the wrong original range when NFKC composes adjacent non-mark code points, corrupting the fuzzy edit target or nearby Unicode content.
[P1] The same-root open branch at fix(edit): preserve unrelated lines during fuzzy text matching #90060 remains viable, so maintainers should land one canonical repaired branch and close or supersede the other.

Maintainer options:

Fix Unicode offset alignment before merge (recommended)
Update the mapper and regression coverage so fuzzy targets after decomposed Hangul Jamo or similar NFKC-composed clusters map to the correct original range.
Choose the canonical branch
If maintainers prefer fix(edit): preserve unrelated lines during fuzzy text matching #90060, pause this PR and move the repaired mapper and tests to the chosen branch instead.

Copy recommended automerge instruction

@clawsweeper automerge

Special instructions:
Repair `src/agents/sessions/tools/edit-diff.ts` so normalized-to-original offset mapping handles NFKC composition across adjacent non-mark code points, not only base-plus-combining-mark segments. Add a regression in `src/agents/sessions/tools/edit-diff.test.ts` with original `\u1100\u1161 \u2018X\u2019 tail`, oldText `'X'`, newText `'Y'`, and an expected output that preserves the surrounding Unicode content while replacing only the fuzzy target. Preserve the original-content base behavior and existing tests, then run `node scripts/run-vitest.mjs src/agents/sessions/tools/edit-diff.test.ts`, `node scripts/run-vitest.mjs src/agents/sessions/tools/edit.test.ts`, and `git diff --check`.

Next step before merge

[P2] A narrow automated repair is appropriate because the remaining blocker is localized to normalized-offset mapping and regression coverage in the edit helper.

Security
Cleared: The diff changes TypeScript edit-helper logic and colocated Vitest tests only; no dependency, workflow, secret, package, or supply-chain surface is changed.

Review findings

[P1] Handle NFKC composition beyond combining marks — src/agents/sessions/tools/edit-diff.ts:237-240

Review details

Best possible solution:

Repair the mapper to use full normalized-to-original alignment for NFKC expansion and composition, then land one canonical fix for #89994.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main still has a source-level reproduction through whole-file fuzzy normalization, and a no-write probe of the PR mapper shows a concrete wrong splice after decomposed Hangul Jamo.

Is this the best way to solve the issue?

No, not yet. Keeping original content as the write/diff base is the right layer, but the normalized-offset mapper must handle broader NFKC composition before this is the best merge shape.

Full review comments:

[P1] Handle NFKC composition beyond combining marks — src/agents/sessions/tools/edit-diff.ts:237-240
This mapper only groups a base code point with following combining marks, but NFKC can also compose adjacent non-mark characters. For example, 가 ‘X’ tail normalizes to 가 'X' tail; the PR logic maps the fuzzy 'X' match one code unit early and can produce 가'Y'’ tail. Build the offset map from a full NFKC alignment, or otherwise cover composed clusters such as Hangul Jamo before splicing.
Confidence: 0.93

Overall correctness: patch is incorrect
Overall confidence: 0.91

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 6830aa39eaa1.

Label changes

Label justifications:

P0: The PR targets a source-backed silent user-file content corruption bug in the core edit tool, and the remaining mapper defect can still corrupt fuzzy edit ranges.
merge-risk: 🚨 other: The diff changes central Unicode edit range mapping, and the remaining NFKC composition gap is a merge-specific content-corruption risk outside the owned merge-risk categories.
rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🦞 diamond lobster and patch quality is 🧂 unranked krab.
status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (live_output): The PR body includes copied live gateway before/after output showing a fuzzy edit changed the target line while preserving unrelated Arabic text and dash characters.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes copied live gateway before/after output showing a fuzzy edit changed the target line while preserving unrelated Arabic text and dash characters.

Evidence reviewed

PR surface:

Source +136, Tests +210. Total +346 across 2 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	1	161	25	+136
Tests	1	210	0	+210
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	2	371	25	+346

Acceptance criteria:

[P1] node scripts/run-vitest.mjs src/agents/sessions/tools/edit-diff.test.ts.
[P1] node scripts/run-vitest.mjs src/agents/sessions/tools/edit.test.ts.
[P1] git diff --check.

What I checked:

Current main still has the reported bug path: Current main sets baseContent to whole-file normalizeForFuzzyMatch(normalizedContent) whenever any initial edit used fuzzy matching, so unrelated lines can still be normalized before splicing. (src/agents/sessions/tools/edit-diff.ts:218, 6830aa39eaa1)
Runtime caller writes helper output: The edit tool writes newContent from applyEditsToNormalizedContent and generates the model/user diff from baseContent versus newContent, making the helper the disk-write and diff-honesty boundary. (src/agents/sessions/tools/edit.ts:412, 6830aa39eaa1)
Latest release remains affected: Tag v2026.6.10 still contains the whole-file fuzzy-normalized baseContent selection before splicing, so the central bug is not shipped-fixed. (src/agents/sessions/tools/edit-diff.ts:218, aa69b12d0086)
PR implements the intended fix layer: At PR head, baseContent remains the original normalized content and fuzzy matches are mapped through mapNormalizedOffsetToOriginal before replacement ranges are spliced. (src/agents/sessions/tools/edit-diff.ts:316, d3dff49fa182)
PR mapper still has a Unicode composition gap: The mapper groups one base code point plus following combining marks only; a no-write probe with 가 ‘X’ tail normalizing to 가 'X' tail maps the fuzzy target to offsets that produce 가'Y'’ tail. (src/agents/sessions/tools/edit-diff.ts:237, d3dff49fa182)
Related canonical work: The PR uses closing syntax for Bug: a fuzzy edit silently normalizes and rewrites unrelated lines across the whole file #89994; fix(edit): preserve unrelated lines during fuzzy text matching #90060 remains an open same-root candidate, and fix: scope fuzzy-edit normalization to matched region only (#89994) #96814 was closed as superseded by this branch.

Likely related people:

obviyus: git blame maps the current edit-diff helper snapshot to commit ace22feb3fe2, whose committer is Ayaan Zaidi; live PR metadata also shows obviyus merged the containing repair batch. (role: current implementation committer and adjacent merger; confidence: medium; commits: ace22feb3fe2, 31a0f97dd944; files: src/agents/sessions/tools/edit-diff.ts, src/agents/sessions/tools/edit.ts)
SebTardif: Authored the older open same-root PR that changes the same edit-diff helper for the same fuzzy-edit corruption report, and this branch explicitly builds on that work. (role: same-root candidate fix author; confidence: high; commits: 502fb5ae7ea0; files: src/agents/sessions/tools/edit-diff.ts, src/agents/sessions/tools/edit-diff.test.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

rizwan-saddal

unused variable though not from my change but removed

rizwan-saddal · 2026-06-25T03:44:47Z

@clawsweeper re-review

clawsweeper · 2026-06-25T03:44:50Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

rizwan-saddal

Mergerable, significantly improve my personal Assistance too

rizwan-saddal · 2026-06-25T04:06:01Z

@clawsweeper re-review

clawsweeper · 2026-06-25T04:06:04Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@SebTardif

Adopted from PR openclaw#90060 (by @SebTardif) with ClawSweeper P1 fix: The original PR's NFKC fast-path checked length equality (nfkcLine.length === origLine.length) to skip offset mapping. This is incorrect — NFKC can preserve total code-unit length while shifting internal positions. For example, "ﬁ X é" (U+FB01 + X + e+combining-acute) normalizes to "fi X é" — both are 6 code units, but X shifts from codepoint 2 to 3. Fixed by comparing string identity (nfkcLine === origLine) instead of length. Added regression test for this exact length-neutral mixed case. Also includes the original PR's 5 commits: - Preserve original content as baseContent (no whole-file normalization) - Map fuzzy-normalized offsets back to original via per-line segment mapper - Handle NFKC expansion, composition, and partial ligature matches - Use Array.from instead of spread for Unicode surrogate-pair safety Fixes openclaw#89994. Upstream PR: openclaw#90060

rizwan-saddal · 2026-06-25T18:44:48Z

@clawsweeper re-review

clawsweeper · 2026-06-25T18:44:52Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Previous oldText 'fi X' matched the ﬁ ligature itself (NFKC maps fi↔ﬁ), so the replacement consumed the unrelated ligature. Use smart quotes to force the fuzzy-match path while keeping the match region after the ligature, verifying that the offset mapper correctly preserves both ﬁ and e+combining-acute in unrelated positions.

The fuzzy match consumes the smart quotes from the original, so newText must provide the desired output form. This correctly tests that the NFKC offset mapper preserves unrelated content (ligature + combining accent).

openclaw-barnacle Bot added agents Agent runtime and tooling size: M triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. labels Jun 25, 2026

rizwan-saddal commented Jun 25, 2026

View reviewed changes

rizwan-saddal force-pushed the fix/edit-fuzzy-v2 branch 2 times, most recently from 6a37a6b to 0aed70e Compare June 25, 2026 03:30

openclaw-barnacle Bot added app: android App: android app: web-ui App: web-ui size: L and removed size: M labels Jun 25, 2026

clawsweeper Bot added the merge-risk: 🚨 automation 🚨 May affect CI, automerge, proof capture, label sync, or maintainer automation. label Jun 25, 2026

rizwan-saddal commented Jun 25, 2026

View reviewed changes

rizwan-saddal force-pushed the fix/edit-fuzzy-v2 branch from 8162ccd to d426bab Compare June 25, 2026 04:08

openclaw-barnacle Bot added size: M and removed app: android App: android app: web-ui App: web-ui size: L triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. labels Jun 25, 2026

rizwan-saddal added 2 commits June 25, 2026 09:23

fix(lint): remove unused variable in edit-diff regression test

b5ddf0c

rizwan-saddal force-pushed the fix/edit-fuzzy-v2 branch from fcefadb to b5ddf0c Compare June 25, 2026 05:23

Merge branch 'main' into fix/edit-fuzzy-v2

62692de

clawsweeper Bot mentioned this pull request Jun 25, 2026

fix: scope fuzzy-edit normalization to matched region only (#89994) #96814

Closed

rizwan-saddal added 2 commits June 25, 2026 22:26

chore: trigger CI re-run after PR body update

741c386

fix(test): force NFKC fuzzy path in length-neutral regression

808c25a

clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. labels Jun 26, 2026

rizwan-saddal added 2 commits June 26, 2026 05:44

fix(test): use smart quotes in newText to preserve original form

d3dff49

The fuzzy match consumes the smart quotes from the original, so newText must provide the desired output form. This correctly tests that the NFKC offset mapper preserves unrelated content (ligature + combining accent).

Merge branch 'main' into fix/edit-fuzzy-v2

c6c8cd7

rizwan-saddal closed this Jun 26, 2026

rizwan-saddal deleted the fix/edit-fuzzy-v2 branch June 27, 2026 00:13

clawsweeper Bot mentioned this pull request Jun 30, 2026

fix(edit): preserve unrelated lines during fuzzy text matching #90060

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(edit): preserve unrelated lines during fuzzy text matching#96636

fix(edit): preserve unrelated lines during fuzzy text matching#96636
rizwan-saddal wants to merge 8 commits into
openclaw:mainfrom
MuftahFree:fix/edit-fuzzy-v2

rizwan-saddal commented Jun 25, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jun 25, 2026 •

edited

Loading

Uh oh!

rizwan-saddal left a comment

Uh oh!

rizwan-saddal commented Jun 25, 2026

Uh oh!

clawsweeper Bot commented Jun 25, 2026 •

edited

Loading

Uh oh!

rizwan-saddal left a comment

Uh oh!

rizwan-saddal commented Jun 25, 2026

Uh oh!

clawsweeper Bot commented Jun 25, 2026 •

edited

Loading

Uh oh!

rizwan-saddal commented Jun 25, 2026

Uh oh!

clawsweeper Bot commented Jun 25, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

rizwan-saddal commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Why This Change Was Made

User Impact

Evidence

Uh oh!

clawsweeper Bot commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rizwan-saddal left a comment

Choose a reason for hiding this comment

Uh oh!

rizwan-saddal commented Jun 25, 2026

Uh oh!

clawsweeper Bot commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rizwan-saddal left a comment

Choose a reason for hiding this comment

Uh oh!

rizwan-saddal commented Jun 25, 2026

Uh oh!

clawsweeper Bot commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rizwan-saddal commented Jun 25, 2026

Uh oh!

clawsweeper Bot commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

rizwan-saddal commented Jun 25, 2026 •

edited

Loading

clawsweeper Bot commented Jun 25, 2026 •

edited

Loading

clawsweeper Bot commented Jun 25, 2026 •

edited

Loading

clawsweeper Bot commented Jun 25, 2026 •

edited

Loading

clawsweeper Bot commented Jun 25, 2026 •

edited

Loading