Skip to content

apply_patch verification fails on canonically-equivalent Unicode — seekSequence needs an NFC pass #31651

Closed as not planned

Description

@beks-m

Description

apply_patch fails with Failed to find expected lines in <file> when the file on disk and the model's context lines are canonically-equivalent Unicode but different byte sequences (NFC vs NFD). None of the 4 passes in seekSequence (packages/opencode/src/patch/index.ts:460) handle this:

  1. exact
  2. rstrip
  3. trim
  4. normalizeUnicode + trim — but normalizeUnicode (index.ts:418) only maps smart quotes, dashes, ellipsis and NBSP; it never touches combining characters.

So a file containing decomposed й (и U+0438 + U+0306) can never be matched by a patch containing precomposed й (U+0439), even though the strings are canonically equivalent and render identically. Models essentially always emit NFC, so any NFD content on disk makes the file permanently unpatchable via apply_patch — the model re-reads, retries, fails again, and burns tokens in a loop.

We hit this in production on Cyrillic markdown files (tables with rows like | Район | Астана | ... |). Related-but-different reports: #2904 (auto-closed stale), #31422 (formatter drift, closed not-planned), #27282/#11687 (Windows). None cover canonical equivalence.

Proposed fix

A 5th pass in seekSequence, after the current pass 4:

// Pass 5: NFC-normalized match (canonical equivalence — no false positives)
const nfc = tryMatch(
  lines, pattern, startIndex,
  (a, b) => a.normalize("NFC").trim() === b.normalize("NFC").trim(),
  eof,
)
return nfc

This is safe: NFC normalization followed by exact equality only matches strings Unicode defines as the same text. Happy to send a PR with this + tests if you'd take it.

Plugins

None

OpenCode version

1.15.4 (gap still present at current HEAD of packages/opencode/src/patch/index.ts)

Steps to reproduce

Engine-level (deterministic, no model needed):

import { Patch } from "./packages/opencode/src/patch" // deriveNewContentsFromChunks

const fileContent = "# Сводка\nРайон: Астана\n".normalize("NFD") // й stored decomposed
const chunk = {
  old_lines: ["Район: Астана".normalize("NFC")], // models emit NFC
  new_lines: ["Район: Алматы"],
  is_end_of_file: false,
}
// -> throws: Failed to find expected lines

End-to-end:

  1. node -e 'require("fs").writeFileSync("test.md", "# Сводка\nРайон: Астана\n".normalize("NFD"))'
  2. Ask any model to change Астана to Алматы in test.md using apply_patch.
  3. apply_patch verification failed: Failed to find expected lines in test.md — and every retry fails the same way, since re-reading doesn't change the canonical-equivalence mismatch.

Operating System

Linux (Debian-based container)

Terminal

headless (server)

Activity

  1. added a commit that references this issue on Jul 9, 2026
    609ae12
  2. leizd commented on Aug 3, 2026

    @leizd

    Hi, I'd like to work on this.

    I confirmed that the current dev branch still lacks canonical Unicode
    normalization in seekSequence(). I'll add a focused NFC matching pass and
    regression tests covering NFC/NFD-equivalent text, while ensuring genuinely
    different strings do not match.

  3. github-actions commented on Oct 3, 2026

    @github-actions
    Contributor

    To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please open a new one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions