Skip to content

apply_diff: *** End of File hunks corrupt or mis-locate edits when the input ends in a newline #4660

Closed

Description

@mmjerge

Please read this first

Describe the bug

apply_diff mishandles *** End of File hunks whenever the input ends in a newline, which is nearly every real source file. Two symptoms, one cause.

1. An append at end of file inserts a blank line and drops the trailing newline.

apply_diff("a\nb\n", "@@\n+c\n*** End of File")
# returns 'a\nb\n\nc'
# expected 'a\nb\nc\n'

2. An EOF hunk with repeated context silently patches the first match instead of the last.

apply_diff("x\nfoo\ny\nfoo\n", " foo\n+added\n*** End of File")
# returns 'x\nfoo\nadded\ny\nfoo\n'   (after the FIRST foo)
# expected 'x\nfoo\ny\nfoo\nadded\n'  (after the LAST foo)

Both are silent.

Neither reproduces when the input lacks a trailing newline. apply_diff("a\nb", ...) returns 'a\nb\nc' and apply_diff("x\nfoo\ny\nfoo", ...) appends after the last foo, both correct. The trailing newline is the trigger.

Mechanism

_parse_update_diff and _apply_chunks split the input with input.split("\n"). For text ending in a newline, this yields a phantom empty final element: "a\nb\n" becomes ["a", "b", ""].

_find_context (src/agents/apply_diff.py:333-341) anchors an EOF hunk at len(lines) - len(context) and counts that phantom element as a line:

  • With empty context, the anchor resolves to len(lines), i.e. after the phantom "". The insert lands past the empty element, so the output gains a blank line and loses its final newline.
  • With non-empty context, the end-anchored window always includes the phantom "", so the comparison can never match. The code falls through to _find_context_core(lines, context, start), a forward search that returns the first occurrence. The fuzz + 10000 penalty that marks this as a degraded match is accumulated into parser.fuzz and then discarded. apply_diff never surfaces it, so the wrong-location edit is invisible.

Debug information

  • Agents SDK version: v0.22.0, and main at 5f9f4f0
  • Python version: 3.10.16 (repo default via uv)
  • Operating system: macOS (darwin, arm64)
  • Model and model provider: n/a — apply_diff is pure text manipulation, no model or network involved
  • Does the issue reproduce with the latest Agents SDK release? Yes, and on main.
  • Does the issue occur consistently or intermittently? Consistently, deterministically.

Repro steps

from agents import apply_diff

# 1. append at EOF
print(repr(apply_diff("a\nb\n", "@@\n+c\n*** End of File")))
# 'a\nb\n\nc'      expected 'a\nb\nc\n'

# 2. ambiguous context with an EOF marker
print(repr(apply_diff("x\nfoo\ny\nfoo\n", " foo\n+added\n*** End of File")))
# 'x\nfoo\nadded\ny\nfoo\n'   expected 'x\nfoo\ny\nfoo\nadded\n'

# controls: same diffs, no trailing newline in the input, both correct
print(repr(apply_diff("a\nb", "@@\n+c\n*** End of File")))          # 'a\nb\nc'
print(repr(apply_diff("x\nfoo\ny\nfoo", " foo\n+added\n*** End of File")))  # '...foo\nadded'

Expected behavior

An *** End of File hunk should treat the phantom trailing element as the end of the file rather than as a line:

  1. An append at EOF yields 'a\nb\nc\n', preserving the trailing newline and adding no blank line.
  2. Repeated context resolves to the last occurrence, which is what the marker is for.

tests/test_apply_diff.py currently has no *** End of File coverage, which is why this went unnoticed.

The JS SDK carries the same logic in packages/agents-core/src/utils/applyDiff.ts (findContext, lines 327-333) and looks affected the same way, though I have only verified the Python side.

I have a fix with regression tests and will open a PR shortly.

Activity

  1. lllakshit commented on Aug 25, 2026

    @lllakshit
    Contributor

    Opened #4661 with a locator fix and regression tests for the phantom trailing split element, including CRLF. Happy to adjust if you already have a preferred patch.

  2. mmjerge commented on Aug 25, 2026

    @mmjerge
    ContributorAuthor

    @lllakshit thanks let's keep yours

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions