Skip to content

[Android] After switching between snapshot and snapshot -i, the settle diff compares different views #4486

Closed

Description

@informramiz

Environment: agent-device 0.21.36, macOS (Apple silicon). Android 17 emulator (Google APIs, arm64), Settings app; iOS 27.2 Simulator (iPhone 18 Pro). Found while benchmarking agent-device against an adb/AXe stack: https://codebyramiz.com/blog/mobile-mcp-agent-device/

Steps: On the Device name confirmation dialog: snapshot -i (OK is @e5), then a full snapshot (where @e3 is the status-bar clock), then press @e3 (meant as OK from the earlier -i output).

Actual: It tapped (115, 37), the clock. That follows the "latest output wins" rule, so that part is my mistake. But the settle diff after the press listed the dialog title and warning as removed (-10) while the dialog was still on screen; the next wait text reported Current surface: Cancel, OK. The diff seems to compare the full-tree baseline with an interactive-only view.

Suggestion: Diff against the same view mode as the baseline. Optionally, keep ref numbers stable across both views (-i as a filter), or warn when a press follows a view-mode switch, since an agent switching views can make the same mistake.

Activity

  1. thymikee commented on Nov 13, 2026

    @thymikee
    Member

    Triage: bug in the settle diff. Switching from snapshot -i to a full snapshot creates a new ref frame, so the changed ref numbers are expected. #4500 makes a settle on a ref from a plain full snapshot compare full trees, removing the false wrapper-node removals. Other selector, point and scroll/back projection cases remain outside that fix.

  2. added a commit that references this issue on Nov 13, 2026
    d387a45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions