You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 8539d87
Browse filesBrowse the repository at this point in the historyBrowse files
fix(daemon): run find focus through the shared touch-point decisions (#4223)
* fix(daemon): aim find focus at the parent-owned touch point
The find focus seam dispatches its own point instead of re-entering the
interaction leaf, so it skipped the touch-point decision the acting paths
run. A match fully tiled by interactive children now refuses before
dispatch, and a child owning the center moves the aim to free parent
space. Missing or zero-area rects get the typed invalid-bounds refusal.
Move resolveNodeTouchPoint to @agent-device/selectors/node-touch-point so
the daemon and command-runtime share one owner, and close the
parentOwnedTouchPoint cell of the find-focus-dispatch guarantee row.
Refs #3734
* chore(gates): declare the selectors node-touch-point subpath
The package-boundaries enumeration lists every selectors subpath, so the new export has to be declared for the R11 layering rule to accept it.
* fix(contracts): describe the find focus touch point in its waivers
The offscreen and nonHittable waiver reasons still said the seam dispatches a rect center.
* fix(daemon): aim find focus and type at a nested text field
The touch-point rule treats a nested text field as a competitor to avoid,
so a match wrapping a field reported focus success beside the field and
the typed text then went wherever the focus already was. A sole enabled,
visible field now becomes the target, with the same touch-point, covered
and keyboard checks. Several fields refuse with AMBIGUOUS_MATCH and the
field candidates.
Refs #3734
* style(test): format the find focus contract scenarios
* fix(daemon): keep find focus on the field the locator named
A disabled field promotes to its parent on iOS, and the parent's other
field then counted as the sole one, so find type wrote into a field the
locator never named. The located field now decides first: an enabled one
stays the target and a disabled one refuses with target_disabled, as does
a match holding only disabled fields.
Count only outermost on-screen typed fields, so a hosted text field, a
scrolled-away field and an untyped child no longer change the target. The
rule moves to the find description so MCP clients see it, and it names
web text boxes as unrecognized.
Refs #3734
* fix(daemon): refuse a disabled field reached through promotion
The disabled check only read the located node, so a same-rect wrapper over
a disabled field was still focused, and a label inside one was promoted
past it to an enabled sibling. find type then wrote into another field and
reported success. The field decision now walks from the located node to the
promoted one, and the nearest field on that path is the named field.
The find help, selector docs and nonHittable waiver now state that a named
text field stays the target.
Refs #3734
* fix(selectors): count only measurable fields and read password facts
A field with no usable bounds counted as on-screen, so it could take the
focus or turn a single field into an ambiguity before the touch point could
reject it. A password fact on an unrecognized type was ignored, so focus
went to the container instead.
Say in the help and docs that a named field wins over its neighbours and
that target_disabled is a reason, and record why the nested-field refusal
follows find's refusal contract for printed refs.
Refs #3734
* docs(selectors): split the find focus rule and name the covered refusal
Refs #3734
* docs(selectors): qualify which fields refuse find focus
Refs #3734
* docs(selectors): name the covered refusal's reason
Refs #3734
'Scroll in a direction, or toward the top/bottom edge of scrollable content. Set until to a selector to reach an off-screen target in one command rather than a scroll-and-check loop. The optional amount is the finger-path fraction of the viewport axis, honored up to 0.8 of it; directional scrolls reduce release momentum, while app scroll physics determine the final content offset. A visible keyboard shortens the swiped band instead of being dismissed; when too little is left, the command refuses with scroll_keyboard_occludes_surface. A directional scroll also reports the movement it observed as movement: moved, at-edge, unchanged, or unobserved when the two reads could not back a claim either way; an unchanged surface inside a container that still hides content in that direction refuses with scroll_no_progress rather than repeating the requested distance. The movement field is absent where a tier verifies per pass (top/bottom, until), where the runtime cannot read a screen, or where a settle observation or a replay already owns that observation.',
78
78
get: 'Read text or accessibility attributes from a snapshot ref or selector without changing the app. Use format text for visible content or attrs for the element attribute map.',
79
79
is: 'Check whether a selector satisfies a UI predicate such as visible, hidden, exists, absent, editable, selected, focused, or text. `absent` passes only when one readable, complete, settled, unscoped, full-depth accessibility capture has zero matches. Use wait when the condition may appear asynchronously.',
80
-
find: 'Find by text/label/value/role/id and run action; the action defaults to click. The role locator matches snapshot kind vocabulary (button, text, text-field, switch, ...); pre-reconciliation leaf spellings (statictext, edittext, textview) stay accepted on the nodes that carried them during a deprecation window.',
80
+
find: 'Find by text/label/value/role/id and run action; the action defaults to click. The role locator matches snapshot kind vocabulary (button, text, text-field, switch, ...); pre-reconciliation leaf spellings (statictext, edittext, textview) stay accepted on the nodes that carried them during a deprecation window. The focus and type actions aim at a text field: the match or the field holding it, even beside other fields, or else the one enabled on-screen field inside the match. Several enabled on-screen fields inside the match refuse with AMBIGUOUS_MATCH and the field candidates, and a disabled target refuses with reason target_disabled. Web text boxes are not recognized as text fields.',
81
81
gesture:
82
82
'Perform a structured pan, fling, swipe, pinch, rotate, transform, tap, or drag gesture. Select the gesture kind, then provide only the inputs that apply to that kind. A tap touches 2 to 5 points at once; repeat it with count and pauseMs.',
'gap: the `findAct` row declares `offscreen: "ignore"` because find ranks across the whole tree and prefers on-screen matches rather than refusing off-screen ones (#1625), and focus/type never re-enter the acting leaf where the runtime rows refuse. So the seam dispatches at a captured rect center the acting paths would reject as outside the root viewport — a closed drawer or an off-viewport carousel is a silent no-op focus here.',
577
+
'gap: the `findAct` row declares `offscreen: "ignore"` because find ranks across the whole tree and prefers on-screen matches rather than refusing off-screen ones (#1625), and focus/type never re-enter the acting leaf where the runtime rows refuse. So the seam dispatches at a captured touch point the acting paths would reject as outside the root viewport — a closed drawer or an off-viewport carousel is a silent no-op focus here.',
578
578
trackingIssue: FIND_FOCUS_DISPATCH_GAPS_ISSUE,
579
579
},
580
+
// The rule treats a nested text field as a competitor to avoid, but this seam means to focus
581
+
// one, so `resolveFocusTarget` chooses the node the rule then aims inside (#3734).
580
582
parentOwnedTouchPoint: {
581
-
kind: 'waived',
582
-
reason:
583
-
'gap: the seam aims `centerOfRect` of the resolved node rather than the parent-owned point `resolveInteractionTouchPoint` chooses, so a match whose center is tiled by an independently interactive descendant activates that descendant instead of failing closed — and the keyboard guard above measures the same center the tap uses, which is the point actually dispatched but not the point the acting paths would aim.',
// The guarantee is promotion AND annotation. This seam runs only the first half, so the
587
587
// cell cannot claim `runtime` for the whole guarantee (the same shape `native-ref` classifies
588
588
// the other way round, annotation without promotion).
589
589
nonHittable: {
590
590
kind: 'waived',
591
591
reason:
592
-
"gap: the promotion half does run — the `findAct` row declares `hittable-ancestor-below-root`, so the seam dispatches the promoted ancestor's center (regression-proven in packages/daemon/src/execution/interaction/internal/__tests__/find.test.ts) — but the annotation half never does: the response is the focus runtime's point payload, so a match still reporting `hittable: false` after promotion is accepted silently, where press/click/fill answer with `targetHittable: false` plus a hint. Closing it needs the annotation composed at this seam, which is response-shape work the `responseIdentity` waiver below also records.",
592
+
"gap: the promotion half does run — the `findAct` row declares `hittable-ancestor-below-root`, so the seam dispatches the promoted ancestor's touch point unless `resolveFocusTarget` chooses a text field instead (regression-proven in packages/daemon/src/execution/interaction/internal/__tests__/find.test.ts) — but the annotation half never does: the response is the focus runtime's point payload, so a match still reporting `hittable: false` after promotion is accepted silently, where press/click/fill answer with `targetHittable: false` plus a hint. Closing it needs the annotation composed at this seam, which is response-shape work the `responseIdentity` waiver below also records.",
593
593
trackingIssue: FIND_FOCUS_DISPATCH_GAPS_ISSUE,
594
594
},
595
595
// One construction site per dispatch kind, both shared with the plain commands: the focus
"gap: the seam dispatches a rect center captured one runner round trip earlier without the #3471 re-observation, so a focus issued right after an action that animated the surface can aim a position the target has already vacated and report success on the vacancy. The gate itself is the acting leaf's (`guardResolvedTargetAgainstMotion`), and this route does not pass through it.",
639
+
"gap: the seam dispatches a touch point captured one runner round trip earlier without the #3471 re-observation, so a focus issued right after an action that animated the surface can aim a position the target has already vacated and report success on the vacancy. The gate itself is the acting leaf's (`guardResolvedTargetAgainstMotion`), and this route does not pass through it.",
0 commit comments