Skip to content

Commit b9a59ea

Browse files
committed
fix(ios): confirm post-tap focus from a signal iOS can actually answer
Adversarial review of the previous commit found that carrying the dispatch point forward was not enough on its own. On iOS `focusedTextInput` is intentionally nil, so the readiness wait the tap path calls can never succeed there: it spent `readinessTimeout` and then repeated the gesture unconditionally. Repeating it at a point the keyboard may since have moved away from its field is the stale-frame hazard this repo refuses elsewhere, and on the reported Flutter field the pre-fix code never reached a second gesture at all, because the failed read aborted the command first. Poll for the software keyboard instead, which is the focus evidence iOS gives without re-reading the resolved element: the wait ends as soon as a keyboard appears, keeps the same ceiling before repeating the gesture so a merely late keyboard is not answered with a gesture a healthy app did not need (#1874), and a keyboard already up at the tap is read as confirmation rather than as a question the repeat should answer. `waitForFocusedTextInput` had no caller left. Also reads that could not be observed: `coordinateTapTextInputIdentityAt` now says when a probe resolved an input whose identity it could not capture, so the resulting loss of tap-to-type authorization is distinguishable from a point that held no input, and the two doc comments that over-claimed what the tap-side fix covers are scoped to what it does.
1 parent 486be67 commit b9a59ea

5 files changed

Lines changed: 83 additions & 49 deletions

File tree

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandExecution.swift‎

Lines changed: 13 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -68,10 +68,12 @@ extension RunnerTests {
6868
let isTextEntry = isTextEntryElement(element)
6969
let touchPoint = expectedPoint ?? CGPoint(x: frame.midX, y: frame.midY)
7070
// Classified here, before the gesture: after it, the element answers its own query only as
71-
// long as the app kept it in the same shape (#3060).
71+
// long as the app kept it in the same shape (#3060). The keyboard state travels with it for
72+
// the same reason — read after the gesture, it cannot say what the gesture caused.
7273
let tappedInput = isTextEntry
7374
? TappedTextInput(element: element, dispatchPoint: touchPoint)
7475
: nil
76+
let keyboardVisibleBeforeTap = tappedInput != nil && isKeyboardVisible(app: activeApp)
7577
let touchFrame = resolvedTouchVisualizationFrame(
7678
app: activeApp,
7779
x: touchPoint.x,
@@ -94,7 +96,11 @@ extension RunnerTests {
9496
}) {
9597
case .performed(let timing):
9698
if let tappedInput {
97-
confirmTextEntryFocusAfterTap(app: activeApp, tapped: tappedInput)
99+
confirmTextEntryFocusAfterTap(
100+
app: activeApp,
101+
tapped: tappedInput,
102+
keyboardVisibleBeforeTap: keyboardVisibleBeforeTap
103+
)
98104
}
99105
rememberTextEntryTap(tappedInput)
100106
return gestureResponse(
@@ -130,7 +136,11 @@ extension RunnerTests {
130136
return response
131137
}
132138
if let tappedInput {
133-
confirmTextEntryFocusAfterTap(app: activeApp, tapped: tappedInput)
139+
confirmTextEntryFocusAfterTap(
140+
app: activeApp,
141+
tapped: tappedInput,
142+
keyboardVisibleBeforeTap: keyboardVisibleBeforeTap
143+
)
134144
}
135145
rememberTextEntryTap(tappedInput)
136146
return gestureResponse(

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextEntry.swift‎

Lines changed: 8 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -141,22 +141,19 @@ extension RunnerTests {
141141

142142
/// The text input a tap dispatched onto, classified BEFORE the gesture ran.
143143
///
144-
/// A tap resolves its element through a query and keeps an index-bound handle to it, so any later
145-
/// read of that handle re-runs the query. ADR 0005 keeps a tap's outcome independent of that
144+
/// A tap resolves its element through a query and keeps a handle bound to that query's index, so any
145+
/// read of the handle re-runs the query. ADR 0005 keeps a tap's outcome independent of that
146146
/// bookkeeping, and #3060 shows what that buys: a Flutter `TextInputSemanticsObject` reports
147147
/// `TextField` through the legacy attributes and `Other` through the modern one, so taking focus
148148
/// makes the element stop answering the query that found it, the re-run answers `No matches found
149149
/// for Element at index 1 from input {(TextField)}`, and XCTest records a failure that ends the
150-
/// runner session for a tap the dispatch had already landed. Any other post-gesture read of a
151-
/// target that moved or stopped resolving — the hostile-tree shapes behind #758, #1105 and #2995 —
152-
/// fails the same way.
150+
/// runner session for a tap the dispatch had already landed.
153151
///
154-
/// So the handle travels forward as an address to type into and nothing else. The question the tap
155-
/// used to ask of it afterwards — does this element take text? — is settled at construction, where
156-
/// the tap already held the answer: the selector route asks its own `isTextEntryElement`, and the
157-
/// coordinate route classifies the identity its probe read inside issue containment before the
158-
/// gesture. Same predicate, one read earlier, so a tap authorizes later typing exactly when it used
159-
/// to and never pays for the answer with its session.
152+
/// So neither tap route reads it after dispatching. The question they used to ask then — does this
153+
/// element take text? — is settled here instead, at the same call sites that made the gesture: the
154+
/// selector route from its own `isTextEntryElement`, the coordinate route from the identity its probe
155+
/// read inside issue containment. The handle itself travels forward as an address to type into, which
156+
/// the next `type` reads under that command's own rules.
160157
struct TappedTextInput {
161158
let element: XCUIElement
162159
/// The point the tap was dispatched at, which the tap already proved dispatchable. The

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextEntryReadiness.swift‎

Lines changed: 35 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,9 @@ import XCTest
22

33
// Text-entry readiness: whether the target is actually able to receive text yet. The waits, the
44
// keyboard signals they read, and the focus corroboration that ends the hardware-keyboard window.
5-
// RunnerTests+TextEntryFocus.swift is the only caller of the two non-private entries here; nothing
6-
// in this file reaches back into it.
5+
// Callers are the text-typing paths in RunnerTests+TextEntry.swift and RunnerTests+TextEntryFocus.swift,
6+
// plus the tap command in RunnerTests+CommandExecution.swift, which confirms focus after its gesture.
7+
// Nothing in this file reaches back into them.
78
extension RunnerTests {
89
func focusedTextInput(app: XCUIApplication) -> XCUIElement? {
910
#if os(iOS)
@@ -80,29 +81,46 @@ extension RunnerTests {
8081
#endif
8182
}
8283

83-
/// Confirms that the tap put keyboard focus on its input, and re-taps once at the point the tap was
84-
/// dispatched at when focus did not arrive. Both facts it acts on come from `TappedTextInput`, so
85-
/// confirming focus never re-reads the element the tap resolved (#3060).
86-
func confirmTextEntryFocusAfterTap(app: XCUIApplication, tapped: TappedTextInput) {
84+
/// Confirms the tap put keyboard focus on its input, and repeats the gesture once when it did not.
85+
///
86+
/// The signal is a software keyboard appearing, which is the focus evidence iOS gives without
87+
/// re-reading the element the tap resolved. The wait it replaced polled `focusedTextInput`, which is
88+
/// intentionally nil on iOS, so it could never succeed: it spent `readinessTimeout` and then re-tapped
89+
/// whatever the tap had achieved (#3060). Polling for a keyboard ends that wait as soon as the tap
90+
/// clearly worked, and keeps the same ceiling before repeating the gesture, because on a loaded host a
91+
/// keyboard is merely late and a re-tap issued too early is a gesture a healthy app did not need
92+
/// (#1874).
93+
///
94+
/// A keyboard already up when the tap started is read as confirmation and nothing repeats the gesture:
95+
/// its visibility cannot say whether focus moved, and a second tap would land on the keyboard or on
96+
/// whatever keyboard avoidance slid into the point. Where no keyboard appears at all — a responder with
97+
/// a custom `inputView`, or a field that never took focus — the repeat at `tapped.dispatchPoint` is
98+
/// kept, since that point is the one the tap already proved dispatchable and a gesture is the only
99+
/// remaining way to move focus.
100+
func confirmTextEntryFocusAfterTap(
101+
app: XCUIApplication,
102+
tapped: TappedTextInput,
103+
keyboardVisibleBeforeTap: Bool
104+
) {
87105
#if os(iOS)
88-
if waitForFocusedTextInput(app: app, timeout: TextEntryTiming.readinessTimeout) != nil {
106+
if keyboardVisibleBeforeTap {
107+
return
108+
}
109+
if softwareKeyboardAppears(app: app) {
89110
return
90111
}
91-
let point = tapped.dispatchPoint
92-
_ = tapAt(app: app, x: point.x, y: point.y)
93-
_ = waitForFocusedTextInput(app: app, timeout: TextEntryTiming.readinessTimeout)
112+
_ = tapAt(app: app, x: tapped.dispatchPoint.x, y: tapped.dispatchPoint.y)
113+
_ = softwareKeyboardAppears(app: app)
94114
#endif
95115
}
96116

97-
private func waitForFocusedTextInput(app: XCUIApplication, timeout: TimeInterval) -> XCUIElement? {
98-
let deadline = Date().addingTimeInterval(timeout)
99-
while Date() < deadline {
100-
if let focused = focusedTextInput(app: app) {
101-
return focused
102-
}
117+
/// Whether a software keyboard shows up within the readiness window, returning as soon as one does.
118+
private func softwareKeyboardAppears(app: XCUIApplication) -> Bool {
119+
let deadline = Date().addingTimeInterval(TextEntryTiming.readinessTimeout)
120+
while !isKeyboardVisible(app: app), Date() < deadline {
103121
sleepFor(TextEntryTiming.pollInterval)
104122
}
105-
return focusedTextInput(app: app)
123+
return isKeyboardVisible(app: app)
106124
}
107125

108126
func keyboardBecameVisible(app: XCUIApplication, wasVisibleAtEntry: Bool) -> Bool {

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextInputProbe.swift‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -33,9 +33,14 @@ extension RunnerTests {
3333
/// the app may have changed the element out from under the query (#3060).
3434
func coordinateTapTextInputIdentityAt(app: XCUIApplication, x: Double, y: Double) -> TextInputAtPoint? {
3535
let probed = containingTextInputProbeIssues(fallback: nil) { shouldStop -> TextInputAtPoint? in
36-
guard let element = queryTextInputs(app: app, point: CGPoint(x: x, y: y), shouldStop: shouldStop).first,
37-
case .input(let identity) = probeTextEntryInput(element)
38-
else {
36+
guard let element = queryTextInputs(app: app, point: CGPoint(x: x, y: y), shouldStop: shouldStop).first else {
37+
return nil
38+
}
39+
guard case .input(let identity) = probeTextEntryInput(element) else {
40+
// The query resolved this element and the tap will still be dispatched onto it, but this read
41+
// could not capture what it is, so the tap cannot authorize later typing. Say so, rather than
42+
// leaving the omission indistinguishable from a point that held no input.
43+
NSLog("AGENT_DEVICE_RUNNER_COORDINATE_TAP_TEXT_IDENTITY_UNAVAILABLE point=(%.0f,%.0f)", x, y)
3944
return nil
4045
}
4146
return TextInputAtPoint(element: element, identity: identity)

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/UnitTests/RunnerTests+CommandExecutionTests.swift‎

Lines changed: 19 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -115,18 +115,19 @@ extension RunnerTests {
115115
)
116116
}
117117

118-
// #3060: a Flutter password field whose legacy accessibility attributes still say `TextField` once
119-
// focused while its modern ones say `Other`, so it stops answering the query that resolved it. The
120-
// runner classified its resolved element AFTER dispatching, which re-ran that query, and XCTest
121-
// answered `No matches found for Element at index 1 from input {(TextField)}` — a recorded failure
122-
// that failed the command and restarted the runner for a tap that had already landed, on a field
123-
// that visibly took focus. The fixture stops answering the query on focus, the shape both channels
124-
// leave behind, so each route below can be held to the fact it used to violate.
125-
126-
/// The coordinate route, end to end through the transport: the same synthesized `tap` the issue's
127-
/// `click @ref` dispatched. A tap that lands must report `tapped` and must leave no recorded failure
128-
/// for the dispatch path to convert into `XCTEST_RECORDED_FAILURE` and a runner restart, while still
129-
/// booking the witness that authorizes the next bare `type`.
118+
// #3060: a Flutter password field reports `TextField` through the legacy accessibility attributes and
119+
// `Other` through the modern ones once it takes focus, so it stops answering the type-bound query that
120+
// resolved it. The runner classified its resolved element AFTER dispatching, which re-ran that query,
121+
// and XCTest answered `No matches found for Element at index 1 from input {(TextField)}` — a recorded
122+
// failure that failed the command and restarted the runner for a tap the dispatch had already landed.
123+
// The fixture reaches the same consequence by leaving the tree on focus, and its app-side delegate
124+
// witness lets the test prove the gesture took focus when the accessibility tree no longer can.
125+
126+
/// The route the issue measured: `click @ref` and `press <x> <y>` both reach a coordinate tap, whose
127+
/// probe handle is bound to the index it held in the type-bound query — the shape of the log's
128+
/// `Element at index 1 from input {(TextField)}`. A tap that lands must report `tapped` and leave no
129+
/// recorded failure for the dispatch path to convert into `XCTEST_RECORDED_FAILURE` and a runner
130+
/// restart, while still booking the witness that the next bare `type` is addressed by.
130131
@MainActor
131132
func testCoordinateTapOnInputThatLeavesTheTreeOnFocusReportsTapped() throws {
132133
app.launchArguments = [
@@ -169,9 +170,12 @@ extension RunnerTests {
169170
)
170171
}
171172

172-
/// The selector route, which additionally confirmed focus by re-reading the element's type and frame
173-
/// after the gesture. Synthesized so the gesture is a real touch that takes focus, which is what
174-
/// moves the fixture out of the tree.
173+
/// The same invariant on the selector route, which settled its input's class and its focus state with
174+
/// two reads after the gesture (`isTextEntryElement` and `element.frame`). Both routes must settle what
175+
/// a gesture can invalidate before dispatching it, so this holds the second one to it: on this fixture
176+
/// the route's identifier-bound handle stops resolving once the field hides, and a read of it then
177+
/// records a failure the command inherits. `click @ref` and `press <x> <y>` in the report both reached
178+
/// the coordinate route above, so this is the generalized rule rather than the reported lookup.
175179
@MainActor
176180
func testSelectorTapOnInputThatLeavesTheTreeOnFocusRecordsNoFailure() throws {
177181
app.launchArguments = [

0 commit comments

Comments
 (0)