Skip to content

Commit ea437cb

Browse files
committed
fix(ios): stop a tap's post-gesture lookup from recording an XCTest failure (#3060)
A tap resolves its element through a query and keeps a handle bound to that query, so reading the handle afterwards re-runs the query. A Flutter password field reports `TextField` through the legacy accessibility attributes and `Other` through the modern ones once it takes focus, so it stops answering the query that found it, XCTest answers `No matches found for Element at index 1 from input {(TextField)}` and RECORDS a failure, and the dispatch path converts that into XCTEST_RECORDED_FAILURE plus an invalidated target. The runner restarts for a tap the dispatch had already landed and the field had visibly taken focus, which is the 7-14 s per tap the report measures. ADR 0005 already keeps a tap's outcome independent of this bookkeeping. Two reads depended on it. `rememberTextEntryTap` re-derived `isTextEntryElement` after the gesture even though both tap routes had already classified the element before dispatching it — the selector route from its own `isTextEntryElement`, the coordinate route from a probe that enumerates only text-entry element types — so the re-check could only ever agree or die. Ask it of nothing: the parameter is now the caller's classification, and the tap books its witness from that. The post-tap readiness wait read `element.elementType` and then `element.frame`, which are the two reads that record. Read the same facts through `snapshot()`, which answers the query the same way but through the throwing channel and records nothing — the remedy the report asked for, and already how `probeTextEntryInput` and `withElement` reach an element. Measured on a handle whose element had left its query: `snapshot()` records nothing and throws 10008, where `.elementType` and `.frame` each record one failure and raise. The frame is still read after the wait, so the focus repeat keeps aiming at where the field is rather than where the tap started: focusing a field moves the layout, and a pre-gesture point can name a different field or none. Both new cases are measured red on the base with the report's own signature, recorded at `isTextEntryElement` and at the readiness reads, and green here. The fixture's field leaves the accessibility tree on focus, the consequence both attribute channels produce, and an app-side delegate witness lets the test prove the gesture took focus when the tree no longer can.
1 parent 2f933e5 commit ea437cb

5 files changed

Lines changed: 172 additions & 13 deletions

File tree

‎apple/runner/AgentDeviceRunner/AgentDeviceRunner/AgentDeviceRunnerApp.m‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,9 @@ @interface AgentDeviceRunnerViewController : UIViewController
6767
@property(nonatomic, assign) NSUInteger replacementAlertActions;
6868
@property(nonatomic, strong) UILabel *textEntryWriteBackStatus;
6969
@property(nonatomic, strong) UILabel *textEntryDigitSlots;
70+
// Filled from the fixture field's own delegate callbacks so a test can tell that a gesture really
71+
// took focus from an input that has stopped answering the accessibility query that found it.
72+
@property(nonatomic, strong) UILabel *textEntryFocusWitness;
7073
@property(nonatomic, assign) NSUInteger textEntryWriteBacks;
7174
@property(nonatomic, copy, nullable) NSString *textEntryRenderedValue;
7275
@property(nonatomic, assign) NSTimeInterval textEntryLastEditTime;
@@ -340,6 +343,20 @@ - (void)agentDeviceTextEntryDidChange:(UITextField *)textField {
340343
}
341344
}
342345

346+
// Stops answering the element-type query that found it, the moment it takes focus, under
347+
// `--agent-device-text-entry-unqueryable-on-focus`. #3060's Flutter password field reaches this shape
348+
// because its two accessibility channels disagree once focused — `TextField` through the legacy
349+
// attributes, `Other` through the modern ones — and what either channel leaves behind is the same:
350+
// the tap's element answers the query that resolved it up to the gesture and refuses it after, so a
351+
// read of it after dispatching records an XCTest failure instead of describing the tap that already
352+
// landed. The field keeps first-responder, so a test still needs a witness that is not the
353+
// accessibility tree to tell the gesture landed: `agent-device-text-entry-focus` is a label this
354+
// method fills in from the field's own delegate callback.
355+
- (void)agentDeviceTextEntryBecomesUnqueryable:(UITextField *)textField {
356+
self.textEntryFocusWitness.text = @"focus";
357+
textField.hidden = YES;
358+
}
359+
343360
// Moves the field up by its own height plus the gap below it when it gains focus, the way keyboard
344361
// avoidance or a bottom sheet extending above the keyboard does, so the neighbouring field slides
345362
// into the point the focus tap hit.
@@ -417,6 +434,19 @@ - (void)viewDidLoad {
417434
[textField.widthAnchor constraintEqualToConstant:240],
418435
[textField.heightAnchor constraintEqualToConstant:AgentDeviceTextEntryFieldHeight],
419436
]];
437+
if ([NSProcessInfo.processInfo.arguments containsObject:@"--agent-device-text-entry-unqueryable-on-focus"]) {
438+
self.textEntryFocusWitness = [[UILabel alloc] init];
439+
self.textEntryFocusWitness.accessibilityIdentifier = @"agent-device-text-entry-focus";
440+
self.textEntryFocusWitness.translatesAutoresizingMaskIntoConstraints = NO;
441+
[self.view addSubview:self.textEntryFocusWitness];
442+
[NSLayoutConstraint activateConstraints:@[
443+
[self.textEntryFocusWitness.centerXAnchor constraintEqualToAnchor:self.view.centerXAnchor],
444+
[self.textEntryFocusWitness.topAnchor constraintEqualToAnchor:textField.bottomAnchor constant:24],
445+
]];
446+
[textField addTarget:self
447+
action:@selector(agentDeviceTextEntryBecomesUnqueryable:)
448+
forControlEvents:UIControlEventEditingDidBegin];
449+
}
420450
if ([NSProcessInfo.processInfo.arguments containsObject:@"--agent-device-text-entry-moves-on-focus"]) {
421451
textField.text = @"stale";
422452
[textField addTarget:self

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextEntry.swift‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -265,7 +265,13 @@ extension RunnerTests {
265265
}
266266

267267
func isTextEntryElement(_ element: XCUIElement) -> Bool {
268-
switch element.elementType {
268+
Self.isTextEntryElementType(element.elementType)
269+
}
270+
271+
/// The element types that take text. Split out so a caller holding a `snapshot()` can classify it
272+
/// without a second read of the element, which a query-bound handle can record a failure for (#3060).
273+
static func isTextEntryElementType(_ elementType: XCUIElement.ElementType) -> Bool {
274+
switch elementType {
269275
case .textField, .secureTextField, .searchField, .textView:
270276
return true
271277
default:

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextEntryFocus.swift‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,16 @@ import XCTest
66
// is RunnerTests+TextEntryReadiness.swift's question, and this file asks it rather than answering
77
// it.
88
extension RunnerTests {
9+
/// Books the tap as the address for the next bare `type`. `element` is a text input the caller
10+
/// classified BEFORE dispatching, or nil when the tap hit nothing worth typing into; that is the whole
11+
/// contract, and this function must not re-check it. A tap holds a handle bound to the query that
12+
/// resolved the element, so asking the handle re-runs that query, and an element that stopped answering
13+
/// it — a Flutter password field once focus makes its accessibility attributes disagree — makes XCTest
14+
/// record a failure. That failure ends the runner session for a tap the dispatch had already landed
15+
/// (#3060), and ADR 0005 already keeps a tap's outcome independent of this bookkeeping.
916
@MainActor
1017
func rememberTextEntryTap(_ element: XCUIElement?) {
11-
guard let element, isTextEntryElement(element) else {
18+
guard let element else {
1219
clearRememberedTextEntryTap()
1320
return
1421
}

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextEntryReadiness.swift‎

Lines changed: 27 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -80,21 +80,37 @@ extension RunnerTests {
8080
#endif
8181
}
8282

83+
/// Gives a tapped text input one readiness window to take focus, then taps it once more if focus did
84+
/// not arrive. Both facts it acts on come from a `snapshot()`, the one way to read a query-bound handle
85+
/// without risking the session (#3060): an element that stopped answering its query makes
86+
/// `element.elementType` or `element.frame` record an XCTest failure, and `didRecordXCTestFailure` turns
87+
/// that into `XCTEST_RECORDED_FAILURE` plus an invalidated target for a tap the dispatch had already
88+
/// landed. `snapshot()` answers the same questions through the throwing channel and records nothing,
89+
/// which is why `probeTextEntryInput` and `withElement` already read elements this way.
90+
///
91+
/// The frame is read again after the wait rather than reused, because focusing a field can move the
92+
/// layout: the repeat aims at where the field is then, which is the moment the read it replaced ran at.
8393
func waitForTextEntryReadinessAfterTap(app: XCUIApplication, element: XCUIElement) {
8494
#if os(iOS)
85-
switch element.elementType {
86-
case .textField, .secureTextField, .searchField, .textView:
87-
if waitForFocusedTextInput(app: app, timeout: TextEntryTiming.readinessTimeout) != nil {
88-
return
89-
}
90-
let frame = element.frame
91-
if !frame.isEmpty {
92-
_ = tapAt(app: app, x: frame.midX, y: frame.midY)
93-
_ = waitForFocusedTextInput(app: app, timeout: TextEntryTiming.readinessTimeout)
94-
}
95-
default:
95+
guard let snapshot = safely("TEXT_ENTRY_READINESS_SNAPSHOT", { try? element.snapshot() }) else {
96+
return
97+
}
98+
guard Self.isTextEntryElementType(snapshot.elementType) else {
9699
return
97100
}
101+
if waitForFocusedTextInput(app: app, timeout: TextEntryTiming.readinessTimeout) != nil {
102+
return
103+
}
104+
// Read again rather than reusing the frame above: focusing a field can move the layout, and the
105+
// repeat wants where the field is now, which is the same moment the read it replaced happened at.
106+
guard let settled = safely("TEXT_ENTRY_READINESS_FRAME", { try? element.snapshot() }) else {
107+
return
108+
}
109+
let frame = settled.frame
110+
if !frame.isEmpty {
111+
_ = tapAt(app: app, x: frame.midX, y: frame.midY)
112+
_ = waitForFocusedTextInput(app: app, timeout: TextEntryTiming.readinessTimeout)
113+
}
98114
#endif
99115
}
100116

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/UnitTests/RunnerTests+CommandExecutionTests.swift‎

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,106 @@ extension RunnerTests {
114114
"focus held by another element must read as a refusal, never as this element's focus"
115115
)
116116
}
117+
118+
// #3060: a Flutter password field reports `TextField` through the legacy accessibility attributes and
119+
// `Other` through the modern ones once it takes focus, so it stops answering the type-bound query that
120+
// resolved it. The runner classified its resolved element AFTER dispatching, which re-ran that query,
121+
// and XCTest answered `No matches found for Element at index 1 from input {(TextField)}` — a recorded
122+
// failure that failed the command and restarted the runner for a tap the dispatch had already landed.
123+
// The fixture reaches the same consequence by leaving the tree on focus, and its app-side delegate
124+
// witness lets the test prove the gesture took focus when the accessibility tree no longer can.
125+
126+
/// The route the issue measured: `click @ref` and `press <x> <y>` both reach a coordinate tap, whose
127+
/// probe handle is bound to the index it held in the type-bound query — the shape of the log's
128+
/// `Element at index 1 from input {(TextField)}`. A tap that lands must report `tapped` and leave no
129+
/// recorded failure for the dispatch path to convert into `XCTEST_RECORDED_FAILURE` and a runner
130+
/// restart, while still booking the witness that the next bare `type` is addressed by.
131+
///
132+
/// The witness assertion pins how it is booked, not that a following `type` uses it: this fixture's
133+
/// field is gone from the tree by then, so the next `type` would refuse the handle. What the tap must
134+
/// not need is a read of the handle to decide whether to book it.
135+
@MainActor
136+
func testCoordinateTapOnInputThatLeavesTheTreeOnFocusReportsTapped() throws {
137+
app.launchArguments = [
138+
"--agent-device-text-entry-regression",
139+
"--agent-device-text-entry-unqueryable-on-focus",
140+
]
141+
app.launch()
142+
defer {
143+
clearSnapshotXCTestChannelPenalty(reason: "test-cleanup")
144+
invalidateCachedTarget(reason: "unit_test_cleanup")
145+
app.terminate()
146+
}
147+
XCTAssertTrue(app.waitForExistence(timeout: appExistenceTimeout))
148+
let field = app.textFields["agent-device-hardware-keyboard-input"]
149+
XCTAssertTrue(field.waitForExistence(timeout: appExistenceTimeout))
150+
let frame = field.frame
151+
mainOwned.app = app
152+
mainOwned.bundleId = "com.callstack.agentdevice.runner"
153+
mainOwned.processIdentifier = try XCTUnwrap(Self.processIdentifier(of: app))
154+
clearSnapshotXCTestChannelPenalty(reason: "fresh-runner")
155+
156+
let failures = currentXCTestFailureCount()
157+
let tap = try runnerCommandFixture(
158+
#"{"appBundleId":"com.callstack.agentdevice.runner","command":"tap","commandId":"tap-unqueryable-on-focus","x":\#(frame.midX),"y":\#(frame.midY),"synthesized":true}"#
159+
)
160+
let response = try execute(command: tap)
161+
162+
XCTAssertEqual(response.data?.message, "tapped", String(describing: response.error))
163+
XCTAssertFalse(didRecordXCTestFailure(since: failures))
164+
// The app's own delegate callbacks, not the accessibility tree: without this, "nothing recorded a
165+
// failure" would also be satisfied by a gesture that never reached the field at all.
166+
XCTAssertEqual(
167+
app.staticTexts["agent-device-text-entry-focus"].label,
168+
"focus",
169+
"the tap must have taken focus from the field it resolved"
170+
)
171+
XCTAssertNotNil(
172+
textEntryTapWitness,
173+
"the tap still authorizes the next bare type from what it knew before dispatching"
174+
)
175+
}
176+
177+
/// The same invariant on the selector route, which read the element three times after its gesture: the
178+
/// type again inside `rememberTextEntryTap`, then the type and the frame in the readiness wait. On this
179+
/// fixture the route's identifier-bound handle stops resolving once the field hides, so each of those
180+
/// recorded a failure the command inherits. `click @ref` and `press <x> <y>` in the report both reached
181+
/// the coordinate route above, so this holds the second route to the same rule rather than reproducing
182+
/// the reported lookup.
183+
@MainActor
184+
func testSelectorTapOnInputThatLeavesTheTreeOnFocusRecordsNoFailure() throws {
185+
app.launchArguments = [
186+
"--agent-device-text-entry-regression",
187+
"--agent-device-text-entry-unqueryable-on-focus",
188+
]
189+
app.launch()
190+
defer {
191+
invalidateCachedTarget(reason: "unit_test_cleanup")
192+
app.terminate()
193+
}
194+
XCTAssertTrue(app.waitForExistence(timeout: appExistenceTimeout))
195+
XCTAssertTrue(
196+
app.textFields["agent-device-hardware-keyboard-input"].waitForExistence(timeout: appExistenceTimeout)
197+
)
198+
mainOwned.app = app
199+
mainOwned.bundleId = "com.callstack.agentdevice.runner"
200+
mainOwned.processIdentifier = try XCTUnwrap(Self.processIdentifier(of: app))
201+
202+
let failures = currentXCTestFailureCount()
203+
let tap = try runnerCommandFixture(
204+
#"{"appBundleId":"com.callstack.agentdevice.runner","command":"tap","commandId":"selector-tap-unqueryable-on-focus","selectorKey":"id","selectorValue":"agent-device-hardware-keyboard-input","synthesized":true}"#
205+
)
206+
let response = try execute(command: tap)
207+
208+
XCTAssertEqual(response.data?.message, "tapped", String(describing: response.error))
209+
XCTAssertFalse(didRecordXCTestFailure(since: failures))
210+
XCTAssertEqual(
211+
app.staticTexts["agent-device-text-entry-focus"].label,
212+
"focus",
213+
"the tap must have taken focus from the field it resolved"
214+
)
215+
XCTAssertNotNil(textEntryTapWitness)
216+
}
117217
#endif
118218
}
119219
#endif

0 commit comments

Comments
 (0)