Skip to content

Commit c53e88b

Browse files
authored
fix(ios): stop a tap's post-gesture lookup from recording an XCTest failure (#3060) (#3237)
A tap holds a handle bound to the query that resolved its element, so reading that handle re-runs the query. Flutter's TextInputSemanticsObject reports TextField through the legacy accessibility attributes and Other through the modern ones once the field is involved, so a focused password field stops answering the query that found it, XCTest records "No matches found for Element at index 1 from input {(TextField)}", and didRecordXCTestFailure turns that into XCTEST_RECORDED_FAILURE plus an invalidated target. The tap had already landed: the report measured 7-14s per tap, all of it runner restart. rememberTextEntryTap re-derived isTextEntryElement after the gesture even though both tap routes had already classified the element before dispatching, so the check could only agree or die. Its parameter is now the caller's classification, which for the coordinate route comes from queryTextInputs -- it enumerates only text-entry element types, so the classification does not depend on the snapshot's modern type. The post-tap readiness wait read element.elementType and element.frame, the two reads that record; the frame now comes from snapshot(), which answers the query through the throwing channel and records nothing, which is already how probeTextEntryInput and withElement reach an element. Readiness no longer classifies at all: the caller decided, and re-deriving it post-dispatch would classify differently from the caller because elementType answers through the legacy attributes while snapshot().elementType answers through the modern ones. The frame is still read after the wait, so the focus repeat aims at where the field is now rather than where the tap started. Runner-only, no CLI/daemon/API change. A tap that did not land still fails and a genuinely recorded failure still invalidates and surfaces.
1 parent 14e1bd7 commit c53e88b

4 files changed

Lines changed: 161 additions & 12 deletions

File tree

‎apple/runner/AgentDeviceRunner/AgentDeviceRunner/AgentDeviceRunnerApp.m‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,9 @@ @interface AgentDeviceRunnerViewController : UIViewController
6767
@property(nonatomic, assign) NSUInteger replacementAlertActions;
6868
@property(nonatomic, strong) UILabel *textEntryWriteBackStatus;
6969
@property(nonatomic, strong) UILabel *textEntryDigitSlots;
70+
// Filled from the fixture field's own delegate callbacks so a test can tell that a gesture really
71+
// took focus from an input that has stopped answering the accessibility query that found it.
72+
@property(nonatomic, strong) UILabel *textEntryFocusWitness;
7073
@property(nonatomic, assign) NSUInteger textEntryWriteBacks;
7174
@property(nonatomic, copy, nullable) NSString *textEntryRenderedValue;
7275
@property(nonatomic, assign) NSTimeInterval textEntryLastEditTime;
@@ -340,6 +343,20 @@ - (void)agentDeviceTextEntryDidChange:(UITextField *)textField {
340343
}
341344
}
342345

346+
// Stops answering the element-type query that found it, the moment it takes focus, under
347+
// `--agent-device-text-entry-unqueryable-on-focus`. #3060's Flutter password field reaches this shape
348+
// because its two accessibility channels disagree once focused — `TextField` through the legacy
349+
// attributes, `Other` through the modern ones — and what either channel leaves behind is the same:
350+
// the tap's element answers the query that resolved it up to the gesture and refuses it after, so a
351+
// read of it after dispatching records an XCTest failure instead of describing the tap that already
352+
// landed. The field keeps first-responder, so a test still needs a witness that is not the
353+
// accessibility tree to tell the gesture landed: `agent-device-text-entry-focus` is a label this
354+
// method fills in from the field's own delegate callback.
355+
- (void)agentDeviceTextEntryBecomesUnqueryable:(UITextField *)textField {
356+
self.textEntryFocusWitness.text = @"focus";
357+
textField.hidden = YES;
358+
}
359+
343360
// Moves the field up by its own height plus the gap below it when it gains focus, the way keyboard
344361
// avoidance or a bottom sheet extending above the keyboard does, so the neighbouring field slides
345362
// into the point the focus tap hit.
@@ -417,6 +434,19 @@ - (void)viewDidLoad {
417434
[textField.widthAnchor constraintEqualToConstant:240],
418435
[textField.heightAnchor constraintEqualToConstant:AgentDeviceTextEntryFieldHeight],
419436
]];
437+
if ([NSProcessInfo.processInfo.arguments containsObject:@"--agent-device-text-entry-unqueryable-on-focus"]) {
438+
self.textEntryFocusWitness = [[UILabel alloc] init];
439+
self.textEntryFocusWitness.accessibilityIdentifier = @"agent-device-text-entry-focus";
440+
self.textEntryFocusWitness.translatesAutoresizingMaskIntoConstraints = NO;
441+
[self.view addSubview:self.textEntryFocusWitness];
442+
[NSLayoutConstraint activateConstraints:@[
443+
[self.textEntryFocusWitness.centerXAnchor constraintEqualToAnchor:self.view.centerXAnchor],
444+
[self.textEntryFocusWitness.topAnchor constraintEqualToAnchor:textField.bottomAnchor constant:24],
445+
]];
446+
[textField addTarget:self
447+
action:@selector(agentDeviceTextEntryBecomesUnqueryable:)
448+
forControlEvents:UIControlEventEditingDidBegin];
449+
}
420450
if ([NSProcessInfo.processInfo.arguments containsObject:@"--agent-device-text-entry-moves-on-focus"]) {
421451
textField.text = @"stale";
422452
[textField addTarget:self

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextEntryFocus.swift‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,16 @@ import XCTest
66
// is RunnerTests+TextEntryReadiness.swift's question, and this file asks it rather than answering
77
// it.
88
extension RunnerTests {
9+
/// Books the tap as the address for the next bare `type`. `element` is a text input the caller
10+
/// classified BEFORE dispatching, or nil when the tap hit nothing worth typing into; that is the whole
11+
/// contract, and this function must not re-check it. A tap holds a handle bound to the query that
12+
/// resolved the element, so asking the handle re-runs that query, and an element that stopped answering
13+
/// it — a Flutter password field once focus makes its accessibility attributes disagree — makes XCTest
14+
/// record a failure. That failure ends the runner session for a tap the dispatch had already landed
15+
/// (#3060), and ADR 0005 already keeps a tap's outcome independent of this bookkeeping.
916
@MainActor
1017
func rememberTextEntryTap(_ element: XCUIElement?) {
11-
guard let element, isTextEntryElement(element) else {
18+
guard let element else {
1219
clearRememberedTextEntryTap()
1320
return
1421
}

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextEntryReadiness.swift‎

Lines changed: 23 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -80,21 +80,33 @@ extension RunnerTests {
8080
#endif
8181
}
8282

83+
/// Gives a tapped text input one readiness window to take focus, then taps it once more if focus did
84+
/// not arrive. The caller has already decided this element is a text input, so this function must not
85+
/// re-derive that fact from the element: both tap routes classify BEFORE dispatching, and asking the
86+
/// handle afterwards re-runs its query. On a Flutter password field that re-run is exactly what fails
87+
/// (#3060): XCTest refuses the element with "computed TextField from legacy attributes vs Other from
88+
/// modern attribute", so a post-dispatch classification would also classify differently from the caller.
89+
///
90+
/// The frame still has to be read, and reading it through `element.frame` records an XCTest failure when
91+
/// the element has stopped answering its query; `didRecordXCTestFailure` turns that into
92+
/// `XCTEST_RECORDED_FAILURE` plus an invalidated target for a tap the dispatch had already landed.
93+
/// `snapshot()` answers the same question through the throwing channel and records nothing, which is why
94+
/// `probeTextEntryInput` and `withElement` already read elements this way. It is read again after the
95+
/// wait rather than reused, because focusing a field moves the layout: the repeat aims at where the
96+
/// field is then, which is the moment the read it replaced ran at.
8397
func waitForTextEntryReadinessAfterTap(app: XCUIApplication, element: XCUIElement) {
8498
#if os(iOS)
85-
switch element.elementType {
86-
case .textField, .secureTextField, .searchField, .textView:
87-
if waitForFocusedTextInput(app: app, timeout: TextEntryTiming.readinessTimeout) != nil {
88-
return
89-
}
90-
let frame = element.frame
91-
if !frame.isEmpty {
92-
_ = tapAt(app: app, x: frame.midX, y: frame.midY)
93-
_ = waitForFocusedTextInput(app: app, timeout: TextEntryTiming.readinessTimeout)
94-
}
95-
default:
99+
if waitForFocusedTextInput(app: app, timeout: TextEntryTiming.readinessTimeout) != nil {
96100
return
97101
}
102+
guard let settled = safely("TEXT_ENTRY_READINESS_FRAME", { try? element.snapshot() }) else {
103+
return
104+
}
105+
let frame = settled.frame
106+
if !frame.isEmpty {
107+
_ = tapAt(app: app, x: frame.midX, y: frame.midY)
108+
_ = waitForFocusedTextInput(app: app, timeout: TextEntryTiming.readinessTimeout)
109+
}
98110
#endif
99111
}
100112

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/UnitTests/RunnerTests+CommandExecutionTests.swift‎

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,106 @@ extension RunnerTests {
114114
"focus held by another element must read as a refusal, never as this element's focus"
115115
)
116116
}
117+
118+
// #3060: a Flutter password field reports `TextField` through the legacy accessibility attributes and
119+
// `Other` through the modern ones once it takes focus, so it stops answering the type-bound query that
120+
// resolved it. The runner classified its resolved element AFTER dispatching, which re-ran that query,
121+
// and XCTest answered `No matches found for Element at index 1 from input {(TextField)}` — a recorded
122+
// failure that failed the command and restarted the runner for a tap the dispatch had already landed.
123+
// The fixture reaches the same consequence by leaving the tree on focus, and its app-side delegate
124+
// witness lets the test prove the gesture took focus when the accessibility tree no longer can.
125+
126+
/// The route the issue measured: `click @ref` and `press <x> <y>` both reach a coordinate tap, whose
127+
/// probe handle is bound to the index it held in the type-bound query — the shape of the log's
128+
/// `Element at index 1 from input {(TextField)}`. A tap that lands must report `tapped` and leave no
129+
/// recorded failure for the dispatch path to convert into `XCTEST_RECORDED_FAILURE` and a runner
130+
/// restart, while still booking the witness that the next bare `type` is addressed by.
131+
///
132+
/// The witness assertion pins how it is booked, not that a following `type` uses it: this fixture's
133+
/// field is gone from the tree by then, so the next `type` would refuse the handle. What the tap must
134+
/// not need is a read of the handle to decide whether to book it.
135+
@MainActor
136+
func testCoordinateTapOnInputThatLeavesTheTreeOnFocusReportsTapped() throws {
137+
app.launchArguments = [
138+
"--agent-device-text-entry-regression",
139+
"--agent-device-text-entry-unqueryable-on-focus",
140+
]
141+
app.launch()
142+
defer {
143+
clearSnapshotXCTestChannelPenalty(reason: "test-cleanup")
144+
invalidateCachedTarget(reason: "unit_test_cleanup")
145+
app.terminate()
146+
}
147+
XCTAssertTrue(app.waitForExistence(timeout: appExistenceTimeout))
148+
let field = app.textFields["agent-device-hardware-keyboard-input"]
149+
XCTAssertTrue(field.waitForExistence(timeout: appExistenceTimeout))
150+
let frame = field.frame
151+
mainOwned.app = app
152+
mainOwned.bundleId = "com.callstack.agentdevice.runner"
153+
mainOwned.processIdentifier = try XCTUnwrap(Self.processIdentifier(of: app))
154+
clearSnapshotXCTestChannelPenalty(reason: "fresh-runner")
155+
156+
let failures = currentXCTestFailureCount()
157+
let tap = try runnerCommandFixture(
158+
#"{"appBundleId":"com.callstack.agentdevice.runner","command":"tap","commandId":"tap-unqueryable-on-focus","x":\#(frame.midX),"y":\#(frame.midY),"synthesized":true}"#
159+
)
160+
let response = try execute(command: tap)
161+
162+
XCTAssertEqual(response.data?.message, "tapped", String(describing: response.error))
163+
XCTAssertFalse(didRecordXCTestFailure(since: failures))
164+
// The app's own delegate callbacks, not the accessibility tree: without this, "nothing recorded a
165+
// failure" would also be satisfied by a gesture that never reached the field at all.
166+
XCTAssertEqual(
167+
app.staticTexts["agent-device-text-entry-focus"].label,
168+
"focus",
169+
"the tap must have taken focus from the field it resolved"
170+
)
171+
XCTAssertNotNil(
172+
textEntryTapWitness,
173+
"the tap still authorizes the next bare type from what it knew before dispatching"
174+
)
175+
}
176+
177+
/// The same invariant on the selector route, which read the element three times after its gesture: the
178+
/// type again inside `rememberTextEntryTap`, then the type and the frame in the readiness wait. On this
179+
/// fixture the route's identifier-bound handle stops resolving once the field hides, so each of those
180+
/// recorded a failure the command inherits. `click @ref` and `press <x> <y>` in the report both reached
181+
/// the coordinate route above, so this holds the second route to the same rule rather than reproducing
182+
/// the reported lookup.
183+
@MainActor
184+
func testSelectorTapOnInputThatLeavesTheTreeOnFocusRecordsNoFailure() throws {
185+
app.launchArguments = [
186+
"--agent-device-text-entry-regression",
187+
"--agent-device-text-entry-unqueryable-on-focus",
188+
]
189+
app.launch()
190+
defer {
191+
invalidateCachedTarget(reason: "unit_test_cleanup")
192+
app.terminate()
193+
}
194+
XCTAssertTrue(app.waitForExistence(timeout: appExistenceTimeout))
195+
XCTAssertTrue(
196+
app.textFields["agent-device-hardware-keyboard-input"].waitForExistence(timeout: appExistenceTimeout)
197+
)
198+
mainOwned.app = app
199+
mainOwned.bundleId = "com.callstack.agentdevice.runner"
200+
mainOwned.processIdentifier = try XCTUnwrap(Self.processIdentifier(of: app))
201+
202+
let failures = currentXCTestFailureCount()
203+
let tap = try runnerCommandFixture(
204+
#"{"appBundleId":"com.callstack.agentdevice.runner","command":"tap","commandId":"selector-tap-unqueryable-on-focus","selectorKey":"id","selectorValue":"agent-device-hardware-keyboard-input","synthesized":true}"#
205+
)
206+
let response = try execute(command: tap)
207+
208+
XCTAssertEqual(response.data?.message, "tapped", String(describing: response.error))
209+
XCTAssertFalse(didRecordXCTestFailure(since: failures))
210+
XCTAssertEqual(
211+
app.staticTexts["agent-device-text-entry-focus"].label,
212+
"focus",
213+
"the tap must have taken focus from the field it resolved"
214+
)
215+
XCTAssertNotNil(textEntryTapWitness)
216+
}
117217
#endif
118218
}
119219
#endif

0 commit comments

Comments
 (0)