Skip to content

Commit 50e8335

Browse files
thymikeeclaude
andcommitted
test(ios): guard that the device back fling queries no windows in the gap, and word the device timing from its measurement
Co-Authored-By: Claude Opus 5.5 <[email protected]>
1 parent a3d69c2 commit 50e8335

5 files changed

Lines changed: 50 additions & 11 deletions

File tree

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -173,11 +173,11 @@ extension RunnerTests {
173173
}
174174

175175
/// The back gesture a device takes: a hold-free 200 ms fling from 2 pt inside the leading edge to
176-
/// 0.9 of the width at mid-height, synthesized on the context's screen route without reading the
177-
/// tree (#4221). On an iPhone 17 Pro it popped Settings > General and a header-less React Native
176+
/// 0.9 of the width at mid-height, synthesized on the context's screen route, so it is safe on a
177+
/// tree XCTest cannot serialize (#4221). On an iPhone 17 Pro it popped Settings > General and a header-less React Native
178178
/// screen 5 times in 5 each, where XCTest's press-and-drag, which holds at the edge for at least
179-
/// 0.1 s and then moves at 500 px/s, missed 2 in 10. The plan is resolved here, so the returned
180-
/// dispatch runs only the fling.
179+
/// 0.1 s and then moves at 500 px/s, missed 2 in 10. The plan and every tree and window read it
180+
/// needs are resolved here, so the returned dispatch runs only the fling.
181181
static let deviceBackFlingDurationMs = 200.0
182182

183183
@MainActor

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/UnitTests/RunnerTests+NavigationTests.swift‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -128,11 +128,11 @@ extension RunnerTests {
128128
XCTAssertGreaterThanOrEqual(slowCaptures, 4, "settle, just-before, and after samples were all slowed")
129129
}
130130

131-
/// A device's sample cost: every capture takes 0.4 s and the contact runs 0.2 s, as a physical
132-
/// iPhone's capture and fling do. The still span grows with the capture, so the pop reads as
133-
/// performed.
131+
/// A device's timing on the simulator's edge touch: every capture takes 0.4 s and the contact is
132+
/// delayed to 0.2 s, as long as a physical iPhone's capture and fling. The still span grows with the
133+
/// capture, so the pop reads as performed.
134134
@MainActor
135-
func testInAppBackWithSlowCapturesAndAFlingLongContactIsPerformed() throws {
135+
func testInAppBackWithSlowCapturesAndALongerContactIsPerformed() throws {
136136
let status = try launchBackFixture(["--agent-device-back-regression"])
137137
defer { endBackFixture() }
138138
let restoreCapture = try slowScreenshots(by: 0.4) {}

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/UnitTests/RunnerTests+SynthesizedInteractionTests.swift‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -265,6 +265,42 @@ extension RunnerTests {
265265
XCTAssertEqual(RunnerCoordinatePressCounter.drags, 0, "the XCTest press-and-drag never ran")
266266
}
267267

268+
/// On a readable tree the device back fling does all its tree and window reads while it is prepared,
269+
/// so the fling itself, which runs after the just-before sample, queries no windows and presses no
270+
/// XCTest coordinate.
271+
@MainActor
272+
func testDeviceBackFlingOnAReadableTreeQueriesNoWindowsWhileItFlings() throws {
273+
app.launch()
274+
mainOwned.app = app
275+
defer {
276+
invalidateCachedTarget(reason: "unit_test_cleanup")
277+
app.terminate()
278+
}
279+
let context = try XCTUnwrap(try synthesizedCoordinateContext(app: app))
280+
guard !context.treeUnreadable else { return XCTFail("the fixture tree must be readable") }
281+
guard case .ready(let fling) = try prepareDeviceBackFling(app: app, context: context) else {
282+
return XCTFail("a readable tree resolves the fling")
283+
}
284+
let selector = NSSelectorFromString("allElementsBoundByIndex")
285+
let query = try XCTUnwrap(class_getInstanceMethod(XCUIElementQuery.self, selector))
286+
let originalQuery = method_getImplementation(query)
287+
var queries = 0
288+
let countedQuery = imp_implementationWithBlock({ (target: AnyObject) -> NSArray in
289+
queries += 1
290+
typealias Query = @convention(c) (AnyObject, Selector) -> NSArray
291+
return unsafeBitCast(originalQuery, to: Query.self)(target, selector)
292+
} as @convention(block) (AnyObject) -> NSArray)
293+
method_setImplementation(query, countedQuery)
294+
defer {
295+
method_setImplementation(query, originalQuery)
296+
imp_removeBlock(countedQuery)
297+
}
298+
299+
guard case .sent = try fling() else { return XCTFail("the fling must be sent") }
300+
301+
XCTAssertEqual(queries, 0, "no windows query runs inside the gap")
302+
}
303+
268304
/// A synthesis that fails before its record is handed to XCTest touched nothing; one that fails
269305
/// after may have run.
270306
func testSynthesisFailureMapsByWhetherTheRecordWasDispatched() {

‎apple/runner/RUNNER_PROTOCOL.md‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -153,11 +153,14 @@ contact, and one more sample, taken right before the swipe, must still match the
153153
| Platform | Resolved before the last sample | Left in the gap | Contact allowance |
154154
| --- | --- | --- | --- |
155155
| Simulator | the edge-touch plan, its display, and the HID touchscreen sender | the 100 ms edge contact | 0.2 s |
156-
| Device | the synthesized fling plan (2 pt inside the leading edge to 0.9 of the width, no hold), which reads no tree | the 200 ms fling | 0.4 s |
156+
| Device | the synthesized fling plan (2 pt inside the leading edge to 0.9 of the width, no hold), with every tree and window read it needs | the 200 ms fling | 0.4 s |
157+
158+
On both, every tree and window read happens during that preparation, before the just-before sample.
157159

158160
Settling takes at most 8 samples, so the wait before the swipe is at most 1.75 s of delays plus
159161
eight captures, then the preparation and the last sample: about 2.6 s with the 0.1 s captures of a
160-
local simulator, and about 5 s with the 0.4 s captures of a physical iPhone. Usually a still screen
162+
local simulator, and about 5 s with the 0.4 s captures measured on one physical iPhone (iPhone 17 Pro,
163+
iOS 27.0.1; https://github.com/callstack/agent-device/pull/4460#issuecomment-6828546541). Usually a still screen
161164
settles after 3 samples.
162165

163166
Only a change from the last sample to the sample after the swipe counts, and only when the time

‎website/docs/docs/commands.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -124,7 +124,7 @@ iOS Simulator launch behavior:
124124
Back, orientation, and hardware buttons:
125125

126126
- `back` uses app-owned back navigation: the back keyevent on Android, the visible back control on macOS, and the Siri Remote Menu button on tvOS. `--in-app` names this default explicitly.
127-
- On iOS, `back` is the left-edge back swipe, with or without `--in-app` or `--system`. It succeeds only when the app's screen, drawn as a 48×96 grayscale grid with its top 7 rows left out (in portrait, the status bar and Dynamic Island band, whether the status bar is shown or hidden; in landscape the iPhone status bar is hidden and the Dynamic Island sits at a side edge, outside those rows), differs in more than 1% of the remaining cells by more than 24 of 256 levels. A small spinner usually stays under that; a larger animation can still count as a change. The change counted is the one between the sample taken immediately before the swipe and the one taken about 0.25 s after it, so a screen that changes on its own inside that gap, after it looked still, can be misread as a back navigation; verify with `snapshot` when the screen has auto-advancing content. Before it swipes, it waits for the screen to stop changing: up to 8 samples 0.25 s apart, which is 1.75 s plus eight screen captures: about 2.6 s on a simulator and about 5 s on a physical iPhone.
127+
- On iOS, `back` is the left-edge back swipe, with or without `--in-app` or `--system`. It succeeds only when the app's screen, drawn as a 48×96 grayscale grid with its top 7 rows left out (in portrait, the status bar and Dynamic Island band, whether the status bar is shown or hidden; in landscape the iPhone status bar is hidden and the Dynamic Island sits at a side edge, outside those rows), differs in more than 1% of the remaining cells by more than 24 of 256 levels. A small spinner usually stays under that; a larger animation can still count as a change. The change counted is the one between the sample taken immediately before the swipe and the one taken about 0.25 s after it, so a screen that changes on its own inside that gap, after it looked still, can be misread as a back navigation; verify with `snapshot` when the screen has auto-advancing content. Before it swipes, it waits for the screen to stop changing: up to 8 samples 0.25 s apart, which is 1.75 s plus eight screen captures: about 2.6 s on a simulator, and about 5 s on a physical iPhone, based on the 0.4 s capture measured on one iPhone 17 Pro with iOS 27.0.1 ([PR #4460 evidence](https://github.com/callstack/agent-device/pull/4460#issuecomment-6828546541)).
128128
- `reason: back_swipe_no_change`: the swipe was sent and the screen did not change. Press the visible back or close control.
129129
- `reason: back_swipe_unverified`: the swipe may or may not have navigated, because the screen kept changing before the swipe, the swipe took longer than the screen had been still, or the screen could not be sampled after it. Check with `snapshot`.
130130
- `reason: back_swipe_not_dispatched`: the swipe was refused before any touch went down, so nothing reached the device.

0 commit comments

Comments
 (0)