Skip to content

Commit 5d4fd44

Browse files
thiagobrezclaude
andcommitted
fix(ios): warm the synthesized-input digitizer before the first gesture
XCTest attaches its HID digitizer lazily, on the first synthesized event a runner process posts. On a cold or loaded simulator that attach can lag several seconds behind the synthesizeWithError call that triggers it. When the first synthesized gesture of a process is timed (a drag with an activation hold, a paced pan), the touch-down then lands seconds into a window whose later samples were scheduled relative to the intended touch-down, so the app reconstructs a malformed gesture even though synthesizeWithError reported success. Once attached, the digitizer stays attached for the runner process, so every later gesture — including after a target relaunch — lands on schedule. Force the one-time attach with a throwaway synthesized contact before the first real gesture. The gesture's own timings are unchanged; the warm-up only moves the unavoidable one-time attach cost off the first user gesture, and is a no-op on a warm host. Runs once per runner process, before the first `gesture` command. Co-Authored-By: Claude Fable 5.1 <[email protected]>
1 parent bd08e6e commit 5d4fd44

4 files changed

Lines changed: 97 additions & 0 deletions

File tree

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandExecution.swift‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2266,6 +2266,10 @@ extension RunnerTests {
22662266
error: ErrorPayload(code: "INVALID_ARGS", message: validationError)
22672267
)
22682268
}
2269+
// Attach the synthesized-input HID digitizer once per process before the first timed gesture,
2270+
// so this gesture's touch-down is not delayed by the one-time attach latency (see
2271+
// RunnerTests+SynthesizedInputWarmup).
2272+
ensureSynthesizedInputWarmed(app: activeApp)
22692273
switch plannedGestureExecution(for: plan) {
22702274
case .fastSwipe:
22712275
// Validation above guarantees a non-empty, single-pointer path for this execution kind.
Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
import XCTest
2+
3+
// One-time warm-up of the private XCTest synthesized-input pipeline.
4+
//
5+
// XCTest attaches its HID digitizer to the system lazily, on the first synthesized event a runner
6+
// process posts. On a warm host that attach is immediate; on a cold or loaded simulator it can lag
7+
// several seconds behind the synthesizeWithError call that triggered it. When the first synthesized
8+
// gesture of a process is a *timed* one (a drag with an activation hold, a paced pan), the touch-down
9+
// then lands seconds into a window whose later samples were scheduled relative to the intended
10+
// touch-down, so the gesture the app reconstructs is malformed even though synthesizeWithError
11+
// reported success. Once the digitizer is attached, every later synthesized gesture in the process —
12+
// including after a target relaunch — lands on schedule.
13+
//
14+
// Absorbing that first-attach latency with a throwaway synthesized contact, before the first real
15+
// gesture runs, keeps the real gesture's timing intact. This is not tolerance widening: the gesture's
16+
// own timings are unchanged; the warm-up only moves the unavoidable one-time attach cost off the
17+
// first user gesture. It is a no-op on a warm host (the attach it forces has already happened).
18+
19+
extension RunnerTests {
20+
/// Whether the synthesized-input pipeline still needs its one-time warm-up. Pure so the
21+
/// once-per-process contract is testable without a live pipeline.
22+
func shouldWarmSynthesizedInput(alreadyWarmed: Bool) -> Bool {
23+
!alreadyWarmed
24+
}
25+
26+
/// The throwaway warm-up contact point: the top-center of the touch reference frame, in the
27+
/// status-bar band. The digitizer attaches on the first synthesis regardless of where the contact
28+
/// lands, so the point is chosen to be inert — the status-bar band does not forward touches to app
29+
/// content — rather than meaningful. Returns nil when the frame is unusable.
30+
func synthesizedInputWarmupPoint(referenceFrame: CGRect) -> CGPoint? {
31+
guard referenceFrame.width > 0, referenceFrame.height > 0 else { return nil }
32+
return CGPoint(x: referenceFrame.midX, y: referenceFrame.minY + 1)
33+
}
34+
35+
/// Best-effort: force the HID digitizer attach once per runner process, before the first real
36+
/// synthesized gesture. Never fails the caller — a warm-up that cannot resolve a frame or that the
37+
/// synthesizer refuses leaves the flag set (a real gesture would hit the same condition) so the
38+
/// warm-up is not retried on every gesture.
39+
func ensureSynthesizedInputWarmed(app: XCUIApplication) {
40+
#if os(iOS)
41+
guard shouldWarmSynthesizedInput(alreadyWarmed: didWarmSynthesizedInput) else { return }
42+
didWarmSynthesizedInput = true
43+
let frame = resolvedTouchReferenceFrame(app: app, appFrame: app.frame)
44+
guard let point = synthesizedInputWarmupPoint(referenceFrame: frame) else {
45+
NSLog("AGENT_DEVICE_RUNNER_SYNTHESIZED_INPUT_WARMUP outcome=skipped reason=no_reference_frame")
46+
return
47+
}
48+
let startedAt = ProcessInfo.processInfo.systemUptime
49+
let outcome = synthesizedTapAt(app: app, x: Double(point.x), y: Double(point.y))
50+
let elapsedMs = (ProcessInfo.processInfo.systemUptime - startedAt) * 1000
51+
switch outcome {
52+
case .performed:
53+
NSLog("AGENT_DEVICE_RUNNER_SYNTHESIZED_INPUT_WARMUP outcome=performed elapsedMs=%.0f", elapsedMs)
54+
case .unsupported(let message, _):
55+
NSLog(
56+
"AGENT_DEVICE_RUNNER_SYNTHESIZED_INPUT_WARMUP outcome=unsupported elapsedMs=%.0f detail=%@",
57+
elapsedMs,
58+
message
59+
)
60+
}
61+
#endif
62+
}
63+
}

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests.swift‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,13 @@ final class RunnerTests: XCTestCase {
4848
var currentApp: XCUIApplication?
4949
var currentBundleId: String?
5050
var currentAppProcessIdentifier: Int?
51+
// Set once, the first time this runner process synthesizes a touch gesture. The private
52+
// XCTest event pipeline attaches its HID digitizer lazily on that first synthesis, and on a
53+
// cold simulator that attach can lag seconds behind the call that triggered it. Because the
54+
// attach is a runner-process/host-level connection rather than app state, it survives target
55+
// relaunches, so warming it once per process is enough. Deliberately not cleared by
56+
// invalidateCachedTarget.
57+
var didWarmSynthesizedInput = false
5158
// iOS does not reliably expose hasKeyboardFocus for a bare type request, especially when
5259
// hardware-keyboard input hides the software keyboard. A successful tap on a concrete text
5360
// input is a scoped witness for the immediately-following bare type; lifecycle and non-text
Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
import XCTest
2+
3+
extension RunnerTests {
4+
#if AGENT_DEVICE_RUNNER_UNIT_TESTS && os(iOS)
5+
func testSynthesizedInputWarmupRunsOnceThenNotAgain() {
6+
XCTAssertTrue(shouldWarmSynthesizedInput(alreadyWarmed: false), "cold process must warm")
7+
XCTAssertFalse(shouldWarmSynthesizedInput(alreadyWarmed: true), "an already-warmed process must not warm again")
8+
}
9+
10+
func testSynthesizedInputWarmupPointSitsInTheStatusBarBandOfTheFrame() throws {
11+
let frame = CGRect(x: 10, y: 20, width: 300, height: 600)
12+
let point = try XCTUnwrap(synthesizedInputWarmupPoint(referenceFrame: frame))
13+
XCTAssertEqual(point.x, frame.midX, accuracy: 0.001, "warm-up contact is horizontally centered")
14+
XCTAssertEqual(point.y, frame.minY + 1, accuracy: 0.001, "warm-up contact sits in the top band, off app content")
15+
XCTAssertTrue(frame.contains(point), "warm-up contact stays inside the reference frame")
16+
}
17+
18+
func testSynthesizedInputWarmupPointIsUnavailableForAnEmptyFrame() {
19+
XCTAssertNil(synthesizedInputWarmupPoint(referenceFrame: .zero))
20+
XCTAssertNil(synthesizedInputWarmupPoint(referenceFrame: CGRect(x: 0, y: 0, width: 0, height: 100)))
21+
}
22+
#endif
23+
}

0 commit comments

Comments
 (0)