Skip to content

Commit b18479a

Browse files
thymikeeclaude
andcommitted
fix(ios): open the sole-screen gesture route on a full-screen iPad app
A deep app tree makes XCTest's windows query fail with kAXErrorIllegalArgument, and the synthesized-gesture route that recovers from it accepted only an iPhone: the runner refused to name a screen it could not prove the app filled, so every tap, long press, scroll, swipe, and back fallback on a hostile-tree iPad app failed. iPad hosts run the same hostile trees as iPhone hosts. RunnerApplicationFillsScreen now answers that proof in-process: it compares the AXRuntime primary app's frame against the system app's frame, the same comparison the Simulator snapshot bridge refuses a windowed capture on (app-window-not-full-screen, ADR 0004). Neither read walks the app's tree. The sole-screen rule keeps one owner (RunnerResolveSoleScreen) and opens on an iPad only while the evidence says fills-screen; a windowed or Stage Manager app, and any unknown, keeps requiring a resolved window. The back fallback point now derives from the screen's own top safe-area inset instead of a frame-height fraction, which sat below the iPad's navigation bar. Co-Authored-By: Claude <[email protected]>
1 parent b08b5e7 commit b18479a

16 files changed

Lines changed: 591 additions & 59 deletions

‎.github/workflows/xctest-nightly.yml‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -146,3 +146,92 @@ jobs:
146146
include-hidden-files: true
147147
path: .tmp/xctest-nightly
148148
if-no-files-found: warn
149+
150+
# The full suite above runs on an iPhone, where the sole-screen rule is decided by the idiom
151+
# alone. The iPad half of the rule — the route opens only while the in-process full-screen
152+
# evidence proves the app fills its screen — can only execute on an iPad. This leg boots one,
153+
# runs the sole-screen tests, and asserts the evidence line the probe prints: no other lane can
154+
# emit it, so a revert to idiom-only routing turns this red instead of silently green.
155+
ipad-sole-screen:
156+
name: iOS Runner Sole-Screen Evidence (iPad)
157+
runs-on: macos-26
158+
timeout-minutes: 30
159+
env:
160+
# Byte-identical inputs to the job above, so both restore the same runner cache entry.
161+
IOS_RUNTIME_VERSION: '26.2'
162+
AGENT_DEVICE_XCUITEST_ARCHS: 'arm64'
163+
AGENT_DEVICE_XCUITEST_INCLUDE_UNIT_TESTS: '1'
164+
AGENT_DEVICE_IOS_RUNNER_DERIVED_PATH: ${{ github.workspace }}/.tmp/ios-runner-derived
165+
steps:
166+
- name: Checkout
167+
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
168+
169+
- name: Setup toolchain
170+
uses: ./.github/actions/setup-node-pnpm
171+
with:
172+
cache-dependency-path: |
173+
pnpm-lock.yaml
174+
examples/test-app/pnpm-lock.yaml
175+
176+
- name: Restore and build iOS XCTest runner
177+
uses: ./.github/actions/setup-apple-runner-build
178+
with:
179+
derived-path: ${{ env.AGENT_DEVICE_IOS_RUNNER_DERIVED_PATH }}
180+
cache-key-prefix: ios-runner-prebuilt
181+
cache-key-suffix: -ios-${{ env.IOS_RUNTIME_VERSION }}-arm64
182+
gate: swift-runner-ios
183+
xcuitest-platform: ios
184+
xcuitest-destination: generic/platform=iOS Simulator
185+
186+
- name: Boot iPad test simulator
187+
id: ipad-simulator
188+
uses: ./.github/actions/boot-ios-test-simulator
189+
with:
190+
runtime-version: ${{ env.IOS_RUNTIME_VERSION }}
191+
preferred-device-name: iPad Pro 11-inch (M4)
192+
193+
- name: Run the sole-screen tests on iPad
194+
run: |
195+
set -euo pipefail
196+
UDID='${{ steps.ipad-simulator.outputs.simulator-udid }}'
197+
# The boot action falls back to any simulator when the preferred name is missing. This
198+
# leg proves an iPad-only rule, so a non-iPad fallback must fail here, not at the grep.
199+
xcrun simctl list devices -j | node -e '
200+
const fs = require("node:fs");
201+
const udid = process.argv[1];
202+
const payload = JSON.parse(fs.readFileSync(0, "utf8"));
203+
const device = Object.values(payload.devices ?? {})
204+
.flat()
205+
.find((entry) => entry.udid === udid);
206+
if (!device?.name?.startsWith("iPad")) {
207+
console.error(`Booted ${device?.name ?? udid}; this leg needs an iPad simulator`);
208+
process.exit(1);
209+
}
210+
' "$UDID"
211+
XCTESTRUN_PATH="$(find "$AGENT_DEVICE_IOS_RUNNER_DERIVED_PATH/Build/Products" -maxdepth 1 -name '*.xctestrun' -print -quit)"
212+
test -n "$XCTESTRUN_PATH"
213+
mkdir -p .tmp/xctest-nightly-ipad
214+
xcodebuild test-without-building \
215+
-xctestrun "$XCTESTRUN_PATH" \
216+
-destination "platform=iOS Simulator,id=${{ steps.ipad-simulator.outputs.simulator-udid }}" \
217+
-only-testing:AgentDeviceRunnerUITests/RunnerTests/testLongPressOnAnUnserializableTreeIsSynthesizedOnTheSoleScreen \
218+
-only-testing:AgentDeviceRunnerUITests/RunnerTests/testGestureViewportOnAnUnserializableTreeIsTheSoleScreen \
219+
-only-testing:AgentDeviceRunnerUITests/RunnerTests/testSoleScreenDisplayIDIsNamedOnlyForAnAppThatFillsItsScreen \
220+
-only-testing:AgentDeviceRunnerUITests/RunnerTests/testInAppBackVerifiesSoleScreenWhenXCTestCannotReadFramesOrEnumerate \
221+
-only-testing:AgentDeviceRunnerUITests/RunnerTests/testInAppBackStillAsksForWindowsWhereNoSoleScreenAnswers \
222+
-only-testing:AgentDeviceRunnerUITests/RunnerTests/testRankedInAppBackUsesSynthesisWithoutXCTestTap \
223+
2>&1 | tee .tmp/xctest-nightly-ipad/runner.log
224+
# The probe prints this line only when a non-phone idiom asks it. Its absence means the
225+
# iPad never probed (a revert to idiom-only routing), not that a test failed.
226+
grep -q 'AGENT_DEVICE_RUNNER_FULL_SCREEN_EVIDENCE result=fills-screen' \
227+
.tmp/xctest-nightly-ipad/runner.log \
228+
|| { echo 'The iPad sole-screen route never produced full-screen evidence.'; exit 1; }
229+
230+
- name: Upload iPad log
231+
if: always()
232+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
233+
with:
234+
name: xctest-nightly-ipad-${{ github.run_id }}-${{ github.run_attempt }}
235+
include-hidden-files: true
236+
path: .tmp/xctest-nightly-ipad
237+
if-no-files-found: warn

‎apple/runner/AgentDeviceRunner/AgentDeviceRunner/AgentDeviceRunnerApp.m‎

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -661,17 +661,36 @@ - (void)viewDidLoad {
661661
self.navigationBackStatus = label;
662662
label.accessibilityIdentifier = @"agent-device-back-status";
663663
if ([NSProcessInfo.processInfo.arguments containsObject:@"--navigation-bar"]) {
664-
UINavigationBar *bar = [[UINavigationBar alloc] initWithFrame:CGRectMake(0, 62, self.view.bounds.size.width, 44)];
664+
// Same band rule as the plain fixture below: the bar sits under the screen's own top inset,
665+
// which differs by family (59 pt on an iPhone 17 Pro, 32 pt on an iPad Pro), so the bar
666+
// anchors to the safe-area top instead of a hardcoded iPhone y.
667+
UINavigationBar *bar = [[UINavigationBar alloc] init];
668+
bar.translatesAutoresizingMaskIntoConstraints = NO;
665669
UINavigationItem *item = [[UINavigationItem alloc] initWithTitle:@"Thread"];
666670
item.leftBarButtonItem = [[UIBarButtonItem alloc] initWithTitle:@"Go back" style:UIBarButtonItemStylePlain target:self action:@selector(navigationBackPressed)];
667671
bar.items = @[item];
668672
[self.view addSubview:bar];
673+
[NSLayoutConstraint activateConstraints:@[
674+
[bar.topAnchor constraintEqualToAnchor:self.view.safeAreaLayoutGuide.topAnchor],
675+
[bar.leadingAnchor constraintEqualToAnchor:self.view.leadingAnchor],
676+
[bar.trailingAnchor constraintEqualToAnchor:self.view.trailingAnchor],
677+
[bar.heightAnchor constraintEqualToConstant:44],
678+
]];
669679
} else {
680+
// The bar band sits under the screen's own top inset, which differs by family (59 pt on an
681+
// iPhone 17 Pro, 32 pt on an iPad Pro). Pinning the control to the safe-area top keeps this
682+
// fixture inside the band the fallback aims at, on either idiom.
670683
UIButton *back = [UIButton buttonWithType:UIButtonTypeSystem];
671-
back.frame = CGRectMake(12, 62, 88, 44);
684+
back.translatesAutoresizingMaskIntoConstraints = NO;
672685
[back setTitle:@"Go back" forState:UIControlStateNormal];
673686
[back addTarget:self action:@selector(navigationBackPressed) forControlEvents:UIControlEventTouchUpInside];
674687
[self.view addSubview:back];
688+
[NSLayoutConstraint activateConstraints:@[
689+
[back.leadingAnchor constraintEqualToAnchor:self.view.safeAreaLayoutGuide.leadingAnchor constant:12],
690+
[back.topAnchor constraintEqualToAnchor:self.view.safeAreaLayoutGuide.topAnchor],
691+
[back.widthAnchor constraintEqualToConstant:88],
692+
[back.heightAnchor constraintEqualToConstant:44],
693+
]];
675694
}
676695
}
677696

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerAppScreenCapture.swift‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -133,11 +133,12 @@ extension RunnerTests {
133133
}
134134

135135
/// Samples the same sole-screen route that synthesized gestures use when the app tree refuses
136-
/// its window. This route cannot name a screen on an iPad or a multi-screen iPhone.
137-
func captureSoleAppScreen() -> Result<CapturedAppScreen, RunnerAppScreenCaptureFailure> {
136+
/// its window. On an iPad the route opens only while the app provably fills its sole screen, so
137+
/// a windowed app keeps refusing instead of sampling the wrong frame.
138+
func captureSoleAppScreen(app: XCUIApplication) -> Result<CapturedAppScreen, RunnerAppScreenCaptureFailure> {
138139
var screen: AnyObject?
139140
var displayID: UInt = 0
140-
guard RunnerResolveSoleScreen(&screen, &displayID) == nil,
141+
guard RunnerResolveSoleScreen(app, &screen, &displayID) == nil,
141142
let captureScreen = screen as? XCUIScreen
142143
else {
143144
return .failure(.unresolvedScreen)

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerSynthesizedGesture.m‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -333,7 +333,7 @@ static RunnerTapSynthesisStatus RunnerTrySynthesizeTap(
333333
NSUInteger displayID = 0;
334334
// A nil window is the caller's sole-screen route: the app tree could not name its window.
335335
NSString *displayError = resolvedWindow == nil
336-
? RunnerResolveSoleScreenDisplayID(&displayID)
336+
? RunnerResolveSoleScreenDisplayID(application, &displayID)
337337
: RunnerResolveWindowDisplayID(resolvedWindow, &displayID);
338338
if (displayError != nil) return displayError;
339339
NSInteger interfaceOrientation =

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Navigation.swift‎

Lines changed: 43 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -179,18 +179,52 @@ extension RunnerTests {
179179
return candidate.midY >= window.minY && candidate.midY <= maxY
180180
}
181181

182-
static func topLeadingNavigationFallbackPoint(in frame: CGRect) -> CGPoint? {
182+
/// The point the top-leading `back` fallback aims at: the leading slot of the bar that sits
183+
/// directly under the top inset of the frame the tap lands in. A fraction of the frame height
184+
/// cannot serve both families — an iPhone 17 Pro measures a 59-pt inset with its bar at y 62-116,
185+
/// while an iPad Pro 11" measures 32 with its bar at y 32-86, where the iPhone's height-derived 96
186+
/// lands below the bar. `22` is half of UIKit's standard 44-pt top bar. The screen's inset counts
187+
/// only when the frame starts at the screen's own top (the sole-screen route and a maximized
188+
/// window); a window that starts below the screen top places its bar at its own top instead. The
189+
/// bounds keep a runtime that reports no inset inside a plausible bar, and a large inset from
190+
/// leaving it.
191+
static func topLeadingNavigationFallbackPoint(
192+
in frame: CGRect,
193+
topSafeAreaInset: CGFloat
194+
) -> CGPoint? {
183195
guard SnapshotGeometry.isPositiveFinite(frame) else {
184196
return nil
185197
}
186198
// Aim at the standard leading navigation slot, bounded for compact and tablet widths.
187199
let xOffset = min(max(frame.width * 0.08, 28), 44)
188-
// Sit below the status/dynamic-island region, inside the bar's leading control: on a 402x874
189-
// iPhone the bar spans y 62-116 and its back control 62-106.
190-
let yOffset = min(max(frame.height * 0.1, 56), 96)
200+
// A screen-top frame takes the screen's inset and the band bounds; a window that starts lower
201+
// places its bar at its own top, so the point centers a standard bar there instead.
202+
if frame.minY > 1 {
203+
return CGPoint(x: frame.minX + xOffset, y: frame.minY + 22)
204+
}
205+
let yOffset = min(max(topSafeAreaInset + 22, 56), 96)
191206
return CGPoint(x: frame.minX + xOffset, y: frame.minY + yOffset)
192207
}
193208

209+
/// The screen's top inset, read in-process from the runner's own window scene. The status-bar /
210+
/// Dynamic Island band is a screen-level fact, so the runner's full-screen host window reports the
211+
/// one the target app sees — and unlike any element frame it answers on a tree XCTest cannot
212+
/// serialize. The largest top inset among the scene's windows wins: an alert or keyboard window
213+
/// reports none, and the window that carries the band is the one to ask.
214+
private func screenTopSafeAreaInset() -> CGFloat {
215+
#if os(iOS)
216+
let insets = UIApplication.shared.connectedScenes
217+
.compactMap { $0 as? UIWindowScene }
218+
.flatMap(\.windows)
219+
.map(\.safeAreaInsets.top)
220+
let inset = insets.max() ?? 0
221+
NSLog("AGENT_DEVICE_RUNNER_BACK_FALLBACK_TOP_INSET read=%.1f windows=%zd", inset, insets.count)
222+
return inset
223+
#else
224+
return 0
225+
#endif
226+
}
227+
194228
@MainActor
195229
private func tapTopLeadingNavigationFallback(
196230
app: XCUIApplication,
@@ -201,7 +235,10 @@ extension RunnerTests {
201235
app: app,
202236
policy: synthesizedGesturePolicy(.coordinateTap),
203237
windowQueryRejected: windowQueryRejected
204-
), let point = Self.topLeadingNavigationFallbackPoint(in: context.referenceFrame) else {
238+
), let point = Self.topLeadingNavigationFallbackPoint(
239+
in: context.referenceFrame,
240+
topSafeAreaInset: screenTopSafeAreaInset()
241+
) else {
205242
return .unavailable
206243
}
207244
let before = captureNavigationFallbackVisualState(app: app, context: context)
@@ -238,7 +275,7 @@ extension RunnerTests {
238275
return Self.navigationFallbackSample(
239276
resolvingApp: {
240277
context.resolvedWindow == nil
241-
? self.captureSoleAppScreen()
278+
? self.captureSoleAppScreen(app: app)
242279
: self.captureResolvedAppScreen(app: app)
243280
},
244281
systemSurface: { self.captureResolvedAppScreen(app: self.springboard) },

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+SynthesizedInteraction.swift‎

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -342,8 +342,8 @@ extension RunnerTests {
342342
}
343343

344344
/// The viewport the portable gesture planner lays pointer paths in: the frame of the synthesized
345-
/// context its record is routed through, so a tree with no window on an iPhone with one screen
346-
/// plans on that screen, the same route a synthesized tap or scroll takes.
345+
/// context its record is routed through, so a tree with no window whose app provably fills its
346+
/// sole screen plans on that screen, the same route a synthesized tap or scroll takes.
347347
@MainActor
348348
func gestureViewportFrame(app: XCUIApplication) -> CGRect {
349349
#if os(iOS)
@@ -517,14 +517,18 @@ extension RunnerTests {
517517

518518

519519
#if os(iOS)
520-
/// The reference frame of the sole-screen route, read without the app's accessibility tree: on an
521-
/// iPhone with one screen a full-screen app's window is that screen. A deep app tree makes the
522-
/// windows query fail with `kAXErrorIllegalArgument`, and without this route every synthesized
523-
/// tap and scroll on such a screen failed even though synthesis itself needs no tree. Anything
524-
/// that can show an app in part of a screen (iPad, more than one screen) keeps requiring a window.
520+
/// The reference frame of the sole-screen route, read without the app's accessibility tree: when
521+
/// the app provably fills the device's only screen, its window is that screen. A deep app tree
522+
/// makes the windows query fail with `kAXErrorIllegalArgument`, and without this route every
523+
/// synthesized tap and scroll on such a screen failed even though synthesis itself needs no tree.
524+
/// `RunnerResolveSoleScreen` owns which devices prove that — an iPhone always, an iPad only while
525+
/// the app fills the screen — so this reads the same answer the record's routing takes, and
526+
/// anything else (a windowed iPad, a multi-screen foldable) keeps requiring a window.
525527
@MainActor
526528
func soleScreenReferenceFrame(app: XCUIApplication) -> CGRect? {
527-
guard UIDevice.current.userInterfaceIdiom == .phone, XCUIScreen.screens.count == 1 else {
529+
var resolvedScreen: AnyObject?
530+
var resolvedDisplayID: UInt = 0
531+
guard RunnerResolveSoleScreen(app, &resolvedScreen, &resolvedDisplayID) == nil else {
528532
return nil
529533
}
530534
let screen = UIScreen.main

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerXCTestEventBridge.h‎

Lines changed: 45 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
#import <Foundation/Foundation.h>
2+
#import <CoreGraphics/CoreGraphics.h>
23

34
NS_ASSUME_NONNULL_BEGIN
45

@@ -79,17 +80,57 @@ FOUNDATION_EXPORT NSString * _Nullable RunnerResolveWindowDisplayID(
7980

8081
// Reads the display ID of the device's only screen, for a gesture whose app window XCTest cannot
8182
// resolve because the app's accessibility tree is too deep to serialize. On an iPhone with exactly
82-
// one screen that screen hosts the full-screen app, so no window is needed to name it. Anywhere else
83-
// this returns a message and leaves `displayID` untouched: an iPad can show the app in part of its
84-
// screen, and only a window can say which panel lights a foldable's app.
85-
FOUNDATION_EXPORT NSString * _Nullable RunnerResolveSoleScreenDisplayID(NSUInteger *displayID);
83+
// one screen that screen hosts the full-screen app, so no window is needed to name it. On an iPad
84+
// the same route opens only while the app provably fills that screen, decided in-process by the
85+
// evidence seam below without the app's tree. Anywhere else this returns a message and leaves
86+
// `displayID` untouched: a windowed iPad app sits in part of its screen, and only a window can say
87+
// which panel lights a foldable's app.
88+
FOUNDATION_EXPORT NSString * _Nullable RunnerResolveSoleScreenDisplayID(
89+
id application,
90+
NSUInteger *displayID
91+
);
8692

8793
// Resolves the screen and display identity under the same sole-screen rule used by synthesis.
8894
FOUNDATION_EXPORT NSString * _Nullable RunnerResolveSoleScreen(
95+
id application,
8996
id _Nullable *screen,
9097
NSUInteger *displayID
9198
);
9299

100+
// What the full-screen proof concluded. The decision is a function of these values, so a test
101+
// supplies them directly instead of restating the device rule.
102+
typedef NS_ENUM(NSUInteger, RunnerFullScreenEvidence) {
103+
// The target app is the accessibility primary app and its frame is the screen's frame.
104+
RunnerFullScreenEvidenceFillsScreen,
105+
// The target app is not the accessibility primary app, so no screen belongs to it to fill.
106+
RunnerFullScreenEvidenceNotPrimaryApp,
107+
// The primary app's frame differs from the system app's frame: an iPad window that shows the app
108+
// in part of the screen (Windowed Apps or Stage Manager).
109+
RunnerFullScreenEvidenceWindowed,
110+
// The runtime, the app, or the system app would not report a usable frame, which proves nothing.
111+
RunnerFullScreenEvidenceUnknown,
112+
};
113+
114+
// Decides the evidence from the values the AXRuntime probe reads. AXRuntime reports the primary
115+
// app's window and the system app, which spans the screen, in one native space; this is the same
116+
// comparison the Simulator snapshot bridge refuses a capture on (`app-window-not-full-screen`,
117+
// ADR 0004). Neither read walks the app's tree, so the decision survives a tree XCTest cannot
118+
// serialize. A pid that cannot name a process (<= 0) or an unreadable frame answers Unknown, never
119+
// a guess.
120+
FOUNDATION_EXPORT RunnerFullScreenEvidence RunnerFullScreenEvidenceFromForeground(
121+
pid_t targetProcessID,
122+
pid_t primaryProcessID,
123+
CGRect appFrame,
124+
CGRect screenFrame
125+
);
126+
127+
// The sole-screen rule with one owner: an iPhone always routes through its only screen; any other
128+
// idiom routes only while the evidence proves the app fills that screen.
129+
FOUNDATION_EXPORT BOOL RunnerSoleScreenRouteOpensForEvidence(
130+
BOOL isPhoneIdiom,
131+
RunnerFullScreenEvidence evidence
132+
);
133+
93134
FOUNDATION_EXPORT NSString * _Nullable RunnerRequireClass(
94135
Class cls,
95136
NSString *className,

0 commit comments

Comments
 (0)