Skip to content

Commit 2965c9c

Browse files
committed
docs(ios): record #2659 spike on reading the app's interface orientation from the AX bridge
Measures all three candidate routes on iPhone 17 Pro (iOS 26.2). Route 1's XC_kAXXCAttributeApplicationOrientation resolves to id 1503 but the guest's snapshot channel returns 0 and XCTest's own reader fails kAXErrorServerNotFound; route 3's only working read (BKHIDServicesGetCurrentDeviceOrientation) is device orientation, which diverges from the foreground app's interface orientation on a rotation-locked app; route 2 is a keyboard-band heuristic. No cheap, reliable interface-orientation fact is available to the guest, so #2653's detect-and-refuse layer and the ADR 0004 refusal stand.
1 parent 91652a8 commit 2965c9c

1 file changed

Lines changed: 200 additions & 0 deletions

File tree

Lines changed: 200 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,200 @@
1+
# #2659 spike: can the Simulator AX bridge guest read the app's interface orientation?
2+
3+
Time-boxed spike, 2026-09-18, on iPhone 17 Pro (iOS 26.2 Simulator, UDID
4+
5AF10197-87C1-4799-835E-3C6CBF9F3163), macOS host with Xcode 26.2. The one
5+
question: **can the Simulator AX bridge guest learn the foreground app's
6+
`UIInterfaceOrientation` (1 portrait, 2 upside-down, 3 landscape-right, 4
7+
landscape-left) cheaply and reliably at capture time**, so the bridge can
8+
normalize a quarter-turned subtree the way the runner does and the
9+
detection-only layer from PR #2653 comes out.
10+
11+
Method: throwaway Objective-C guests compiled exactly like the bridge
12+
(`xcrun --sdk iphonesimulator clang -arch arm64 -mios-simulator-version-min=15.0
13+
-fobjc-arc`), spawned with `xcrun simctl spawn <udid> <bin> <app-pid>`, that
14+
dlopen the same `AXRuntime` / `XCTAutomationSupport` the bridge dlopens and read
15+
orientation through each candidate route. Orientation was forced with
16+
`agent-device orientation <portrait|landscape-left|landscape-right>` (which drives
17+
`XCUIDevice.shared.orientation` and asserts the observed device orientation),
18+
so each reading is taken on a screen whose orientation is known. The subject app
19+
is `com.callstack.agentdevicelab` (`examples/test-app`, supports all four
20+
orientations). None of this probe code is merged; the essential source is in the
21+
Appendix.
22+
23+
## Verdict
24+
25+
**No.** No route gives the app's `UIInterfaceOrientation` cheaply *and* reliably
26+
from the guest. The attribute the spike hoped for exists and resolves, but the
27+
guest's snapshot channel returns 0 for it and XCTest's own reader errors out;
28+
the one cheap service read returns the *device* orientation, which is the wrong
29+
quantity and provably diverges from the app's interface orientation. The
30+
detect-and-refuse layer #2653 added on the bridge side, and the ADR 0004 "refuse
31+
the screen" decision, stay. The delete-the-layer follow-up plan is **not**
32+
unlocked.
33+
34+
## Route 1 — an AX attribute in the bridge's own vocabulary
35+
36+
Candidate names were fed through the same resolver the bridge already uses,
37+
`XCAXAccessibilityAttributesForStringAttributes`, one name at a time:
38+
39+
| name | resolves? | attribute id |
40+
|---|---|---|
41+
| `XC_kAXXCAttributeApplicationOrientation` | yes | **1503** |
42+
| `XC_kAXXCAttributeOrientation` | no | -1 |
43+
| `XC_kAXXCAttributeInterfaceOrientation` | no | -1 |
44+
| `XC_kAXXCAttributeFrame` (control) | yes | 2003 |
45+
46+
The string `XC_kAXXCAttributeApplicationOrientation` is real in
47+
`XCTAutomationSupport`'s table, next to `appOrientationForElement:error:` and
48+
"Fetching application interface orientation". So a name exists. The value does
49+
not reach the guest:
50+
51+
- Requesting attribute 1503 in `userTestingSnapshotForElement:options:error:` —
52+
the exact call the bridge uses — returns **0 (`UIInterfaceOrientationUnknown`)**
53+
on the `UIApplication` root and on every window, in portrait, landscape-left,
54+
and landscape-right, while the sibling `...Frame` attribute in the *same*
55+
snapshot correctly reports `(0,0,402,874)` in portrait and `(0,0,874,402)` in
56+
landscape. The snapshot channel knows the geometry but not the orientation.
57+
58+
- XCTest's own reader `-[XCTAccessibilityFramework appOrientationForElement:error:]`
59+
— the very class the bridge instantiates via `initForRemoteAccess` — fails from
60+
a guest: with the pid's `XCAccessibilityElement` it returns `0` and
61+
`Error getting app orientation kAXErrorServerNotFound`; with the raw
62+
`AXUIElement` it returns `0` with no error. The runner reads a real value only
63+
because it holds a live XCUIApplication session; the bridge's remote-access
64+
channel cannot ask for that attribute and get it back.
65+
66+
Per-read cost was irrelevant (the failing reads run in ~0.04 ms) because no value
67+
is produced.
68+
69+
## Route 2 — inference from the tree already captured
70+
71+
Reached only because routes 1 and 3 (checked first, in cheapness order) returned
72+
no fact. The signal route 2 would use is present and matches the fixture: in
73+
landscape the app's own `UIWindow`/`UITextEffectsWindow` report `(0,0,874,402)`
74+
while `UIRemoteKeyboardWindow` (and, in the same capture, `UIInputSetContainerView`)
75+
report the quarter-turned `(0,0,402,874)`. That says "some surface arrived turned"
76+
but not "the app is at interface orientation N": it is an assumption about docked
77+
keyboards, it is blind with no keyboard up, it cannot name an arbitrary rotated
78+
surface, and it cannot tell portrait from portrait-upside-down. Fallback material
79+
only, and weaker than routes 1 and 3 precisely because those were checked first
80+
and a heuristic is all that is left once they return no fact.
81+
82+
## Route 3 — a SpringBoard / BackBoard service call
83+
84+
Symbols checked with `nm -gU` on the iOS 26.2 Simulator runtime binaries and
85+
confirmed with `dlopen`/`dlsym` from inside the guest.
86+
87+
- **SpringBoardServices**: `SBSGetActiveInterfaceOrientation` is not exported
88+
under that C name. `SBGetInterfaceOrientation` and the MIG
89+
`_SBSGetActiveInterfaceOrientation` resolve, but called as `long (void)` they
90+
return a fixed **`0x10000003`** in every orientation — not a
91+
`UIInterfaceOrientation`. Usable as-is: no.
92+
- **BackBoardServices**: `BKHIDServicesGetCurrentDeviceOrientation` works and is
93+
cheap — median **0.014–0.015 ms** over 11 warm reads (≈ a cached read, far
94+
below a Mach round trip). It tracks the **device** orientation
95+
(`UIDeviceOrientation`), not the app's interface orientation.
96+
97+
What the guest actually measures, beside the foreground app's own root window in
98+
the same capture:
99+
100+
| screen (set via the runner) | device read `BKHIDServices…` | test-app root window |
101+
|---|---|---|
102+
| portrait | 1 | `(0,0,402,874)` |
103+
| device landscape-left | 3 | `(0,0,874,402)` |
104+
| device landscape-right | 4 | `(0,0,874,402)` |
105+
106+
The test-app supports every orientation and autorotates, so its *interface*
107+
orientation is the mirror of the device orientation, and because
108+
`UIDeviceOrientation` and `UIInterfaceOrientation` give the mirrored landscape pair
109+
the same integers (device landscapeLeft 3 ↔ interface landscapeRight 3; device
110+
landscapeRight 4 ↔ interface landscapeLeft 4), the device integer happens to equal
111+
the runner's `interfaceOrientation` integer here. (The interface integer is that
112+
documented mapping, not a value the spike printed — no host surface exposes the
113+
runner's read; the window column is the measured geometry, which cannot tell
114+
landscape-left from landscape-right on its own.) It reads the wrong quantity, and
115+
that bites:
116+
117+
- With a **rotation-locked** app foreground — `com.apple.Preferences`, whose root
118+
reports `SwiftUIApplication` — and the device rotated to landscape-left, the
119+
device read is **3** while the app's root window stays portrait
120+
**`(0,0,402,874)`**. The device is landscape; the foreground app's interface is
121+
portrait. A bridge that fed 3 into the rotation table would turn a portrait
122+
screen that never turned.
123+
124+
So route 3's only working read is device orientation, which is not the app's
125+
interface orientation and cannot be made to be from a guest (UIKit keeps
126+
per-app interface orientation inside the app's own process, which is exactly why
127+
the runner reads it with a live session).
128+
129+
## What this changes
130+
131+
Nothing merges from the spike. The bridge's detect-and-refuse layer #2653 added
132+
stays, because the question it answers — "is this subtree quarter-turned and can I name the
133+
orientation to turn it back?" — still has a "no, and I cannot cheaply name it"
134+
answer on the guest side. The ADR 0004 amendment "A producer that cannot name the
135+
app's interface orientation refuses the screen" is now backed by measurements
136+
rather than by the absence of a check.
137+
138+
If a future attempt wants to unlock the delete-the-layer plan, the bar is a guest
139+
read that (a) survives a rotation-locked foreground app and (b) names the
140+
*foreground app's* interface orientation, not the device's. Route 1's attribute is
141+
the right handle to keep an eye on — if a future bridge ever populates 1503 in the
142+
snapshot the way the runner reads it, `tree.ts` could port
143+
`CoordinateSpaceRotation.oriented(rect:in:interfaceOrientation:)`, pin it against the fixture's
144+
`rotationCases`, and retire `isQuarterTurnedWindowFrame`,
145+
`unresolvedCoordinateSpaceWindows`, `window-coordinate-space-unresolved`, and the
146+
ADR 0004 refusal decision. That handle does not work today.
147+
148+
## Out of scope / not covered
149+
150+
- **Physical devices.** The AX bridge is Simulator-only; nothing here speaks to
151+
the `usbmux`/`network` runner path.
152+
- The runner's literal `interfaceOrientationForApplication:` integer was not read
153+
from the host side — no command or snapshot field exposes it. Interface-space
154+
truth was witnessed by the captured window geometry and by the rotation-locked
155+
Settings divergence above, not by a printed runner value.
156+
- A rotation-locked *React Native* app was not measured directly; `examples/test-app`
157+
autorotates, so it cannot itself show the device/interface split. The Settings
158+
case demonstrates that class of divergence on a system app.
159+
- `SBGetInterfaceOrientation` was called as `long (void)`; a display-id or
160+
out-param signature was not chased, but its constant return across three known
161+
orientations rules it out as a plain interface-orientation source regardless.
162+
163+
## Appendix — throwaway probe (not merged)
164+
165+
`probe.m` spawned via `xcrun simctl spawn <udid> probe <pid>`; the load-bearing
166+
pieces:
167+
168+
```objc
169+
// Route 1: resolve the name the bridge's own resolver understands.
170+
AttributeNumbersForNamesFn attrForNames = dlsym(RTLD_DEFAULT, "XCAXAccessibilityAttributesForStringAttributes");
171+
// attrForNames(@[@"XC_kAXXCAttributeApplicationOrientation"]) -> @[1503]
172+
173+
// Route 1: read it back off the app-root snapshot (same call the bridge uses).
174+
// attributes=[1503,2003], maxDepth=1 -> root attributes[@1503] == 0 in every orientation.
175+
176+
// Route 1: XCTest's own reader, same class the bridge instantiates.
177+
SEL ao = NSSelectorFromString(@"appOrientationForElement:error:");
178+
long v = ((long(*)(id,SEL,id,NSError**))objc_msgSend)(framework, ao, elementOrRaw, &err);
179+
// element -> 0 + kAXErrorServerNotFound; raw -> 0.
180+
181+
// Route 3: the cheap device-orientation read (device space, not interface space).
182+
void *bbs = dlopen(".../BackBoardServices.framework/BackBoardServices", RTLD_NOW);
183+
long(*get)(void) = dlsym(bbs, "BKHIDServicesGetCurrentDeviceOrientation"); // portrait=1, LL=3, LR=4
184+
```
185+
186+
Force the screen with the built CLI against an isolated state dir, then read:
187+
188+
```bash
189+
agent-device orientation landscape-left --session spikesess --platform ios --udid <udid> --state-dir <dir>
190+
xcrun simctl spawn <udid> .../probe <app-pid>
191+
```
192+
193+
Symbol discovery that shaped the candidates (host-side):
194+
195+
```bash
196+
RR="/Library/Developer/CoreSimulator/Volumes/iOS_23C54/.../iOS 26.2.simruntime/Contents/Resources/RuntimeRoot"
197+
strings -a "$RR/Developer/Library/PrivateFrameworks/XCTAutomationSupport.framework/XCTAutomationSupport" | grep -i orientation # -> XC_kAXXCAttributeApplicationOrientation
198+
nm -gU "$RR/System/Library/PrivateFrameworks/SpringBoardServices.framework/SpringBoardServices" | grep -i orient # -> SBGetInterfaceOrientation, _SBSGetActiveInterfaceOrientation
199+
nm -gU "$RR/System/Library/PrivateFrameworks/BackBoardServices.framework/BackBoardServices" | grep -i orient # -> BKHIDServicesGetCurrentDeviceOrientation
200+
```

0 commit comments

Comments
 (0)