|
| 1 | +# #2659 spike: can the Simulator AX bridge guest read the app's interface orientation? |
| 2 | + |
| 3 | +Time-boxed spike, 2026-09-18, on iPhone 17 Pro (iOS 26.2 Simulator, UDID |
| 4 | +5AF10197-87C1-4799-835E-3C6CBF9F3163), macOS host with Xcode 26.2. The one |
| 5 | +question: **can the Simulator AX bridge guest learn the foreground app's |
| 6 | +`UIInterfaceOrientation` (1 portrait, 2 upside-down, 3 landscape-right, 4 |
| 7 | +landscape-left) cheaply and reliably at capture time**, so the bridge can |
| 8 | +normalize a quarter-turned subtree the way the runner does and the |
| 9 | +detection-only layer from PR #2653 comes out. |
| 10 | + |
| 11 | +Method: throwaway Objective-C guests compiled exactly like the bridge |
| 12 | +(`xcrun --sdk iphonesimulator clang -arch arm64 -mios-simulator-version-min=15.0 |
| 13 | +-fobjc-arc`), spawned with `xcrun simctl spawn <udid> <bin> <app-pid>`, that |
| 14 | +dlopen the same `AXRuntime` / `XCTAutomationSupport` the bridge dlopens and read |
| 15 | +orientation through each candidate route. Orientation was forced with |
| 16 | +`agent-device orientation <portrait|landscape-left|landscape-right>` (which drives |
| 17 | +`XCUIDevice.shared.orientation` and asserts the observed device orientation), |
| 18 | +so each reading is taken on a screen whose orientation is known. The subject app |
| 19 | +is `com.callstack.agentdevicelab` (`examples/test-app`, supports all four |
| 20 | +orientations). None of this probe code is merged; the essential source is in the |
| 21 | +Appendix. |
| 22 | + |
| 23 | +## Verdict |
| 24 | + |
| 25 | +**No.** No route gives the app's `UIInterfaceOrientation` cheaply *and* reliably |
| 26 | +from the guest. The attribute the spike hoped for exists and resolves, but the |
| 27 | +guest's snapshot channel returns 0 for it and XCTest's own reader errors out; |
| 28 | +the one cheap service read returns the *device* orientation, which is the wrong |
| 29 | +quantity and provably diverges from the app's interface orientation. The |
| 30 | +detect-and-refuse layer #2653 added on the bridge side, and the ADR 0004 "refuse |
| 31 | +the screen" decision, stay. The delete-the-layer follow-up plan is **not** |
| 32 | +unlocked. |
| 33 | + |
| 34 | +## Route 1 — an AX attribute in the bridge's own vocabulary |
| 35 | + |
| 36 | +Candidate names were fed through the same resolver the bridge already uses, |
| 37 | +`XCAXAccessibilityAttributesForStringAttributes`, one name at a time: |
| 38 | + |
| 39 | +| name | resolves? | attribute id | |
| 40 | +|---|---|---| |
| 41 | +| `XC_kAXXCAttributeApplicationOrientation` | yes | **1503** | |
| 42 | +| `XC_kAXXCAttributeOrientation` | no | -1 | |
| 43 | +| `XC_kAXXCAttributeInterfaceOrientation` | no | -1 | |
| 44 | +| `XC_kAXXCAttributeFrame` (control) | yes | 2003 | |
| 45 | + |
| 46 | +The string `XC_kAXXCAttributeApplicationOrientation` is real in |
| 47 | +`XCTAutomationSupport`'s table, next to `appOrientationForElement:error:` and |
| 48 | +"Fetching application interface orientation". So a name exists. The value does |
| 49 | +not reach the guest: |
| 50 | + |
| 51 | +- Requesting attribute 1503 in `userTestingSnapshotForElement:options:error:` — |
| 52 | + the exact call the bridge uses — returns **0 (`UIInterfaceOrientationUnknown`)** |
| 53 | + on the `UIApplication` root and on every window, in portrait, landscape-left, |
| 54 | + and landscape-right, while the sibling `...Frame` attribute in the *same* |
| 55 | + snapshot correctly reports `(0,0,402,874)` in portrait and `(0,0,874,402)` in |
| 56 | + landscape. The snapshot channel knows the geometry but not the orientation. |
| 57 | + |
| 58 | +- XCTest's own reader `-[XCTAccessibilityFramework appOrientationForElement:error:]` |
| 59 | + — the very class the bridge instantiates via `initForRemoteAccess` — fails from |
| 60 | + a guest: with the pid's `XCAccessibilityElement` it returns `0` and |
| 61 | + `Error getting app orientation kAXErrorServerNotFound`; with the raw |
| 62 | + `AXUIElement` it returns `0` with no error. The runner reads a real value only |
| 63 | + because it holds a live XCUIApplication session; the bridge's remote-access |
| 64 | + channel cannot ask for that attribute and get it back. |
| 65 | + |
| 66 | +Per-read cost was irrelevant (the failing reads run in ~0.04 ms) because no value |
| 67 | +is produced. |
| 68 | + |
| 69 | +## Route 2 — inference from the tree already captured |
| 70 | + |
| 71 | +Reached only because routes 1 and 3 (checked first, in cheapness order) returned |
| 72 | +no fact. The signal route 2 would use is present and matches the fixture: in |
| 73 | +landscape the app's own `UIWindow`/`UITextEffectsWindow` report `(0,0,874,402)` |
| 74 | +while `UIRemoteKeyboardWindow` (and, in the same capture, `UIInputSetContainerView`) |
| 75 | +report the quarter-turned `(0,0,402,874)`. That says "some surface arrived turned" |
| 76 | +but not "the app is at interface orientation N": it is an assumption about docked |
| 77 | +keyboards, it is blind with no keyboard up, it cannot name an arbitrary rotated |
| 78 | +surface, and it cannot tell portrait from portrait-upside-down. Fallback material |
| 79 | +only, and weaker than routes 1 and 3 precisely because those were checked first |
| 80 | +and a heuristic is all that is left once they return no fact. |
| 81 | + |
| 82 | +## Route 3 — a SpringBoard / BackBoard service call |
| 83 | + |
| 84 | +Symbols checked with `nm -gU` on the iOS 26.2 Simulator runtime binaries and |
| 85 | +confirmed with `dlopen`/`dlsym` from inside the guest. |
| 86 | + |
| 87 | +- **SpringBoardServices**: `SBSGetActiveInterfaceOrientation` is not exported |
| 88 | + under that C name. `SBGetInterfaceOrientation` and the MIG |
| 89 | + `_SBSGetActiveInterfaceOrientation` resolve, but called as `long (void)` they |
| 90 | + return a fixed **`0x10000003`** in every orientation — not a |
| 91 | + `UIInterfaceOrientation`. Usable as-is: no. |
| 92 | +- **BackBoardServices**: `BKHIDServicesGetCurrentDeviceOrientation` works and is |
| 93 | + cheap — median **0.014–0.015 ms** over 11 warm reads (≈ a cached read, far |
| 94 | + below a Mach round trip). It tracks the **device** orientation |
| 95 | + (`UIDeviceOrientation`), not the app's interface orientation. |
| 96 | + |
| 97 | +What the guest actually measures, beside the foreground app's own root window in |
| 98 | +the same capture: |
| 99 | + |
| 100 | +| screen (set via the runner) | device read `BKHIDServices…` | test-app root window | |
| 101 | +|---|---|---| |
| 102 | +| portrait | 1 | `(0,0,402,874)` | |
| 103 | +| device landscape-left | 3 | `(0,0,874,402)` | |
| 104 | +| device landscape-right | 4 | `(0,0,874,402)` | |
| 105 | + |
| 106 | +The test-app supports every orientation and autorotates, so its *interface* |
| 107 | +orientation is the mirror of the device orientation, and because |
| 108 | +`UIDeviceOrientation` and `UIInterfaceOrientation` give the mirrored landscape pair |
| 109 | +the same integers (device landscapeLeft 3 ↔ interface landscapeRight 3; device |
| 110 | +landscapeRight 4 ↔ interface landscapeLeft 4), the device integer happens to equal |
| 111 | +the runner's `interfaceOrientation` integer here. (The interface integer is that |
| 112 | +documented mapping, not a value the spike printed — no host surface exposes the |
| 113 | +runner's read; the window column is the measured geometry, which cannot tell |
| 114 | +landscape-left from landscape-right on its own.) It reads the wrong quantity, and |
| 115 | +that bites: |
| 116 | + |
| 117 | +- With a **rotation-locked** app foreground — `com.apple.Preferences`, whose root |
| 118 | + reports `SwiftUIApplication` — and the device rotated to landscape-left, the |
| 119 | + device read is **3** while the app's root window stays portrait |
| 120 | + **`(0,0,402,874)`**. The device is landscape; the foreground app's interface is |
| 121 | + portrait. A bridge that fed 3 into the rotation table would turn a portrait |
| 122 | + screen that never turned. |
| 123 | + |
| 124 | +So route 3's only working read is device orientation, which is not the app's |
| 125 | +interface orientation and cannot be made to be from a guest (UIKit keeps |
| 126 | +per-app interface orientation inside the app's own process, which is exactly why |
| 127 | +the runner reads it with a live session). |
| 128 | + |
| 129 | +## What this changes |
| 130 | + |
| 131 | +Nothing merges from the spike. The bridge's detect-and-refuse layer #2653 added |
| 132 | +stays, because the question it answers — "is this subtree quarter-turned and can I name the |
| 133 | +orientation to turn it back?" — still has a "no, and I cannot cheaply name it" |
| 134 | +answer on the guest side. The ADR 0004 amendment "A producer that cannot name the |
| 135 | +app's interface orientation refuses the screen" is now backed by measurements |
| 136 | +rather than by the absence of a check. |
| 137 | + |
| 138 | +If a future attempt wants to unlock the delete-the-layer plan, the bar is a guest |
| 139 | +read that (a) survives a rotation-locked foreground app and (b) names the |
| 140 | +*foreground app's* interface orientation, not the device's. Route 1's attribute is |
| 141 | +the right handle to keep an eye on — if a future bridge ever populates 1503 in the |
| 142 | +snapshot the way the runner reads it, `tree.ts` could port |
| 143 | +`CoordinateSpaceRotation.oriented(rect:in:interfaceOrientation:)`, pin it against the fixture's |
| 144 | +`rotationCases`, and retire `isQuarterTurnedWindowFrame`, |
| 145 | +`unresolvedCoordinateSpaceWindows`, `window-coordinate-space-unresolved`, and the |
| 146 | +ADR 0004 refusal decision. That handle does not work today. |
| 147 | + |
| 148 | +## Out of scope / not covered |
| 149 | + |
| 150 | +- **Physical devices.** The AX bridge is Simulator-only; nothing here speaks to |
| 151 | + the `usbmux`/`network` runner path. |
| 152 | +- The runner's literal `interfaceOrientationForApplication:` integer was not read |
| 153 | + from the host side — no command or snapshot field exposes it. Interface-space |
| 154 | + truth was witnessed by the captured window geometry and by the rotation-locked |
| 155 | + Settings divergence above, not by a printed runner value. |
| 156 | +- A rotation-locked *React Native* app was not measured directly; `examples/test-app` |
| 157 | + autorotates, so it cannot itself show the device/interface split. The Settings |
| 158 | + case demonstrates that class of divergence on a system app. |
| 159 | +- `SBGetInterfaceOrientation` was called as `long (void)`; a display-id or |
| 160 | + out-param signature was not chased, but its constant return across three known |
| 161 | + orientations rules it out as a plain interface-orientation source regardless. |
| 162 | + |
| 163 | +## Appendix — throwaway probe (not merged) |
| 164 | + |
| 165 | +`probe.m` spawned via `xcrun simctl spawn <udid> probe <pid>`; the load-bearing |
| 166 | +pieces: |
| 167 | + |
| 168 | +```objc |
| 169 | +// Route 1: resolve the name the bridge's own resolver understands. |
| 170 | +AttributeNumbersForNamesFn attrForNames = dlsym(RTLD_DEFAULT, "XCAXAccessibilityAttributesForStringAttributes"); |
| 171 | +// attrForNames(@[@"XC_kAXXCAttributeApplicationOrientation"]) -> @[1503] |
| 172 | + |
| 173 | +// Route 1: read it back off the app-root snapshot (same call the bridge uses). |
| 174 | +// attributes=[1503,2003], maxDepth=1 -> root attributes[@1503] == 0 in every orientation. |
| 175 | + |
| 176 | +// Route 1: XCTest's own reader, same class the bridge instantiates. |
| 177 | +SEL ao = NSSelectorFromString(@"appOrientationForElement:error:"); |
| 178 | +long v = ((long(*)(id,SEL,id,NSError**))objc_msgSend)(framework, ao, elementOrRaw, &err); |
| 179 | +// element -> 0 + kAXErrorServerNotFound; raw -> 0. |
| 180 | + |
| 181 | +// Route 3: the cheap device-orientation read (device space, not interface space). |
| 182 | +void *bbs = dlopen(".../BackBoardServices.framework/BackBoardServices", RTLD_NOW); |
| 183 | +long(*get)(void) = dlsym(bbs, "BKHIDServicesGetCurrentDeviceOrientation"); // portrait=1, LL=3, LR=4 |
| 184 | +``` |
| 185 | + |
| 186 | +Force the screen with the built CLI against an isolated state dir, then read: |
| 187 | + |
| 188 | +```bash |
| 189 | +agent-device orientation landscape-left --session spikesess --platform ios --udid <udid> --state-dir <dir> |
| 190 | +xcrun simctl spawn <udid> .../probe <app-pid> |
| 191 | +``` |
| 192 | + |
| 193 | +Symbol discovery that shaped the candidates (host-side): |
| 194 | + |
| 195 | +```bash |
| 196 | +RR="/Library/Developer/CoreSimulator/Volumes/iOS_23C54/.../iOS 26.2.simruntime/Contents/Resources/RuntimeRoot" |
| 197 | +strings -a "$RR/Developer/Library/PrivateFrameworks/XCTAutomationSupport.framework/XCTAutomationSupport" | grep -i orientation # -> XC_kAXXCAttributeApplicationOrientation |
| 198 | +nm -gU "$RR/System/Library/PrivateFrameworks/SpringBoardServices.framework/SpringBoardServices" | grep -i orient # -> SBGetInterfaceOrientation, _SBSGetActiveInterfaceOrientation |
| 199 | +nm -gU "$RR/System/Library/PrivateFrameworks/BackBoardServices.framework/BackBoardServices" | grep -i orient # -> BKHIDServicesGetCurrentDeviceOrientation |
| 200 | +``` |
0 commit comments