Skip to content

Commit 616e843

Browse files
committed
docs(ios): record the open-pose Duo evidence and the overlay exporter defect
The operator opened the Duo mid-verification, so the open pose is now evidence rather than a gap: the capture names `LCD-1` at 951x669, taps resolve inside that surface, typed text reads back, and a pre-fold ref is refused instead of replayed at the new point size. What stays open is the quarter turn and the runner fallback, both restated as gaps. `recordVideo` gets the per-panel table that proves `--display` is honored (2006x2852 luma 241.42 for the lit panel against 1398x2034 luma 0.00 for the dark one), plus the separate finding that the touch-overlay exporter returns a 480x336 black video for the inner panel's rot90 track — reproducible by feeding a raw simctl capture straight into the untouched `recording-overlay.swift`, and fine on a non-rotated iPhone 17 export. Also drop the pose and role wording from the decision table, since the inventory no longer carries either.
1 parent 4780b46 commit 616e843

2 files changed

Lines changed: 51 additions & 21 deletions

File tree

‎docs/adr/0025-foldable-apple-panels.md‎

Lines changed: 48 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -41,21 +41,24 @@ reader of CoreDevice display info.
4141
guessing dark would aim a capture at a panel that may be showing nothing.
4242
- **Panel identity.** A device is multi-screen when it reports more than one `type: integrated`
4343
panel. CoreDevice marks one panel `primary`; that panel is the outer display, because it is the
44-
panel that stays lit while the device is closed. The inner display is the largest remaining panel,
45-
because a foldable's inner surface encloses the outer one. A third and later panel stays unroled
46-
rather than being labelled `inner`, and a device with no `primary` panel gets **no** roles at all:
47-
inventing them from CoreDevice's array order would silently invert the naming. When the primary
48-
panel is also the largest, the `primary` attribute still decides and the conflict is emitted as a
49-
diagnostic rather than silently repaired by a geometry rule.
44+
panel that stays lit while the device is closed. `primary` and the lit count are the whole model:
45+
the inventory labels panels `outer`/`inner` nowhere, because no caller consumes a label — every
46+
decision is made from `primary` plus panel power, and a name derived from array order or from
47+
geometry would be a second source of truth ready to invert silently. When the primary panel is
48+
also the largest, the `primary` attribute still decides and the conflict is emitted as
49+
`apple_display_primary_geometry_conflict` rather than silently repaired by a geometry rule.
5050
- **Capture target.** `resolveAppleCaptureDisplay` returns a panel for every multi-panel device and
5151
`undefined` only for a single-panel device or an unresolved inventory. When panel power identifies
52-
exactly one lit panel that panel is named; otherwise the `primary` panel is named anyway, the
53-
pose is `unknown`, and `apple_display_capture_ambiguous` is emitted. A multi-panel device never
52+
exactly one lit panel that panel is named; otherwise the `primary` panel is named anyway,
53+
`ambiguous` is set, and `apple_display_capture_ambiguous` is emitted. A multi-panel device never
5454
receives a display-less capture, because the implicit default is precisely the black-and-exit-0
5555
failure being fixed; naming a possibly-dark panel is a visible, diagnosable miss instead.
56-
- **Pose.** `deriveDisplayPose` reads panel power only: exactly one lit panel and it is
57-
`primary` → `closed`; exactly one lit panel and it is not → `fully-open`; anything else
58-
(both, neither, no `primary`) → `unknown`.
56+
- **No pose field.** Panel power cannot separate Apple's `UIHinge.Status.fullyOpen` from
57+
`.partiallyOpen` — both leave the inner panel lit and the outer panel dark — so a derived pose
58+
could only ever mean "not closed" while looking like a fact. The inventory therefore reports what
59+
it measured (`primary`, panel power, geometry, orientation) and leaves pose to the app under test,
60+
which can read `UIHinge.status` exactly. A caller that needs the pose asks the operator and
61+
re-snapshots.
5962
- **No cache.** The probe runs per capture. Which panel is lit is precisely what an operator
6063
changes by folding or opening the device, so a cached inventory would resume capturing the dark
6164
panel — the exact failure being fixed. The probe runs under
@@ -72,6 +75,7 @@ whose content is black, so every downstream consumer trusts a capture of nothing
7275
| Capture path | Before | After |
7376
| --- | --- | --- |
7477
| `screenshot` on a closed Duo | 669x951 @1x, mean luma 0.09 (black) | 466x678 @1x, mean luma 65.8 (the lit outer panel) |
78+
| `screenshot` on an open Duo | 951x669 @1x, mean luma 241 (correct by luck: the highest screen ID is the lit inner panel) | 951x669 @1x naming `LCD-1` explicitly |
7579
| `screenshot` on iPhone 17 | 402x874 @1x, mean luma 104.9 | unchanged |
7680

7781
This is the failure behind reports that the tool "picks the wrong screen" on iPhone Duo. It was
@@ -145,11 +149,36 @@ coordinates do not survive it. `agent-device help foldable` states this and stat
145149
report which poses remain unverified rather than assume a pose was set.
146150

147151
`simctl io recordVideo` has the same implicit-display default as `screenshot`, so recording names
148-
the lit panel through the same resolver.
152+
the lit panel through the same resolver. On an open Duo the 27.1 toolchain accepts the panel name
153+
and honors it per panel; sampled mean luma over the whole frame:
149154

150-
## Accepted evidence gaps
155+
| `recordVideo` argv | exported size | mean luma |
156+
| --- | --- | --- |
157+
| `--display=LCD-1` (lit inner) | 2006x2852 | 241.42 |
158+
| `--display=LCD` (dark outer) | 1398x2034 | 0.00 |
159+
| no `--display` | 2006x2852 | 241.42 |
160+
161+
`record start`/`record stop` exit 0 in both poses, and with `--hide-touches` the export keeps the
162+
captured geometry (`2006x2852`, mean luma 241.42). Without it, the touch-overlay exporter is broken
163+
for the inner panel's `rot90` track: it returns a `480x336` black video, and feeding an untouched
164+
raw `simctl` capture straight into `recording-overlay.swift` reproduces a `0x0` zero-duration
165+
output. That exporter defect is independent of panel selection — the same pipeline exports a
166+
non-rotated iPhone 17 recording intact (`1206x2622`, mean luma 228.30) — and is tracked separately.
167+
168+
## Verified on a booted Duo
169+
170+
Closed pose: `screenshot` moved from `669x951` luma 0.09 to `466x678` luma 65.8; a tap on Safari's
171+
address field at `(191, 620)` opened the keyboard; and on the `examples/test-app` dev build a
172+
41-node `snapshot -i`, a tap that dismissed the dev-menu sheet at `(345, 301)`, and a Catalog-tab tap
173+
at `(128, 626)` that settled `+19 -15`.
174+
175+
Open pose, after an operator opened the device: the capture names `LCD-1` at `951x669 @1x`,
176+
`snapshot -i` returns Safari's nodes on that surface, `tap @e4` resolves to `(590, 478)` inside it,
177+
and text sent with `type` is found again by `find text` — hit testing and read-back both follow the
178+
lit panel. A ref issued before the fold is refused afterwards as an expired frame rather than
179+
replayed at the new point size, which is the pose-change rule working as designed.
151180

152-
Both gaps need an open-pose device, which no available command can produce.
181+
## Accepted evidence gaps
153182

154183
- **Runner screenshot fallback.** `XCUIScreen.main` is hardcoded in the runner's `screenshot`
155184
command and in the synthesized-gesture reference frame. The `simctl` path is the default for iOS
@@ -158,11 +187,9 @@ Both gaps need an open-pose device, which no available command can produce.
158187
`active` flag — only `screens` and `mainScreen` — so choosing the lit panel there cannot be done
159188
officially without pairing screenshots against panel geometry, which is the pixel heuristic this
160189
ADR rejects. Left unresolved rather than guessed.
161-
- **Inner-panel snapshot and interaction.** Closed-pose capture, snapshot, taps, and density are
162-
verified on a booted Duo, on both a system app and a React Native one: 466x678 outer, luma 65.8
163-
after vs 0.09 before; a tap on Safari's address field at 191,620 opened the keyboard; and on the
164-
`examples/test-app` dev build a 41-node `snapshot -i`, a tap that dismissed the dev-menu sheet
165-
(345,301), and a Catalog-tab tap (128,626) that settled `+19 -15`. Open-pose snapshot frames, tap
166-
routing, and the quarter-turn detector (`currentOrientation: rot90` on the inner panel) are
167-
unverified.
190+
- **Quarter-turn detection.** Both Duo panels report `currentOrientation: rot90`, and no available
191+
path rotates a foldable, so the orientation half of the inventory is carried but never exercised
192+
against a changed value.
193+
- **Pose control.** No official host API sets the hinge angle, so both poses above depend on an
194+
operator opening or closing the device in Device Hub.
168195

‎docs/agents/device-verification.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,9 @@ toolchain per command:
7070
capturable — a capture of the dark panel exits 0 and writes an all-black PNG.
7171
- Pose is not scriptable. Ask the operator to fold or open the device in Device Hub, then
7272
re-snapshot; refs and coordinates do not survive the pose change.
73+
- Record a foldable with `record start --hide-touches`. The touch-overlay exporter returns a small
74+
all-black video for the inner panel's `rot90` track, which reads as a capture failure but is not
75+
one; the raw `simctl` capture behind it is correct. See ADR 0025.
7376
- An app must adopt the UIScene lifecycle to launch on iOS 27.1 at all: a legacy
7477
`UIApplicationDelegate` app traps at launch inside
7578
`___UIApplicationEvaluateRuntimeIssueForNoSceneLifecycleAdoption`, which reads like a broken

0 commit comments

Comments
 (0)