Skip to content

Commit ec7aecb

Browse files
committed
refactor(ios): keep the window-content translation with the capture that runs it [ #3899 ]
1 parent a8dbd01 commit ec7aecb

5 files changed

Lines changed: 30 additions & 68 deletions

File tree

‎apple/snapshot-presentation/Tests/AgentDeviceSnapshotPresentationTests/CoordinateSpaceTests.swift‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -317,9 +317,11 @@ final class CoordinateSpaceTests: XCTestCase {
317317
}
318318

319319
/// Golden parity table (#2612, #3899): every case in contracts/fixtures/window-coordinate-space.json
320-
/// must agree with the vitest twin
321-
/// (packages/platform-apple/src/snapshot-source/window-coordinate-space.test.ts). Add
322-
/// cases there, never fork the rule. Detection is replayed under both landscape turns, because
320+
/// must agree here, and the quarter-turn rows with the vitest twin
321+
/// (packages/platform-apple/src/snapshot-source/window-coordinate-space.test.ts). The
322+
/// window-content rows replay here alone: the host channel's guest refuses a windowed tree
323+
/// before its decoder can read one, so this capture is the rule's only production owner. Add
324+
/// cases to the table, never fork the rule. Detection is replayed under both landscape turns, because
323325
/// which window declares the native space cannot depend on which way the app is turned; the
324326
/// rotation rows carry their own orientation.
325327
func testWindowCoordinateSpaceMatchesGoldenParityTable() throws {

‎contracts/fixtures/window-coordinate-space.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{
2-
"description": "Golden table for the coordinate space a captured window's own box declares (#2612), the twin of tap-keyboard-occlusion-policy.json on the geometry that guard reads. iOS hosts some system surfaces in the device's native (portrait-up) space even while the app is rotated, so their whole subtree arrives quarter-turned: measured on iPhone 17 Pro (iOS 26.2) with the system keyboard up over a landscape app frame (0,0,874,402), UIRemoteKeyboardWindow reports its own box as (0,0,402,874) — the app's box with the two side lengths swapped — while the app's own UIWindow, UITextEffectsWindow and a dev-menu overlay window all report (0,0,874,402); in portrait the same keyboard window reports (0,0,402,874) against a (0,0,402,874) app frame, i.e. nothing is turned. RULE 1 (detection) is enforced twice, in packages/contracts/src/window-coordinate-space.ts and in SnapshotGeometrySpace inside apple/snapshot-presentation/Sources/AgentDeviceSnapshotPresentation/SnapshotCoordinateSpace.swift, and decides from side lengths alone: a window is quarter-turned relative to an app frame when the app frame is not square (abs(app.width - app.height) > quarterTurnTolerance), both frames are finite, positive and non-empty, and the window's side lengths match the app's on the swapped axes (abs(window.width - app.height) <= quarterTurnTolerance && abs(window.height - app.width) <= quarterTurnTolerance). Origins are deliberately never compared: the turn is a statement about which axis a box runs along, and a window may legitimately be placed elsewhere. A square app frame cannot be told from its own quarter turn, so it is left alone rather than guessed at. The tolerance is 1 because measured captures match the swap to the point and this absorbs float representation only. RULE 2 (rotation) is the capture's way back for the runner's own UIInterfaceOrientation raw values — 1 portrait, 2 portraitUpsideDown, 3 landscapeRight, 4 landscapeLeft, and 0 unknown, where portrait and unknown publish what the platform reported rather than inventing a turn — implemented by CoordinateSpaceRotation.oriented(rect:in:interfaceOrientation:) in that same Swift file, whose rect helper turns opposite corners of the reported box and re-orders them so the result keeps a top-left origin. `rotationCases` are native -> oriented rows: the landscape ones measured on the device above, the 180 degree and identity ones derived from that table. Capture turns back with the exact inverse of what dispatch turns forward, so each row also has to come back to its own `native` box under the forward mapping; that is why one measured-looking landscape-left pairing whose native and oriented boxes are identical is absent — the table computes no such row. Infinity has no JSON spelling, so the unusable frame a platform hands back when it could not resolve a box is named {\"infinite\": true} and decodes to CGRect.infinite on the Swift side and to an infinite rect on the TypeScript side. RULE 3 (content translation) is the other space a captured subtree arrives in, and the one rule in this file that reads an origin. An iPad window that does not fill the screen (Windowed Apps, Stage Manager) reports its own screen box while the subtree under it arrives anchored at its own top-left: measured on iPad Pro 11-inch (M5, iOS 27.0) with a Calendar window at screen (224,207,386x682), the application root reports (0,0,386,682) and its Add button (335,14,37,36) for a control drawn at screen (559,221). Published as reported, that tree mixes two spaces, clips its own content, and moves every tap by the window origin (#3899). SnapshotContentTranslation.offset(reportedBox:viewportBox:) reads the carrier box against the viewport box the platform reported, and names the viewport origin as the offset only when the carrier is the app's own window: a (0,0)-anchored box whose side lengths match the viewport's within the same tolerance, while the viewport starts somewhere else on the screen. A carrier that is not the app's own window names no space, so its numbers are left alone: the keyboard window and a SpringBoard overlay under a windowed app report the screen's own box and are already in screen space. A full-screen window, an iPhone, and each foldable panel of ADR 0025 report origins at zero on both sides, so the rule reads no offset there and every published rect stays exactly as it was measured before. `windowContentCases` are reportedBox + viewportBox to offset rows, replayed against that Swift function and against windowContentTranslation in packages/platform-apple/src/snapshot-source/window-coordinate-space.ts; a null offset means the rule declares no translation. Because a translation is the one thing a rotation must not also do, `rotationCases` pin the turn as axes-only: `native` and `oriented` never move a box's origin, and a row whose app frame is offset from the screen comes back at the same place as its own zero-origin twin.",
2+
"description": "Golden table for the coordinate space a captured window's own box declares (#2612), the twin of tap-keyboard-occlusion-policy.json on the geometry that guard reads. iOS hosts some system surfaces in the device's native (portrait-up) space even while the app is rotated, so their whole subtree arrives quarter-turned: measured on iPhone 17 Pro (iOS 26.2) with the system keyboard up over a landscape app frame (0,0,874,402), UIRemoteKeyboardWindow reports its own box as (0,0,402,874) — the app's box with the two side lengths swapped — while the app's own UIWindow, UITextEffectsWindow and a dev-menu overlay window all report (0,0,874,402); in portrait the same keyboard window reports (0,0,402,874) against a (0,0,402,874) app frame, i.e. nothing is turned. RULE 1 (detection) is enforced twice, in packages/contracts/src/window-coordinate-space.ts and in SnapshotGeometrySpace inside apple/snapshot-presentation/Sources/AgentDeviceSnapshotPresentation/SnapshotCoordinateSpace.swift, and decides from side lengths alone: a window is quarter-turned relative to an app frame when the app frame is not square (abs(app.width - app.height) > quarterTurnTolerance), both frames are finite, positive and non-empty, and the window's side lengths match the app's on the swapped axes (abs(window.width - app.height) <= quarterTurnTolerance && abs(window.height - app.width) <= quarterTurnTolerance). Origins are deliberately never compared: the turn is a statement about which axis a box runs along, and a window may legitimately be placed elsewhere. A square app frame cannot be told from its own quarter turn, so it is left alone rather than guessed at. The tolerance is 1 because measured captures match the swap to the point and this absorbs float representation only. RULE 2 (rotation) is the capture's way back for the runner's own UIInterfaceOrientation raw values — 1 portrait, 2 portraitUpsideDown, 3 landscapeRight, 4 landscapeLeft, and 0 unknown, where portrait and unknown publish what the platform reported rather than inventing a turn — implemented by CoordinateSpaceRotation.oriented(rect:in:interfaceOrientation:) in that same Swift file, whose rect helper turns opposite corners of the reported box and re-orders them so the result keeps a top-left origin. `rotationCases` are native -> oriented rows: the landscape ones measured on the device above, the 180 degree and identity ones derived from that table. Capture turns back with the exact inverse of what dispatch turns forward, so each row also has to come back to its own `native` box under the forward mapping; that is why one measured-looking landscape-left pairing whose native and oriented boxes are identical is absent — the table computes no such row. Infinity has no JSON spelling, so the unusable frame a platform hands back when it could not resolve a box is named {\"infinite\": true} and decodes to CGRect.infinite on the Swift side and to an infinite rect on the TypeScript side. RULE 3 (content translation) is the other space a captured subtree arrives in, and the one rule in this file that reads an origin. An iPad window that does not fill the screen (Windowed Apps, Stage Manager) reports its own screen box while the subtree under it arrives anchored at its own top-left: measured on iPad Pro 11-inch (M5, iOS 27.0) with a Calendar window at screen (224,207,386x682), the application root reports (0,0,386,682) and its Add button (335,14,37,36) for a control drawn at screen (559,221). Published as reported, that tree mixes two spaces, clips its own content, and moves every tap by the window origin (#3899). SnapshotContentTranslation.offset(reportedBox:viewportBox:) reads the carrier box against the viewport box the platform reported, and names the viewport origin as the offset only when the carrier is the app's own window: a (0,0)-anchored box whose side lengths match the viewport's within the same tolerance, while the viewport starts somewhere else on the screen. A carrier that is not the app's own window names no space, so its numbers are left alone: the keyboard window and a SpringBoard overlay under a windowed app report the screen's own box and are already in screen space. A full-screen window, an iPhone, and each foldable panel of ADR 0025 report origins at zero on both sides, so the rule reads no offset there and every published rect stays exactly as it was measured before. `windowContentCases` are reportedBox + viewportBox to offset rows, replayed against that Swift function alone; a null offset means the rule declares no translation. The host decoder (packages/platform-apple/src/snapshot-source/window-coordinate-space.ts) replays only the quarter-turn rows: its channel's guest refuses a windowed tree (app-window-not-full-screen, #3899) before that decoder can read one, so the translation has exactly one producer-side owner. Because a translation is the one thing a rotation must not also do, `rotationCases` pin the turn as axes-only: `native` and `oriented` never move a box's origin, and a row whose app frame is offset from the screen comes back at the same place as its own zero-origin twin.",
33
"constants": {
44
"quarterTurnTolerance": 1
55
},

‎docs/adr/0004-ios-snapshot-backend-strategy.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -649,7 +649,7 @@ The carrier's box must be a `(0,0)`-anchored copy of the viewport box the platfo
649649
Then the subtree was reported in the carrier's content space, and the viewport's origin is the one vector that carries it to the screen.
650650
Any other carrier box names no space, so the rule leaves its numbers alone: the keyboard window and the SpringBoard overlay above report the screen's own box and are already in screen space.
651651
A full-screen window, an iPhone, and each foldable panel of ADR 0025 report origins at zero on both sides, so the rule reads no vector there and every rect is unchanged.
652-
`SnapshotContentTranslation.offset(reportedBox:viewportBox:)` is that rule, replayed from `contracts/fixtures/window-coordinate-space.json` in both languages.
652+
`SnapshotContentTranslation.offset(reportedBox:viewportBox:)` is that rule, replayed from `contracts/fixtures/window-coordinate-space.json` in the capture's language alone: the host bridge refuses a windowed tree (`app-window-not-full-screen`) before its decoder can read one, so the translation has one producer-side owner and the host's parity replay covers only the rules that decoder reads.
653653

654654
Rotation and translation stay separate rules, and the split is the point.
655655
`CoordinateSpaceRotation` turns a box's axes and never moves its origin: the digitizer's origin is the display's own, which is where a placed window's screen-space points are already measured from.

‎packages/platform-apple/src/snapshot-source/window-coordinate-space.test.ts‎

Lines changed: 4 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,6 @@ import type { Point, Rect } from '@agent-device/kernel/snapshot';
66
import {
77
isQuarterTurnedWindowFrame,
88
WINDOW_QUARTER_TURN_TOLERANCE,
9-
windowContentTranslation,
109
} from './window-coordinate-space.ts';
1110

1211
// ADR 0011 Layer 2 golden parity table (#2612): the SAME JSON is asserted against the Swift twin
@@ -107,21 +106,10 @@ test('the quarter-turn window rule agrees with every golden parity table case',
107106
}
108107
});
109108

110-
test('the window-content translation rule agrees with every golden parity table case', () => {
111-
for (const fixture of loadTable().windowContentCases) {
112-
const translation = windowContentTranslation(
113-
toRect(fixture.reportedBox),
114-
toRect(fixture.viewportBox),
115-
);
116-
assert.deepEqual(
117-
translation,
118-
fixture.offset === null
119-
? null
120-
: { offsetX: fixture.offset.width, offsetY: fixture.offset.height },
121-
fixture.name,
122-
);
123-
}
124-
});
109+
// `windowContentCases` is replayed in the Swift capture alone (CoordinateSpaceTests): this decoder
110+
// never receives a windowed tree — the bridge guest refuses it first (#3899) — so a second replay
111+
// here would pin arithmetic no production reader runs. The table still loads here, and its rows are
112+
// read for the name-uniqueness guard below.
125113

126114
test('the quarter-turn tolerance belongs to the table, not this file', () => {
127115
assert.equal(loadTable().constants.quarterTurnTolerance, WINDOW_QUARTER_TURN_TOLERANCE);

‎packages/platform-apple/src/snapshot-source/window-coordinate-space.ts‎

Lines changed: 19 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,9 @@ import { isPositiveFiniteRect } from '@agent-device/kernel/rect';
2121
* anchored at its own top-left: the application root reads `(0,0,386,682)` and its `Add` button
2222
* `(335,14,37,36)` — the button's screen position `(559,221)` minus the window origin. Published as
2323
* reported, that tree mixes two spaces, clips its own content, and moves every tap by the window
24-
* origin (#3899). `windowContentTranslation` names the one vector that puts the subtree back.
24+
* origin (#3899). That translation belongs to the capture that performs it — the runner's Swift
25+
* `SnapshotContentTranslation` — not to this decoder, because the guest refuses a windowed tree
26+
* before this file can read one; see the note at the foot of this file.
2527
*
2628
* A rule that reads a box reads what the box can say. The quarter turn is a statement about which
2729
* axis a box runs along, so it compares side lengths and nothing else. A content translation is a
@@ -31,12 +33,13 @@ import { isPositiveFiniteRect } from '@agent-device/kernel/rect';
3133
* A full-screen window, an iPhone, and each foldable panel report origins at zero on both sides, so
3234
* the rule reads no translation there and every published rect stays as the platform reported it.
3335
*
34-
* The same rules are enforced in Swift for the capture that produces these trees, and one table
35-
* proves they are one rule: `contracts/fixtures/window-coordinate-space.json` is replayed here and
36-
* against `SnapshotGeometrySpace` and `SnapshotContentTranslation` in
37-
* `apple/snapshot-presentation/Sources/AgentDeviceSnapshotPresentation/SnapshotCoordinateSpace.swift`. The table also
38-
* pins the way back (`CoordinateSpaceRotation.oriented(rect:in:interfaceOrientation:)`, Swift-only
39-
* today); change any rule only through that table.
36+
* RULE 1 is enforced in both languages, because both channels read it. RULE 3 (the content
37+
* translation) and RULE 2 (the rotation back, `CoordinateSpaceRotation.oriented(rect:in:interfaceOrientation:)`)
38+
* live with the runner's capture alone. One table proves the shared rules are one rule:
39+
* `contracts/fixtures/window-coordinate-space.json` is replayed here and against
40+
* `SnapshotGeometrySpace`, `SnapshotContentTranslation`, and `CoordinateSpaceRotation` in
41+
* `apple/snapshot-presentation/Sources/AgentDeviceSnapshotPresentation/SnapshotCoordinateSpace.swift`;
42+
* change any rule only through that table.
4043
*
4144
* Why this is a detector and not a repair: the bridge guest cannot read the app's interface
4245
* orientation. Measured in the #2659 spike (verdict on the issue, write-up in the diff of #2667):
@@ -71,44 +74,13 @@ export function isQuarterTurnedWindowFrame(windowFrame: Rect, appFrame: Rect): b
7174
);
7275
}
7376

74-
/** The vector that carries a window-relative subtree into the screen space the viewport names. */
75-
export type WindowContentTranslation = {
76-
offsetX: number;
77-
offsetY: number;
78-
};
79-
80-
/**
81-
* The vector from a carrier's own box to the screen space it is placed at, or null when the carrier's
82-
* box declares no translation. The carrier's subtree was reported in the carrier's content space, so
83-
* adding the vector to each reported rect publishes the subtree in the viewport's screen space (#3899).
84-
*
85-
* Null on anything that is not the app's own placed window: a box that is not a `(0,0)`-anchored copy
86-
* of the viewport's side lengths names no space, and a viewport that already starts at the screen
87-
* origin needs no move. Replayed against `windowContentCases` in
88-
* `contracts/fixtures/window-coordinate-space.json`, in the Swift twin and here.
77+
/*
78+
* A window that does not fill the screen (#3899) is translated by the capture that owns it, and this
79+
* file is not that capture. The bridge guest refuses a windowed tree (`app-window-not-full-screen`)
80+
* before this decoder ever sees one, so no host tree arrives in the window's content space and there
81+
* is nothing here to translate. The rule lives with the producer that converts —
82+
* `SnapshotContentTranslation` beside the Swift capture — and `windowContentCases` in
83+
* `contracts/fixtures/window-coordinate-space.json` is replayed there alone. The quarter-turn
84+
* detection above stays in both languages because this decoder does read it: it is what refuses a
85+
* turned tree the reader cannot resolve.
8986
*/
90-
export function windowContentTranslation(
91-
reportedBox: Rect,
92-
viewportBox: Rect,
93-
): WindowContentTranslation | null {
94-
if (!isPositiveFiniteRect(reportedBox) || !isPositiveFiniteRect(viewportBox)) return null;
95-
if (
96-
Math.abs(reportedBox.x) > WINDOW_QUARTER_TURN_TOLERANCE ||
97-
Math.abs(reportedBox.y) > WINDOW_QUARTER_TURN_TOLERANCE
98-
) {
99-
return null;
100-
}
101-
if (
102-
Math.abs(viewportBox.x) <= WINDOW_QUARTER_TURN_TOLERANCE &&
103-
Math.abs(viewportBox.y) <= WINDOW_QUARTER_TURN_TOLERANCE
104-
) {
105-
return null;
106-
}
107-
if (
108-
Math.abs(reportedBox.width - viewportBox.width) > WINDOW_QUARTER_TURN_TOLERANCE ||
109-
Math.abs(reportedBox.height - viewportBox.height) > WINDOW_QUARTER_TURN_TOLERANCE
110-
) {
111-
return null;
112-
}
113-
return { offsetX: viewportBox.x, offsetY: viewportBox.y };
114-
}

0 commit comments

Comments
 (0)