Skip to content

Commit 77fe281

Browse files
committed
refactor(ios): drop the bridge keyboard-band fact; keep one reader in kernel/record without the rect edge
Review follow-ups on #2670. - The Simulator AX bridge half is removed. deriveKeyboardBandFactFromTree ran the same resolveVisibleKeyboardSurface rule the guard already runs on the same decoded nodes, so a bridge 'visible' equalled the tree band and a bridge 'absent' equalled 'no-keyboard'; the field on the bridge acquisition carrier, its adapter wiring and its tests go with it. The runner is the only producer that measures the band directly, and it is the only one that publishes the fact. - readSnapshotKeyboardBandFact stays the single reader for both seams and lives in kernel/record, which every caller already evaluates. The isPositiveFiniteRect import that grew nine closure budgets is replaced by the same check inlined as one expression, so record.ts keeps its merge-base closure and no entry grows. (A separate kernel module was tried first; it adds itself to src/cli.ts's closure and the ratchet rejects that too.) - Typos and the orphaned doc comment above readObject fixed; the bridge-path presentation test whose carrier no longer exists is removed (the runner path is covered by snapshot-state.test.ts).
1 parent 5da5150 commit 77fe281

14 files changed

Lines changed: 18 additions & 224 deletions

File tree

‎packages/capture-kit/src/snapshot/ios-snapshot-runtime.ts‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,6 @@ export function presentIosSnapshotAcquisition(
4949
...(acquired.acquisition.truncated === undefined
5050
? {}
5151
: { truncated: acquired.acquisition.truncated }),
52-
...(acquired.acquisition.keyboard ? { keyboard: acquired.acquisition.keyboard } : {}),
5352
...snapshotWarnings(acquired.acquisition.residue),
5453
};
5554
} catch (error) {

‎packages/contracts/src/ios-snapshot.ts‎

Lines changed: 1 addition & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,4 @@
1-
import type {
2-
RawSnapshotNode,
3-
Rect,
4-
SnapshotKeyboardBandFact,
5-
SnapshotNode,
6-
} from '@agent-device/kernel/snapshot';
1+
import type { RawSnapshotNode, Rect, SnapshotNode } from '@agent-device/kernel/snapshot';
72

83
export type IosSnapshotProducer =
94
| 'apple-runner'
@@ -157,12 +152,6 @@ type IosSnapshotAcquisitionForIntent<Intent extends IosAcquisitionIntent> = Read
157152
nodes: readonly RawSnapshotNode[];
158153
truncated?: boolean;
159154
viewport: IosViewportEvidence;
160-
/**
161-
* The keyboard band this producer measured for this capture, derived from the tree it decoded
162-
* rather than from a second query (#2660). The tap-path guard prefers it over re-deriving the band
163-
* itself; an acquisition that says nothing leaves the guard on the tree-derived rule.
164-
*/
165-
keyboard?: SnapshotKeyboardBandFact;
166155
lineage: IosSnapshotLineage;
167156
residue: readonly IosAcquisitionResidue[];
168157
}>;

‎packages/contracts/src/tap-keyboard-occlusion.test.ts‎

Lines changed: 0 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,6 @@ import type {
88
SnapshotKeyboardBandFact,
99
} from '@agent-device/kernel/snapshot';
1010
import {
11-
deriveKeyboardBandFactFromTree,
1211
resolveKeyboardTapOcclusion,
1312
TAP_KEYBOARD_OCCLUDES_TARGET_DETAILS,
1413
TAP_KEYBOARD_OCCLUDES_TARGET_REASON,
@@ -225,31 +224,3 @@ test('an unmeasurable fact leaves every golden-table decision exactly as the tre
225224
assert.equal(withFact.kind, fixture.expected.kind, `${fixture.name}: kind`);
226225
}
227226
});
228-
229-
test('the fact a producer derives from its own tree is the band the rule measures', () => {
230-
// The bridge publishes exactly this (#2660), so the guard's preferred path and the rule it
231-
// replaces have to agree case by case — including the cases where the honest answer is that the
232-
// tree cannot be measured.
233-
for (const fixture of loadTable().cases) {
234-
const fact = deriveKeyboardBandFactFromTree({
235-
nodes: fixture.nodes as RawSnapshotNode[],
236-
viewport: fixture.viewport,
237-
});
238-
if (fixture.expected.frame) {
239-
assert.equal(fact.kind, 'visible', `${fixture.name}: expected a measured band`);
240-
assert.deepEqual(
241-
(fact as { frame: Rect }).frame,
242-
fixture.expected.frame,
243-
`${fixture.name}: band frame`,
244-
);
245-
} else if (fixture.expected.kind === 'no-keyboard') {
246-
assert.equal(fact.kind, 'absent', `${fixture.name}: a tree with no keyboard proves absence`);
247-
} else {
248-
assert.equal(
249-
fact.kind,
250-
'unmeasurable',
251-
`${fixture.name}: a keyboard this tree cannot measure is not an absent one`,
252-
);
253-
}
254-
}
255-
});

‎packages/contracts/src/tap-keyboard-occlusion.ts‎

Lines changed: 4 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -18,11 +18,10 @@ import { normalizeType } from './snapshot-text.ts';
1818
* too. The result was a silent misfire: pressing an element behind the keyboard reported success
1919
* while the touch activated a key (#2589).
2020
*
21-
* Two producers answer this question, in this order. A capture whose producer measured the band
22-
* directly publishes a {@link SnapshotKeyboardBandFact} beside its tree, and the guard measures the
23-
* tap point against that band — a point-in-rect check with no geometry to believe. The Apple runner
24-
* does this from `app.keyboards.firstMatch`, which answers in the app's own orientation space, and
25-
* the Simulator AX bridge does it for the portrait trees it is willing to serve (#2660).
21+
* A capture whose producer measured the band directly publishes a {@link SnapshotKeyboardBandFact}
22+
* beside its tree, and the guard measures the tap point against that band — a point-in-rect check
23+
* with no geometry to believe. The Apple runner does this from `app.keyboards.firstMatch`, which
24+
* answers in the app's own orientation space (#2660).
2625
*
2726
* Otherwise the band is derived from the captured tree every acting path already holds, so the guard
2827
* costs no round trip. This is the path for Android's input method nodes and for the producers that
@@ -316,27 +315,6 @@ function resolveVisibleKeyboardSurface(
316315
};
317316
}
318317

319-
/**
320-
* The fact a producer publishes when the tree it decoded is the only evidence it has (#2660): the
321-
* band this module's rule measures, or an honest statement that this tree cannot be measured.
322-
*
323-
* This is the bridge's answer, and it deliberately reuses the tree rule rather than inventing a
324-
* second one — the bridge reads the app's own process, so in portrait the keyboard window's subtree
325-
* is already in the app's space and the rule computes the band it always did. A landscape capture
326-
* with the keyboard up is refused by the bridge and served by the runner, which measures the band
327-
* itself, so the bridge never has to answer that case (#2612, #2653).
328-
*/
329-
export function deriveKeyboardBandFactFromTree(params: {
330-
nodes: readonly RawSnapshotNode[];
331-
viewport: Rect | null;
332-
}): SnapshotKeyboardBandFact {
333-
const surface = resolveVisibleKeyboardSurface(params.nodes, params.viewport);
334-
if (surface) return { kind: 'visible', frame: surface.frame };
335-
return params.nodes.some(isKeyboardAnchorNode)
336-
? { kind: 'unmeasurable', reason: 'tree-keyboard-band-unmeasurable' }
337-
: { kind: 'absent' };
338-
}
339-
340318
/**
341319
* Whether this tap point belongs to the visible keyboard rather than to app content behind it.
342320
*

‎packages/kernel/src/record.test.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ test('a measured band, a proven absence, and a stated failure read back as thems
3030

3131
// The order is load-bearing: a payload that is not even an object cannot also have an unrecognized
3232
// kind, and an `unmeasurable` is checked for its reason before a `visible` is checked for its frame.
33-
test('a payload that cannot be place names one reason, resolved from shape to kind to fields', () => {
33+
test('a payload that cannot be placed names one reason, resolved from shape to kind to fields', () => {
3434
const cases: ReadonlyArray<readonly [unknown, string]> = [
3535
['visible', 'malformed-fact'],
3636
[42, 'malformed-fact'],

‎packages/kernel/src/record.ts‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
11
import { AppError } from '@agent-device/kernel/errors';
22
import type { DeviceKind, DeviceTarget, PublicPlatform } from '@agent-device/kernel/device';
3-
import { isPositiveFiniteRect } from '@agent-device/kernel/rect';
43
import type { Point, Rect, SnapshotKeyboardBandFact } from '@agent-device/kernel/snapshot';
54

65
function readRequired<T>(
@@ -85,7 +84,7 @@ export function parseRect(value: unknown): Rect | undefined {
8584
}
8685

8786
/**
88-
* Reads a producer-declared keyboard band (#2660) out of an undtyped payload, shared by every seam
87+
* Reads a producer-declared keyboard band (#2660) out of an untyped payload, shared by every seam
8988
* that receives one: the Apple runner's wire reader, the daemon's Node client reader. `undefined`
9089
* means the producer published no fact at all, which is how a tier that never reads the keyboard is
9190
* distinguished from one that measured and reported.
@@ -108,7 +107,14 @@ export function readSnapshotKeyboardBandFact(value: unknown): SnapshotKeyboardBa
108107
}
109108
if (value.kind === 'visible') {
110109
const frame = parseRect(value.frame);
111-
return isPositiveFiniteRect(frame)
110+
// Same rule as `isPositiveFiniteRect` in kernel/rect, inlined: this module is a leaf that
111+
// many facades evaluate, and a rect import here would land in every one of their closures.
112+
const plottable =
113+
frame !== undefined &&
114+
[frame.x, frame.y, frame.width, frame.height].every(Number.isFinite) &&
115+
frame.width > 0 &&
116+
frame.height > 0;
117+
return plottable
112118
? { kind: 'visible', frame }
113119
: { kind: 'unmeasurable', reason: 'invalid-visible-frame' };
114120
}

‎packages/kernel/src/snapshot.ts‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -281,9 +281,8 @@ export type HiddenContentHint = {
281281
*
282282
* A keyboard is its own system surface, so it never reaches the tree as a covering sibling of app
283283
* content, and a consumer that wants to refuse a tap behind it has to learn where it is from
284-
* somewhere (#2589). Every producer that can measure the band directly — the Apple runner's
285-
* `app.keyboards` query, and the Simulator AX bridge for the portrait tree it is willing to serve —
286-
* publishes one fact per capture and says nothing else about it. A consumer therefore gets three
284+
* somewhere (#2589). A producer that can measure the band directly — the Apple runner, from its
285+
* `app.keyboards` query — publishes one fact per capture and says nothing else about it. A consumer therefore gets three
287286
* answers and no fourth: a band in the same space as every node rect, a proven absence, or a
288287
* producer that could not look.
289288
*

‎packages/platform-apple/src/snapshot-source/adapter.test.ts‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -61,9 +61,6 @@ test('the Simulator AX source returns raw acquisition facts and discloses unsupp
6161
kind: 'reported',
6262
rect: { x: 0, y: 0, width: 390, height: 844 },
6363
});
64-
// The tree this host answers with holds no keyboard, and the bridge publishes that as a proven
65-
// absence rather than silence (#2660): the guard is allowed to act on it.
66-
assert.deepEqual(result.acquisition.keyboard, { kind: 'absent' });
6764
assert.deepEqual(result.acquisition.lineage, {
6865
targetId: 'target-1',
6966
generation: 'generation-1',

‎packages/platform-apple/src/snapshot-source/adapter.ts‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -243,7 +243,6 @@ function createAcquisition(
243243
nodes,
244244
truncated,
245245
viewport: decoded.viewport,
246-
keyboard: decoded.keyboard,
247246
lineage,
248247
residue,
249248
};

‎packages/platform-apple/src/snapshot-source/tree.test.ts‎

Lines changed: 0 additions & 109 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
11
import assert from 'node:assert/strict';
22
import { test } from 'vitest';
3-
import { resolveKeyboardTapOcclusion } from '@agent-device/contracts/tap-keyboard-occlusion';
43
import { SnapshotSourceError } from './errors.ts';
54
import { decodeSnapshotBridgeTree } from './tree.ts';
65
import type { SnapshotSourceLimits } from './types.ts';
@@ -329,111 +328,3 @@ test('a window reporting the app box quarter-turned is counted as an unresolved
329328
0,
330329
);
331330
});
332-
333-
// The keyboard band the bridge publishes (#2660). The bridge cannot query the keyboard the way the
334-
// runner does, so what it publishes has to BE the band the shared rule measures over the tree it
335-
// decoded — anything else would move the tap guard's decision without anyone deciding to.
336-
337-
const PORTRAIT_KEYBOARD_TREE = {
338-
[application]: 'Application',
339-
[frame]: { X: 0, Y: 0, Width: 402, Height: 874 },
340-
[children]: [
341-
{
342-
[application]: 'UIRemoteKeyboardWindow',
343-
[automationType]: 4,
344-
[frame]: { X: 0, Y: 583, Width: 402, Height: 233 },
345-
[children]: [
346-
{
347-
[automationType]: 19,
348-
[frame]: { X: 0, Y: 583, Width: 402, Height: 233 },
349-
[children]: [0, 1, 2, 3].map((column) => ({
350-
[automationType]: 20,
351-
[frame]: { X: column * 100.5, Y: 600, Width: 100.5, Height: 60 },
352-
[children]: [],
353-
})),
354-
},
355-
],
356-
},
357-
],
358-
};
359-
360-
test('a portrait keyboard-up capture publishes the band the tree rule measures', () => {
361-
const decoded = decodeSnapshotBridgeTree(PORTRAIT_KEYBOARD_TREE, { truncated: false }, limits);
362-
363-
// Reported key plane bottoms out at 816 of an 874 pt viewport, and the band still owns the strip
364-
// below it: the same frame the tap guard computed from this tree before the fact existed.
365-
assert.deepEqual(decoded.keyboard, {
366-
kind: 'visible',
367-
frame: { x: 0, y: 583, width: 402, height: 291 },
368-
});
369-
370-
const point = { x: 201, y: 832 };
371-
const measuredByTheRule = resolveKeyboardTapOcclusion({
372-
nodes: decoded.nodes,
373-
viewport: decoded.viewport.kind === 'missing' ? null : decoded.viewport.rect,
374-
point,
375-
});
376-
assert.equal(measuredByTheRule.kind, 'occluded');
377-
assert.deepEqual(
378-
decoded.keyboard.kind === 'visible' ? decoded.keyboard.frame : null,
379-
measuredByTheRule.surface.frame,
380-
'the published band must be the band the rule measures',
381-
);
382-
});
383-
384-
test('a tree with no keyboard window publishes a proven absence', () => {
385-
const decoded = decodeSnapshotBridgeTree(
386-
{
387-
[application]: 'Application',
388-
[frame]: { X: 0, Y: 0, Width: 402, Height: 874 },
389-
[children]: [
390-
{
391-
[application]: 'UIWindow',
392-
[automationType]: 4,
393-
[frame]: { X: 0, Y: 0, Width: 402, Height: 874 },
394-
[children]: [],
395-
},
396-
],
397-
},
398-
{ truncated: false },
399-
limits,
400-
);
401-
402-
assert.deepEqual(decoded.keyboard, { kind: 'absent' });
403-
});
404-
405-
test('a keyboard this tree cannot measure is published as unmeasurable, not as absent', () => {
406-
// An iPad floating keyboard reports a band the rule refuses to extend to the bottom edge. The
407-
// screen is not clear of a keyboard, and the guard has to be able to say it abstained.
408-
const decoded = decodeSnapshotBridgeTree(
409-
{
410-
[application]: 'Application',
411-
[frame]: { X: 0, Y: 0, Width: 402, Height: 874 },
412-
[children]: [
413-
{
414-
[application]: 'UIRemoteKeyboardWindow',
415-
[automationType]: 4,
416-
[frame]: { X: 21, Y: 480, Width: 360, Height: 200 },
417-
[children]: [
418-
{
419-
[automationType]: 19,
420-
[frame]: { X: 21, Y: 480, Width: 360, Height: 200 },
421-
[children]: [0, 1, 2].map((column) => ({
422-
[automationType]: 20,
423-
[frame]: { X: 21 + column * 120, Y: 500, Width: 120, Height: 60 },
424-
[children]: [],
425-
})),
426-
},
427-
],
428-
},
429-
],
430-
},
431-
{ truncated: false },
432-
limits,
433-
);
434-
435-
assert.deepEqual(decoded.keyboard, {
436-
kind: 'unmeasurable',
437-
reason: 'tree-keyboard-band-unmeasurable',
438-
});
439-
});

0 commit comments

Comments
 (0)