Skip to content

Commit 44050b4

Browse files
committed
fix(replay): demote non-unique ids from writer identity/selector chain (#1269)
Android list-row GET replays bind the wrong row because the recorder uses the non-unique framework resource id `android:id/title` (matchCount 11 on Settings root) as primary identity; positional drift then makes the identity verifier correctly refuse with `identity-mismatch`. Demote an id from identity whenever it matches more than one node in the record-time tree (capture-time uniqueness, not an `android:id/*` namespace check — a reused RN FlatList testID hits the same class on iOS). Applied in both places a recorded id feeds identity: - `computeTargetEvidence` (session-target-evidence.ts): the `target-v1` identity tuple falls back to role+label when the id's own capture-time match count exceeds one, reusing the existing `filterIdentitySet` domain machinery (an empty ancestry degrades it to a plain id scan). - `buildSelectorChainForNode` (selectors/build.ts): the recorded selector chain omits a non-unique id rather than leading with it. Every writer call site (get/press/fill recording, plus the divergence-suggestion path) now passes the record-time tree so the check has something to count against; omitting it preserves prior behavior for isolated-node callers (tests). Resolver-side `resolveSelectorChain` and live press/fill resolution are untouched per ADR 0012 (disclosed-not-changed disambiguation) — this is writer/replay-scoped only. Amends ADR 0012 decision 3: an id may serve as identity (and lead the selector chain) only when it uniquely denotes the target in the record-time tree. Adds fixtures: an Android duplicated-`android:id/title` list (the measured repro) and an iOS/RN duplicated-testID FlatList shape, both demoted and still verifying via the now-selective label; a regression case confirming an already-unique id is unaffected. Out of scope: the Android list-*press* class (matchCount 12, label-less `role="linearlayout"` container with no id at all to demote) needs a separate design decision — deriving identity from the labeled descendant. Tracked as a follow-up, not attempted here.
1 parent e58cbcd commit 44050b4

8 files changed

Lines changed: 344 additions & 3 deletions

File tree

‎docs/adr/0012-interactive-replay.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,11 @@ Accepted (2026-07-10); partially implemented (last updated 2026-07-13). See [Mig
1919
filters, not scopings; and `screen.refs` is ranked within the byte cap (foreign-window dismiss targets
2020
ahead of app content; mass-covered app nodes surfaced rather than emptied) instead of sliced in document
2121
order, so a captured overlay is never buried past the cap (#1264).
22+
- Decision 3 amendment, id demotion under non-unique capture-time match — a recorded id no longer serves
23+
as identity, or leads the selector chain, unless it is unique in the record-time tree; a non-unique id
24+
(a shared Android framework resource id such as `android:id/title`, or a reused RN `FlatList` `testID`)
25+
falls back to role+label in both `computeTargetEvidence`'s `target-v1` tuple and
26+
`buildSelectorChainForNode`'s chain (#1269).
2227

2328
**Accepted but NOT yet implemented** (this amendment; tracked by #1235 — repair-transaction lifecycle):
2429
the R7 repair-transaction keep-alive and its distinct `resume.repairSessionHeld` signal, the ARMED →
@@ -316,6 +321,14 @@ or, when the recording carries no `id`, when their normalized roles are equal an
316321
are equal (label absent on both sides counts as equal; label present on exactly one side is a mismatch).
317322
A recorded `id` never matches a node without that id.
318323

324+
> **Amendment (#1269).** An id may serve as identity (and lead the selector chain) only when it uniquely
325+
> denotes the target in the record-time tree: the writer computes the id's own capture-time match count
326+
> (independent of ancestry) and, whenever more than one node in the record-time tree carries the recorded
327+
> id, demotes it — falling back to role+label exactly as an unrecorded id already does. A shared
328+
> Android framework resource id (`android:id/title`, present on every titled list row) is the measured
329+
> case, but the rule is capture-time uniqueness, not an `android:id/*` namespace heuristic: a reused RN
330+
> `FlatList` `testID` hits the same demotion on iOS.
331+
319332
**Ancestry.** The chain is the nearest **K = 8** ancestors of the target, ordered **leaf→root** (nearest
320333
ancestor first), each entry `{ role, label? }` under the same normalization (`role` may be the empty
321334
string when the node has no type; `label` is omitted when empty). Truncation drops entries from the

‎src/commands/interaction/runtime/resolution.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -403,6 +403,7 @@ function describeResolvedInteractionNode(
403403
node,
404404
selectorChain: buildSelectorChainForNode(node, runtime.backend.platform, {
405405
action: action === 'fill' ? 'fill' : 'click',
406+
nodes,
406407
}),
407408
refLabel: resolveRefLabel(node, nodes),
408409
...describeNonHittableTarget(node, action),

‎src/commands/interaction/runtime/selector-read.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -210,6 +210,7 @@ export const getCommand: RuntimeCommand<GetCommandOptions, GetCommandResult> = a
210210
);
211211
const selectorChain = buildSelectorChainForNode(resolved.node, runtime.backend.platform, {
212212
action: 'get',
213+
nodes: capture.snapshot.nodes,
213214
});
214215
const target = { kind: 'ref' as const, ref: `@${resolved.ref}` };
215216
const preActionNodes = capture.snapshot.nodes;
@@ -233,6 +234,7 @@ export const getCommand: RuntimeCommand<GetCommandOptions, GetCommandResult> = a
233234

234235
const selectorChain = buildSelectorChainForNode(resolved.node, runtime.backend.platform, {
235236
action: 'get',
237+
nodes: resolved.capture.snapshot.nodes,
236238
});
237239

238240
if (options.property === 'attrs') {

‎src/daemon/__tests__/session-target-evidence.test.ts‎

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -466,3 +466,126 @@ test('computeTargetEvidence: a parent cycle records unverifiable, never verified
466466
assert.ok(evidence);
467467
assert.equal(evidence.verification, 'unverifiable');
468468
});
469+
470+
// ---------------------------------------------------------------------------
471+
// #1269: ADR 0012 decision 3 amendment — an id is identity only when it
472+
// uniquely denotes the target in the record-time tree. A shared framework
473+
// resource id (Android's `android:id/title`, present on every titled list
474+
// row) is not selective: the recorded ancestry is label-less generic
475+
// containers (linearlayout/recyclerview/framelayout) contributing no
476+
// disambiguation, so an id-led identity set spans every row and replay binds
477+
// the wrong one under positional drift. The fix demotes the id back to
478+
// role+label whenever it matches more than one node at record time — the
479+
// same fallback an unrecorded id already uses.
480+
// ---------------------------------------------------------------------------
481+
482+
function androidSettingsListFixture(): SnapshotNode[] {
483+
return toSnapshotNodes([
484+
{ index: 0, type: 'FrameLayout', depth: 0 },
485+
{ index: 1, type: 'RecyclerView', depth: 1, parentIndex: 0 },
486+
// Each row is a label-less LinearLayout wrapper (real Android list
487+
// shape) whose title TextView shares one framework resource id.
488+
{ index: 2, type: 'LinearLayout', depth: 2, parentIndex: 1 },
489+
{
490+
index: 3,
491+
type: 'TextView',
492+
identifier: 'android:id/title',
493+
label: 'Network & internet',
494+
rect: { x: 0, y: 100, width: 300, height: 48 },
495+
depth: 3,
496+
parentIndex: 2,
497+
},
498+
{ index: 4, type: 'LinearLayout', depth: 2, parentIndex: 1 },
499+
{
500+
index: 5,
501+
type: 'TextView',
502+
identifier: 'android:id/title',
503+
label: 'Connected devices',
504+
rect: { x: 0, y: 148, width: 300, height: 48 },
505+
depth: 3,
506+
parentIndex: 4,
507+
},
508+
{ index: 6, type: 'LinearLayout', depth: 2, parentIndex: 1 },
509+
{
510+
index: 7,
511+
type: 'TextView',
512+
identifier: 'android:id/title',
513+
label: 'Apps',
514+
rect: { x: 0, y: 196, width: 300, height: 48 },
515+
depth: 3,
516+
parentIndex: 6,
517+
},
518+
]);
519+
}
520+
521+
test('computeTargetEvidence: a shared Android framework id (matchCount 3 > 1) is demoted to role+label, and still verifies via the now-selective label', () => {
522+
const nodes = androidSettingsListFixture();
523+
const winner = findByLabel(nodes, 'Network & internet');
524+
const evidence = computeTargetEvidence({ node: winner, preActionNodes: nodes });
525+
assert.ok(evidence);
526+
assert.equal(
527+
evidence.id,
528+
undefined,
529+
'the non-unique android:id/title must not be recorded as identity',
530+
);
531+
assert.equal(evidence.role, 'textview');
532+
assert.equal(evidence.label, 'Network & internet');
533+
assert.deepEqual(evidence.ancestry[0], { role: 'linearlayout' });
534+
// With the id demoted, role+label uniquely isolates this row among the
535+
// three sharing the id, so the record-time self-check still verifies —
536+
// this is the measured "get-only stripped-id: 20/20 clean" fix.
537+
assert.equal(evidence.verification, 'verified');
538+
});
539+
540+
test('computeTargetEvidence: an RN FlatList reusing one testID across rows (iOS/RN, matchCount > 1) is demoted the same way', () => {
541+
// The class is not Android-specific: an RN `FlatList` `renderItem` that
542+
// assigns the SAME testID to every row reproduces one shared accessibility
543+
// identifier across siblings on iOS too. Capture-time uniqueness — not an
544+
// `android:id/*` namespace check — is the rule.
545+
const nodes = toSnapshotNodes([
546+
{ index: 0, type: 'Application', depth: 0 },
547+
{ index: 1, type: 'ScrollView', identifier: 'contacts-list', depth: 1, parentIndex: 0 },
548+
{
549+
index: 2,
550+
type: 'Cell',
551+
identifier: 'contact-row',
552+
label: 'Ada Lovelace',
553+
rect: { x: 0, y: 0, width: 320, height: 60 },
554+
depth: 2,
555+
parentIndex: 1,
556+
},
557+
{
558+
index: 3,
559+
type: 'Cell',
560+
identifier: 'contact-row',
561+
label: 'Grace Hopper',
562+
rect: { x: 0, y: 60, width: 320, height: 60 },
563+
depth: 2,
564+
parentIndex: 1,
565+
},
566+
{
567+
index: 4,
568+
type: 'Cell',
569+
identifier: 'contact-row',
570+
label: 'Katherine Johnson',
571+
rect: { x: 0, y: 120, width: 320, height: 60 },
572+
depth: 2,
573+
parentIndex: 1,
574+
},
575+
]);
576+
const winner = findByLabel(nodes, 'Grace Hopper');
577+
const evidence = computeTargetEvidence({ node: winner, preActionNodes: nodes });
578+
assert.ok(evidence);
579+
assert.equal(evidence.id, undefined, 'the shared testID must not be recorded as identity');
580+
assert.equal(evidence.role, 'cell');
581+
assert.equal(evidence.label, 'Grace Hopper');
582+
assert.equal(evidence.verification, 'verified');
583+
});
584+
585+
test('computeTargetEvidence: a unique id is still recorded as identity (already-clean case unchanged)', () => {
586+
const nodes = toolbarFixture();
587+
const winner = findByLabel(nodes, 'Save');
588+
const evidence = computeTargetEvidence({ node: winner, preActionNodes: nodes });
589+
assert.ok(evidence);
590+
assert.equal(evidence.id, 'save');
591+
});

‎src/daemon/handlers/session-replay-divergence.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -557,6 +557,7 @@ export function buildReplayDivergenceSuggestionForNode(params: {
557557
const selectorChain = buildSelectorChainForNode(node, session.device.platform, {
558558
action:
559559
action.command === 'fill' ? 'fill' : isTouchTargetCommand(action.command) ? 'click' : 'get',
560+
nodes,
560561
});
561562
const role = formatRole(node.type ?? 'Element');
562563
const label = displayLabel(node, role);

‎src/daemon/session-target-evidence.ts‎

Lines changed: 29 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ export function computeTargetEvidence(
4545
const { node, preActionNodes: nodes } = capture;
4646
if (typeof node.index !== 'number') return undefined;
4747
const byIndex = buildIndexMap(nodes);
48-
const identity = boundedLocalIdentity(node);
48+
const identity = demoteNonUniqueId(boundedLocalIdentity(node), nodes, byIndex);
4949
const ancestryWalk = buildAncestryChain(node, byIndex, TARGET_ANNOTATION_MAX_ANCESTRY);
5050
const fullAncestry = ancestryWalk.chain;
5151
const sibling = computeSiblingOrdinal(nodes, node);
@@ -237,6 +237,34 @@ export function filterIdentitySet(
237237
});
238238
}
239239

240+
/**
241+
* ADR 0012 decision 3 amendment (#1269): an id is identity only when it
242+
* uniquely denotes the target in the record-time tree. `boundedLocalIdentity`
243+
* reads a node's id unconditionally, but a shared framework resource id
244+
* (Android's `android:id/title` matching every list row is the measured
245+
* case — #1269) is not selective: on replay the id-led identity set spans
246+
* every row, position drifts, and verification correctly refuses a
247+
* confident bind. Passing an empty `ancestry` to `filterIdentitySet`
248+
* degrades its ancestry-prefix check to vacuously-true, so it returns
249+
* exactly the nodes sharing this id anywhere in the tree — the id's own
250+
* capture-time match count, independent of structural context. When that
251+
* count is greater than one, fall back to role+label, exactly the identity
252+
* an unrecorded id already computes. The rule is capture-time uniqueness,
253+
* not an id-namespace heuristic: a reused RN `FlatList` `testID` hits the
254+
* same demotion on iOS.
255+
*/
256+
function demoteNonUniqueId(
257+
identity: LocalIdentity,
258+
nodes: readonly SnapshotNode[],
259+
byIndex: Map<number, SnapshotNode>,
260+
): LocalIdentity {
261+
if (identity.id === undefined) return identity;
262+
const idMatchCount = filterIdentitySet(nodes, byIndex, identity, []).length;
263+
if (idMatchCount <= 1) return identity;
264+
const { role, label } = identity;
265+
return { role, ...(label !== undefined ? { label } : {}) };
266+
}
267+
240268
function computeDisambiguationDomain(params: {
241269
nodes: readonly SnapshotNode[];
242270
byIndex: Map<number, SnapshotNode>;

‎src/selectors/build.test.ts‎

Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
import { test } from 'vitest';
2+
import assert from 'node:assert/strict';
3+
import type { SnapshotNode } from '../kernel/snapshot.ts';
4+
import { buildNodes } from '../__tests__/test-utils/snapshot-builders.ts';
5+
import { buildSelectorChainForNode } from './build.ts';
6+
7+
function findByLabel(nodes: SnapshotNode[], label: string): SnapshotNode {
8+
const found = nodes.find((node) => node.label === label);
9+
if (!found) throw new Error(`fixture missing node with label ${label}`);
10+
return found;
11+
}
12+
13+
// ---------------------------------------------------------------------------
14+
// #1269: a recorded id only leads the chain when it is unique in the
15+
// record-time tree. `android:id/title` — a shared Android framework
16+
// resource id present on every titled list row — is the measured repro
17+
// shape; matchCount reflects how many nodes in `nodes` carry the id.
18+
// ---------------------------------------------------------------------------
19+
20+
function androidSettingsListFixture(): SnapshotNode[] {
21+
return buildNodes([
22+
{ index: 0, type: 'FrameLayout', depth: 0 },
23+
{ index: 1, type: 'RecyclerView', depth: 1, parentIndex: 0 },
24+
// Each row is a label-less LinearLayout wrapper (real Android list shape)
25+
// whose title TextView shares the same framework resource id.
26+
{ index: 2, type: 'LinearLayout', depth: 2, parentIndex: 1 },
27+
{
28+
index: 3,
29+
type: 'TextView',
30+
identifier: 'android:id/title',
31+
label: 'Network & internet',
32+
rect: { x: 0, y: 100, width: 300, height: 48 },
33+
depth: 3,
34+
parentIndex: 2,
35+
},
36+
{ index: 4, type: 'LinearLayout', depth: 2, parentIndex: 1 },
37+
{
38+
index: 5,
39+
type: 'TextView',
40+
identifier: 'android:id/title',
41+
label: 'Connected devices',
42+
rect: { x: 0, y: 148, width: 300, height: 48 },
43+
depth: 3,
44+
parentIndex: 4,
45+
},
46+
{ index: 6, type: 'LinearLayout', depth: 2, parentIndex: 1 },
47+
{
48+
index: 7,
49+
type: 'TextView',
50+
identifier: 'android:id/title',
51+
label: 'Apps',
52+
rect: { x: 0, y: 196, width: 300, height: 48 },
53+
depth: 3,
54+
parentIndex: 6,
55+
},
56+
]);
57+
}
58+
59+
test('buildSelectorChainForNode: a shared Android framework id (matchCount > 1) is demoted — chain leads with role+label, not id', () => {
60+
const nodes = androidSettingsListFixture();
61+
const row = findByLabel(nodes, 'Network & internet');
62+
const chain = buildSelectorChainForNode(row, 'android', { action: 'get', nodes });
63+
assert.deepEqual(chain, [
64+
'role="textview" label="Network & internet"',
65+
'label="Network & internet"',
66+
]);
67+
assert.ok(
68+
!chain.some((entry) => entry.startsWith('id=')),
69+
'a non-unique id must never lead (or appear in) the recorded selector chain',
70+
);
71+
});
72+
73+
test('buildSelectorChainForNode: an RN FlatList reusing one testID across rows (iOS/RN, matchCount > 1) is demoted the same way', () => {
74+
// RN's common FlatList antipattern: `renderItem` assigns the SAME testID to
75+
// every row. On iOS this surfaces as one accessibility identifier shared by
76+
// every cell — the same non-selective-id class as the Android framework
77+
// resource id above, reproduced on a different platform to prove the rule
78+
// is capture-time uniqueness, not an `android:id/*` namespace check.
79+
const nodes = buildNodes([
80+
{ index: 0, type: 'Application', depth: 0 },
81+
{ index: 1, type: 'ScrollView', identifier: 'contacts-list', depth: 1, parentIndex: 0 },
82+
{
83+
index: 2,
84+
type: 'Cell',
85+
identifier: 'contact-row',
86+
label: 'Ada Lovelace',
87+
rect: { x: 0, y: 0, width: 320, height: 60 },
88+
depth: 2,
89+
parentIndex: 1,
90+
},
91+
{
92+
index: 3,
93+
type: 'Cell',
94+
identifier: 'contact-row',
95+
label: 'Grace Hopper',
96+
rect: { x: 0, y: 60, width: 320, height: 60 },
97+
depth: 2,
98+
parentIndex: 1,
99+
},
100+
{
101+
index: 4,
102+
type: 'Cell',
103+
identifier: 'contact-row',
104+
label: 'Katherine Johnson',
105+
rect: { x: 0, y: 120, width: 320, height: 60 },
106+
depth: 2,
107+
parentIndex: 1,
108+
},
109+
]);
110+
const row = findByLabel(nodes, 'Grace Hopper');
111+
const chain = buildSelectorChainForNode(row, 'ios', { action: 'get', nodes });
112+
assert.deepEqual(chain, ['role="cell" label="Grace Hopper"', 'label="Grace Hopper"']);
113+
assert.ok(!chain.some((entry) => entry.startsWith('id=')));
114+
});
115+
116+
test('buildSelectorChainForNode: a unique id still leads the chain (existing already-clean behavior is preserved)', () => {
117+
const nodes = buildNodes([
118+
{ index: 0, type: 'Window', depth: 0 },
119+
{
120+
index: 1,
121+
type: 'Button',
122+
identifier: 'save',
123+
label: 'Save',
124+
rect: { x: 10, y: 10, width: 40, height: 20 },
125+
depth: 1,
126+
parentIndex: 0,
127+
},
128+
]);
129+
const node = findByLabel(nodes, 'Save');
130+
const chain = buildSelectorChainForNode(node, 'ios', { action: 'get', nodes });
131+
assert.deepEqual(chain, ['id="save"', 'role="button" label="Save"', 'label="Save"']);
132+
});
133+
134+
test('buildSelectorChainForNode: without a record-time tree, an id is trusted as-is (back-compat default)', () => {
135+
const nodes = buildNodes([
136+
{
137+
index: 0,
138+
type: 'Button',
139+
identifier: 'save',
140+
label: 'Save',
141+
rect: { x: 10, y: 10, width: 40, height: 20 },
142+
depth: 0,
143+
},
144+
]);
145+
const chain = buildSelectorChainForNode(nodes[0]!, 'ios', { action: 'get' });
146+
assert.deepEqual(chain, ['id="save"', 'role="button" label="Save"', 'label="Save"']);
147+
});

0 commit comments

Comments
 (0)