Skip to content

Commit 5a05a3e

Browse files
committed
fix(commands): keep clickability evidence across the snapshot state merge (#3182)
The state seam added the backend's keyboard and viewport facts to the state without carrying the private clickability evidence over, and that evidence is retained by object identity. Android Maestro taps then read `exact-evidence-not-retained`, skip clickable-first ordering, and tap the document-order duplicate of a requested id — the Android smoke lane's no-index `tapOn` failure, which is this PR's, not a flake. The mutation that makes the new test fail: remove the copy and the evidence read on the published result is undefined.
1 parent 8ff568a commit 5a05a3e

2 files changed

Lines changed: 41 additions & 2 deletions

File tree

‎src/commands/capture/runtime/snapshot.test.ts‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,10 @@ import {
1515
import { makeSnapshotState } from '@agent-device/selectors/snapshot-geometry-fixtures';
1616
import type { PostGestureOutcome } from '@agent-device/kernel/snapshot';
1717
import { snapshotViewportSizeFrom } from '@agent-device/kernel/rect';
18+
import {
19+
attachSnapshotClickabilityEvidence,
20+
readSnapshotClickabilityEvidence,
21+
} from '@agent-device/contracts/capture';
1822
import { formatPostGestureOutcomeWarning } from '@agent-device/capture-kit/post-gesture-stability';
1923

2024
test('runtime snapshot captures nodes and updates the session baseline', async () => {
@@ -835,3 +839,35 @@ test('runtime snapshot leaves the viewport off when the producer measured no box
835839

836840
assert.equal('viewport' in result, false);
837841
});
842+
843+
// The Android helper's clickability evidence is retained by object identity, and this command
844+
// merges the backend's facts into a copy of the state it hands over (#3182). A copy that forgets
845+
// the evidence is silent until Maestro's clickable-first ordering finds no exact evidence, falls
846+
// back to document order, and taps an inert duplicate of a tapped id (Android smoke lane).
847+
test('runtime snapshot carries backend clickability evidence through the response merge', async () => {
848+
const clickability = {
849+
kind: 'exact',
850+
provider: 'android-helper',
851+
clickableByNodeIndex: new Map([
852+
[0, false],
853+
[1, true],
854+
]),
855+
} as const;
856+
const state = makeSnapshotState(
857+
[
858+
{ index: 0, depth: 0, type: 'TextView', label: 'Inert duplicate target' },
859+
{ index: 1, depth: 0, type: 'Button', label: 'Clickable duplicate target' },
860+
],
861+
{ backend: 'android', producer: 'android-uiautomator' },
862+
);
863+
attachSnapshotClickabilityEvidence(state, clickability);
864+
const device = createSnapshotOnlyDevice({
865+
snapshot: state,
866+
viewport: snapshotViewportSizeFrom({ x: 0, y: 0, width: 1080, height: 2400 }),
867+
});
868+
869+
const result = await device.capture.snapshot({ session: 'default' });
870+
871+
assert.deepEqual(readSnapshotClickabilityEvidence(result), clickability);
872+
assert.deepEqual(result.viewport, { width: 1080, height: 2400 });
873+
});

‎src/commands/capture/runtime/snapshot.ts‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -209,12 +209,15 @@ function normalizeBackendSnapshot(
209209
// A backend may hand over state it built itself, and the response-level facts it publishes beside
210210
// that state travel with this capture: returning the state alone would drop a producer's own
211211
// keyboard band or viewport box (#3182) on the one path where the backend named them itself.
212+
// The copy must carry the private clickability evidence with it: that fact is retained by object
213+
// identity, so a spread that forgets it makes Maestro's clickable-first ordering fall back to
214+
// divergence and tap the document-order (inert) duplicate of a tapped id.
212215
if (result.snapshot) {
213-
return {
216+
return copySnapshotClickabilityEvidence(result.snapshot, {
214217
...result.snapshot,
215218
...(result.keyboard ? { keyboard: result.keyboard } : {}),
216219
...(result.viewport ? { viewport: result.viewport } : {}),
217-
};
220+
});
218221
}
219222
return {
220223
nodes: result.nodes ?? [],

0 commit comments

Comments
 (0)