Skip to content

Commit eab7f34

Browse files
committed
fix(selectors): retire the stored tree after a side effect or a native read
Selector reads reuse the session's stored tree for 750 ms after its capture. Two events outdated that tree without replacing it: a wait text satisfied by the owner's native text reading after an earlier miss capture, and a mutation that captured nothing. The next get or is then reused the pre-change tree and reported Selector did not match. Every side-effect seam (expireRefFrame) and every native text reading now marks the session's current tree outdated, and the session cache tier refuses an outdated tree. The mark is keyed by the tree, so it survives the session record copies that reset the per-session runtime revision. Closes #2866
1 parent 9fac266 commit eab7f34

7 files changed

Lines changed: 140 additions & 12 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,13 @@
22

33
## Unreleased
44

5+
- Fixed (mobile): a `get`, `is`, `find`, or `wait` selector read no longer answers from a tree that
6+
a later command outdated. Selector reads reuse the session's stored tree for 750 ms after its
7+
capture. Two cases outdated that tree without replacing it: a `wait text` satisfied by the
8+
owner's native text reading after an earlier miss capture, and a mutation that captured nothing
9+
(for example a coordinate `press`). The next read then reused the pre-change tree and could
10+
report `Selector did not match` for an element that was on screen. A side effect or a native read
11+
now retires the stored tree, and the next read captures again.
512
- Fixed (mobile): a read taken right after a `scroll`, `swipe`, or `gesture swipe` no longer reports
613
a definite miss when the surface never settled. When post-gesture stabilization ran out of budget
714
on a surface still moving, `is visible` answered a plain `selector_not_found` and `is absent`

‎docs/agents/selector-capture.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,9 @@ These are cross-route behavior requirements; their rationale and owning decision
99
resolution; ambiguity and other runner failures remain failures.
1010
- Regular selector reads are capture-backed. `@ref` resolves against its authorized ref frame,
1111
while `get`, `is`, `find`, and `wait` selectors capture through the backend. Polling bypasses the
12-
snapshot cache, as do active freshness recovery and stabilization.
12+
snapshot cache, as do active freshness recovery and stabilization. The cache serves a stored tree
13+
only while it is the newest observation: a side-effect seam or a native read such as `wait
14+
text`'s owner text reading retires it.
1315
- Sparse capture verdicts are observable failures and never replace the session snapshot. Only a
1416
user-facing snapshot may publish a fallback screenshot; internal polling must not create one
1517
artifact per attempt.

‎src/daemon/__tests__/wait-runtime.test.ts‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ import { unavailableDeploymentSnapshotAndShutdownOperationFacts } from '../../__
2626
import type { BindDeviceRuntime, InspectDeviceRuntimeFacts } from '../request-runtime-binding.ts';
2727
import { handleSnapshotCommands } from '../handlers/snapshot.ts';
2828
import { resolveBoundSelectorCapture } from '../selector-capture-binding.ts';
29+
import { dispatchGetViaRuntime } from '../selector-runtime.ts';
2930
import type { DaemonRequest } from '../daemon-request.ts';
3031

3132
const webDevice = {
@@ -389,6 +390,53 @@ test('a text wait is satisfied by the owner native reading when the tree never c
389390
expect(harness.captureSnapshot).toHaveBeenCalled();
390391
});
391392

393+
test('a read after a natively satisfied text wait captures instead of reusing the older tree', async () => {
394+
// Poll 1: the native reading misses and the capture still shows the previous screen, which it
395+
// publishes to the session. The app then navigates, and poll 2's native reading sees the
396+
// destination, so the stored tree is older than the observation that satisfied the wait.
397+
let nativeReads = 0;
398+
const harness = waitRuntimeHarness({
399+
findText: available,
400+
findTextAnswers: () => {
401+
nativeReads += 1;
402+
return nativeReads > 1;
403+
},
404+
nodesPerPoll: [
405+
[{ index: 0, depth: 0, type: 'StaticText', label: 'Home' }],
406+
[
407+
{ index: 0, depth: 0, type: 'StaticText', label: 'Automation lab' },
408+
{ index: 1, depth: 0, type: 'StaticText', label: 'cold.start' },
409+
],
410+
],
411+
});
412+
const {
413+
response: waited,
414+
session,
415+
sessionStore,
416+
} = await runWait(['text', 'Automation lab', '2000'], harness);
417+
expect(waited).toMatchObject({ ok: true, data: { text: 'Automation lab' } });
418+
expect(harness.captureSnapshot).toHaveBeenCalledOnce();
419+
420+
const read = await dispatchGetViaRuntime({
421+
req: {
422+
command: 'get',
423+
positionals: ['text', 'label="cold.start"'],
424+
token: 't',
425+
session: session.name,
426+
flags: {},
427+
meta: { requestId: 'wait-runtime-get' },
428+
} as unknown as DaemonRequest,
429+
sessionName: session.name,
430+
logPath: '/tmp/daemon.log',
431+
sessionStore,
432+
inspectFacts: harness.inspectFacts,
433+
bindDevice: harness.bindDevice,
434+
});
435+
436+
expect(read).toMatchObject({ ok: true, data: { text: 'cold.start' } });
437+
expect(harness.captureSnapshot).toHaveBeenCalledTimes(2);
438+
});
439+
392440
test('a satisfied native reading short-circuits the poll without capturing', async () => {
393441
const harness = waitRuntimeHarness({
394442
findText: available,

‎src/daemon/interaction/internal/__tests__/interaction-touch-press.test.ts‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
import { test, expect, vi, beforeEach } from 'vitest';
22
import { attachRefs } from '@agent-device/kernel/snapshot';
3+
import { makeIosSession } from '../../../../__tests__/test-utils/session-factories.ts';
34
import { makeSessionStore } from '../../../../__tests__/test-utils/store-factory.ts';
45
import { handleInteractionCommands } from '../../index.ts';
56
import {
@@ -391,3 +392,55 @@ test('#1654: the shared guards still run on the pre-resolved node', async () =>
391392
}
392393
expect(readPressPoint(mockTapPoint)).toBeUndefined();
393394
});
395+
396+
test('a read right after a press captures the post-tap screen instead of reusing the pre-tap tree', async () => {
397+
const sessionStore = makeSessionStore();
398+
const sessionName = 'read-after-press';
399+
sessionStore.set(sessionName, makeIosSession(sessionName, { appBundleId: 'com.example.app' }));
400+
const screen = (step: string) => ({
401+
backend: 'xctest' as const,
402+
producer: 'apple-runner' as const,
403+
nodes: [
404+
{ index: 0, depth: 0, type: 'Application', rect: { x: 0, y: 0, width: 393, height: 852 } },
405+
{
406+
index: 1,
407+
depth: 1,
408+
parentIndex: 0,
409+
type: 'Button',
410+
label: 'Next',
411+
rect: { x: 24, y: 120, width: 120, height: 44 },
412+
enabled: true,
413+
hittable: true,
414+
},
415+
{
416+
index: 2,
417+
depth: 1,
418+
parentIndex: 0,
419+
type: 'StaticText',
420+
label: step,
421+
identifier: 'step',
422+
rect: { x: 24, y: 220, width: 320, height: 24 },
423+
},
424+
],
425+
});
426+
mockCaptureSnapshotForSession
427+
.mockResolvedValueOnce(screen('Step 1'))
428+
.mockResolvedValue(screen('Step 2'));
429+
const run = async (command: string, positionals: string[]) =>
430+
await handleInteractionCommands({
431+
req: { token: 't', session: sessionName, command, positionals, flags: {} },
432+
sessionName,
433+
sessionStore,
434+
contextFromFlags,
435+
...getRuntimeBindings(),
436+
});
437+
438+
expect(await run('is', ['visible', 'label=Next'])).toMatchObject({ ok: true });
439+
expect(await run('press', ['84', '142'])).toMatchObject({ ok: true });
440+
expect(mockCaptureSnapshotForSession).toHaveBeenCalledTimes(1);
441+
442+
const read = await run('get', ['text', 'id="step"']);
443+
444+
expect(read).toMatchObject({ ok: true, data: { text: 'Step 2' } });
445+
expect(mockCaptureSnapshotForSession).toHaveBeenCalledTimes(2);
446+
});

‎src/daemon/ref-frame.ts‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ import {
1010
import type { SessionState } from './session-state.ts';
1111

1212
const runtimeRevisions = new WeakMap<SessionState, number>();
13+
const outdatedObservations = new WeakSet<SnapshotState>();
1314

1415
/**
1516
* ADR 0014 session ref-frame lifetime — the authorization model for mutation
@@ -89,10 +90,25 @@ export function refFrameTree(session: SessionState): SnapshotState | undefined {
8990
*/
9091
export function expireRefFrame(session: SessionState): void {
9192
advanceSessionRuntimeRevision(session);
93+
markSessionSnapshotOutdated(session);
9294
session.refFrame = expiredRefFrame(refFrame(session));
9395
session.snapshotScopeSource = undefined;
9496
}
9597

98+
/**
99+
* Record that the device was observed or changed after the session's stored tree was
100+
* captured: a side-effect seam above, or a native read that produces no tree (such as `wait
101+
* text`'s owner text reading). An outdated tree stays the session's latest stored observation,
102+
* but a selector read never reuses it in place of a capture.
103+
*/
104+
export function markSessionSnapshotOutdated(session: SessionState): void {
105+
if (session.snapshot) outdatedObservations.add(session.snapshot);
106+
}
107+
108+
export function isOutdatedObservation(snapshot: SnapshotState): boolean {
109+
return outdatedObservations.has(snapshot);
110+
}
111+
96112
/**
97113
* Monotonic, daemon-private revision for side-effect lineage. Unlike the
98114
* client-visible snapshot/ref generations, this advances for every possible

‎src/daemon/selector-capture-runtime.ts‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ import { captureSnapshot } from './snapshot-capture.ts';
1515
import { setSessionSnapshot } from './session-snapshot.ts';
1616
import { getActiveAndroidSnapshotFreshness } from './session-snapshot-freshness.ts';
1717
import { isPostGestureStabilizationPending } from './deferred-interaction-outcome.ts';
18+
import { isOutdatedObservation } from './ref-frame.ts';
1819
import type { BoundSelectorCapture } from './selector-capture-binding.ts';
1920
import { buildRuntimeCaptureInput } from './snapshot-runtime-capture-input.ts';
2021
import { isLegacySparseIosInteractiveSnapshot } from '@agent-device/selectors/absence-observation';
@@ -258,7 +259,7 @@ function reusableSessionSnapshot(params: {
258259
}): SnapshotState | undefined {
259260
const { session, timestamp, request } = params;
260261
const snapshot = session?.snapshot;
261-
if (!snapshot) return undefined;
262+
if (!snapshot || isOutdatedObservation(snapshot)) return undefined;
262263
if (!canUseSessionSnapshotCache(session, request)) return undefined;
263264
if (!isFreshSelectorSnapshot(snapshot, timestamp)) return undefined;
264265
if (snapshot.presentationKey !== presentationKeyFor(request)) return undefined;

‎src/daemon/selector-runtime-backend.ts‎

Lines changed: 11 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ import { createDaemonRuntimeSessionStore } from './runtime-session.ts';
1212
import { contextFromFlags, type BoundContextFromFlags } from './context.ts';
1313
import { readTextForNode } from './selector-text-runtime.ts';
1414
import { setSessionSnapshot } from './session-snapshot.ts';
15+
import { markSessionSnapshotOutdated } from './ref-frame.ts';
1516
import { SessionStore } from './session-store.ts';
1617
import type { DaemonRequest, DaemonResponse } from './daemon-request.ts';
1718
import type { SessionState } from './session-state.ts';
@@ -220,16 +221,16 @@ function createSelectorBackend(params: SelectorRuntimeDeviceParams): AgentDevice
220221
// reports `found: false` and the poll consults the canonical tree.
221222
...(boundFindText
222223
? {
223-
findText: async (context: BackendCommandContext, text: string) => ({
224-
found: (
225-
await boundFindText({
226-
text,
227-
options: { appBundleId: session?.appBundleId, surface: session?.surface },
228-
execution: runnerExecution,
229-
...(context.signal ? { signal: context.signal } : {}),
230-
})
231-
).found,
232-
}),
224+
findText: async (context: BackendCommandContext, text: string) => {
225+
const { found } = await boundFindText({
226+
text,
227+
options: { appBundleId: session?.appBundleId, surface: session?.surface },
228+
execution: runnerExecution,
229+
...(context.signal ? { signal: context.signal } : {}),
230+
});
231+
if (session) markSessionSnapshotOutdated(session);
232+
return { found };
233+
},
233234
}
234235
: {}),
235236
};

0 commit comments

Comments
 (0)