Skip to content

Commit 96e1a80

Browse files
committed
feat(wait): carry a per-poll timeline in timeout failures
A wait timeout said `reason`, `readableCaptures`, and `waitedMs`, so a failure could not say where its budget went: the runs behind #2343 spent a 10s budget on one poll (5.8s runner findText on a fresh app, 3.4s of target discovery, a fallback cancelled at the deadline) and reported the same `wait_capture_stalled` as a dead runner. The failure details now carry `captures` and `polls[]`, one entry per poll with `startedMs` on the wait's own clock, `durationMs`, and a typed outcome (readable, unreadable, deadline, runner-restart), next to the unchanged reason and the request-log link. Long waits keep the first five and last twenty-five polls so the response stays compact.
1 parent 27a97ee commit 96e1a80

4 files changed

Lines changed: 97 additions & 0 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,10 @@
22

33
## Unreleased
44

5+
- Added: `wait` timeout errors carry a per-poll timeline in `error.details` (`captures`,
6+
`polls[]` with `startedMs`, `durationMs`, and a typed `outcome`: readable, unreadable, deadline,
7+
runner-restart) next to the unchanged `reason`, so a failure says where its budget went without
8+
opening the request log. Long waits keep the first five and last twenty-five polls.
59
- Fixed: iOS snapshots no longer report `truncated: true` merely because a later backend produced
610
them. The runner stamped every recovered capture as truncated — including a complete private-AX
711
tree taken while the XCTest channel was penalized as slow — so a strict `is absent` / `wait absent`

‎src/commands/interaction/runtime/wait-polling.test.ts‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,3 +41,61 @@ test('poll delay observes both runtime and command cancellation', async () => {
4141
await assert.rejects(sleeping, reason);
4242
}
4343
});
44+
45+
test('failure evidence carries every poll on the wait clock with its typed outcome', async () => {
46+
let currentMs = 0;
47+
const runtime = {
48+
clock: { now: () => currentMs, sleep: async (_durationMs: number) => {} },
49+
} as AgentDeviceRuntime;
50+
const unreadable = new Error('content verdict');
51+
const polling = createWaitPolling(runtime, {}, 60, SELECTOR_PIPELINE_POLICIES.wait, {
52+
isUnreadableError: (error) => error === unreadable,
53+
});
54+
55+
await polling.capture(async () => {
56+
currentMs += 20;
57+
return 'seen';
58+
});
59+
await polling.capture(async () => {
60+
currentMs += 10;
61+
throw unreadable;
62+
});
63+
// The third capture outlives the remaining real-time budget: the deadline cancels it.
64+
const last = await polling.capture(async () => {
65+
await new Promise((resolve) => setTimeout(resolve, 120));
66+
currentMs += 40;
67+
return 'late';
68+
});
69+
70+
assert.equal(last.timedOut, true);
71+
const evidence = polling.failureEvidence();
72+
assert.equal(evidence.captures, 3);
73+
assert.equal(evidence.readableCaptures, 1);
74+
assert.deepEqual(evidence.polls, [
75+
{ startedMs: 0, durationMs: 20, outcome: 'readable' },
76+
{ startedMs: 20, durationMs: 10, outcome: 'unreadable' },
77+
{ startedMs: 30, durationMs: 40, outcome: 'deadline' },
78+
]);
79+
});
80+
81+
test('a long wait keeps its first polls and its last polls in the failure evidence', async () => {
82+
let currentMs = 0;
83+
const runtime = {
84+
clock: { now: () => currentMs, sleep: async (_durationMs: number) => {} },
85+
} as AgentDeviceRuntime;
86+
const polling = createWaitPolling(runtime, {}, 100_000, SELECTOR_PIPELINE_POLICIES.wait);
87+
for (let index = 0; index < 40; index += 1) {
88+
await polling.capture(async () => {
89+
currentMs += 100;
90+
return index;
91+
});
92+
}
93+
94+
const evidence = polling.failureEvidence();
95+
assert.equal(evidence.captures, 40);
96+
assert.equal(evidence.polls.length, 30);
97+
assert.equal(evidence.polls[0]?.startedMs, 0);
98+
assert.equal(evidence.polls[4]?.startedMs, 400);
99+
assert.equal(evidence.polls[5]?.startedMs, 1_500);
100+
assert.equal(evidence.polls.at(-1)?.startedMs, 3_900);
101+
});

‎src/commands/interaction/runtime/wait-polling.ts‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,30 @@ export const DEFAULT_WAIT_TIMEOUT_MS = SELECTOR_PIPELINE_POLICIES.wait.poll.defa
1717

1818
export type WaitPollDeadline = 'capture-stalled' | 'capture-truncated' | 'runner-restart-exhausted';
1919

20+
/**
21+
* How one poll ended: a readable capture, an unreadable content verdict the wait rode out, the
22+
* deadline cancelling the capture in flight, or that cancellation carrying runner-restart
23+
* evidence. Whether a readable capture matched is the caller's verdict, not the poll's.
24+
*/
25+
export type WaitPollOutcome = 'readable' | 'unreadable' | 'deadline' | 'runner-restart';
26+
27+
/** One poll on the wait's own clock: when it started after the wait began and how long it ran. */
28+
export type WaitPollRecord = {
29+
startedMs: number;
30+
durationMs: number;
31+
outcome: WaitPollOutcome;
32+
};
33+
34+
/** Keeps a long wait's failure compact: the first polls carry the cold-start cost, the last the end. */
35+
const WAIT_POLL_TIMELINE_HEAD = 5;
36+
const WAIT_POLL_TIMELINE_TAIL = 25;
37+
2038
export type WaitFailureEvidence = {
2139
timeoutMs: number;
2240
readableCaptures: number;
41+
/** Every poll attempted, readable or not. */
42+
captures: number;
43+
polls: WaitPollRecord[];
2344
waitedMs: number;
2445
runnerRestarted?: true;
2546
runnerRestartReason?: string;
@@ -77,12 +98,16 @@ export function createWaitPolling(
7798
const timeoutMs = requestedTimeoutMs ?? budget.defaultTimeoutMs;
7899
const startedAtMs = now(runtime);
79100
const unreadable = createUnreadablePollTracker(classification.isUnreadableError);
101+
const polls: WaitPollRecord[] = [];
80102
let timeoutEvidence: Partial<WaitFailureEvidence> = {};
81103
const remainingMs = () => Math.max(0, timeoutMs - (now(runtime) - startedAtMs));
82104

83105
return {
84106
capture: async <T>(capture: (signal: AbortSignal) => Promise<T>) => {
85107
let captureWasReadable = false;
108+
const startedMs = now(runtime) - startedAtMs;
109+
const recordPoll = (outcome: WaitPollOutcome) =>
110+
polls.push({ startedMs, durationMs: now(runtime) - startedAtMs - startedMs, outcome });
86111
const result = await runWithinWaitDeadline(
87112
runtime,
88113
options,
@@ -96,9 +121,11 @@ export function createWaitPolling(
96121
);
97122
if (!result.timedOut) {
98123
if (captureWasReadable) unreadable.recordReadableCapture();
124+
recordPoll(captureWasReadable ? 'readable' : 'unreadable');
99125
return result;
100126
}
101127
const runnerRestart = runnerRestartTimeoutEvidence(result.error);
128+
recordPoll(runnerRestart ? 'runner-restart' : 'deadline');
102129
timeoutEvidence = runnerRestart ?? {};
103130
// A capture that only becomes readable after its deadline is not evidence for this wait.
104131
// Count only captures that completed before runWithinWaitDeadline returned a timeout.
@@ -119,6 +146,8 @@ export function createWaitPolling(
119146
failureEvidence: (): WaitFailureEvidence => ({
120147
timeoutMs,
121148
readableCaptures: unreadable.readableCaptures(),
149+
captures: polls.length,
150+
polls: compactPollTimeline(polls),
122151
waitedMs: now(runtime) - startedAtMs,
123152
...timeoutEvidence,
124153
}),
@@ -131,6 +160,11 @@ export function createWaitPolling(
131160
};
132161
}
133162

163+
function compactPollTimeline(polls: readonly WaitPollRecord[]): WaitPollRecord[] {
164+
if (polls.length <= WAIT_POLL_TIMELINE_HEAD + WAIT_POLL_TIMELINE_TAIL) return [...polls];
165+
return [...polls.slice(0, WAIT_POLL_TIMELINE_HEAD), ...polls.slice(-WAIT_POLL_TIMELINE_TAIL)];
166+
}
167+
134168
function waitCaptureStalledError(message: string, evidence: WaitFailureEvidence): AppError {
135169
return new AppError('COMMAND_FAILED', message, {
136170
reason: WAIT_REASONS.captureStalled,

‎website/docs/docs/commands.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -419,6 +419,7 @@ agent-device alert dismiss
419419
- Because `wait @ref` is text-based after resolution, duplicate labels can match a different element than the original ref target.
420420
- `wait` shares the selector/snapshot resolution flow used by `click`, `fill`, `get`, and `is`.
421421
- Wait failures carry a structured `error.details.reason` in `--json` output: `wait_target_absent` proves a positive wait never found a match; `wait_target_present` means strict `wait absent` reached its deadline with valid captures that still contained matches; `predicate_failed` means strict `wait absent` could not prove absence because no valid capture arrived, with the final observation/diagnostic preserved; `wait_capture_stalled` means no readable capture arrived and is retriable; `wait_deadline_exceeded` means a later capture consumed the remaining budget after an earlier readable capture; `wait_landmark_identity_mismatch` is a replay destination-guard refusal; and `wait_stable_timeout` means the UI did not settle. Use `readableCaptures`, `waitedMs`, `matches`, and `firstMatch` instead of parsing error text. `firstMatch` carries identity/text evidence only; absence failures do not claim visibility or rect evidence.
422+
- Wait failures also carry `captures` (every poll attempted), `readableCaptures`, and `polls`, one entry per poll with `startedMs` on the wait's own clock, `durationMs`, and `outcome` (`readable`, `unreadable`, `deadline`, or `runner-restart`), so a timeout says where its budget went; long waits keep the first five and last twenty-five polls. `logPath` links the full request log.
422423
- `alert` inspects or handles system alerts on iOS simulator, macOS desktop, and Android native/runtime permission dialogs.
423424
- `alert` without an action is equivalent to `alert get`.
424425
- `accept` and `dismiss` are sent once on every platform. A lost or unconfirmed response is reported as an error and never replayed; run `alert get` before acting again.

0 commit comments

Comments
 (0)