Skip to content

Commit 5b1e1c1

Browse files
committed
fix(alerts): close the window holes the review found and cap the budget
The shared tree-alert driver consumed an answer from a poll that started after its window closed (the loop tests settled before remaining, and an expired-at-start probe rode unguarded), and its first probe paid the Android helper cold start inside the action budget, aborting the very read that leg was about to press on — the executed smoke failure. The first observation is now read-first, unarmed, consumed once, and arms the window behind it; a later probe starting past the deadline never captures at all, and the press rides the armed window. - contracts: a mix of ranked and unranked role candidates refuses instead of letting the one known label win (selectRankedAlertButton- ByRole); MAX_ALERT_WINDOW_MS names the timer ceiling the daemon refuses above (INVALID_ARGS) and the driver defaults above. - android: same-label dedup keys on the physical center, so nested nodes collapse and two distinct OKs reach the shared refusal; the input dispatch option type owns cancellation only, and the tap forward picks fields so allowFailure cannot reach the transport. - docs: alert-window-expired/none/undecided reasons stated exactly per phase, Limrun's non-abortable transport described as post-capture and pre-press guards with the hung-request limit named. Prose at the driver trims to invariants.
1 parent 2aa8370 commit 5b1e1c1

14 files changed

Lines changed: 414 additions & 106 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
- Add `settings mobile-data on|off` for adb-backed Android targets. It switches mobile data through `svc data` and reports the `mobileData` state stored in the global `mobile_data` row; multi-SIM read-back is not covered yet. Every other target refuses it before anything runs, and `--app` is refused because the switch is device-wide.
66
- Limrun iOS sessions now serve `alert get/wait/accept/dismiss` instead of refusing with `unsupported-provider-mode`. System alerts are read from the same element-tree capture snapshots ride — a blank-label, process-exclusive application root hosting a text-and-buttons sheet — and accepted or dismissed by tapping the button whose English label states the asked-for action; a capture that is alert-shaped but unresolvable (degraded buttons, competing sheets, an unverifiable owner) is reported as undecided and never pressed or counted as dismissed. The one shape no capture fact separates is an app dialog under a blanked root shared by no other process; the label-only acting legs answer it with a refusal, never a guess.
7-
- The tree-based alert backends (Android and Limrun iOS) now share one alert driver and one English button-label vocabulary declared as a per-role preference table: `alert accept/dismiss` classify the same label the same way on both platforms, so Android now recognizes iOS variant grants (`Allow Once`, `Allow All the Time`, `Allow While Using App`, `Allow Notifications`) and `Ask App Not to Track` as acting choices, and a sheet offering two grants (Android 11+'s "While using the app"/"Only this time", iOS's "Allow Once"/"Allow While Using App") resolves by the declared preference order rather than tree order or refusal. A choice refuses only when the capture names no single physical button for the action — the same label at two points — and each backend's own policy answers then (Android's neutral-button/Back fallback, Limrun's typed refusal). `alert not found` and the no-matching-button refusal now carry the same typed `details.reason` on both platforms. `alert get` reports its outcome as a typed `status` (`visible`/`none`/`undecided`) so clients switch on states instead of parsing the message, and every alert budget now bounds the capture and press inside it: Android hands the leg's cancellation to the adb executor (which kills the child), and a capture that lands after its window closed — the case Limrun's non-abortable transport can produce — is refused rather than consumed or pressed.
7+
- The tree-based alert backends (Android and Limrun iOS) now share one alert driver and one English button-label vocabulary declared as a per-role preference table: `alert accept/dismiss` classify the same label the same way on both platforms, so Android now recognizes iOS variant grants (`Allow Once`, `Allow All the Time`, `Allow While Using App`, `Allow Notifications`) and `Ask App Not to Track` as acting choices, and a sheet offering two grants (Android 11+'s "While using the app"/"Only this time", iOS's "Allow Once"/"Allow While Using App") resolves by the declared preference order rather than tree order or refusal. A choice refuses only when the capture names no single physical button for the action — the same label at two points — and each backend's own policy answers then (Android's neutral-button/Back fallback, Limrun's typed refusal). `alert not found` and the no-matching-button refusal now carry the same typed `details.reason` on both platforms. `alert get` reports its outcome as a typed `status` (`visible`/`none`/`undecided`) so clients switch on states instead of parsing the message. Every alert budget now bounds the capture and press inside it, with per-transport honesty: Android hands the leg's cancellation to the adb executor, which kills the child in flight; Limrun's client exposes no abort, so its guards answer post-capture and pre-press — a late answer is refused and nothing is pressed, but an already-sent request runs on. An `alert wait` budget above 2147483647 ms is refused as `INVALID_ARGS` rather than misfiring a timer.
88
- Limrun iOS snapshots no longer fail when the instance reports a numeric or boolean accessibility value (the system photo picker's tabs report `1`/`0`): the provider seam normalizes variant scalars to text, the way the Apple tree does. Limrun nodes now also publish `selected: true` for nodes carrying the `Selected` trait, so snapshot output and `selected:` selectors can tell a picked photo from an unpicked one.
99
- `settings` refuses a setting the target does not serve before anything changes, with `UNSUPPORTED_OPERATION`, `settings <setting> is not supported on this device`, and the target's `error.details.reason` (such as `unsupported-platform-leaf`). This replaces the macOS `INVALID_ARGS` refusal and the Apple `setting-unsupported-on-leaf` reason, and now also covers settings that used to reach the device first, such as Android `reset-keychain` or Apple `animations`.
1010
- `settings` refuses, before touching the device, a state its setting does not accept (`Invalid <setting> state: …`), a permission action or mode its target does not take (such as `permission deny accessibility`), and a stray extra argument (such as `clear-app-state <app> <extra>`). The CLI and the daemon accept the same state spellings, in any casing or surrounding whitespace.

‎packages/contracts/src/alert-contract.test.ts‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,23 @@ describe('selectRankedAlertButtonByRole', () => {
3030
expect(selectRankedAlertButtonByRole(buttons, 'accept')).toBeNull();
3131
});
3232

33+
it('refuses a mix of ranked and unranked candidates rather than preferring the known label', () => {
34+
// The unranked one's role came from a platform id classifier, so it is a real grant the
35+
// table cannot order. Letting the one known English label win would press an arbitrary
36+
// grant of an unordered pair.
37+
const buttons: Button[] = [
38+
{ label: 'OK', role: 'accept' },
39+
{ label: 'Unrecognized grant', role: 'accept' },
40+
];
41+
expect(selectRankedAlertButtonByRole(buttons, 'accept')).toBeNull();
42+
});
43+
44+
it('keeps the declared order over mixed English variants of distinct grants', () => {
45+
// The closest positive: both candidates are ranked, so this is a preference, not ambiguity.
46+
const buttons = [button('Allow Once'), button('Allow')];
47+
expect(selectRankedAlertButtonByRole(buttons, 'accept')?.label).toBe('Allow');
48+
});
49+
3350
it('returns null when no button carries the role', () => {
3451
expect(selectRankedAlertButtonByRole([button('Settings')], 'accept')).toBeNull();
3552
});

‎packages/contracts/src/alert-contract.ts‎

Lines changed: 19 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,15 @@ export const ALERT_POLL_INTERVAL_MS = 300;
22
export const DEFAULT_ALERT_TIMEOUT_MS = 10_000;
33
export const ALERT_ACTION_RETRY_MS = 2_000;
44

5+
/**
6+
* The largest alert budget the legs accept. The window signal rides `AbortSignal.timeout`, whose
7+
* Node ceiling is 2^31-1 ms: larger values are rejected outright there, and its underlying timer
8+
* overflows to a 1 ms firing when merely exceeded. The daemon refuses a larger `alert wait`
9+
* budget as `INVALID_ARGS`, and a tree driver reads one as the bounded default — never a
10+
* silently shortened window.
11+
*/
12+
export const MAX_ALERT_WINDOW_MS = 2_147_483_647;
13+
514
/**
615
* The one alert failure the family retries on: the backend looked and there was no alert *yet*.
716
* Both Apple backends state it in typed form — the XCTest runner as this `ErrorPayload.code`
@@ -104,22 +113,23 @@ function alertButtonPreferenceRank(label: string, role: 'accept' | 'dismiss'): n
104113

105114
/**
106115
* The one choice boundary for a shared choice: pick the action's role candidates by the declared
107-
* preference table. A single candidate wins outright; several resolve when one outscores the
108-
* rest; a tie on the same entry (the same label at two points) or vocabulary-ranked-free
109-
* multiplicity returns null, because the capture then names no single physical button for the
110-
* action. Distinct labels that share a role are not a refusal: the table ranks them.
116+
* preference table. A single candidate wins outright. Several resolve only when every one of
117+
* them carries a vocabulary rank and one rank outscores the rest: a mix of ranked and
118+
* id-classified grants cannot be ordered by the table, so it names no single physical button and
119+
* is refused rather than resolved by tree order or by one side's label happening to be known.
111120
*/
112121
export function selectRankedAlertButtonByRole<
113122
TButton extends Readonly<{ role: AlertButtonRole; label: string }>,
114123
>(buttons: readonly TButton[], role: 'accept' | 'dismiss'): TButton | null {
115124
const matching = buttons.filter((button) => button.role === role);
116125
if (matching.length === 1) return matching[0] ?? null;
117126
if (matching.length === 0) return null;
118-
const ranked = matching
119-
.map((button) => ({ button, rank: alertButtonPreferenceRank(button.label, role) }))
120-
.filter((entry): entry is { button: TButton; rank: number } => entry.rank !== null)
121-
.sort((a, b) => a.rank - b.rank);
122-
if (ranked.length === 0) return null;
127+
const ranked = matching.map((button) => ({
128+
button,
129+
rank: alertButtonPreferenceRank(button.label, role),
130+
}));
131+
if (ranked.some((entry) => entry.rank === null)) return null;
132+
ranked.sort((a, b) => (a.rank ?? 0) - (b.rank ?? 0));
123133
const [first, second] = ranked;
124134
if (!first || (second && second.rank === first.rank)) return null;
125135
return first.button;

‎packages/daemon/src/execution/handlers/__tests__/snapshot-alert.test.ts‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -140,3 +140,34 @@ for (const action of ['accept', 'dismiss']) {
140140
expect(successor?.actions).toEqual([]);
141141
});
142142
}
143+
144+
test('alert wait forwards its timeout budget to the runtime leg', async () => {
145+
const sessionStore = makeSessionStore();
146+
const sessionName = 'ios-sim';
147+
sessionStore.publish(sessionName, makeSession(sessionName));
148+
149+
await handleSnapshotCommands(sessionName, sessionStore, ['wait', '1500']);
150+
151+
// The ceiling test below proves the refusal; this proves a legal budget crosses the runtime
152+
// seam whole. (The Apple runner leg reads the state as `get` and waits on its own deadline.)
153+
expect(mockRunnerCommand.mock.calls[0]?.[1]).toMatchObject({
154+
command: 'alert',
155+
timeoutMs: 1_500,
156+
});
157+
});
158+
159+
test('alert wait refuses a budget past the window ceiling instead of arming a misfired timer', async () => {
160+
// Above 2^31-1 ms Node's timer overflows: `AbortSignal.timeout` throws outright and a bare
161+
// setTimeout fires after 1 ms. The refusal must be the typed argument error, never either.
162+
const sessionStore = makeSessionStore();
163+
const sessionName = 'ios-sim';
164+
sessionStore.publish(sessionName, makeSession(sessionName));
165+
166+
await expect(
167+
handleSnapshotCommands(sessionName, sessionStore, ['wait', '4294967296']),
168+
).rejects.toThrow(/exceeds the maximum/);
169+
await expect(
170+
handleSnapshotCommands(sessionName, sessionStore, ['wait', '2147483648']),
171+
).rejects.toThrow(/exceeds the maximum/);
172+
expect(mockRunnerCommand).not.toHaveBeenCalled();
173+
});

‎packages/daemon/src/execution/handlers/snapshot-alert.ts‎

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,9 @@ import type { AlertRuntimeInput } from '@agent-device/contracts/alert-runtime';
33
import {
44
type AlertAction,
55
DEFAULT_ALERT_TIMEOUT_MS as DEFAULT_TIMEOUT_MS,
6+
MAX_ALERT_WINDOW_MS,
67
} from '@agent-device/contracts/alert-contract';
8+
import { AppError } from '@agent-device/kernel/errors';
79
import type { BoundDeviceRuntime } from '@agent-device/contracts/platform-runtime';
810
import {
911
alertAcceptUse,
@@ -151,7 +153,7 @@ export async function handleAlertCommand(
151153
session?.trace?.outPath,
152154
);
153155
const data = await bound.execute({
154-
timeoutMs: parseTimeout(req.positionals?.[1]) ?? DEFAULT_TIMEOUT_MS,
156+
timeoutMs: alertWaitTimeoutMs(action, req.positionals?.[1]),
155157
...alertTarget(session),
156158
execution: runtimeExecutionFromContext(context),
157159
});
@@ -179,3 +181,21 @@ function normalizeAlertAction(action: string | undefined): AlertAction {
179181
if (action === 'accept' || action === 'dismiss' || action === 'wait') return action;
180182
return 'get';
181183
}
184+
185+
/**
186+
* The `alert wait` budget before it reaches any leg. Past the contract's window ceiling the
187+
* budget cannot be armed as a timer at all, so the command refuses the argument instead of
188+
* starting a wait that would misfire; the driver's own default remains the last line for
189+
* budgets that arrive by other routes.
190+
*/
191+
function alertWaitTimeoutMs(action: AlertAction, rawTimeout: string | undefined): number {
192+
const timeoutMs = parseTimeout(rawTimeout) ?? DEFAULT_TIMEOUT_MS;
193+
if (action === 'wait' && timeoutMs > MAX_ALERT_WINDOW_MS) {
194+
throw new AppError(
195+
'INVALID_ARGS',
196+
`alert wait timeoutMs exceeds the maximum of ${MAX_ALERT_WINDOW_MS} ms`,
197+
{ hint: 'Pass a shorter timeout, or run repeated alert waits.' },
198+
);
199+
}
200+
return timeoutMs;
201+
}

‎packages/platform-android/src/__tests__/alert-detection.test.ts‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,3 +108,35 @@ test('the Android 11+ permission sheet resolves its two grants by the preference
108108
);
109109
assert.equal(chooseAndroidAlertButton(candidate?.buttons ?? [], 'dismiss')?.label, 'Deny');
110110
});
111+
112+
test('nested nodes over one button center deduplicate to one candidate', () => {
113+
// uiautomator commonly nests the clickable wrapper and the label node at the same center.
114+
// They are one physical button, and collapsing them must not read as ambiguity.
115+
const candidate = findAndroidAlertCandidate([
116+
node(0, 'android.app.AlertDialog'),
117+
text(1, 'Discard draft?', 'android:id/alertTitle'),
118+
button(2, 'OK', 'android:id/button1', { x: 52, y: 612 }),
119+
node(3, 'android.widget.TextView', {
120+
label: 'OK',
121+
parentIndex: 2,
122+
rect: { x: 52, y: 612, width: 128, height: 52 },
123+
}),
124+
]);
125+
126+
assert.deepEqual(candidate?.alert.buttons, ['OK']);
127+
assert.equal(chooseAndroidAlertButton(candidate?.buttons ?? [], 'accept')?.label, 'OK');
128+
});
129+
130+
test('the same label at two distinct centers keeps two candidates so the choice refuses', () => {
131+
// Two spatially distinct OKs are two physical buttons; the capture names no single one, so
132+
// the shared ranked boundary refuses rather than taking the tree-first candidate.
133+
const candidate = findAndroidAlertCandidate([
134+
node(0, 'android.app.AlertDialog'),
135+
text(1, 'Which OK?', 'android:id/alertTitle'),
136+
button(2, 'OK', 'android:id/button1', { x: 52, y: 612 }),
137+
button(3, 'OK', 'android:id/button1_copy', { x: 240, y: 612 }),
138+
]);
139+
140+
assert.deepEqual(candidate?.alert.buttons, ['OK', 'OK']);
141+
assert.equal(chooseAndroidAlertButton(candidate?.buttons ?? [], 'accept'), null);
142+
});

‎packages/platform-android/src/__tests__/alert.test.ts‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@ import { test, vi } from 'vitest';
22
import assert from 'node:assert/strict';
33
import type { DeviceInfo } from '@agent-device/kernel/device';
44
import { deviceShellArgv, type ShellWord } from '@agent-device/kernel/device-shell';
5+
import { isRequestCanceledError } from '@agent-device/kernel/errors';
6+
import type { RawSnapshotNode } from '@agent-device/kernel/snapshot';
57
import { button, node, text } from './alert-fixtures.ts';
68

79
const runAndroidAdb = vi.fn(async (_device: DeviceInfo, _args: readonly string[]) => ({
@@ -194,3 +196,57 @@ test('wait with a spent budget still reports an alert that is already visible',
194196
assert.equal(result.kind, 'alertWait');
195197
assert.equal(result.alert?.title, 'Automation confirmation');
196198
});
199+
200+
test('the read-first capture rides the caller cancellation and dies with it', async () => {
201+
// Android's transport is cancellable, so the invariant here is the actual child-process cut:
202+
// the capture seam receives the leg's live AbortSignal, and aborting the caller cuts the
203+
// in-flight capture before any press exists.
204+
runAndroidAdb.mockClear();
205+
const caller = new AbortController();
206+
const seamSignals: AbortSignal[] = [];
207+
let rejectCapture: ((error: Error) => void) | undefined;
208+
const result = handleAndroidAlert(device, 'dismiss', {
209+
signal: caller.signal,
210+
captureNodes: (seam) =>
211+
new Promise<RawSnapshotNode[]>((_resolve, reject) => {
212+
seamSignals.push(seam.signal);
213+
rejectCapture = reject;
214+
seam.signal.addEventListener('abort', () => reject(new Error('adb killed')), {
215+
once: true,
216+
});
217+
}),
218+
});
219+
result.catch(() => {});
220+
await Promise.resolve();
221+
caller.abort();
222+
rejectCapture?.(new Error('adb killed'));
223+
224+
await assert.rejects(result, (error: unknown) => isRequestCanceledError(error));
225+
assert.equal(seamSignals.length, 1);
226+
assert.equal(seamSignals[0]?.aborted, true);
227+
assert.equal(runAndroidAdb.mock.calls.length, 0);
228+
});
229+
230+
test('a slow first read is not cut off by its own action budget', async () => {
231+
// The Android smoke timeline: the session's first capture pays the snapshot-helper cold
232+
// start (install + session bring-up), landing well past the 2s action budget. That read is
233+
// the leg's read-first observation — consumed whatever it costs — and the press rides the
234+
// window it arms behind it. Without the read-first exemption this is the executed CI
235+
// failure: `alert-window-expired, dispatched: no` with the dialog plainly on screen.
236+
runAndroidAdb.mockClear();
237+
let reads = 0;
238+
const slowWarmRead = async () => {
239+
reads += 1;
240+
if (reads > 1) return runAndroidAdb.mock.calls.length === 0 ? dialog : [];
241+
// 2.5 driven seconds: past the 2s lookup window the old guarded first probe enforced.
242+
await new Promise<void>((resolve) => setTimeout(resolve, 2_500));
243+
return dialog;
244+
};
245+
246+
const result = await underFakeTimers(
247+
async () => await handleAndroidAlert(device, 'dismiss', { captureNodes: slowWarmRead }),
248+
);
249+
250+
assert.equal(result.kind, 'alertHandled');
251+
assert.equal(runAndroidAdb.mock.calls.length, 1);
252+
});

‎packages/platform-android/src/__tests__/input-actions.test.ts‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,12 +4,14 @@ import {
44
backAndroid,
55
homeAndroid,
66
longPressAndroid,
7+
pressAndroid,
78
pressAndroidEnter,
89
pressAndroidTvRemote,
910
probeAndroidDisplayRotation,
1011
scrollAndroid,
1112
setAndroidOrientation,
1213
} from '../input-actions.ts';
14+
import { AppError } from '@agent-device/kernel/errors';
1315
import { ANDROID_EMULATOR } from './test-utils/device-fixtures.ts';
1416
import { withFakeAdb } from './test-utils/fake-adb.ts';
1517
import { withAndroidAdbProvider } from '../adb-executor.ts';
@@ -371,6 +373,41 @@ test('pressAndroidEnter presses the ENTER keyevent', async () => {
371373
);
372374
});
373375

376+
test('pressAndroid surfaces a failed tap as a typed dispatch error, not a false success', async () => {
377+
// The device route raises the failed `adb shell input tap` as its own AppError; the input
378+
// seam must hand that through with dispatch evidence, never as a tap that happened.
379+
await withFakeAdb(
380+
(args) =>
381+
args[1] === 'input'
382+
? { stdout: '', stderr: 'INJECT_EVENTS security exception', exitCode: 1 }
383+
: undefined,
384+
async ({ device }) => {
385+
await assert.rejects(pressAndroid(device, 100, 200), (error: unknown) => {
386+
assert.ok(error instanceof AppError);
387+
assert.equal(error.code, 'COMMAND_FAILED');
388+
assert.equal(error.details?.dispatched, 'unknown');
389+
assert.equal(error.details?.exitCode, 1);
390+
return true;
391+
});
392+
},
393+
);
394+
});
395+
396+
test('pressAndroid dispatches no failure suppression even when one is smuggled past the type', async () => {
397+
// The option type is the interface; an untyped caller cannot widen it back. The tap still
398+
// fails typed: `allowFailure` never reaches the transport.
399+
await withFakeAdb(
400+
(args) =>
401+
args[1] === 'input' ? { stdout: '', stderr: 'adb: device offline', exitCode: 1 } : undefined,
402+
async ({ device }) => {
403+
await assert.rejects(
404+
pressAndroid(device, 100, 200, { allowFailure: true } as { signal?: AbortSignal }),
405+
(error: unknown) => error instanceof AppError && error.code === 'COMMAND_FAILED',
406+
);
407+
},
408+
);
409+
});
410+
374411
// The orientation settle polls at its own interval; the clock is the assertion, not the wait.
375412
vi.mock('@agent-device/host-kit/retry', () => ({ sleep: async () => {} }));
376413

0 commit comments

Comments
 (0)