Skip to content

Commit 287cc18

Browse files
authored
fix(gestures): fail pre-removal gesture forms at .ad parse time (#1216) (#1393)
* fix(gestures): fail pre-removal gesture forms at .ad parse time (#1216) #1315 removed the timed forms of `swipe`, `gesture fling`, and `gesture swipe` and `gesture rotate`'s `velocity`, but shipped without the migration guide, the repository sweep, or the parse-time error that issue #1216's own checklist gates a removal on. The sweep finds what that left behind: both `06-swipe-gestures.ad` integration fixtures still carried the 5-argument swipe and would fail at replay, two tests still asserted the removed shapes, and two branches still read the retired positional. Argument arity for every public gesture syntax now lives in one table keyed off the canonical `GESTURE_KINDS`, so a new kind cannot skip it and a form removed from the CLI is removed from `.ad` in the same edit. Both callers read it: the CLI argv parse, and a new `.ad` preflight. A stale script now fails when it is parsed — before the replay executes any device action — naming the line and computing its rewrite, instead of running up to that step and failing as a repairable divergence. The preflight checks arity only: `${VAR}` tokens resolve after planning, and interpolation never splits a token, so the count is decidable while the values are not. Deleting the dead duration read in `readSwipeGeometry` would have left `replay export` emitting no duration, handing Maestro's 400ms default to a gesture the script runs at 100ms, so the export now states `duration: 100`. `.ad` positional gesture parsing is NOT removed. Its only remaining callers are the CLI argv parse and the `.ad` line parse, both the current public syntax rather than a bridge to an older one, so there is nothing to migrate off. ADR 0013 records that and drops the "compatibility" framing that made it read as debt. Both migrated fixtures verified on real devices with the repo's own CLI: iOS simulator 34.9s, Android emulator 45.9s. * fix(gestures): reject removed swipe input at the Node/MCP boundary Review findings on d88c6ed. P1: `interactionDaemonWriters.swipe` hand-projects five fields, so a JavaScript caller's `durationMs` was dropped before the daemon's `readSwipeInput` could reject it and a default-duration fling ran instead — the exact silent reinterpretation the guide promises does not happen. `gesture` was already safe because its writer runs `readGestureInput` -> `readGesturePayload`, which rejects the removed keys; `swipe` was the one surface with no reader of its own. The rejection now lives in contracts and is shared by the client writer and the daemon handler, so there is one rule and one message. The SDK regression covers all four removed keys and asserts the transport is never reached; reverting the writer call fails it on `swipe durationMs`. P2: the preflight's retired-slot test required a numeric token, so `swipe 197 650 197 300 ${DURATION}` fell back to bare usage text. An unresolved `${VAR}` now counts as the retired slot and is carried into the pan rewrite, while a stray flag or word stays a plain usage error. P2: the removal shipped in 0.20.0, not 0.21 — removal commit 6d99914 is contained in tag v0.20.0. The guide said 0.21 because the CHANGELOG still files it under `Unreleased`; the tag is the truth (headings lag several releases repo-wide, so that is pre-existing and left alone). The `.ad` grep recipe now matches variable-backed durations too. * docs(gestures): make the migration sweep and MCP claim accurate Re-review findings on 328bad8 (both migration-guide accuracy). The saved-script sweep matched only the five-argument `swipe` form; it missed timed `gesture fling`, timed `gesture swipe`, and `gesture rotate ... velocity`, so the repository-cleanliness step was incomplete. Provide one grep per retired form (number-or-`${VAR}` token), each verified to flag the removed shape and skip the valid one. The MCP section claimed the structured rejection carries the CLI's concrete replacement command. It does not: `readGesturePayload` and `assertNoRemovedSwipeInput` return a message that names the removed key and the replacement command (e.g. `gesture fling does not accept durationMs; use gesture pan for timed movement`) but not the fully-substituted coordinate rewrite, because the structured request carries no positional string to rewrite. Describe what the structured path actually returns. * docs(gestures): make the .ad sweep parser-aligned Re-review finding on 4fd5d39: the documented sweep required literal single spaces and unquoted numeric tokens, but the `.ad` tokenizer separates on any whitespace (`/\s/`, so tabs too) and accepts double-quoted tokens, so tab-separated or quoted-duration stale lines the parser rejects were missed. Rewrite the patterns to follow the tokenizer: `[[:space:]]+` between tokens and a numeric slot that accepts a bare or double-quoted number (optionally signed) or `${VAR}`. Requiring a digit in the numeric slot keeps a trailing flag like `--count` from being read as the retired positional. Verified against a fixture of tab/space/quoted/quoted-var/negative encodings that the four greps flag exactly the lines `parseReplayScriptDetailed` rejects and none it accepts, and that the live repo sweeps clean. Also state plainly what a regex cannot promise: the parser is the authoritative gate — every retired form is rejected at parse time before any device action, so running the suite finds every stale line by construction and a missed grep can never reach execution. The grep stays a bulk pre-flight, and the "every affected line" claim is scoped to that.
1 parent 256887f commit 287cc18

17 files changed

Lines changed: 600 additions & 66 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,9 @@
1313
- `gesture swipe preset` no longer accepts a trailing `durationMs` positional; use `gesture pan` for timed movement.
1414
- `gesture rotate degrees [x] [y]` no longer accepts a trailing `velocity` positional; rotation pacing is derived from `degrees`.
1515
- MCP/Node schemas no longer advertise `velocity` or `durationMs` on `swipe`/`fling`/`gesture swipe`; `durationMs` remains on `gesture pan` and `gesture transform`.
16+
- A `.ad` script that still carries a removed positional now fails when the script is parsed, before the replay executes any device action, naming the line and its rewrite (for example `swipe accepts 4 arguments: x1 y1 x2 y2 (line 6). The trailing durationMs positional was removed: use "gesture pan 197 650 0 -350 300" ...`). Previously the script ran up to that step and then failed as a replay divergence.
17+
- Published the [gesture migration guide](https://agent-device.dev/docs/migrating-gestures) covering CLI, Node.js, MCP, and saved `.ad` recordings, plus the deprecation policy the next such removal follows (#1216).
18+
- `replay export` now writes an explicit `duration: 100` for `swipe` — the canonical fling duration — instead of omitting it and letting Maestro apply its own 400ms default. Maestro flows replayed by `agent-device` are unaffected; a timed Maestro `swipe` still normalizes to `gesture pan` with the `endpoint-hold` profile.
1619
- Breaking: the deprecated `rotate` CLI command alias has been removed. Use `orientation` instead; invoking `rotate` now fails with `rotate was renamed to orientation; for the two-finger gesture use: gesture rotate`.
1720
- Breaking (ADR 0014, session ref-frame lifetime): a mutation through an `@ref` now expires the session's ref frame, so a later ref mutation without a fresh observation fails closed with a typed `details.reason` (`ref_frame_expired`, `ref_generation_mismatch`, `plain_ref_requires_complete_frame`, or `ref_not_issued`) instead of acting on a possibly-navigated screen. A ref-oriented sequence that performs several mutations must re-`snapshot` between them, consume an honestly issued `--settle` ref in pinned `@eN~s<gen>` form, or use selectors. Enforcement applies on every platform, not just iOS. Legacy hand-written `.ad` scripts that reuse several bare refs from one snapshot must capture between mutations or use selectors.
1821
- Ref reads resolve against the authorized ref frame's source tree, so an internal read-only capture (including Android freshness) can no longer retarget an admitted `@ref` by positional coincidence. Read-only ref consumers keep the structured staleness warning while the frame retains the ref's evidence.

‎docs/adr/0013-unified-gesture-plans.md‎

Lines changed: 19 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -96,9 +96,25 @@ Platform adapters consume the canonical plan:
9696
The `Interactor` and backend expose one compositional `performGesture(plan)` primitive instead of a
9797
method per semantic alias. The old scalar Apple and Android multi-touch executors and the
9898
public-command alias-to-positionals-to-reparse route are deleted. `.ad` keeps its established
99-
positional syntax through one named replay compatibility codec; CLI, Node.js, and MCP send
100-
structured input. Providers should compose transport/device bindings with the shared platform
101-
adapter rather than reimplement the interaction runtime.
99+
positional syntax through one named codec; CLI, Node.js, and MCP send structured input. Providers
100+
should compose transport/device bindings with the shared platform adapter rather than reimplement
101+
the interaction runtime.
102+
103+
That `.ad` codec is the script format's own syntax, not a compatibility shim, and is not scheduled
104+
for removal (amended 2026-07, issue #1216). Its only remaining callers are the CLI argv parse and
105+
the `.ad` line parse — both the current public syntax — so there is nothing left to migrate off.
106+
A structured `.ad` gesture payload was rejected: it would make recordings unreadable and ungreppable
107+
for no behavioral gain.
108+
109+
Argument arity for both callers comes from one table (`PUBLIC_GESTURE_SYNTAX`), so a form removed
110+
from the CLI is removed from `.ad` in the same edit. A `.ad` script that still carries a removed
111+
positional fails when the script is parsed, before the replay executes any device action, with the
112+
offending line and its rewrite. Arity is the only thing checked at parse time: `${VAR}` tokens
113+
resolve after planning, and interpolation never splits a token, so the count is decidable while the
114+
values are not. The removal process these inputs follow — announce, warn for one minor release,
115+
publish the migration guide, prove the repository is clean, then remove the branch and its tests —
116+
is documented in the public
117+
[gesture migration guide](https://agent-device.dev/docs/migrating-gestures).
102118

103119
Repeated coordinate swipes are bounded at the public command contract and daemon trust boundary.
104120
Individual count and pause limits prevent pathological fields, while the combined planned gesture

‎src/__tests__/client.test.ts‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -673,6 +673,55 @@ test('interactions.rotateGesture rejects partial centers on the client side', as
673673
assert.equal(setup.calls.length, 0);
674674
});
675675

676+
test('removed gesture inputs are rejected client-side, never dropped from the projection', async () => {
677+
const setup = createTransport(async () => {
678+
throw new Error('transport should not run for invalid input');
679+
});
680+
const client = createAgentDeviceClient(setup.config, { transport: setup.transport });
681+
// A JavaScript caller (or a stale compiled build) can still reach these keys
682+
// past the removed compile-time fields. `swipe` has no structured reader of
683+
// its own, so before #1216 its writer silently dropped `durationMs` and ran a
684+
// default-duration fling instead of failing.
685+
const removed: Array<[string, () => Promise<unknown>, string]> = [
686+
[
687+
'swipe durationMs',
688+
() =>
689+
client.interactions.swipe({
690+
from: { x: 197, y: 650 },
691+
to: { x: 197, y: 300 },
692+
durationMs: 300,
693+
} as Parameters<typeof client.interactions.swipe>[0]),
694+
'swipe does not accept durationMs; use gesture pan for timed movement',
695+
],
696+
[
697+
'fling durationMs',
698+
() =>
699+
client.interactions.fling({ direction: 'down', x: 100, y: 200, durationMs: 300 } as never),
700+
'gesture fling does not accept durationMs; use gesture pan for timed movement',
701+
],
702+
[
703+
'gesture swipe durationMs',
704+
() => client.interactions.swipeGesture({ preset: 'left', durationMs: 300 } as never),
705+
'gesture swipe does not accept durationMs; use gesture pan for timed movement',
706+
],
707+
[
708+
'rotate velocity',
709+
() => client.interactions.rotateGesture({ degrees: 35, velocity: 800 } as never),
710+
'gesture rotate does not accept velocity; rotation pacing derives from degrees',
711+
],
712+
];
713+
714+
for (const [label, call, message] of removed) {
715+
await assert.rejects(
716+
call,
717+
(error: unknown) =>
718+
error instanceof AppError && error.code === 'INVALID_ARGS' && error.message === message,
719+
label,
720+
);
721+
}
722+
assert.equal(setup.calls.length, 0);
723+
});
724+
676725
test('interactions.pan projects one- and two-finger requests through typed gesture input', async () => {
677726
const setup = createTransport(async () => ({ ok: true, data: { message: 'Panned' } }));
678727
const client = createAgentDeviceClient(setup.config, { transport: setup.transport });

‎src/commands/interaction/interactions.ts‎

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,10 @@ import {
1111
readInteractionTargetFromPositionals,
1212
} from '../../core/interaction-positionals.ts';
1313
import { AppError } from '../../kernel/errors.ts';
14-
import { swipePayloadFromPositionals } from '../../contracts/gesture-normalization.ts';
14+
import {
15+
assertNoRemovedSwipeInput,
16+
swipePayloadFromPositionals,
17+
} from '../../contracts/gesture-normalization.ts';
1518
import type { ScrollInputDirection } from './runtime/gestures.ts';
1619
import {
1720
commonInputFromFlags,
@@ -124,14 +127,16 @@ export const interactionDaemonWriters = {
124127
longpress: direct(PUBLIC_COMMANDS.longPress, (input) =>
125128
longPressPositionals(input as LongPressOptions),
126129
),
127-
swipe: (input) =>
128-
request(PUBLIC_COMMANDS.swipe, [], input, {
130+
swipe: (input) => {
131+
assertNoRemovedSwipeInput(input);
132+
return request(PUBLIC_COMMANDS.swipe, [], input, {
129133
from: input.from,
130134
to: input.to,
131135
count: input.count,
132136
pauseMs: input.pauseMs,
133137
pattern: input.pattern,
134-
}),
138+
});
139+
},
135140
focus: direct(PUBLIC_COMMANDS.focus, (input) => [String(input.x), String(input.y)]),
136141
type: direct(PUBLIC_COMMANDS.type, (input) => typePositionals(input as TypeTextOptions)),
137142
fill: direct(PUBLIC_COMMANDS.fill, (input) => fillPositionals(input as FillOptions)),

‎src/compat/maestro/__tests__/export-flow.test.ts‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ screenshot "./artifacts/checkout"
4343
test('exports coordinate gestures and sleep waits with warnings', () => {
4444
const result = exportReplayScriptToMaestro(`open com.example.app
4545
click 120 240
46-
swipe 200 700 200 200 300 --count 2
46+
swipe 200 700 200 200 --count 2
4747
wait 500
4848
`);
4949

@@ -52,8 +52,8 @@ wait 500
5252
[
5353
'launchApp',
5454
{ tapOn: { point: '120,240' } },
55-
{ swipe: { start: '200,700', end: '200,200', duration: 300 } },
56-
{ swipe: { start: '200,700', end: '200,200', duration: 300 } },
55+
{ swipe: { start: '200,700', end: '200,200', duration: 100 } },
56+
{ swipe: { start: '200,700', end: '200,200', duration: 100 } },
5757
{ waitForAnimationToEnd: { timeout: 500 } },
5858
],
5959
]);

‎src/compat/maestro/export-flow.ts‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
import type { SessionAction } from '../../daemon/types.ts';
2+
import { GESTURE_FLING_DURATION_MS } from '../../contracts/gesture-plan.ts';
23
import { parseSelectorChain, type Selector } from '../../selectors/index.ts';
34
import type { SelectorTerm } from '../../selectors/parse.ts';
45
import { AppError } from '../../kernel/errors.ts';
@@ -39,7 +40,7 @@ type ActionConverter = (action: SessionAction) => ConvertedAction;
3940
type SwipeGeometry = {
4041
start: string;
4142
end: string;
42-
duration?: number;
43+
duration: number;
4344
};
4445

4546
const TEXT_SELECTOR_KEYS = new Set<string>(MAESTRO_TEXT_SELECTOR_KEYS);
@@ -362,13 +363,18 @@ function convertSwipeAction(action: SessionAction): ConvertedAction {
362363
};
363364
}
364365

366+
/**
367+
* `swipe` carries no duration positional, so the export states the canonical
368+
* fling duration rather than letting Maestro apply its own 400 ms default to a
369+
* gesture the script runs at 100 ms.
370+
*/
365371
function readSwipeGeometry(action: SessionAction): SwipeGeometry | undefined {
366-
const [x1, y1, x2, y2, duration] = action.positionals;
372+
const [x1, y1, x2, y2] = action.positionals;
367373
if (!isNumber(x1) || !isNumber(y1) || !isNumber(x2) || !isNumber(y2)) return undefined;
368374
return {
369375
start: formatMaestroPoint(x1, y1),
370376
end: formatMaestroPoint(x2, y2),
371-
...(duration && isNumber(duration) ? { duration: Number(duration) } : {}),
377+
duration: GESTURE_FLING_DURATION_MS,
372378
};
373379
}
374380

‎src/contracts/gesture-normalization.test.ts‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
import assert from 'node:assert/strict';
22
import { test } from 'vitest';
33
import {
4+
describeReplayGestureArityError,
45
gesturePayloadFromPositionals,
56
gesturePayloadToPositionals,
67
normalizePublicGesture,
@@ -57,6 +58,86 @@ test('gesture recording codec round-trips fling with distance', () => {
5758
assert.deepEqual(gesturePayloadFromPositionals(gesturePayloadToPositionals(payload)), payload);
5859
});
5960

61+
test('a retired trailing positional reports its migration, not a bare usage line', () => {
62+
assert.throws(() => swipePayloadFromPositionals(['197', '650', '197', '300', '300']), {
63+
code: 'INVALID_ARGS',
64+
message:
65+
'swipe accepts 4 arguments: x1 y1 x2 y2. The trailing durationMs positional was removed: use "gesture pan 197 650 0 -350 300" for the same timed drag, or "swipe 197 650 197 300" for a default-duration swipe.',
66+
});
67+
assert.throws(() => gesturePayloadFromPositionals(['fling', 'down', '100', '200', '80', '500']), {
68+
code: 'INVALID_ARGS',
69+
message: /trailing durationMs positional was removed: use "gesture fling down 100 200 80"/,
70+
});
71+
assert.throws(() => gesturePayloadFromPositionals(['swipe', 'left', '500']), {
72+
code: 'INVALID_ARGS',
73+
message: /trailing durationMs positional was removed: use "gesture swipe left"/,
74+
});
75+
assert.throws(() => gesturePayloadFromPositionals(['rotate', '35', '100', '200', '800']), {
76+
code: 'INVALID_ARGS',
77+
message: /trailing velocity positional was removed: use "gesture rotate 35 100 200"/,
78+
});
79+
});
80+
81+
test('a malformed line is a usage error rather than a migration claim', () => {
82+
assert.throws(() => swipePayloadFromPositionals(['1', '2', '3', '4', '5', '6']), {
83+
code: 'INVALID_ARGS',
84+
message: 'swipe accepts 4 arguments: x1 y1 x2 y2.',
85+
});
86+
assert.throws(() => swipePayloadFromPositionals(['1', '2', '3', '4', '--unknown']), {
87+
code: 'INVALID_ARGS',
88+
message: 'swipe accepts 4 arguments: x1 y1 x2 y2.',
89+
});
90+
assert.throws(() => swipePayloadFromPositionals(['1', '2', '3', '4', 'hello']), {
91+
code: 'INVALID_ARGS',
92+
message: 'swipe accepts 4 arguments: x1 y1 x2 y2.',
93+
});
94+
});
95+
96+
test('the .ad preflight names the offending line and leaves live syntax alone', () => {
97+
assert.equal(
98+
describeReplayGestureArityError('swipe', ['197', '650', '197', '300', '300'], 'line 6'),
99+
'swipe accepts 4 arguments: x1 y1 x2 y2 (line 6). The trailing durationMs positional was removed: use "gesture pan 197 650 0 -350 300" for the same timed drag, or "swipe 197 650 197 300" for a default-duration swipe.',
100+
);
101+
assert.equal(
102+
describeReplayGestureArityError('gesture', ['pan', '110', '443', '48', '0', '500'], 'line 2'),
103+
undefined,
104+
);
105+
assert.equal(describeReplayGestureArityError('click', ['120', '240'], 'line 2'), undefined);
106+
});
107+
108+
test('a variable-backed retired positional still gets the migration', () => {
109+
assert.equal(
110+
describeReplayGestureArityError('swipe', ['197', '650', '197', '300', '${DURATION}'], 'line 6'),
111+
'swipe accepts 4 arguments: x1 y1 x2 y2 (line 6). The trailing durationMs positional was removed: use "gesture pan 197 650 0 -350 ${DURATION}" for the same timed drag, or "swipe 197 650 197 300" for a default-duration swipe.',
112+
);
113+
assert.equal(
114+
describeReplayGestureArityError(
115+
'swipe',
116+
['${X1}', '${Y1}', '${X2}', '${Y2}', '${DURATION}'],
117+
'line 6',
118+
),
119+
'swipe accepts 4 arguments: x1 y1 x2 y2 (line 6). The trailing durationMs positional was removed: use "gesture pan x1 y1 dx dy durationMs" for the same timed drag, or "swipe ${X1} ${Y1} ${X2} ${Y2}" for a default-duration swipe.',
120+
);
121+
assert.match(
122+
describeReplayGestureArityError('gesture', ['rotate', '35', '195', '443', '${V}'], 'line 4') ??
123+
'',
124+
/trailing velocity positional was removed: use "gesture rotate 35 195 443"/,
125+
);
126+
});
127+
128+
test('the .ad preflight defers to dispatch for values it cannot know yet', () => {
129+
// `${VAR}` tokens resolve after planning, and interpolation never splits a
130+
// token, so only the argument count is decidable at parse time.
131+
assert.equal(
132+
describeReplayGestureArityError('swipe', ['${X1}', '${Y1}', '${X2}', '${Y2}'], 'line 3'),
133+
undefined,
134+
);
135+
assert.equal(
136+
describeReplayGestureArityError('gesture', ['${KIND}', '1', '2', '3'], 'line 4'),
137+
undefined,
138+
);
139+
});
140+
60141
test('pinch and rotate syntax rejects a partial origin', () => {
61142
assert.throws(() => gesturePayloadFromPositionals(['pinch', '1.5', '100']), {
62143
code: 'INVALID_ARGS',

0 commit comments

Comments
 (0)