Skip to content

Commit 0677309

Browse files
authored
fix(android): honor scroll releaseBehavior to stop the fling overshoot (#2372)
* fix(android): honor scroll releaseBehavior to stop the fling overshoot scrollAndroid ignored ScrollReleaseBehavior and always released the pan like a fling, so a controlled `scroll` overshot its requested distance by ~60% on both a plain RecyclerView and an RN ScrollView. Android's VelocityTracker treats a truly stationary release as a resampled duplicate and ignores it, so a hold alone can't stop the fling. For the default 'controlled' release, append a short tail after the pan's endpoint that holds the scroll axis exactly fixed (zero velocity there) while nudging the orthogonal axis every frame so no two consecutive samples repeat. 'inertial' (the scroll top/bottom edge passes) is unchanged. Live-measured on a Pixel 9 Pro XL emulator: displacement lands within touch-slop of the requested drag instead of overshooting it. Fixes #2371 * fix(android): cap the controlled-release tail at the gesture duration ceiling The release tail was appended after buildGesturePlan's own duration validation, so a maximum-duration (10000ms) scroll silently produced a 10160ms plan beyond GESTURE_DURATION_MAX_MS. Cap the tail so the dispatched plan never exceeds that shared ceiling, shrinking (and, at the exact maximum, dropping) the tail near the boundary instead of truncating the requested move. Add regression coverage for the exact maximum and near-maximum cases. Also fix the stale android-lifecycle provider-integration expectation: a plain scroll resolves to 'controlled' release by default, so its dispatched plan is now 510ms (350ms move + 160ms tail), not 350ms. * fix(android): reject a controlled scroll that leaves the release tail no room, instead of shrinking it The previous fix capped the tail near GESTURE_DURATION_MAX_MS, which silently dropped the fling-suppressing braking step for scrolls close to the ceiling -- defeating the point of the fix exactly where a long scroll needs it. A controlled scroll's own accepted duration range now stops CONTROLLED_RELEASE_TAIL_MS short of that shared ceiling (9840ms), rejecting anything past it with a clear INVALID_ARGS error instead of truncating the move or the tail. Every accepted controlled scroll now runs the full, unshortened tail; 'inertial' releases are unaffected and keep the full 10000ms range. Also drop the multi-line explanatory comment on the corrected android-lifecycle.test.ts expectation (a same-line comment instead) so the fix doesn't grow that file past its line-count tripwire. * docs(scroll): describe reduced momentum and the Android controlled-scroll limit The scroll docs only carried the generic app-physics caveat, not the release-behavior contract itself. State plainly that scroll releases with reduced momentum rather than an exact stop, and document Android's controlled-release durationMs ceiling (9840ms, reserved headroom for the braking tail added in #2371) and that a longer request is rejected rather than silently truncated.
1 parent 0627190 commit 0677309

5 files changed

Lines changed: 321 additions & 21 deletions

File tree

‎packages/platform-android/src/__tests__/input-actions.test.ts‎

Lines changed: 203 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
import { test, vi } from 'vitest';
22
import assert from 'node:assert/strict';
3+
import { GESTURE_SAMPLE_INTERVAL_MS } from '@agent-device/contracts/gesture-plan';
4+
import { GESTURE_DURATION_MAX_MS } from '@agent-device/contracts/gesture-plan-types';
35
import {
46
backAndroid,
57
homeAndroid,
@@ -79,7 +81,14 @@ test('scrollAndroid accepts sub-frame public durations at the Android planner mi
7981
async () => {
8082
const outputs: Record<string, unknown>[] = [];
8183
for (const durationMs of [0, 15]) {
82-
outputs.push(await scrollAndroid(ANDROID_EMULATOR, 'down', { durationMs }));
84+
// 'inertial' keeps the injected plan's durationMs equal to the honored move time, so the
85+
// flooring this test targets is not conflated with the 'controlled' release tail below.
86+
outputs.push(
87+
await scrollAndroid(ANDROID_EMULATOR, 'down', {
88+
durationMs,
89+
releaseBehavior: 'inertial',
90+
}),
91+
);
8392
}
8493
return outputs;
8594
},
@@ -95,6 +104,199 @@ test('scrollAndroid accepts sub-frame public durations at the Android planner mi
95104
);
96105
});
97106

107+
test('scrollAndroid defaults to a controlled release: a quivering tail past the pan endpoint', async () => {
108+
const touchCalls: Parameters<AndroidTouchInjector>[0][] = [];
109+
await withAndroidAdbProvider(
110+
{
111+
exec: async () => {
112+
throw new Error('adb must not run');
113+
},
114+
gestureViewport: async () => ({ x: 10, y: 20, width: 1080, height: 1920 }),
115+
touch: async (request) => {
116+
touchCalls.push(request);
117+
return { injected: true };
118+
},
119+
},
120+
{ serial: ANDROID_EMULATOR.id },
121+
async () => await scrollAndroid(ANDROID_EMULATOR, 'down', { pixels: 240, durationMs: 120 }),
122+
);
123+
124+
assert.equal(touchCalls.length, 1);
125+
const [touch] = touchCalls;
126+
const samples = touch!.pointers[0]!.samples;
127+
const endpoint = samples.find((sample) => sample.offsetMs === 120)!;
128+
const tail = samples.filter((sample) => sample.offsetMs > 120);
129+
130+
// The plan carries a >=100ms tail past the honored 120ms move; the CLI-facing `durationMs` in
131+
// the command result (asserted below) stays at the honored move time.
132+
assert.equal(touch!.durationMs, 280);
133+
assert.ok(tail.length >= 100 / GESTURE_SAMPLE_INTERVAL_MS);
134+
for (const sample of tail) assert.equal(sample.point.y, endpoint.point.y);
135+
const allPastEndpoint = [endpoint, ...tail];
136+
for (let index = 1; index < allPastEndpoint.length; index += 1) {
137+
assert.notEqual(allPastEndpoint[index]!.point.x, allPastEndpoint[index - 1]!.point.x);
138+
}
139+
});
140+
141+
test('scrollAndroid composes the duration floor with the default controlled-release tail', async () => {
142+
const touchCalls: Parameters<AndroidTouchInjector>[0][] = [];
143+
await withAndroidAdbProvider(
144+
{
145+
exec: async () => {
146+
throw new Error('adb must not run');
147+
},
148+
gestureViewport: async () => ({ x: 0, y: 0, width: 1080, height: 1920 }),
149+
touch: async (request) => {
150+
touchCalls.push(request);
151+
},
152+
},
153+
{ serial: ANDROID_EMULATOR.id },
154+
async () => await scrollAndroid(ANDROID_EMULATOR, 'down', { durationMs: 0 }),
155+
);
156+
157+
// The move floors to the Android planner minimum (16ms) before the tail is appended, not after.
158+
assert.equal(touchCalls[0]!.durationMs, GESTURE_SAMPLE_INTERVAL_MS + 160);
159+
});
160+
161+
test('scrollAndroid runs the full controlled-release tail at the largest duration that still leaves it room', async () => {
162+
const touchCalls: Parameters<AndroidTouchInjector>[0][] = [];
163+
const maxControlledMoveMs = GESTURE_DURATION_MAX_MS - 160;
164+
const result = await withAndroidAdbProvider(
165+
{
166+
exec: async () => {
167+
throw new Error('adb must not run');
168+
},
169+
gestureViewport: async () => ({ x: 0, y: 0, width: 1080, height: 1920 }),
170+
touch: async (request) => {
171+
touchCalls.push(request);
172+
},
173+
},
174+
{ serial: ANDROID_EMULATOR.id },
175+
async () =>
176+
await scrollAndroid(ANDROID_EMULATOR, 'down', {
177+
pixels: 1800,
178+
durationMs: maxControlledMoveMs,
179+
}),
180+
);
181+
182+
// The requested move is honored in full, and the tail always runs at its full length — the
183+
// dispatched plan lands exactly at GESTURE_DURATION_MAX_MS, never past it.
184+
assert.equal(result.durationMs, maxControlledMoveMs);
185+
const [touch] = touchCalls;
186+
assert.equal(touch!.durationMs, GESTURE_DURATION_MAX_MS);
187+
assert.equal(touch!.pointers[0]!.samples.at(-1)!.offsetMs, GESTURE_DURATION_MAX_MS);
188+
});
189+
190+
test('scrollAndroid rejects a controlled-release durationMs that would leave the release tail no room, without shortening the move', async () => {
191+
await withAndroidAdbProvider(
192+
{
193+
exec: async () => {
194+
throw new Error('adb must not run');
195+
},
196+
gestureViewport: async () => ({ x: 0, y: 0, width: 1080, height: 1920 }),
197+
touch: async () => {
198+
throw new Error('touch must not run for a rejected request');
199+
},
200+
},
201+
{ serial: ANDROID_EMULATOR.id },
202+
async () => {
203+
await assert.rejects(
204+
scrollAndroid(ANDROID_EMULATOR, 'down', {
205+
pixels: 1800,
206+
durationMs: GESTURE_DURATION_MAX_MS - 159,
207+
}),
208+
/scroll durationMs must be at most 9840 for a controlled release/,
209+
);
210+
},
211+
);
212+
});
213+
214+
test("scrollAndroid accepts the full GESTURE_DURATION_MAX_MS for an 'inertial' release, which needs no tail", async () => {
215+
const touchCalls: Parameters<AndroidTouchInjector>[0][] = [];
216+
await withAndroidAdbProvider(
217+
{
218+
exec: async () => {
219+
throw new Error('adb must not run');
220+
},
221+
gestureViewport: async () => ({ x: 0, y: 0, width: 1080, height: 1920 }),
222+
touch: async (request) => {
223+
touchCalls.push(request);
224+
},
225+
},
226+
{ serial: ANDROID_EMULATOR.id },
227+
async () =>
228+
await scrollAndroid(ANDROID_EMULATOR, 'down', {
229+
pixels: 1800,
230+
durationMs: GESTURE_DURATION_MAX_MS,
231+
releaseBehavior: 'inertial',
232+
}),
233+
);
234+
235+
const [touch] = touchCalls;
236+
assert.equal(touch!.durationMs, GESTURE_DURATION_MAX_MS);
237+
});
238+
239+
test('scrollAndroid jitters the axis orthogonal to a horizontal scroll, holding the scroll axis fixed', async () => {
240+
const touchCalls: Parameters<AndroidTouchInjector>[0][] = [];
241+
await withAndroidAdbProvider(
242+
{
243+
exec: async () => {
244+
throw new Error('adb must not run');
245+
},
246+
gestureViewport: async () => ({ x: 10, y: 20, width: 1080, height: 1920 }),
247+
touch: async (request) => {
248+
touchCalls.push(request);
249+
return { injected: true };
250+
},
251+
},
252+
{ serial: ANDROID_EMULATOR.id },
253+
async () => await scrollAndroid(ANDROID_EMULATOR, 'left', { pixels: 240, durationMs: 120 }),
254+
);
255+
256+
assert.equal(touchCalls.length, 1);
257+
const [touch] = touchCalls;
258+
const samples = touch!.pointers[0]!.samples;
259+
const endpoint = samples.find((sample) => sample.offsetMs === 120)!;
260+
const tail = samples.filter((sample) => sample.offsetMs > 120);
261+
262+
assert.ok(tail.length >= 100 / GESTURE_SAMPLE_INTERVAL_MS);
263+
// The scroll axis (x, for a horizontal scroll) stays exactly at the endpoint — zero velocity
264+
// there by construction; only the orthogonal axis (y) jitters to dodge the resampling quirk.
265+
for (const sample of tail) assert.equal(sample.point.x, endpoint.point.x);
266+
const allPastEndpoint = [endpoint, ...tail];
267+
for (let index = 1; index < allPastEndpoint.length; index += 1) {
268+
assert.notEqual(allPastEndpoint[index]!.point.y, allPastEndpoint[index - 1]!.point.y);
269+
}
270+
});
271+
272+
test('scrollAndroid honors an inertial release (scroll top/bottom): lifts at the pan endpoint', async () => {
273+
const touchCalls: Parameters<AndroidTouchInjector>[0][] = [];
274+
await withAndroidAdbProvider(
275+
{
276+
exec: async () => {
277+
throw new Error('adb must not run');
278+
},
279+
gestureViewport: async () => ({ x: 10, y: 20, width: 1080, height: 1920 }),
280+
touch: async (request) => {
281+
touchCalls.push(request);
282+
return { injected: true };
283+
},
284+
},
285+
{ serial: ANDROID_EMULATOR.id },
286+
async () =>
287+
await scrollAndroid(ANDROID_EMULATOR, 'down', {
288+
pixels: 240,
289+
durationMs: 120,
290+
releaseBehavior: 'inertial',
291+
}),
292+
);
293+
294+
assert.equal(touchCalls.length, 1);
295+
const [touch] = touchCalls;
296+
assert.equal(touch!.durationMs, 120);
297+
assert.equal(touch!.pointers[0]!.samples.at(-1)!.offsetMs, 120);
298+
});
299+
98300
test('longPressAndroid sends a stationary semantic touch plan', async () => {
99301
const touchCalls: Parameters<AndroidTouchInjector>[0][] = [];
100302
const result = await withAndroidAdbProvider(

‎packages/platform-android/src/input-actions.ts‎

Lines changed: 114 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -3,16 +3,28 @@
33
* IME, and the adb-shell writer — is `text-input.ts`.
44
*/
55
import { DEVICE_ROTATION_SURFACE_INDEX, type DeviceRotation } from '@agent-device/contracts/device';
6-
import { buildGesturePlan } from '@agent-device/contracts/gesture-plan';
7-
import { GESTURE_DURATION_MIN_MS } from '@agent-device/contracts/gesture-plan-types';
8-
import { DEFAULT_MOBILE_SCROLL_DURATION_MS } from '@agent-device/contracts/scroll-command';
6+
import { GESTURE_SAMPLE_INTERVAL_MS, buildGesturePlan } from '@agent-device/contracts/gesture-plan';
7+
import {
8+
GESTURE_DURATION_MAX_MS,
9+
GESTURE_DURATION_MIN_MS,
10+
} from '@agent-device/contracts/gesture-plan-types';
11+
import type {
12+
GesturePlan,
13+
PointerTrajectorySample,
14+
SinglePointerTrajectory,
15+
} from '@agent-device/contracts/gesture-plan-types';
16+
import {
17+
DEFAULT_MOBILE_SCROLL_DURATION_MS,
18+
type ScrollReleaseBehavior,
19+
} from '@agent-device/contracts/scroll-command';
920
import {
1021
type ScrollDirection,
1122
buildScrollGesturePlan,
1223
} from '@agent-device/contracts/scroll-gesture';
1324
import { type TvRemoteButton, toAndroidTvRemoteKeyevent } from '@agent-device/contracts/tv-remote';
1425
import type { DeviceInfo } from '@agent-device/kernel/device';
1526
import { AppError } from '@agent-device/kernel/errors';
27+
import type { Rect } from '@agent-device/kernel/snapshot';
1628
import { sleep } from '@agent-device/host-kit/retry';
1729
import { runAndroidAdb } from './adb.ts';
1830
import { executeAndroidTouchPlan, readAndroidGestureViewport } from './touch-executor.ts';
@@ -163,7 +175,12 @@ export async function focusAndroid(device: DeviceInfo, x: number, y: number): Pr
163175
export async function scrollAndroid(
164176
device: DeviceInfo,
165177
direction: ScrollDirection,
166-
options?: { amount?: number; pixels?: number; durationMs?: number } & AndroidHelperSessionOptions,
178+
options?: {
179+
amount?: number;
180+
pixels?: number;
181+
durationMs?: number;
182+
releaseBehavior?: ScrollReleaseBehavior;
183+
} & AndroidHelperSessionOptions,
167184
): Promise<Record<string, unknown>> {
168185
// The viewport read and the gesture are two helper calls one command apart: giving the read the
169186
// command's session scope keeps both on the same instrumentation.
@@ -192,21 +209,26 @@ export async function scrollAndroid(
192209
options?.durationMs ?? DEFAULT_MOBILE_SCROLL_DURATION_MS,
193210
GESTURE_DURATION_MIN_MS,
194211
);
212+
const releaseBehavior = options?.releaseBehavior ?? 'controlled';
213+
if (releaseBehavior === 'controlled') assertRoomForControlledReleaseTail(durationMs);
214+
const gesturePlan = buildGesturePlan(
215+
{
216+
intent: 'pan',
217+
origin: { x: scrollPlan.x1, y: scrollPlan.y1 },
218+
delta: {
219+
x: scrollPlan.x2 - scrollPlan.x1,
220+
y: scrollPlan.y2 - scrollPlan.y1,
221+
},
222+
durationMs,
223+
},
224+
viewport,
225+
'android',
226+
);
195227
const backend = await executeAndroidTouchPlan(
196228
device,
197-
buildGesturePlan(
198-
{
199-
intent: 'pan',
200-
origin: { x: scrollPlan.x1, y: scrollPlan.y1 },
201-
delta: {
202-
x: scrollPlan.x2 - scrollPlan.x1,
203-
y: scrollPlan.y2 - scrollPlan.y1,
204-
},
205-
durationMs,
206-
},
207-
viewport,
208-
'android',
209-
),
229+
releaseBehavior === 'controlled'
230+
? withControlledReleaseTail(gesturePlan, viewport, direction)
231+
: gesturePlan,
210232
);
211233

212234
return {
@@ -216,6 +238,81 @@ export async function scrollAndroid(
216238
};
217239
}
218240

241+
// Kept an even multiple of GESTURE_SAMPLE_INTERVAL_MS so the tail's last sample lands back on the
242+
// pan's exact endpoint (an odd multiple would still avoid the fling — every consecutive sample
243+
// still differs — but would leave the release 1px off the requested endpoint).
244+
const CONTROLLED_RELEASE_TAIL_MS = 160;
245+
246+
// The dispatched plan (move + tail) must never exceed GESTURE_DURATION_MAX_MS, the same ceiling
247+
// every gesture plan is built under. Rather than silently dropping the tail for a move that
248+
// leaves it no room, a controlled scroll's own accepted range stops short of the shared ceiling
249+
// by the tail's length — the full tail runs for every accepted controlled scroll, and a request
250+
// past this narrower range is rejected with the reason, not truncated.
251+
const CONTROLLED_RELEASE_MAX_MOVE_MS = GESTURE_DURATION_MAX_MS - CONTROLLED_RELEASE_TAIL_MS;
252+
253+
function assertRoomForControlledReleaseTail(durationMs: number): void {
254+
if (durationMs <= CONTROLLED_RELEASE_MAX_MOVE_MS) return;
255+
throw new AppError(
256+
'INVALID_ARGS',
257+
`scroll durationMs must be at most ${CONTROLLED_RELEASE_MAX_MOVE_MS} for a controlled release ` +
258+
`(leaves room for the ${CONTROLLED_RELEASE_TAIL_MS}ms release tail within the ` +
259+
`${GESTURE_DURATION_MAX_MS}ms gesture ceiling)`,
260+
{
261+
hint: "Pass a shorter durationMs, or releaseBehavior 'inertial' if the fling is acceptable.",
262+
},
263+
);
264+
}
265+
266+
/**
267+
* A short, quivering tail appended after a 'controlled' scroll's endpoint, adding
268+
* `CONTROLLED_RELEASE_TAIL_MS` of real time to the gesture. AOSP's `InputConsumer::rewriteMessage`
269+
* collapses a MOVE that repeats the previous coordinates into a "resampled" sample, and
270+
* `VelocityTracker` skips resampled samples — so a truly stationary tail never reaches the
271+
* tracker, and `ScrollView.onTouchEvent` (which computes release velocity before applying UP)
272+
* still flings at the pan's velocity. Nudging the axis orthogonal to the scroll by 1px every frame
273+
* (holding the scroll axis exactly at the endpoint — zero velocity there by construction) keeps
274+
* every sample distinct without adding net travel along either axis. Measured fling-free for
275+
* vertical scrolls on a `RecyclerView` and an RN `ScrollView` (issue #2371); not independently
276+
* verified against every OEM skin or a Compose `LazyColumn`. An 'inertial' release (the
277+
* `scroll top`/`scroll bottom` edge passes) lifts at the pan's endpoint unchanged.
278+
*
279+
* Callers must have already checked `assertRoomForControlledReleaseTail` on the move duration —
280+
* this always appends the full tail.
281+
*/
282+
function withControlledReleaseTail(
283+
plan: GesturePlan,
284+
viewport: Rect,
285+
direction: ScrollDirection,
286+
): GesturePlan {
287+
if (plan.topology !== 'single') return plan;
288+
const steps = CONTROLLED_RELEASE_TAIL_MS / GESTURE_SAMPLE_INTERVAL_MS;
289+
const [pointer] = plan.pointers;
290+
const end = pointer.samples.at(-1)!;
291+
const horizontal = direction === 'left' || direction === 'right';
292+
const jitterBase = horizontal ? end.point.y : end.point.x;
293+
const jitterMin = (horizontal ? viewport.y : viewport.x) + 1;
294+
const jitterMax = (horizontal ? viewport.y + viewport.height : viewport.x + viewport.width) - 1;
295+
const nudged = jitterBase + 1 <= jitterMax ? jitterBase + 1 : Math.max(jitterMin, jitterBase - 1);
296+
const tail: PointerTrajectorySample[] = Array.from({ length: steps }, (_, index) => {
297+
const jitter = index % 2 === 0 ? nudged : jitterBase;
298+
return {
299+
offsetMs: plan.durationMs + (index + 1) * GESTURE_SAMPLE_INTERVAL_MS,
300+
point: horizontal ? { x: end.point.x, y: jitter } : { x: jitter, y: end.point.y },
301+
};
302+
});
303+
const samples: SinglePointerTrajectory['samples'] = [
304+
pointer.samples[0],
305+
pointer.samples[1],
306+
...pointer.samples.slice(2),
307+
...tail,
308+
];
309+
return {
310+
...plan,
311+
durationMs: plan.durationMs + CONTROLLED_RELEASE_TAIL_MS,
312+
pointers: [{ ...pointer, samples }],
313+
};
314+
}
315+
219316
function resolveAndroidUserRotation(orientation: DeviceRotation): string {
220317
const index = DEVICE_ROTATION_SURFACE_INDEX[orientation];
221318
if (index === undefined) {

0 commit comments

Comments
 (0)