Skip to content

Commit 571d5cd

Browse files
committed
fix(scroll): name what a scroll observed and read an edge off one tree
Two pieces the directional-scroll answer needs, with no behavior change on its own. `movement` is the vocabulary a scroll uses to say what happened to the surface: `moved`, `at-edge`, `unchanged` when the visible content did not change, and `unobserved` when the command could not compare anything. It is absent when the runtime cannot read a screen at all, so a caller never has to guess whether a missing claim means "did not move". `readScrollEdgeState` is the one pure door into the edge analyzer — the question `canScrollFurtherAtEdge` and `scrollSurfaceFingerprint` already ask, exposed with the frame of the container that answer was taken on. Without `containerRect`, `canScroll: false` cannot distinguish content that ends here from a tree that named no container, and a caller checking whether its gesture landed inside the list has nothing to check against.
1 parent b38b1f9 commit 571d5cd

5 files changed

Lines changed: 197 additions & 4 deletions

File tree

‎packages/capture-kit/src/snapshot/__tests__/scroll-edge-state-pass-orchestration.test.ts‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,56 @@ test('an amount-based message names the honored travel when the planner reports
7777
);
7878
});
7979

80+
/**
81+
* #2714. A travel figure describes the swipe that was dispatched, not content that moved, so an
82+
* observation of no change outranks it. `unobserved` is the pair to these: the disclosure travels in
83+
* the `movement` field, and the prose keeps the number the caller asked for.
84+
*/
85+
test('formatScrollEdgeMessage: an observed edge answers without a distance', () => {
86+
assert.equal(
87+
formatScrollEdgeMessage({ direction: 'down', passes: 1, amount: 0.75, movement: 'at-edge' }),
88+
'Scrolled down and no hidden content below was detected',
89+
);
90+
assert.equal(
91+
formatScrollEdgeMessage({ direction: 'up', passes: 1, amount: 0.75, movement: 'at-edge' }),
92+
'Scrolled up and no hidden content above was detected',
93+
);
94+
});
95+
96+
test('formatScrollEdgeMessage: an unchanged surface reports the measurement, not the request', () => {
97+
assert.equal(
98+
formatScrollEdgeMessage({ direction: 'left', passes: 1, pixels: 500, movement: 'unchanged' }),
99+
'Scrolled left and the visible content did not change',
100+
);
101+
});
102+
103+
test('formatScrollEdgeMessage: an observation outranks the distance it disagrees with', () => {
104+
assert.equal(
105+
formatScrollEdgeMessage({
106+
direction: 'down',
107+
passes: 1,
108+
amount: 3,
109+
pixels: 5000,
110+
honoredPixels: 640,
111+
movement: 'at-edge',
112+
}),
113+
'Scrolled down and no hidden content below was detected',
114+
);
115+
});
116+
117+
test('formatScrollEdgeMessage: an unobserved movement keeps the distance the caller asked for', () => {
118+
assert.equal(
119+
formatScrollEdgeMessage({
120+
direction: 'down',
121+
passes: 1,
122+
amount: 0.75,
123+
honoredPixels: 656,
124+
movement: 'unobserved',
125+
}),
126+
'Scrolled down by 0.75 of the viewport (656px)',
127+
);
128+
});
129+
80130
// ---------------------------------------------------------------------------
81131
// captureScrollEdgeState: retry-without-scope on an empty scoped capture
82132
// ---------------------------------------------------------------------------
Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
import { test } from 'vitest';
2+
import assert from 'node:assert/strict';
3+
import { captureScrollEdgeState, readScrollEdgeState } from '../scroll-edge-state.ts';
4+
import { scrollNode, windowRoot } from './scroll-edge-state-fixtures.ts';
5+
6+
/**
7+
* The pure door into the edge analyzer. `scroll <dir>` reads an edge decision off a tree it already
8+
* holds to say what its gesture did (#2714), which makes `containerRect` part of the answer: the
9+
* caller has to be able to tell "this content ends here" from "the tree named no container", and
10+
* `canScroll: false` alone cannot tell them apart.
11+
*/
12+
13+
const CONTAINER = { x: 18, y: 178, width: 366, height: 662 };
14+
15+
test('readScrollEdgeState: a resolved container reports the frame the decision was taken on', async () => {
16+
const nodes = [
17+
windowRoot(),
18+
scrollNode(1, { hiddenContentBelow: true, rect: CONTAINER }),
19+
{
20+
ref: 'e3',
21+
index: 2,
22+
parentIndex: 1,
23+
type: 'StaticText',
24+
label: 'Row',
25+
rect: { x: 24, y: 700, width: 300, height: 40 },
26+
},
27+
];
28+
29+
const state = await readScrollEdgeState(nodes, 'bottom');
30+
31+
assert.equal(state.canScroll, true);
32+
assert.deepEqual(state.containerRect, CONTAINER);
33+
});
34+
35+
test('readScrollEdgeState: a tree with no scroll container reports no frame to check a gesture against', async () => {
36+
const nodes = [
37+
windowRoot(),
38+
{
39+
ref: 'e2',
40+
index: 1,
41+
parentIndex: 0,
42+
type: 'Button',
43+
label: 'Tap me',
44+
rect: { x: 20, y: 40, width: 100, height: 40 },
45+
},
46+
];
47+
48+
const state = await readScrollEdgeState(nodes, 'bottom');
49+
50+
assert.equal(state.canScroll, false);
51+
assert.equal(state.containerRect, undefined);
52+
});
53+
54+
test('readScrollEdgeState: a zero-area scrollable resolves to no container and therefore no frame', async () => {
55+
const state = await readScrollEdgeState(
56+
[windowRoot(), scrollNode(1, { hiddenContentBelow: true, rect: { ...CONTAINER, height: 0 } })],
57+
'bottom',
58+
);
59+
60+
assert.equal(state.containerRect, undefined);
61+
});
62+
63+
/**
64+
* The reader must not become a second source of truth: the captured and pure doors disagreeing about
65+
* one tree would mean an edge loop and a movement claim reading different rules.
66+
*/
67+
test('readScrollEdgeState answers what captureScrollEdgeState answers for the same tree', async () => {
68+
const nodes = [windowRoot(), scrollNode(1, { hiddenContentBelow: true, rect: CONTAINER })];
69+
70+
const captured = await captureScrollEdgeState({
71+
edge: 'bottom',
72+
captureNodes: async () => nodes,
73+
});
74+
const read = await readScrollEdgeState(nodes, 'bottom');
75+
76+
assert.equal(read.canScroll, captured.canScroll);
77+
assert.deepEqual(read.containerRect, captured.containerRect);
78+
assert.equal(read.fingerprint, captured.fingerprint);
79+
});

‎packages/capture-kit/src/snapshot/scroll-edge-state.ts‎

Lines changed: 39 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,21 @@
11
import { AppError } from '@agent-device/kernel/errors';
2+
import type { ScrollMovementObservation } from '@agent-device/contracts/scroll-command';
23
import type { ScrollDirection } from '@agent-device/contracts/scroll-gesture';
3-
import type { Point, RawSnapshotNode, SnapshotNode } from '@agent-device/kernel/snapshot';
4+
import type { Point, RawSnapshotNode, Rect, SnapshotNode } from '@agent-device/kernel/snapshot';
45

56
export type ScrollEdge = 'top' | 'bottom';
67

78
export type ScrollEdgeState = {
89
canScroll: boolean;
910
emptySnapshot: boolean;
1011
scope?: string;
12+
/**
13+
* The frame of the container every other field on this record describes, absent when the tree
14+
* named no usable scroll container at all. Without it `canScroll: false` cannot be read as
15+
* "already at this edge": a tree that names no scroller and a scroller with nothing left to
16+
* reveal answer the same way, and only one of them is the end of the content (#2714).
17+
*/
18+
containerRect?: Rect;
1119
/**
1220
* A cheap signature of what is on screen right now. Two consecutive captures with the same
1321
* fingerprint but `canScroll` still true mean the scroll did not move the container — the actuator
@@ -55,6 +63,19 @@ export async function captureScrollEdgeState(params: {
5563
}
5664
}
5765

66+
/**
67+
* Everything one tree can answer about this edge — hidden content left, the container that answer
68+
* was taken on, and the surface signature — with no capture. The one pure door into the analyzer,
69+
* so every reader of an edge decision goes through the same selection rules.
70+
*/
71+
export async function readScrollEdgeState(
72+
nodes: readonly (RawSnapshotNode | SnapshotNode)[],
73+
edge: ScrollEdge,
74+
): Promise<ScrollEdgeState> {
75+
const { analyzeScrollEdgeState } = await import('./scroll-edge-state/selection.ts');
76+
return analyzeScrollEdgeState(nodes, edge);
77+
}
78+
5879
/**
5980
* Is there hidden content left at this edge? The same question `runScrollEdgePasses` loops on,
6081
* exposed for callers with their own stop condition (`scroll --until`) so both read one signal.
@@ -63,8 +84,7 @@ export async function canScrollFurtherAtEdge(
6384
nodes: readonly (RawSnapshotNode | SnapshotNode)[],
6485
edge: ScrollEdge,
6586
): Promise<boolean> {
66-
const { analyzeScrollEdgeState } = await import('./scroll-edge-state/selection.ts');
67-
return analyzeScrollEdgeState(nodes, edge).canScroll;
87+
return (await readScrollEdgeState(nodes, edge)).canScroll;
6888
}
6989

7090
/**
@@ -167,6 +187,10 @@ export async function runScrollEdgePasses<TResult>(params: {
167187
* travel that was asked for: one gesture cannot cross more than the viewport axis minus its edge
168188
* padding, so a large `amount` saturates. Naming the honored distance is what keeps
169189
* `scroll down 3` from reporting a three-viewport scroll it never performed.
190+
*
191+
* `movement` is what the owner measured after the gesture (#2714), and it outranks the requested
192+
* distance: a directional scroll that observed no change cannot answer with a travel figure, since
193+
* that number describes the swipe that was dispatched, not content that moved.
170194
*/
171195
export function formatScrollEdgeMessage(params: {
172196
direction: ScrollDirection;
@@ -175,12 +199,23 @@ export function formatScrollEdgeMessage(params: {
175199
amount?: number | undefined;
176200
pixels?: number | undefined;
177201
honoredPixels?: number | undefined;
202+
movement?: ScrollMovementObservation | undefined;
178203
}): string {
179-
const { direction, edge, passes, amount, pixels, honoredPixels } = params;
204+
const { direction, edge, passes, amount, pixels, honoredPixels, movement } = params;
180205
if (edge && passes === 0) {
181206
return `Already at ${edge}; no hidden content ${edge === 'bottom' ? 'below' : 'above'} detected`;
182207
}
183208
if (edge) return `Scrolled to ${edge} with ${passes} ${direction} passes`;
209+
if (movement === 'at-edge') {
210+
// The same honesty the zero-pass branch above holds: the analyzer reports no hidden content
211+
// DETECTED, which is not a claim that the content provably ends here.
212+
return `Scrolled ${direction} and no hidden content ${
213+
direction === 'down' ? 'below' : 'above'
214+
} was detected`;
215+
}
216+
if (movement === 'unchanged') {
217+
return `Scrolled ${direction} and the visible content did not change`;
218+
}
184219
if (pixels !== undefined) return `Scrolled ${direction} by ${honoredPixels ?? pixels}px`;
185220
if (amount !== undefined) {
186221
return honoredPixels === undefined

‎packages/capture-kit/src/snapshot/scroll-edge-state/selection.ts‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,9 @@ export function analyzeScrollEdgeState(
3838
return {
3939
canScroll,
4040
emptySnapshot: false,
41+
// Every selection path admits only `isUsableRect` containers, so a resolved container always
42+
// has a frame to report and its absence always means no container was resolved.
43+
...(container.rect ? { containerRect: container.rect } : {}),
4144
scope: buildScrollContainerScope(container, nodes),
4245
fingerprint: buildSurfaceFingerprint(container, nodes),
4346
};

‎packages/contracts/src/scroll-command.ts‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,24 @@ export const DEFAULT_IOS_SCROLL_AMOUNT = 0.65;
99

1010
export type ScrollReleaseBehavior = 'controlled' | 'inertial';
1111

12+
/**
13+
* What a directional scroll actually SAW after its gesture, which is the only evidence that can
14+
* back the distance the same response reports (#2714).
15+
*
16+
* - `'moved'`: the post-gesture surface differs from the pre-gesture one, so content did move.
17+
* - `'at-edge'`: the surface is identical and the resolved container names no hidden content in
18+
* that direction — the scroll was a legitimate no-op at the end of the content.
19+
* - `'unchanged'`: the surface is identical and the direction has no end-of-content signal to read
20+
* (a horizontal scroll: the hidden-content analyzer only covers the vertical axis), so the
21+
* response says what it measured without guessing which of the two it was.
22+
* - `'unobserved'`: nothing comparable was available, so the distance rests on the gesture plan
23+
* alone. Callers that need the effect confirmed ask for a capture or a `--settle` observation.
24+
*
25+
* A directional scroll that measures an unchanged surface WITH hidden content still in that
26+
* direction does not answer at all: it fails with `scroll_no_progress`.
27+
*/
28+
export type ScrollMovementObservation = 'moved' | 'at-edge' | 'unchanged' | 'unobserved';
29+
1230
export type ScrollDistanceOptions = {
1331
amount?: number;
1432
pixels?: number;
@@ -118,6 +136,14 @@ export type ScrollCommandResult = {
118136
durationMs?: number;
119137
message?: string;
120138
settle?: SettleObservation;
139+
/**
140+
* The observation that gated this response's distance claim. See
141+
* {@link ScrollMovementObservation}: `scroll` answers with what it measured after the gesture,
142+
* and only `'moved'` and `'at-edge'` confirm the surface's fate. Absent on the tiers that verify
143+
* per pass instead of per gesture (`scroll top`/`bottom` and `--until`), and on platforms whose
144+
* scroll owner never dispatches a swipe (the Linux wheel).
145+
*/
146+
movement?: ScrollMovementObservation;
121147
/**
122148
* Set only when an on-screen keyboard made the owner clip the swipe into the band above it
123149
* (#2500). Absent means the swipe was not clipped, which is not the same claim as `false`: a

0 commit comments

Comments
 (0)