Skip to content

Commit 2cc05f3

Browse files
committed
fix(ios): quote the scroll --until selector for the shell and scope the unmeasured mark to regular snapshots
The recovery hint wrapped the selector with JSON.stringify, and inside double quotes a shell still expands $, backticks and !. It now uses a POSIX single-quote wrap. The help and snapshots.md say the [unmeasured] mark belongs to regular snapshots: --raw returns before the fold.
1 parent 83a50db commit 2cc05f3

4 files changed

Lines changed: 33 additions & 6 deletions

File tree

‎packages/command-runtime/src/commands/interaction/runtime/resolution-touch-point.test.ts‎

Lines changed: 21 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -135,7 +135,7 @@ test('runtime press refuses a selector match with a zero-area rect and dispatche
135135
* A Flutter screen on iOS at the top of its content (#4451), as the Simulator AX bridge presents it:
136136
* the scroll view and its content are siblings, and the content below the fold keeps a 0x0 rect.
137137
*/
138-
function scrolledOutButtonSnapshot() {
138+
function scrolledOutButtonSnapshot(label = 'View all') {
139139
return makeSnapshotState([
140140
{
141141
index: 0,
@@ -169,7 +169,7 @@ function scrolledOutButtonSnapshot() {
169169
parentIndex: 0,
170170
type: 'Button',
171171
role: 'FlutterSemanticsObject',
172-
label: 'View all',
172+
label,
173173
rect: { x: 0, y: 0, width: 0, height: 0 },
174174
hittable: false,
175175
},
@@ -192,7 +192,7 @@ test('runtime press on a zero-area target beside a scroll container names the sc
192192
assert.equal(details?.reason, 'target_bounds_invalid');
193193
assert.equal(details?.dispatched, 'no');
194194
assert.equal(details?.scrollContainerRef, '@e2');
195-
assert.match(String(details?.hint), /scroll <direction> --until "label=\\"View all\\""/);
195+
assert.match(String(details?.hint), /scroll <direction> --until 'label="View all"'/);
196196
return true;
197197
},
198198
);
@@ -212,6 +212,24 @@ test('runtime press on a zero-area target beside a scroll container names the sc
212212
assert.deepEqual(taps, []);
213213
});
214214

215+
test('the scroll a zero-area target is pointed at survives the shell', async () => {
216+
// The hint is copied into a shell: double quotes would expand the `$`, and a bare single-quote
217+
// wrap would end at the apostrophe.
218+
const device = createInteractionDevice(scrolledOutButtonSnapshot("Today's $5 deal"), {
219+
tap: async () => ({ ok: true }),
220+
});
221+
222+
await assert.rejects(
223+
() => device.interactions.press(selector('label="Today\'s $5 deal"'), { session: 'default' }),
224+
(error: unknown) => {
225+
const details = (error as { details?: Record<string, unknown> }).details;
226+
assert.equal(details?.reason, 'target_bounds_invalid');
227+
assert.match(String(details?.hint), /--until 'label="Today'\\''s \$5 deal"'/);
228+
return true;
229+
},
230+
);
231+
});
232+
215233
test('runtime press on a Compose button dispatches its center only once its role node is folded', async () => {
216234
const tree = parseUiHierarchyTree(`<hierarchy>
217235
<node class="android.view.View" resource-id="compose-container" bounds="[53,1856][1027,1982]"

‎packages/command-surface/src/capture/snapshot.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ export const snapshotCommandFacet = defineCommandFacet({
7575
name: SNAPSHOT_COMMAND_NAME,
7676
text: {
7777
summary: 'Capture or diff the accessibility tree',
78-
cliDetail: `Scoped output (--scope) renders every returned ref with its ref generation pin (@eN~s<refsGeneration>) and one Refs generation line, so a pasted ref targets the ref frame that issued it; --json keeps plain refs plus the response-level refsGeneration, and --raw keeps byte-for-byte output. Regular Android ADB snapshots clip app rectangles outside measured status and navigation bars, including launcher-owned navigation; fully covered content is omitted and --raw retains reported bounds. Ref and selector taps honor measured visible bounds even for refs from raw snapshots. Regular iOS snapshots clip overflowing elements to their visible area and omit fully outside elements. In an iOS snapshot presented from an acquired tree (the Simulator AX bridge and provider backends, not the XCTest runner), an element that reports a frame measuring nothing is kept and marked [unmeasured]: nothing confirms it is on screen. If it is inside scrollable content, reach it with scroll <direction> --until <selector> before acting on it. --json reports clipping counts in snapshotQualityNotes with code regular-geometry-clipped; --raw retains reported frames. --observe-only is local iOS snapshot-only: an already-ready compatible runner reads the session app without activation or foreground repair, refuses unavailable observation or a readable system alert over the app, and reports XCUIApplication state rather than verified foreground ownership (#2696). Regular targetActivation disclosure stays unchanged (#2682, #2694). Repeated equivalent unfiltered Android snapshots return a compact unchanged acknowledgement. Use --force-full to re-emit the tree; --json and --raw retain full output. Android Appium JSON snapshots publish a display viewport only when measurements before and after the tree agree, independent of tree extent or keyboard content resizing. On Android, JSON value carries field content rather than static labels; saved selectors and replay scripts using value= for static labels must use label= instead; a reported showing hint means value is empty, and text equal to the placeholder is preserved when hint state is unknown. HarmonyOS reads the on-screen keyboard's own system window and publishes the keyboard field of --json output: the field is visibility-only — visible or hidden per the keyboard window's z-order column; a failed probe, an unreadable or conflicting z-order, or a table without the panel publishes unmeasurable, never a guessed band. When an iOS snapshot reports a screen viewport, inspect rects with snapshot -i --json and press the intended control's center, then verify with diff snapshot -i or snapshot --diff. When a fallback cannot establish the screen origin, viewport is omitted and coordinate-derived element actions refuse with window-origin-unknown before input; the snapshot may warn about the unavailable origin. Use a plain screenshot to choose screen coordinates. An empty text field with no label or value prints its hint as [placeholder:"..."]; label=, text= and find do not match that hint. iOS backend capability contract: ${snapshotBackendCapabilityHelp}.`,
78+
cliDetail: `Scoped output (--scope) renders every returned ref with its ref generation pin (@eN~s<refsGeneration>) and one Refs generation line, so a pasted ref targets the ref frame that issued it; --json keeps plain refs plus the response-level refsGeneration, and --raw keeps byte-for-byte output. Regular Android ADB snapshots clip app rectangles outside measured status and navigation bars, including launcher-owned navigation; fully covered content is omitted and --raw retains reported bounds. Ref and selector taps honor measured visible bounds even for refs from raw snapshots. Regular iOS snapshots clip overflowing elements to their visible area and omit fully outside elements. In a regular (non-raw) iOS snapshot presented from an acquired tree (the Simulator AX bridge and provider backends, not the XCTest runner), an element that reports a frame measuring nothing is kept and marked [unmeasured]: nothing confirms it is on screen. If it is inside scrollable content, reach it with scroll <direction> --until <selector> before acting on it. --json reports clipping counts in snapshotQualityNotes with code regular-geometry-clipped; --raw retains reported frames. --observe-only is local iOS snapshot-only: an already-ready compatible runner reads the session app without activation or foreground repair, refuses unavailable observation or a readable system alert over the app, and reports XCUIApplication state rather than verified foreground ownership (#2696). Regular targetActivation disclosure stays unchanged (#2682, #2694). Repeated equivalent unfiltered Android snapshots return a compact unchanged acknowledgement. Use --force-full to re-emit the tree; --json and --raw retain full output. Android Appium JSON snapshots publish a display viewport only when measurements before and after the tree agree, independent of tree extent or keyboard content resizing. On Android, JSON value carries field content rather than static labels; saved selectors and replay scripts using value= for static labels must use label= instead; a reported showing hint means value is empty, and text equal to the placeholder is preserved when hint state is unknown. HarmonyOS reads the on-screen keyboard's own system window and publishes the keyboard field of --json output: the field is visibility-only — visible or hidden per the keyboard window's z-order column; a failed probe, an unreadable or conflicting z-order, or a table without the panel publishes unmeasurable, never a guessed band. When an iOS snapshot reports a screen viewport, inspect rects with snapshot -i --json and press the intended control's center, then verify with diff snapshot -i or snapshot --diff. When a fallback cannot establish the screen origin, viewport is omitted and coordinate-derived element actions refuse with window-origin-unknown before input; the snapshot may warn about the unavailable origin. Use a plain screenshot to choose screen coordinates. An empty text field with no label or value prints its hint as [placeholder:"..."]; label=, text= and find do not match that hint. iOS backend capability contract: ${snapshotBackendCapabilityHelp}.`,
7979
},
8080
metadata: snapshotCommandMetadata,
8181
run: (client, input) => client.capture.snapshot(input),

‎packages/selectors/src/node-touch-point.ts‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -139,6 +139,15 @@ function isScrollIndicator(node: SnapshotNode): boolean {
139139
return /scroll\s?(bar|indicator)/.test(kind);
140140
}
141141

142+
/**
143+
* POSIX single-quote wrap, so the hint's command survives a paste into a shell whatever the
144+
* selector holds. The kernel's `shellQuote` is the same rule; importing it here would add an eager
145+
* module to this file's closure.
146+
*/
147+
function quoteForShell(value: string): string {
148+
return `'${value.replaceAll("'", String.raw`'\''`)}'`;
149+
}
150+
142151
/**
143152
* The recovery for a target with no usable frame beside a scroll container. The frame that
144153
* refuses the touch point is also the one that would have named the direction, so the hint names
@@ -154,7 +163,7 @@ function scrollContainerRecovery(
154163
if (!container) return {};
155164
const recovery =
156165
'selector' in target
157-
? `run scroll <direction> --until ${JSON.stringify(target.selector)} to bring it on screen, then retry with the same selector.`
166+
? `run scroll <direction> --until ${quoteForShell(target.selector)} to bring it on screen, then retry with the same selector.`
158167
: 'scroll toward it, then retry with a selector (e.g. text=/id=) rather than an @ref — the scroll expires the ref frame, so re-run snapshot -i before reusing any @ref.';
159168
return {
160169
...(container.ref ? { scrollContainerRef: `@${container.ref}` } : {}),

‎website/docs/docs/snapshots.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,7 @@ and disclosure behavior ([#2682](https://github.com/callstack/agent-device/issue
8484
- Off-screen interactive content is collapsed into summaries that show only a few labels, such as `[off-screen below] 3 interactive items: "Privacy", "Battery", "About"`.
8585
- If your target only appears in an off-screen summary, run `scroll <direction> --until <selector>` to stop on it, or `scroll <direction> --settle` to get the diff with fresh refs. See [Interactions](/docs/commands#interactions).
8686
- When agent-device knows which container holds the hidden content, it shows the summary inside that scroll or list container, for example `[content above scroll-area hidden]` or `[content below list hidden]`.
87-
- In an iOS snapshot that agent-device presents from an acquired tree (the Simulator AX bridge and provider backends), an element that reports a frame measuring nothing is kept and marked `[unmeasured]`: nothing confirms it is on screen. A Flutter scroll view keeps its scrolled-out content this way. If the element is inside scrollable content, run `scroll <direction> --until <selector>` before acting on it. Snapshots presented by the XCTest runner do not add this mark.
87+
- In a regular (non-`--raw`) iOS snapshot that agent-device presents from an acquired tree (the Simulator AX bridge and provider backends), an element that reports a frame measuring nothing is kept and marked `[unmeasured]`: nothing confirms it is on screen. A Flutter scroll view keeps its scrolled-out content this way. If the element is inside scrollable content, run `scroll <direction> --until <selector>` before acting on it. Snapshots presented by the XCTest runner do not add this mark.
8888
- Add `-s "<label>"` (or `-s @ref`) to limit output to one part of the screen. See [Scope a snapshot](#scope-a-snapshot).
8989
- Add `-d <depth>` when you only need the upper layers of the hierarchy.
9090
- If `snapshot -i -d <n>` reports no interactive elements at that depth, retry once without `-d` instead of taking more shallow snapshots.

0 commit comments

Comments
 (0)