Skip to content

Commit a552b16

Browse files
committed
feat(snapshot): disclose a cut capture on every platform; raise the iOS bridge node cap to 5000
Every backend sets truncated: true when it cuts a capture at one of its limits, but only JSON carried it. One shared warning now renders from that flag in the cross-platform warnings assembly and tells the agent what fell off (what comes last in document order) and what to do. The iOS Simulator AX bridge cap moves from 1500 to 5000 nodes, the Android helper's bound. Measured on a synthetic 600-row screen, acquisition time did not change with the cap while the 1500 cut dropped the on-screen footer.
1 parent 3cd6341 commit a552b16

7 files changed

Lines changed: 78 additions & 4 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,16 @@
22

33
## Unreleased
44

5+
- Changed: a capture that a backend cut at one of its limits now says so in the snapshot's
6+
warnings, on every platform, instead of only setting `truncated: true` in JSON. The text path
7+
had no disclosure at all, so an agent read a screen missing its footer, tab bar, or the items
8+
after a long list as complete — the backends walk the tree in document order, so what falls
9+
off is what comes last, on screen or not. One shared warning renders from the shared flag; the
10+
limit and dimension stay backend-side.
11+
- Changed: the iOS Simulator AX bridge caps a capture at 5000 nodes, up from 1500, the Android
12+
helper's bound. Measured on a synthetic 600-row screen, acquisition time did not move with the
13+
cap (the native read fetches the whole tree; the cap only stops conversion) while the 1500 cut
14+
dropped the screen's on-screen footer.
515
- Fixed: Custom test reporters reject invalid exit codes, including values such as `256` that
616
could wrap to success and hide a failing suite. `getExitCode` accepts integers from `0` to `255`
717
or `undefined`; JSON output reports an invalid code as one `INVALID_ARGS` error.

‎packages/capture-kit/src/snapshot/snapshot-presentation/quality-warnings.ts‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,20 @@ function customActionCoverageWarning(verdict: SnapshotQualityVerdict): string[]
3333
return lines;
3434
}
3535

36+
/**
37+
* One disclosure for every backend that reports a cut capture: the iOS Simulator bridge and the
38+
* Android helper stop at a node cap, the XCTest runner and the web provider at their own bounds.
39+
* Every one walks the tree in document order, so what falls off is what comes last — footers,
40+
* tab bars, the items after a long list — even when it is on screen. The fact is the shared
41+
* `truncated` flag; the dimension and limit stay backend-side, so the copy names neither.
42+
*/
43+
export function truncatedCaptureWarning(truncated: boolean | undefined): string[] {
44+
if (truncated !== true) return [];
45+
return [
46+
'This capture was cut at a backend limit, so elements later in the tree — footers, tab bars, items after a long list — may be missing even when they are on screen; their refs and selectors cannot be resolved from this snapshot. Navigate or scroll so fewer elements render and re-run, and use screenshot as visual truth for what is missing.',
47+
];
48+
}
49+
3650
export function recoveredSnapshotQualityWarning(
3751
backend: SnapshotQualityVerdict['backend'],
3852
): string {

‎packages/capture-kit/src/snapshot/snapshot-presentation/warnings.test.ts‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
import assert from 'node:assert/strict';
22
import { test } from 'vitest';
33
import { readSnapshotQualityVerdict } from '../../snapshot-quality-verdict.ts';
4-
import { renderSnapshotQualityWarnings } from './quality-warnings.ts';
4+
import { renderSnapshotQualityWarnings, truncatedCaptureWarning } from './quality-warnings.ts';
55

66
const sharedRecoveryReason =
77
'iOS XCTest snapshot failed while serializing the accessibility tree. Error kAXErrorIllegalArgument getting snapshot for element <AXUIElementRef 0x1>';
@@ -270,3 +270,14 @@ test('the coverage line is independent of degradation state', () => {
270270
assert.equal(warnings.length, 1);
271271
assert.match(warnings[0] ?? '', /read for 3 of 8 merged elements/);
272272
});
273+
274+
test('a cut capture is disclosed once, from the shared truncated flag alone', () => {
275+
assert.deepEqual(truncatedCaptureWarning(false), []);
276+
assert.deepEqual(truncatedCaptureWarning(undefined), []);
277+
const [warning, ...rest] = truncatedCaptureWarning(true);
278+
assert.deepEqual(rest, []);
279+
assert.match(warning ?? '', /cut at a backend limit/);
280+
assert.match(warning ?? '', /footers, tab bars/);
281+
assert.match(warning ?? '', /refs and selectors cannot be resolved/);
282+
assert.match(warning ?? '', /screenshot/);
283+
});

‎packages/platform-apple/src/snapshot-source/limits.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ const MAXIMUM_NODES = 10_000;
1010
export const DEFAULT_SNAPSHOT_SOURCE_LIMITS: SnapshotSourceLimits = Object.freeze({
1111
maxRequestBytes: 64 * 1024,
1212
maxResponseBytes: 4 * 1024 * 1024,
13-
maxNodes: 1500,
13+
maxNodes: 5000,
1414
maxTraversalDepth: 64,
1515
maxDurationMs: 5_000,
1616
});

‎src/commands/capture/runtime/snapshot.test.ts‎

Lines changed: 30 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -350,7 +350,7 @@ test('runtime snapshot renders the structured quality verdict and skips legacy d
350350

351351
const result = await device.capture.snapshot({ session: 'default' });
352352

353-
assert.equal(result.warnings?.length, 2);
353+
assert.equal(result.warnings?.length, 3);
354354
assert.match(
355355
String(result.warnings?.[0]),
356356
/Detected an overly complex or slow accessibility tree/,
@@ -359,6 +359,8 @@ test('runtime snapshot renders the structured quality verdict and skips legacy d
359359
assert.match(String(result.warnings?.[0]), /It is OK to continue/);
360360
assert.match(String(result.warnings?.[0]), /snapshotQuality\.reason/);
361361
assert.match(String(result.warnings?.[1]), /@e2 \[Other\] merges many labels/);
362+
// The fixture is a cut capture; the shared disclosure follows the verdict's own warnings.
363+
assert.match(String(result.warnings?.[2]), /cut at a backend limit/);
362364
assert.deepEqual(result.snapshotQuality?.state, 'recovered');
363365
});
364366

@@ -703,3 +705,30 @@ function assertReactNativeOverlayWarning(warnings: string[] | undefined) {
703705
assert.match(warnings[0] ?? '', /agent-device react-native dismiss-overlay/);
704706
assert.match(warnings[0] ?? '', /verifies the overlay is gone/);
705707
}
708+
709+
test('runtime snapshot discloses a cut capture the same way on every backend', async () => {
710+
for (const backend of ['android', 'xctest'] as const) {
711+
const device = createSnapshotOnlyDevice({
712+
nodes: [{ ref: 'e1', index: 0, depth: 0, type: 'Window', label: 'Home' }],
713+
truncated: true,
714+
backend,
715+
});
716+
717+
const result = await device.capture.snapshot({ session: 'default' });
718+
719+
assert.equal(result.truncated, true, backend);
720+
const cut = (result.warnings ?? []).filter((warning) => /cut at a backend limit/.test(warning));
721+
assert.equal(cut.length, 1, `${backend}: ${JSON.stringify(result.warnings)}`);
722+
}
723+
724+
const complete = createSnapshotOnlyDevice({
725+
nodes: [{ ref: 'e1', index: 0, depth: 0, type: 'Window', label: 'Home' }],
726+
truncated: false,
727+
backend: 'android',
728+
});
729+
const result = await complete.capture.snapshot({ session: 'default' });
730+
assert.equal(
731+
(result.warnings ?? []).some((warning) => /cut at a backend limit/.test(warning)),
732+
false,
733+
);
734+
});

‎src/commands/capture/runtime/snapshot.ts‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,10 @@ import {
2121
buildSnapshotDiff,
2222
countSnapshotComparableLines,
2323
} from '@agent-device/capture-kit/snapshot-diff';
24-
import { renderSnapshotQualityWarnings } from '@agent-device/capture-kit/quality-warnings';
24+
import {
25+
renderSnapshotQualityWarnings,
26+
truncatedCaptureWarning,
27+
} from '@agent-device/capture-kit/quality-warnings';
2528
import { buildSnapshotVisibility } from '@agent-device/capture-kit/snapshot-visibility';
2629
import { ANDROID_SYSTEM_SURFACE_DISCLOSURE } from '@agent-device/contracts/android-system-surface-disclosure';
2730
import { formatReactNativeOverlayWarning } from '../../react-native/overlay.ts';
@@ -247,6 +250,7 @@ function buildSnapshotWarnings(params: {
247250
...renderSnapshotQualityWarnings(params.annotations.quality, params.snapshot.nodes),
248251
);
249252
}
253+
warnings.push(...truncatedCaptureWarning(snapshotTruncationForResult(params.snapshot)));
250254
warnings.push(...buildEmptyAndroidInteractiveWarnings(params));
251255
if (!params.annotations.quality) {
252256
// Legacy runners without a structured verdict keep the old daemon-side heuristics.

‎website/docs/docs/commands.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -377,6 +377,12 @@ agent-device get attrs @e1
377377
Android `--raw` is the acquired tree: it also keeps nodes Android marks invisible and stale
378378
application windows. The helper does not report `checked`/`selected` state, and it caps
379379
captures at 5000 nodes before any `--scope` applies (`truncated: true`).
380+
- `truncated: true` means the backend cut the capture at one of its limits — the Android helper
381+
and the iOS Simulator AX bridge at 5000 nodes, the XCTest runner and the web provider at their
382+
own bounds. Every backend walks the tree in document order, so what falls off is what comes
383+
last: footers, tab bars, items after a long list, even when on screen. The snapshot carries a
384+
warning that says so; navigate or scroll so fewer elements render and re-run, and use
385+
`screenshot` as visual truth for the rest.
380386
- `--scope <text|@ref>` returns the subtree of the first node in document order whose label, value,
381387
or identifier contains the scope text (case-insensitive) and whose subtree still has content in
382388
the requested projection, re-rooted at depth 0; no match returns an empty snapshot rather than the

0 commit comments

Comments
 (0)