Skip to content

Commit 7ffc88b

Browse files
author
Cameron Chisholm
committed
feat(screenshot): accept --crop-on on physical iOS devices
1 parent 507ef23 commit 7ffc88b

7 files changed

Lines changed: 31 additions & 8 deletions

File tree

‎contracts/fixtures/screenshot-crop-geometry.json‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,15 @@
1717
"expectedProjection": { "x": 48, "y": 880, "width": 1110, "height": 156 },
1818
"expectedIntersection": { "x": 48, "y": 880, "width": 1110, "height": 156 }
1919
},
20+
{
21+
"name": "iOS physical 3x capture (iPhone 12): a fractional points rect scales by imgW/bounds.width and rounds",
22+
"space": "viewport-points",
23+
"bounds": { "x": 0, "y": 0, "width": 390, "height": 844 },
24+
"rect": { "x": 255.6667, "y": 73.6667, "width": 54.3333, "height": 14.3333 },
25+
"image": { "width": 1170, "height": 2532 },
26+
"expectedProjection": { "x": 767, "y": 221, "width": 163, "height": 43 },
27+
"expectedIntersection": { "x": 767, "y": 221, "width": 163, "height": 43 }
28+
},
2029
{
2130
"name": "Android: device-pixel rects project 1:1 with no bounds lookup",
2231
"space": "device-pixels",

‎packages/capture-kit/src/snapshot-rect-projection.test.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ import {
1414
} from './snapshot-rect-projection.ts';
1515

1616
// Golden geometry table for `screenshot --crop-on`: the numbers are the live
17-
// evidence baseline (iOS simulator 402/1206, Android 1080, macOS 586/1172),
17+
// evidence baseline (iOS simulator 402/1206, iOS physical 390/1170, Android 1080, macOS 586/1172),
1818
// so a projection-law change that moves any accepted cell's crop box turns this
1919
// red before the pixel-identity cross-check could ever notice.
2020

‎packages/daemon/src/execution/__tests__/screenshot-crop-target.test.ts‎

Lines changed: 17 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ import {
33
ANDROID_EMULATOR,
44
IOS_DEVICE,
55
IOS_SIMULATOR,
6+
IPADOS_SIMULATOR,
67
LINUX_DEVICE,
78
MACOS_DEVICE,
89
TVOS_SIMULATOR,
@@ -59,7 +60,11 @@ test('the classifier and the acceptance matrix agree one-to-one, and the accepte
5960
}
6061
expect(CROP_TARGET_DEVICES.map((row) => row.target)).toEqual(matrixTargets);
6162
for (const cell of SCREENSHOT_CROP_TARGET_CELLS) {
62-
if (cell.target === 'ios-simulator' || cell.target === 'android-emulator') {
63+
if (
64+
cell.target === 'ios-simulator' ||
65+
cell.target === 'android-emulator' ||
66+
cell.target === 'ios-physical'
67+
) {
6368
expect(cell.status).toBe('accepted');
6469
} else {
6570
expect(cell).toEqual({
@@ -71,6 +76,12 @@ test('the classifier and the acceptance matrix agree one-to-one, and the accepte
7176
}
7277
});
7378

79+
test('a physical iPad classifies into the ios-physical cell', () => {
80+
expect(classifyScreenshotCropTarget({ ...IPADOS_SIMULATOR, kind: 'device' }, undefined)).toBe(
81+
'ios-physical',
82+
);
83+
});
84+
7485
const MACOS_SURFACE_BACKENDS: Record<SessionSurface, MacOsSurfaceBackend> = {
7586
app: 'xctest',
7687
'frontmost-app': 'macos-helper',
@@ -168,7 +179,9 @@ test('a matrix-rejected target is refused with the pending-evidence reason', ()
168179
});
169180

170181
test('an accepted target with a valid selector passes the policy', () => {
171-
expect(() =>
172-
assertScreenshotCropPolicy({ device: ANDROID_EMULATOR, surface: undefined, ...POLICY_BASE }),
173-
).not.toThrow();
182+
for (const device of [IOS_SIMULATOR, ANDROID_EMULATOR, IOS_DEVICE]) {
183+
expect(() =>
184+
assertScreenshotCropPolicy({ device, surface: undefined, ...POLICY_BASE }),
185+
).not.toThrow();
186+
}
174187
});

‎packages/daemon/src/execution/screenshot-crop-target.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ export const SCREENSHOT_CROP_TARGET_CELLS: readonly CropTargetCell[] = [
4646
{ target: 'android-emulator', status: 'accepted' },
4747
{ target: 'android-device', status: 'rejected', rejectionReason: PENDING },
4848
{ target: 'macos-app-window', status: 'rejected', rejectionReason: PENDING },
49-
{ target: 'ios-physical', status: 'rejected', rejectionReason: PENDING },
49+
{ target: 'ios-physical', status: 'accepted' },
5050
{ target: 'macos-helper', status: 'rejected', rejectionReason: PENDING },
5151
{ target: 'web', status: 'rejected', rejectionReason: PENDING },
5252
{ target: 'linux', status: 'rejected', rejectionReason: PENDING },

‎src/commands/capture/screenshot.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ export const screenshotCommandFacet = defineCommandFacet({
7474
text: {
7575
summary: 'Capture a screenshot',
7676
cliDetail:
77-
"Web defaults to the viewport; use --fullscreen, --full, or -f for the entire page. iOS simulators default to 1x logical-point output; use --pixel-density to request a different screenshot density. macOS app sessions default to the app window; use --fullscreen for full desktop, --scale to downscale, --crop-on <selector> to crop the capture to the frame the selector resolves on the same screen (currently iOS simulators and Android emulators), --overlay-refs to annotate current refs, --normalize-status-bar for deterministic iOS simulator or Android chrome. Android captures immediately by default; --normalize-status-bar also waits for transient UI to fade, and --no-stabilize keeps capture immediate. With the native macOS backend, the front window is the app's frontmost visible, nonempty normal-level (layer-0) window on the current Space: floating palettes and app-modal alerts at window level 8 are skipped, and a sheet is captured as its own window. On macOS, any --surface other than app (desktop, menubar, frontmost-app) always captures the main display and refuses an explicit --fullscreen. On single-display iOS simulators and Android devices, the result reports displayRotation, the rotation the display rendered the capture in, when the device returns a reading in time; otherwise the field is absent.",
77+
"Web defaults to the viewport; use --fullscreen, --full, or -f for the entire page. iOS simulators default to 1x logical-point output; use --pixel-density to request a different screenshot density. macOS app sessions default to the app window; use --fullscreen for full desktop, --scale to downscale, --crop-on <selector> to crop the capture to the frame the selector resolves on the same screen (currently iOS simulators, physical iOS devices, and Android emulators), --overlay-refs to annotate current refs, --normalize-status-bar for deterministic iOS simulator or Android chrome. Android captures immediately by default; --normalize-status-bar also waits for transient UI to fade, and --no-stabilize keeps capture immediate. With the native macOS backend, the front window is the app's frontmost visible, nonempty normal-level (layer-0) window on the current Space: floating palettes and app-modal alerts at window level 8 are skipped, and a sheet is captured as its own window. On macOS, any --surface other than app (desktop, menubar, frontmost-app) always captures the main display and refuses an explicit --fullscreen. On single-display iOS simulators and Android devices, the result reports displayRotation, the rotation the display rendered the capture in, when the device returns a reading in time; otherwise the field is absent.",
7878
},
7979
metadata: screenshotCommandMetadata,
8080
run: (client, input) => client.capture.screenshot(input),

‎src/commands/schema/cli-help-command-usage.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ test('usageForCommand documents screenshot web aliases and stabilization flags',
3030
assert.match(help, /deterministic iOS simulator or Android chrome/);
3131
assert.match(help, /--crop-on <selector-expression>/);
3232
assert.match(help, /crop the capture to the frame of the selector/);
33+
assert.match(help, /currently iOS simulators, physical iOS devices, and Android emulators/);
3334
});
3435

3536
test('usageForCommand documents screenshot diff normalization', async () => {

‎website/docs/docs/commands.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -985,7 +985,7 @@ agent-device record stop # Stop active recording
985985
- `--scale <factor>` resizes both dimensions, from `0.01` to `1` (full resolution). Set `AGENT_DEVICE_SCREENSHOT_SCALE=0.3` or `screenshotScale` in config as a token-conscious default; an explicit `--scale` overrides it. Leave it unset or pass `--scale 1` for pixel-diff baselines. `--max-size` is refused everywhere (CLI, `.ad` scripts, Node options, config, `AGENT_DEVICE_SCREENSHOT_MAX_SIZE`) with guidance to use `--scale`.
986986
- ADB-backed Android screenshots capture immediately by default. Use `--normalize-status-bar` to set deterministic status-bar chrome and wait for transient UI to fade. In the Node client, `stabilize: true` also enables this wait.
987987
- `--overlay-refs` takes a fresh snapshot and burns visible `@eN` refs and their rectangles into the PNG. With `--scale`, refs are drawn at the final size, so avoid scales where labels become unreadable. In `--json` mode each ref also has a screenshot-space `center` for `press <x> <y>` fallback.
988-
- `--crop-on <selector>` crops the capture to the frame the selector resolves on the same screen. It works only on iOS simulators and Android emulators, needs exactly one framed node, cannot be combined with `--overlay-refs` or `--fullscreen`, and warns when the frame is clipped. The crop is re-encoded, so do not byte-compare it with an older crop.
988+
- `--crop-on <selector>` crops the capture to the frame the selector resolves on the same screen. It works only on iOS simulators, physical iOS devices, and Android emulators, needs exactly one framed node, cannot be combined with `--overlay-refs` or `--fullscreen`, and warns when the frame is clipped. The crop is re-encoded, so do not byte-compare it with an older crop.
989989
- `--fullscreen` on macOS applies only to app sessions. Other surfaces (`desktop`, `menubar`, `frontmost-app`) always capture the main display and refuse `--fullscreen` with `INVALID_ARGS`.
990990
- `--normalize-status-bar` temporarily normalizes iOS simulator status-bar chrome for reusable baselines.
991991
- On single-display iOS simulators and Android devices, the result may report `displayRotation` (`portrait`, `portrait-upside-down`, `landscape-left`, or `landscape-right`). It is absent when the device returns no reading in time, on other targets, and on multi-display devices such as foldables. The saved image is always upright.

0 commit comments

Comments
 (0)