Skip to content

Commit dbc5b09

Browse files
thymikeethymikee
committed
fix: unify divergence screen capture with snapshot's full-window scope
Route captureDivergenceObservation through captureSnapshotData — the same function the snapshot command itself builds its capture with (Android's snapshot-helper full-window route with its graceful app-scoped fallback, iOS's bounded system-modal probe path, macOS/Linux surface-scoped branches) — instead of a parallel hand-rolled dispatchCommand call. The chrome filter and meaningful-target filter stay layered on top as filters over that full capture, never as a scoping. Amends ADR-0012 decision 4 to state the invariant: an agent must never see a healthier `screen` in a divergence report than a plain `snapshot` would show it, so a separate-window system overlay (volume dialog, quick-settings shade, permission dialog) must survive into `screen.refs` exactly as `snapshot` would present it. Also fixes the synthetic `volume_dialog_slider` id in snapshot-chrome-android-statusbar.test.ts to the real, live-verified `volume_new_ringer_active_icon_container` id and rewords the test comment to read as a filter-logic unit test rather than a live-capture-path claim, and adds unit coverage for the invariant itself. Fixes #1264
1 parent 0436793 commit dbc5b09

4 files changed

Lines changed: 146 additions & 32 deletions

File tree

‎docs/adr/0012-interactive-replay.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,9 @@ Accepted (2026-07-10); partially implemented (last updated 2026-07-13). See [Mig
1212
`REPLAY_DIVERGENCE` repair-loop fix (#1223).
1313
- Decision 6, the base agent-supervised re-record repair — `replay --save-script` arming, the
1414
post-watermark healed slice, `repairHint`, and the writer's bare-`@ref` fail-loud guard (#1228).
15+
- Decision 4 amendment, `screen`'s capture scope — the divergence `screen` capture is unified with
16+
`snapshot`'s own full-window scope, with the chrome and meaningful-target filters layered on top as
17+
filters, not scopings (#1264).
1518

1619
**Accepted but NOT yet implemented** (this amendment; tracked by #1235 — repair-transaction lifecycle):
1720
the R7 repair-transaction keep-alive and its distinct `resume.repairSessionHeld` signal, the ARMED →
@@ -438,6 +441,22 @@ healthy snapshot digest and the only form that issues actionable refs. `{ state:
438441
hint }` is returned when capture fails or is sparse; it has no refs or generation and must not fall back to
439442
the old session tree. Screen-capture failure never replaces or masks the original replay cause.
440443

444+
> **Amendment (#1264): `screen`'s capture scope.** `refs` is a filtered digest of the exact same tree a
445+
> plain `snapshot` would return at that moment — the capture underneath `screen` is built with `snapshot`'s
446+
> own full-window scope (Android: the snapshot-helper route, with the existing graceful app-scoped fallback
447+
> only when the helper is unavailable; iOS: the same capture semantics as the bounded system-modal probe
448+
> path; macOS/Linux: their surface-scoped branches), never a narrower, app-window-only capture assembled
449+
> separately for divergence reporting. The chrome filter (#1233/#1256, `collectSettleChromeRefs`) and the
450+
> meaningful-target filter (label/id or `hittable`) are layered ON TOP of that full capture as **filters**,
451+
> not as a separate, narrower scoping — a filter may drop a node the full capture contains, but the capture
452+
> itself must never omit content `snapshot` would show. This is a hard invariant: **an agent must never see
453+
> a healthier `screen` in a divergence report than a plain `snapshot` would show it.** Concretely, a
454+
> separate-window system overlay covering the app at the moment of capture — a held volume dialog, a
455+
> persistent quick-settings shade, a permission dialog — must appear in `screen.refs` (its actionable/
456+
> hittable/labeled nodes surviving the filters) exactly as `snapshot` would present it, and `repairHint`
457+
> (decision 6) is computed over that same full, correctly-scoped capture so it is never routed as if the
458+
> app underneath a covering overlay were healthy.
459+
441460
Response levels bound the entire serialized UTF-8 `details.divergence` object, not merely its arrays:
442461
compact (`--level digest`) is at most **8 KiB**, default at most **24 KiB**, and full at most **64 KiB**.
443462
Compact carries at most **8** screen refs and no suggestion entries — it carries `suggestionCount` (the

‎src/core/__tests__/snapshot-chrome-android-statusbar.test.ts‎

Lines changed: 17 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -104,14 +104,16 @@ test('Android non-raw capture: status-bar leaves are recognized as chrome once t
104104
}
105105
});
106106

107-
test('Android actionable systemui overlay (volume dialog) still survives with the status-bar leak fix (#1251)', () => {
108-
// Extend the RAW capture with a disjoint systemui run whose leaf ids look
109-
// like an actionable overlay (volume dialog): no real capture of that
110-
// surface was available, but the point of this test is exactly that the
111-
// new leaf-id set must NOT broaden to "any systemui id". Appended to the
112-
// RAW array (not the already-walked output) so the whole tree — status bar
113-
// AND volume dialog — goes through the real walk together, exactly as a
114-
// single device capture would.
107+
test('Android actionable systemui overlay (volume dialog) still survives the chrome filter (#1251)', () => {
108+
// Filter-logic unit test, NOT a live-capture-path claim (#1264 finding 2): this
109+
// exercises `collectSettleChromeRefs` in isolation over a synthetic systemui
110+
// run appended to an unrelated capture fixture. The leaf ids ARE real,
111+
// live-verified volume-dialog ids (`volume_dialog_container`,
112+
// `volume_new_ringer_active_icon_container`), but appending them to this
113+
// fixture does not reproduce how a live capture reaches the filter — that is
114+
// covered separately by the full-capture invariant test below. The point
115+
// here is narrower: the status-bar/nav-bar leaf-id set (#1251) must NOT
116+
// broaden to "any systemui id" and drop an actionable overlay it is handed.
115117
const rawWithVolumeDialog: RawSnapshotNode[] = [
116118
...ANDROID_IME_CAPTURE_RAW_NODES,
117119
{
@@ -125,22 +127,25 @@ test('Android actionable systemui overlay (volume dialog) still survives with th
125127
parentIndex: 9000,
126128
type: 'android.widget.ImageButton',
127129
bundleId: 'com.android.systemui',
128-
identifier: 'com.android.systemui:id/volume_dialog_slider',
129-
label: 'Media volume',
130+
identifier: 'com.android.systemui:id/volume_new_ringer_active_icon_container',
131+
label: 'Ringer volume',
130132
hittable: true,
131133
},
132134
];
133135
const nodes = attachRefs(walkNonRawAndroidFixture(rawWithVolumeDialog));
134136
const chromeRefs = collectSettleChromeRefs(nodes, 'com.callstack.agentdevicelab');
135137

136138
// The container is non-hittable with a generic id, but its hittable
137-
// `volume_dialog_slider` child keeps it in the walked tree (descendantHittable).
139+
// `volume_new_ringer_active_icon_container` child keeps it in the walked
140+
// tree (descendantHittable).
138141
assert.equal(
139142
chromeRefs.has(refForIdentifier(nodes, 'com.android.systemui:id/volume_dialog_container')),
140143
false,
141144
);
142145
assert.equal(
143-
chromeRefs.has(refForIdentifier(nodes, 'com.android.systemui:id/volume_dialog_slider')),
146+
chromeRefs.has(
147+
refForIdentifier(nodes, 'com.android.systemui:id/volume_new_ringer_active_icon_container'),
148+
),
144149
false,
145150
);
146151
// The status-bar leak fix stays active in the very same tree.

‎src/daemon/handlers/__tests__/session-replay-divergence.test.ts‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -471,3 +471,93 @@ test('buildReplayFailureDivergence excludes Android status-bar/IME chrome from s
471471
expect(screen.refs.some((ref) => ref.label === '+48 555 010 010')).toBe(true); // field-phone
472472
expect(screen.truncated).toBeUndefined();
473473
});
474+
475+
// #1264 invariant: an agent must never see a healthier `screen` in a
476+
// divergence report than a plain `snapshot` would show it. A separate-window
477+
// system overlay (e.g. the volume dialog) that a full `snapshot` capture would
478+
// include must also survive into divergence `screen.refs` — its actionable
479+
// (hittable) content passing the meaningful-target filter — while ordinary
480+
// status-bar/nav-bar/IME chrome in the SAME capture is still excluded. This
481+
// extends the real (walked) Android fixture from the test above with a
482+
// systemui overlay run using real, live-verified volume-dialog leaf ids
483+
// (`volume_dialog_container`, `volume_new_ringer_active_icon_container`, see
484+
// `core/__tests__/snapshot-chrome-android-statusbar.test.ts`), appended to the
485+
// RAW array so the whole tree — app content, status bar, AND the overlay —
486+
// goes through the real non-raw walk together, exactly as a single full
487+
// capture would.
488+
test('buildReplayFailureDivergence: a system-overlay window in the full capture survives into screen.refs while status/nav chrome is filtered (#1264)', async () => {
489+
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-replay-divergence-overlay-'));
490+
const sessionStore = new SessionStore(path.join(root, 'sessions'));
491+
const sessionName = 'default';
492+
sessionStore.set(
493+
sessionName,
494+
makeAndroidSession(sessionName, { appBundleId: 'com.callstack.agentdevicelab' }),
495+
);
496+
497+
const rawWithVolumeDialog = [
498+
...ANDROID_IME_CAPTURE_RAW_NODES,
499+
{
500+
index: 9000,
501+
type: 'android.widget.FrameLayout',
502+
bundleId: 'com.android.systemui',
503+
identifier: 'com.android.systemui:id/volume_dialog_container',
504+
},
505+
{
506+
index: 9001,
507+
parentIndex: 9000,
508+
type: 'android.widget.ImageButton',
509+
bundleId: 'com.android.systemui',
510+
identifier: 'com.android.systemui:id/volume_new_ringer_active_icon_container',
511+
label: 'Ringer volume',
512+
hittable: true,
513+
},
514+
];
515+
516+
mockDispatchCommand.mockResolvedValue({
517+
nodes: walkNonRawAndroidFixture(rawWithVolumeDialog),
518+
truncated: false,
519+
backend: 'android',
520+
});
521+
522+
const action = {
523+
ts: 0,
524+
command: 'press',
525+
positionals: ['label="Full name"'],
526+
flags: {},
527+
result: { selectorChain: ['label="Full name"'] },
528+
};
529+
const divergence = await buildReplayFailureDivergence({
530+
error: { code: 'COMMAND_FAILED', message: 'not hittable' },
531+
action,
532+
index: 0,
533+
sourcePath: path.join(root, 'flow.ad'),
534+
sourceLine: 1,
535+
session: sessionStore.get(sessionName),
536+
sessionName,
537+
sessionStore,
538+
logPath: path.join(root, 'daemon.log'),
539+
responseLevel: 'default',
540+
planActions: [action],
541+
planDigest: 'test-plan-digest',
542+
});
543+
544+
expect(divergence.screen.state).toBe('available');
545+
const screen = divergence.screen as Extract<typeof divergence.screen, { state: 'available' }>;
546+
547+
// The overlay's actionable (hittable) control survives the filters, exactly
548+
// as a plain `snapshot` at the same moment would show it.
549+
expect(screen.refs.some((ref) => ref.label === 'Ringer volume')).toBe(true);
550+
551+
// Ordinary status-bar/mobile/wifi systemui chrome and IME keys, present in
552+
// the SAME capture, are still filtered — the overlay fix does not broaden
553+
// the chrome filter into a no-op.
554+
expect(screen.refs.some((ref) => ref.label === '7:03')).toBe(false); // clock
555+
expect(screen.refs.some((ref) => ref.label === 'T-Mobile, signal full.')).toBe(false); // mobile_combo
556+
expect(screen.refs.some((ref) => ref.label === 'Wifi signal full.')).toBe(false); // wifi_signal
557+
expect(screen.refs.some((ref) => ref.label === 'Shift')).toBe(false); // IME key
558+
559+
// App content underneath, from the same capture, is unaffected by either
560+
// the overlay or the chrome filter.
561+
expect(screen.refs.some((ref) => ref.label === 'Checkout form')).toBe(true);
562+
expect(screen.refs.some((ref) => ref.label === 'review name')).toBe(true); // field-name
563+
});

‎src/daemon/handlers/session-replay-divergence.ts‎

Lines changed: 20 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,12 @@
11
import fs from 'node:fs';
22
import path from 'node:path';
3-
import { dispatchCommand } from '../../core/dispatch.ts';
4-
import { contextFromFlags } from '../context.ts';
53
import { markSessionSnapshotRefsIssued, setSessionSnapshot } from '../session-snapshot.ts';
64
import { isSparseSnapshotQualityVerdict } from '../../snapshot/snapshot-quality.ts';
75
import { displayLabel, formatRole } from '../../snapshot/snapshot-lines.ts';
86
import { redactDiagnosticData } from '../../kernel/redaction.ts';
97
import type { DaemonError, ResponseLevel } from '../../kernel/contracts.ts';
10-
import type { RawSnapshotNode, SnapshotBackend, SnapshotNode } from '../../kernel/snapshot.ts';
11-
import { buildSnapshotState } from './snapshot-capture.ts';
8+
import type { SnapshotNode } from '../../kernel/snapshot.ts';
9+
import { buildSnapshotState, captureSnapshotData } from './snapshot-capture.ts';
1210
import {
1311
buildSelectorChainForNode,
1412
resolveSelectorChain,
@@ -203,6 +201,17 @@ export function toReplayRepairHintCapture(
203201
* sequence (setSessionSnapshot -> markSessionSnapshotRefsIssued -> store).
204202
* Sparse captures do not write back (selector-capture reliability contract),
205203
* so a sparse verdict degrades the whole observation.
204+
*
205+
* ADR 0012 decision 4 amendment (#1264): this reuses `captureSnapshotData` —
206+
* the SAME function the `snapshot` command itself builds its capture with
207+
* (Android: snapshot-helper full-window route with the graceful app-scoped
208+
* fallback; iOS: the bounded system-modal probe path; macOS/Linux: their
209+
* surface-scoped branches) — instead of a parallel hand-rolled dispatch. An
210+
* agent must never see a healthier `screen` in a divergence report than a
211+
* plain `snapshot` would show it, so `screen` cannot be built from a narrower
212+
* capture than `snapshot`'s. The chrome filter (`collectSettleChromeRefs`)
213+
* and the meaningful-target filter below stay layered ON TOP of this full
214+
* capture as FILTERS, never as a narrower scoping.
206215
*/
207216
export async function captureDivergenceObservation(params: {
208217
session: SessionState;
@@ -213,24 +222,15 @@ export async function captureDivergenceObservation(params: {
213222
}): Promise<DivergenceObservation> {
214223
const { session, sessionName, sessionStore, logPath, action } = params;
215224
const snapshotInteractiveOnly = divergenceCaptureInteractiveOnly(action);
225+
const flags = { ...(action.flags ?? {}), snapshotInteractiveOnly };
216226
try {
217-
const data = (await dispatchCommand(session.device, 'snapshot', [], undefined, {
218-
...contextFromFlags(
219-
logPath,
220-
{ ...(action.flags ?? {}), snapshotInteractiveOnly },
221-
session.appBundleId,
222-
session.trace?.outPath,
223-
),
224-
})) as {
225-
nodes?: RawSnapshotNode[];
226-
truncated?: boolean;
227-
backend?: SnapshotBackend;
228-
quality?: unknown;
229-
};
230-
const snapshot = buildSnapshotState(data, {
231-
...(action.flags ?? {}),
232-
snapshotInteractiveOnly,
227+
const data = await captureSnapshotData({
228+
device: session.device,
229+
session,
230+
flags,
231+
logPath,
233232
});
233+
const snapshot = buildSnapshotState(data, flags);
234234
if (isSparseSnapshotQualityVerdict(snapshot.snapshotQuality)) {
235235
return {
236236
state: 'unavailable',

0 commit comments

Comments
 (0)