Skip to content

Commit 191c351

Browse files
committed
fix(daemon): require discriminating overlap for a post-gesture baseline match
PR review on #1563 (P1): interactionSurfaceMatchesBaseline returned true whenever ANY shared entry was frozen, including the application/window viewport root, whose rect is invariant under any gesture. In the exact scope-drift case this PR supports, a broad pre-gesture baseline and a narrow post-gesture selector capture can share only that root after a real, successful scroll — the boolean predicate called that a baseline match and extended the interaction to the 3.5s stale-read cap on zero real evidence. Fix: replace the boolean with classifyBaselineSurfaceEvidence, a subset-tolerant classifier reusing this module's existing InteractionSurfaceChange vocabulary ('changed' | 'unchanged' | 'ambiguous') instead of a bespoke boolean or an Application-only special case. An entry only counts as evidence when it is `discriminating` — excludes the viewport root (minimal local equivalent of snapshot-occlusion.ts's isViewportRoot) and keyboard chrome (minimal local equivalent of snapshot-chrome.ts's keyboard-container check), both computed once at signature-build time since the flat signature-entry representation has no ref/parentIndex to reuse those modules' full ancestor-walk classifiers directly. Zero discriminating overlap is now 'ambiguous' (insufficient evidence) rather than a match, and decidePostGestureStabilityVerdict falls through 'ambiguous' to 'trust' — the safe default, same as 'changed'. Tests: the reviewer's exact shape (signatures sharing only the Application root, with the real content swapped) at three layers — classifyBaselineSurfaceEvidence directly, decidePostGestureStabilityVerdict, and the full capturePostGestureStabilizedResult async loop (proving no cap-tax: settles in 2 capture attempts, not 3.5s). Also: root+one real element both frozen still matches (guards against over-excluding), and keyboard chrome excluded from discriminating overlap. All prior tests kept green unchanged. Counterfactual: reverted to the old boolean predicate and reran — 5 tests went red, including the async regression test, which didn't just fail an assertion but timed out after 5s because the boolean predicate extended the interaction to the 3.5s distrust cap the test's 1s timer advance never covered — exactly the "extends to cap" failure mode the review predicted. Restored, 37/37 green. _Generated by Claude Code_
1 parent a4ad668 commit 191c351

6 files changed

Lines changed: 353 additions & 58 deletions

File tree

‎src/daemon/__tests__/interaction-outcome-policy.test.ts‎

Lines changed: 118 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@ import { test } from 'vitest';
33
import type { SnapshotState } from '@agent-device/kernel/snapshot';
44
import {
55
buildInteractionSurfaceSignature,
6+
classifyBaselineSurfaceEvidence,
67
classifyInteractionSurfaceChange,
7-
interactionSurfaceMatchesBaseline,
88
markPendingInteractionOutcome,
99
stripInternalInteractionFlags,
1010
} from '../interaction-outcome-policy.ts';
@@ -40,55 +40,65 @@ test('classifyInteractionSurfaceChange detects material layout movement', () =>
4040
});
4141

4242
// ---------------------------------------------------------------------------
43-
// interactionSurfaceMatchesBaseline (#1542 defect 2): subset-tolerant baseline
44-
// comparison. Live evidence on checkout-form.ad showed the pre-gesture
45-
// baseline (captured by an earlier `wait`, a broad query) and the post-gesture
46-
// quiet signature (captured by the click's interactive-only selector
47-
// resolution) never line up as whole arrays even when the target element
48-
// never moved — this is the comparison that has to see through that scope
49-
// drift.
43+
// classifyBaselineSurfaceEvidence (#1542 defect 2, #1563 review): subset-
44+
// tolerant, three-valued baseline comparison. Live evidence on
45+
// checkout-form.ad showed the pre-gesture baseline (captured by an earlier
46+
// `wait`, a broad query) and the post-gesture quiet signature (captured by
47+
// the click's interactive-only selector resolution) never line up as whole
48+
// arrays even when the target element never moved — the first version of
49+
// this check has to see through that scope drift.
50+
//
51+
// The #1563 review then caught a SECOND failure mode in that first version
52+
// (a plain "any shared entry frozen" boolean): the viewport root
53+
// (Application/Window) is always present and its rect is invariant under any
54+
// gesture, so a broad baseline and a narrow post-gesture capture can share
55+
// ONLY the root even after a real, successful scroll — and the boolean
56+
// predicate called that a match. `classifyBaselineSurfaceEvidence` requires
57+
// at least one DISCRIMINATING shared entry (excluding the viewport root and
58+
// keyboard chrome) before calling it `'unchanged'`; a root-only (or
59+
// no-discriminating-evidence) overlap is `'ambiguous'` instead.
5060
// ---------------------------------------------------------------------------
5161

52-
test('interactionSurfaceMatchesBaseline matches identical signatures', () => {
62+
test('classifyBaselineSurfaceEvidence reports unchanged for identical signatures', () => {
5363
const baseline = buildInteractionSurfaceSignature(makeSnapshot('Inbox').nodes);
5464
const current = buildInteractionSurfaceSignature(makeSnapshot('Inbox').nodes);
5565

56-
assert.equal(interactionSurfaceMatchesBaseline(baseline, current), true);
66+
assert.equal(classifyBaselineSurfaceEvidence(baseline, current), 'unchanged');
5767
});
5868

59-
test('interactionSurfaceMatchesBaseline treats an empty side as no evidence', () => {
69+
test('classifyBaselineSurfaceEvidence is ambiguous when either side is empty', () => {
6070
const baseline = buildInteractionSurfaceSignature(makeSnapshot('Inbox').nodes);
6171

62-
assert.equal(interactionSurfaceMatchesBaseline([], baseline), false);
63-
assert.equal(interactionSurfaceMatchesBaseline(baseline, []), false);
64-
assert.equal(interactionSurfaceMatchesBaseline([], []), false);
72+
assert.equal(classifyBaselineSurfaceEvidence([], baseline), 'ambiguous');
73+
assert.equal(classifyBaselineSurfaceEvidence(baseline, []), 'ambiguous');
74+
assert.equal(classifyBaselineSurfaceEvidence([], []), 'ambiguous');
6575
});
6676

67-
test('interactionSurfaceMatchesBaseline matches through a broader baseline scope when the shared element is frozen', () => {
77+
test('classifyBaselineSurfaceEvidence reports unchanged through a broader baseline scope when the shared discriminating element is frozen', () => {
6878
// The exact live shape: the baseline came from a broader capture (extra
6979
// "Loading" text node the interactive-only capture never sees), but the
7080
// shared "primary-action" button never moved.
7181
const baseline = buildInteractionSurfaceSignature(makeSnapshotWithExtraText('Inbox', 500).nodes);
7282
const current = buildInteractionSurfaceSignature(makeSnapshot('Inbox', 500).nodes);
7383

74-
assert.equal(interactionSurfaceMatchesBaseline(baseline, current), true);
84+
assert.equal(classifyBaselineSurfaceEvidence(baseline, current), 'unchanged');
7585
});
7686

77-
test('interactionSurfaceMatchesBaseline matches through a broader current scope when the shared element is frozen', () => {
87+
test('classifyBaselineSurfaceEvidence reports unchanged through a broader current scope when the shared discriminating element is frozen', () => {
7888
const baseline = buildInteractionSurfaceSignature(makeSnapshot('Inbox', 500).nodes);
7989
const current = buildInteractionSurfaceSignature(makeSnapshotWithExtraText('Inbox', 500).nodes);
8090

81-
assert.equal(interactionSurfaceMatchesBaseline(baseline, current), true);
91+
assert.equal(classifyBaselineSurfaceEvidence(baseline, current), 'unchanged');
8292
});
8393

84-
test('interactionSurfaceMatchesBaseline detects real movement even through a scope difference', () => {
94+
test('classifyBaselineSurfaceEvidence detects real movement even through a scope difference', () => {
8595
const baseline = buildInteractionSurfaceSignature(makeSnapshotWithExtraText('Inbox', 500).nodes);
8696
const current = buildInteractionSurfaceSignature(makeSnapshot('Inbox', 120).nodes);
8797

88-
assert.equal(interactionSurfaceMatchesBaseline(baseline, current), false);
98+
assert.equal(classifyBaselineSurfaceEvidence(baseline, current), 'changed');
8999
});
90100

91-
test('interactionSurfaceMatchesBaseline is ambiguous (no match) when the signatures share no key', () => {
101+
test('classifyBaselineSurfaceEvidence is ambiguous when the signatures share no key at all', () => {
92102
const baseline = buildInteractionSurfaceSignature([
93103
{
94104
ref: 'e1',
@@ -110,16 +120,100 @@ test('interactionSurfaceMatchesBaseline is ambiguous (no match) when the signatu
110120
},
111121
]);
112122

113-
assert.equal(interactionSurfaceMatchesBaseline(baseline, current), false);
123+
assert.equal(classifyBaselineSurfaceEvidence(baseline, current), 'ambiguous');
114124
});
115125

116-
test('interactionSurfaceMatchesBaseline tolerates tiny rect drift on the shared element', () => {
126+
test('classifyBaselineSurfaceEvidence tolerates tiny rect drift on the shared discriminating element', () => {
117127
const baseline = buildInteractionSurfaceSignature(makeSnapshotWithExtraText('Inbox', 500).nodes);
118128
const current = buildInteractionSurfaceSignature(makeSnapshot('Inbox', 500.4).nodes);
119129

120-
assert.equal(interactionSurfaceMatchesBaseline(baseline, current), true);
130+
assert.equal(classifyBaselineSurfaceEvidence(baseline, current), 'unchanged');
121131
});
122132

133+
// --- #1563 review regression: root-only overlap must NOT read as evidence ---
134+
135+
test('classifyBaselineSurfaceEvidence is ambiguous (NOT unchanged) when a real scroll leaves only the application root shared — the reviewer-caught false-distrust shape', () => {
136+
// baseline = {Application, Pickup@y=500}; current = {Application,
137+
// OtherButton@...} — a genuine, successful scroll replaced every real
138+
// element in view, so the only entry the two signatures still share is the
139+
// always-present, always-identical viewport root. A boolean "any shared
140+
// entry frozen" predicate calls this a baseline match (the root always
141+
// "matches") and would extend the interaction to the 3.5s stale-read
142+
// deadline on zero real evidence — exactly the bug this test pins.
143+
const baseline = buildInteractionSurfaceSignature([
144+
applicationRootNode(),
145+
{
146+
ref: 'e2',
147+
index: 1,
148+
parentIndex: 0,
149+
type: 'Button',
150+
identifier: 'shipping-pickup',
151+
label: 'Pickup',
152+
rect: { x: 20, y: 500, width: 200, height: 44 },
153+
},
154+
]);
155+
const current = buildInteractionSurfaceSignature([
156+
applicationRootNode(),
157+
{
158+
ref: 'e2',
159+
index: 1,
160+
parentIndex: 0,
161+
type: 'Button',
162+
identifier: 'shipping-delivery',
163+
label: 'Delivery',
164+
rect: { x: 20, y: 120, width: 200, height: 44 },
165+
},
166+
]);
167+
168+
assert.equal(classifyBaselineSurfaceEvidence(baseline, current), 'ambiguous');
169+
});
170+
171+
test('classifyBaselineSurfaceEvidence is ambiguous when the current capture is the application root alone', () => {
172+
const baseline = buildInteractionSurfaceSignature(makeSnapshot('Inbox', 500).nodes);
173+
const current = buildInteractionSurfaceSignature([applicationRootNode()]);
174+
175+
assert.equal(classifyBaselineSurfaceEvidence(baseline, current), 'ambiguous');
176+
});
177+
178+
test('classifyBaselineSurfaceEvidence excludes keyboard chrome from discriminating overlap', () => {
179+
const keyboardNode = {
180+
ref: 'e3',
181+
index: 2,
182+
parentIndex: 0,
183+
type: 'Keyboard',
184+
rect: { x: 0, y: 500, width: 390, height: 300 },
185+
};
186+
const baseline = buildInteractionSurfaceSignature([applicationRootNode(), keyboardNode]);
187+
// The keyboard's own container rect never changes; only the app content
188+
// does. A capture sharing just the root and the keyboard container (no
189+
// real content) must not read as a baseline match.
190+
const current = buildInteractionSurfaceSignature([applicationRootNode(), keyboardNode]);
191+
192+
assert.equal(classifyBaselineSurfaceEvidence(baseline, current), 'ambiguous');
193+
});
194+
195+
test('classifyBaselineSurfaceEvidence still reports unchanged when the root AND a real discriminating element both match (guards against over-excluding)', () => {
196+
// Root-sharing alone is not disqualifying — it just cannot be the ONLY
197+
// evidence. Once a real, frozen discriminating element is also shared
198+
// (the ordinary "genuinely stuck" case), the verdict must still be
199+
// 'unchanged', not swing to 'ambiguous' just because the root is present.
200+
const snapshotNodes = makeSnapshot('Inbox', 500).nodes; // [Application, primary-action Button]
201+
const baseline = buildInteractionSurfaceSignature(snapshotNodes);
202+
const current = buildInteractionSurfaceSignature(snapshotNodes);
203+
204+
assert.equal(classifyBaselineSurfaceEvidence(baseline, current), 'unchanged');
205+
});
206+
207+
function applicationRootNode() {
208+
return {
209+
ref: 'e1',
210+
index: 0,
211+
type: 'Application',
212+
label: 'App',
213+
rect: { x: 0, y: 0, width: 390, height: 844 },
214+
};
215+
}
216+
123217
test('markPendingInteractionOutcome stores retry state only for explicit retry flags', () => {
124218
const session = makeSession();
125219
markPendingInteractionOutcome({

‎src/daemon/__tests__/post-gesture-stabilization.test.ts‎

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -209,6 +209,56 @@ test('decidePostGestureStabilityVerdict accepts a baseline-matching signature on
209209
);
210210
});
211211

212+
// --- #1563 review regression: a root-only shared overlap must trust immediately, not tax the cap ---
213+
214+
test('decidePostGestureStabilityVerdict trusts immediately when a real scroll leaves only the application root shared (no cap tax)', () => {
215+
const baseline = buildInteractionSurfaceSignature([
216+
applicationRootNode(),
217+
{
218+
ref: 'e2',
219+
index: 1,
220+
parentIndex: 0,
221+
type: 'Button',
222+
identifier: 'shipping-pickup',
223+
label: 'Pickup',
224+
rect: { x: 20, y: 500, width: 200, height: 44 },
225+
},
226+
]);
227+
const quiet = buildInteractionSurfaceSignature([
228+
applicationRootNode(),
229+
{
230+
ref: 'e2',
231+
index: 1,
232+
parentIndex: 0,
233+
type: 'Button',
234+
identifier: 'shipping-delivery',
235+
label: 'Delivery',
236+
rect: { x: 20, y: 120, width: 200, height: 44 },
237+
},
238+
]);
239+
240+
assert.equal(
241+
decidePostGestureStabilityVerdict({
242+
needsBaselineDistrust: true,
243+
baselineSignature: baseline,
244+
quietSignature: quiet,
245+
elapsedMs: 0, // first quiet match, well before any cap
246+
distrustCapMs: 3_500,
247+
}),
248+
'trust',
249+
);
250+
});
251+
252+
function applicationRootNode() {
253+
return {
254+
ref: 'e-root',
255+
index: 0,
256+
type: 'Application',
257+
label: 'App',
258+
rect: { x: 0, y: 0, width: 390, height: 844 },
259+
};
260+
}
261+
212262
// ---------------------------------------------------------------------------
213263
// capturePostGestureStabilizedResult: the async loop wired to the pure
214264
// decision above. Fake timers keep these instant despite the real 200ms poll
@@ -400,6 +450,44 @@ test('capturePostGestureStabilizedResult catches a frozen target even when the b
400450
assert.equal(settled, 0);
401451
});
402452

453+
test('capturePostGestureStabilizedResult trusts immediately (no cap tax) when a real scroll leaves only the application root shared — #1563 review regression', async () => {
454+
// The reviewer's exact false-distrust shape end to end: the baseline is
455+
// Application + Pickup; every post-gesture read is Application + a
456+
// DIFFERENT button (Delivery) — a genuine, successful scroll that swapped
457+
// every real element. A boolean "any shared entry frozen" predicate would
458+
// call the shared, always-identical Application root a baseline match and
459+
// extend this to the 3.5s stale-read cap on zero real evidence.
460+
vi.useFakeTimers();
461+
const session = makeSession('ios');
462+
session.snapshot = pickupSnapshot(500);
463+
markPostGestureStabilization(session, 'scroll');
464+
465+
const capture = vi.fn(async () => deliverySnapshot(120)); // consistent from the first read: quiet immediately
466+
467+
const resultPromise = withDiagnosticsScope({}, async () => {
468+
const result = await capturePostGestureStabilizedResult({
469+
session,
470+
capture,
471+
readSnapshot: (snapshot) => snapshot,
472+
});
473+
return {
474+
result,
475+
staleAccepts: countDiagnosticEventsByPhase(['post_gesture_snapshot_stale_accept']),
476+
settled: countDiagnosticEventsByPhase(['post_gesture_snapshot_stabilized']),
477+
};
478+
});
479+
480+
await vi.advanceTimersByTimeAsync(1_000);
481+
const { staleAccepts, settled } = await resultPromise;
482+
483+
assert.equal(settled, 1);
484+
assert.equal(staleAccepts, 0);
485+
// Trusted at the first quiet match (initial capture + one poll = 2
486+
// attempts): root-only overlap is ambiguous, not a baseline match, so it
487+
// never pays the 3.5s distrust cap.
488+
assert.equal(capture.mock.calls.length, 2);
489+
});
490+
403491
function pickupSnapshot(y = 500) {
404492
return makeSnapshotState([
405493
{ index: 0, type: 'Application', label: 'App', rect: { x: 0, y: 0, width: 390, height: 844 } },
@@ -414,6 +502,23 @@ function pickupSnapshot(y = 500) {
414502
]);
415503
}
416504

505+
// Same Application root as pickupSnapshot, but a DIFFERENT real element —
506+
// models a genuine, successful scroll that swapped every real element in
507+
// view, so the only entry shared with a pickupSnapshot baseline is the root.
508+
function deliverySnapshot(y = 500) {
509+
return makeSnapshotState([
510+
{ index: 0, type: 'Application', label: 'App', rect: { x: 0, y: 0, width: 390, height: 844 } },
511+
{
512+
index: 1,
513+
parentIndex: 0,
514+
type: 'Button',
515+
identifier: 'shipping-delivery',
516+
label: 'Delivery',
517+
rect: { x: 20, y, width: 200, height: 44 },
518+
},
519+
]);
520+
}
521+
417522
// Broader-scope variant: adds a non-interactive text node an interactive-only
418523
// capture would never return, modeling the real pre-gesture-baseline vs
419524
// post-gesture-selector-capture scope mismatch.

‎src/daemon/handlers/__tests__/snapshot-handler.test.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1137,6 +1137,7 @@ test('captureSnapshot lazily retries pending no-change touch before returning fr
11371137
y: 120,
11381138
width: 160,
11391139
height: 48,
1140+
discriminating: true,
11401141
},
11411142
],
11421143
};
@@ -1279,6 +1280,7 @@ test('captureSnapshot retries pending tap outcome before post-gesture stabilizat
12791280
y: 1301,
12801281
width: 476,
12811282
height: 110,
1283+
discriminating: true,
12821284
},
12831285
],
12841286
};

0 commit comments

Comments
 (0)