Skip to content

Commit 7d22176

Browse files
strzhaoclaude
andcommitted
fix(interaction): omit changedFromBefore when no verify baseline was captured
When the pre-action baseline capture failed, --verify evidence still reported changedFromBefore: false, which reads as an unchanged screen. A missing baseline and a matched baseline are different facts: the field now stays absent when no comparison was made, and the response schema stops requiring it. Present-baseline behavior is unchanged. Co-Authored-By: Claude Code <[email protected]>
1 parent b24ecf2 commit 7d22176

5 files changed

Lines changed: 62 additions & 9 deletions

File tree

‎packages/command-runtime/src/commands/interaction/runtime/interactions.test.ts‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,51 @@ test('runtime click with verify captures a baseline for point targets and report
5757
assert.equal(captureCount, 2);
5858
});
5959

60+
test('runtime click with verify reports unchanged evidence for a matching point baseline', async () => {
61+
const device = createInteractionDevice(selectorSnapshot(), {
62+
captureSnapshot: async () => ({ snapshot: selectorSnapshot() }),
63+
tap: async () => ({ ok: true }),
64+
});
65+
66+
const result = await device.interactions.click(
67+
{ kind: 'point', x: 10, y: 20 },
68+
{ session: 'default', verify: true },
69+
);
70+
71+
assert.equal(result.kind, 'point');
72+
assert.ok(result.evidence);
73+
assert.equal(result.evidence?.changedFromBefore, false);
74+
});
75+
76+
test('runtime click with verify omits changedFromBefore when the baseline capture fails', async () => {
77+
let captureCount = 0;
78+
const device = createInteractionDevice(selectorSnapshot(), {
79+
captureSnapshot: async () => {
80+
captureCount += 1;
81+
// Capture 1 = the pre-action baseline request, which fails on a
82+
// tree-blind surface. Capture 2 = the post-action verify capture.
83+
if (captureCount === 1) throw new Error('interactive capture unavailable');
84+
return { snapshot: selectorSnapshot() };
85+
},
86+
tap: async () => ({ ok: true }),
87+
});
88+
89+
const result = await device.interactions.click(
90+
{ kind: 'point', x: 10, y: 20 },
91+
{ session: 'default', verify: true },
92+
);
93+
94+
assert.equal(result.kind, 'point');
95+
assert.ok(result.evidence);
96+
// No baseline, no claim: the evidence must not read as an unchanged screen,
97+
// because "nothing changed" and "nothing was compared" are different facts
98+
// and the second must not launder the first past a caller watching for a
99+
// coordinate tap that landed on nothing.
100+
assert.equal('changedFromBefore' in (result.evidence ?? {}), false);
101+
assert.equal(typeof result.evidence?.digest, 'string');
102+
assert.equal(captureCount, 2);
103+
});
104+
60105
test('runtime click with verify skips the native ref fast path so evidence can be captured', async () => {
61106
const calls: string[] = [];
62107
let captureCount = 0;

‎packages/command-runtime/src/commands/interaction/runtime/post-action-observation.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -95,8 +95,8 @@ async function observeAfterInteraction(
9595
* Post-action side of `--verify` (#1047): one interactive-only capture through
9696
* the same capture helper the resolution path already uses, digested and then
9797
* discarded. The node tree itself is never attached to the result, only the
98-
* cheap summary. A missing baseline still yields `changedFromBefore: false` —
99-
* no baseline, no claim.
98+
* cheap summary. A missing baseline yields no `changedFromBefore` claim —
99+
* no baseline, no comparison.
100100
*/
101101
async function captureVerifyEvidence(
102102
runtime: AgentDeviceRuntime,

‎packages/command-runtime/src/commands/interaction/runtime/post-action-surface.ts‎

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,8 @@ export function resolvePostActionSurfaceChange(
4242
* `--verify` evidence for one post-action capture (#1047). Same surface: the digest comparison
4343
* against the pre-action baseline, as before. Cross-surface: no digest comparison is made — the
4444
* surface itself was replaced, which is a change by construction, and `surfaceChange` discloses
45-
* that this is what `changedFromBefore` reports.
45+
* that this is what `changedFromBefore` reports. Missing baseline: no comparison is claimed —
46+
* `changedFromBefore` is omitted instead of reading as an unchanged screen.
4647
*/
4748
export function summarizePostActionEvidence(
4849
after: SurfaceScopedNodes,
@@ -51,9 +52,11 @@ export function summarizePostActionEvidence(
5152
const summary = summarizeAxEvidence(after.nodes);
5253
const surfaceChange = resolvePostActionSurfaceChange(baseline, after);
5354
if (surfaceChange) return { ...summary, changedFromBefore: true, surfaceChange };
54-
const changedFromBefore =
55-
baseline !== undefined && summary.digest !== summarizeAxEvidence(baseline.nodes).digest;
56-
return { ...summary, changedFromBefore };
55+
if (baseline === undefined) return summary;
56+
return {
57+
...summary,
58+
changedFromBefore: summary.digest !== summarizeAxEvidence(baseline.nodes).digest,
59+
};
5760
}
5861

5962
/**

‎packages/command-surface/src/interaction/index.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -75,11 +75,11 @@ const interactionEvidenceSchema: JsonSchema = objectSchema(
7575
interactiveNodeCount: numberSchema('Subset of nodeCount the platform reports as hittable.'),
7676
digest: stringSchema('Order-independent digest of the post-action node multiset.'),
7777
changedFromBefore: booleanSchema(
78-
'Whether the post-action digest differs from the pre-action capture digest. false is evidence, not failure. With surfaceChange present, no digest comparison is made: it reports that surface transition.',
78+
'Whether the post-action digest differs from the pre-action capture digest. false is evidence, not failure. Absent when no pre-action baseline was captured, so no comparison is claimed. With surfaceChange present, no digest comparison is made: it reports that surface transition.',
7979
),
8080
surfaceChange: postActionSurfaceChangeSchema,
8181
},
82-
['nodeCount', 'interactiveNodeCount', 'digest', 'changedFromBefore'],
82+
['nodeCount', 'interactiveNodeCount', 'digest'],
8383
);
8484

8585
const responseCostSchema: JsonSchema = objectSchema(

‎packages/contracts/src/interaction.ts‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -232,6 +232,11 @@ export type PostActionSurfaceChange = {
232232
* `changedFromBefore: false` is evidence, not failure — the command still
233233
* succeeded.
234234
*
235+
* The field is ABSENT when no pre-action baseline was captured, so "no comparison
236+
* was made" stays distinguishable from "nothing changed": a missing baseline must
237+
* not read as an unchanged screen, because a coordinate tap that landed on
238+
* nothing would otherwise report the same verdict as one that changed nothing.
239+
*
235240
* When `surfaceChange` is present the two captures describe different surfaces, so the digest
236241
* comparison is not made at all: `changedFromBefore` then reports that transition, which replaced
237242
* the whole observed surface.
@@ -241,7 +246,7 @@ export type InteractionEvidence = {
241246
nodeCount: number;
242247
interactiveNodeCount: number;
243248
digest: string;
244-
changedFromBefore: boolean;
249+
changedFromBefore?: boolean;
245250
surfaceChange?: PostActionSurfaceChange;
246251
};
247252

0 commit comments

Comments
 (0)