Skip to content

Commit e050191

Browse files
authored
fix(interaction): disclose label-fallback when the native-ref preflight recovers by label (#4352)
* fix(interaction): disclose label-fallback when the native-ref preflight recovers by label The native-ref fast path always disclosed exact, even when its preflight recovered a stale @ref through its trailing label. The preflight now returns the disclosure built by buildRefResolution, and dispatch keeps exact only when the preflight observed no tree. The ADR 0011 registry anchor, coverage manifest and resolution-disclosure doc follow. Refs #4332 * docs(interaction): list native-ref label-fallback coverage and correct the exact default * docs(interaction): list native-ref label-fallback in the contract coverage * docs(contracts): note that native-ref label-fallback still dispatches the original ref
1 parent 6ff24d9 commit e050191

7 files changed

Lines changed: 47 additions & 19 deletions

File tree

‎packages/command-runtime/docs/resolution-disclosure.md‎

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -33,11 +33,12 @@ The runtime-ref path can recover a stale or unusable `@ref` through its recorded
3333
This recovery returns `{ source: "ref", phase: "pre-action", kind: "label-fallback" }`.
3434
Runtime-ref label recovery discloses its fallback provenance.
3535

36-
The native-ref fast path always discloses `exact`.
37-
It dispatches the ref handle and forwards the recorded `fallbackLabel` to the backend.
38-
The daemon preflight can itself recover a node by label, but the response still discloses `exact`.
39-
Backend label recovery is also not reflected in the response.
40-
Thus, this disclosure can misstate ref provenance.
36+
The native-ref fast path discloses what its preflight observed.
37+
The preflight resolves the ref against the stored session snapshot through `tryResolveRefNode`.
38+
It discloses `exact` when the ref resolves, and `label-fallback` when it recovers the node by its trailing label.
39+
When the preflight is a no-op, the path dispatches the ref handle and discloses `exact`.
40+
The path forwards the recorded `fallbackLabel` to the backend.
41+
The daemon's touch backend dispatches only the ref, so no backend label recovery occurs on the daemon path.
4142

4243
## Direct-iOS selector
4344

@@ -58,7 +59,7 @@ ADR 0011's matrix has a `resolutionDisclosure` guarantee for every interaction p
5859
- `runtime-selector` enforces the complete pre-action diagnostic shape.
5960
- `runtime-ref` enforces `exact` or `label-fallback` for trailing-label recovery.
6061
- `target-drag` enforces a disclosure for each endpoint: the source and the destination each carry their own `resolution`.
61-
- `native-ref` enforces `exact`.
62+
- `native-ref` enforces `exact` or `label-fallback` for preflight label recovery.
6263
- `maestro-direct-selector` enforces only `{ source: "direct-ios", kind: "not-observed" }`.
6364
- `coordinate` is inapplicable because no element resolves.
6465
- `find-focus-dispatch` is inapplicable because it issues no identity (ADR 0014).
@@ -85,13 +86,15 @@ Layer-3 coverage must claim every enforced/delegated cell:
8586
- Distinct-subtree rejection with candidates
8687
- Exact-ref provenance for runtime and native refs
8788
- Runtime-ref `label-fallback` recovery
89+
- Native-ref preflight `label-fallback` recovery
8890
- Direct-iOS `not-observed` without a snapshot
8991

9092
Matrix and provider contracts cover every `resolutionDisclosure` cell:
9193

9294
- Runtime ambiguity/tiebreak and the five-alternative limit
9395
- Runtime/native exact-ref provenance
9496
- Runtime-ref `label-fallback` disclosure
97+
- Native-ref preflight `label-fallback` disclosure
9598
- Target-drag disclosure for the source and destination endpoints
9699
- Direct-iOS `not-observed`
97100
- Coordinate inapplicability

‎packages/command-runtime/src/commands/interaction/runtime/native-ref-interaction.ts‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ import { SELECTOR_PIPELINE_POLICIES } from '@agent-device/selectors/selector-pip
66
import { surfaceScopedNodes } from '@agent-device/capture-kit/snapshot-baseline';
77
import type {
88
InteractionTarget,
9+
ResolutionDisclosure,
910
ResolvedInteractionTarget,
1011
SurfaceScopedNodes,
1112
} from '@agent-device/contracts/interaction';
@@ -59,6 +60,7 @@ export async function preflightNativeRefInteraction(
5960
hint?: string;
6061
node?: SnapshotNode;
6162
preAction?: SurfaceScopedNodes;
63+
resolution?: ResolutionDisclosure;
6264
}> {
6365
const session = await runtime.sessions.get(options.session ?? 'default');
6466
const storedSnapshot = session?.snapshot;
@@ -95,13 +97,15 @@ export async function preflightNativeRefInteraction(
9597
// evidence source for the fast path, at zero extra capture cost.
9698
node: visibleNode,
9799
preAction: surfaceScopedNodes(storedSnapshot),
100+
resolution: resolved.resolution,
98101
};
99102
}
100103

101104
/**
102105
* ADR 0011 native-ref dispatch, shared by click/fill/hover @ref: run the
103106
* preflight guards against the stored node, hand the ref to the backend as
104-
* its own element handle, and return the exact-ref result envelope. Callers
107+
* its own element handle, and return the ref result envelope. It discloses the
108+
* preflight's resolution, or `exact` when the preflight produced none. Callers
105109
* decide WHEN the path applies (backend capability, no non-default options,
106110
* no replay guard, no settle baseline); this owns only the dispatch itself so
107111
* the three commands cannot drift on preflight or disclosure.
@@ -128,8 +132,8 @@ export async function dispatchNativeRefInteraction(
128132
return {
129133
kind: 'ref',
130134
target: { kind: 'ref', ref: target.ref },
131-
resolution: EXACT_REF_RESOLUTION,
132135
...preflight,
136+
resolution: preflight.resolution ?? EXACT_REF_RESOLUTION,
133137
...(formattedBackendResult ? { backendResult: formattedBackendResult } : {}),
134138
};
135139
}

‎packages/command-runtime/src/commands/interaction/runtime/resolution-disclosure.ts‎

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -25,10 +25,7 @@ export type ResolvedRefNode = {
2525
const RESOLUTION_DIAGNOSTIC_STRING_BYTE_CAP = 256;
2626
const MAX_RESOLUTION_ALTERNATIVES = 5;
2727

28-
/**
29-
* A successful `@ref` lookup names exactly one node; label recovery discloses label-fallback instead.
30-
* ADR 0011 registry anchor: interaction-guarantees.ts cites it as a `via` symbol.
31-
*/
28+
/** A successful `@ref` lookup names exactly one node; label recovery discloses label-fallback instead. */
3229
export const EXACT_REF_RESOLUTION: ResolutionDisclosure = {
3330
source: 'ref',
3431
phase: 'pre-action',
@@ -41,7 +38,7 @@ const LABEL_FALLBACK_REF_RESOLUTION: ResolutionDisclosure = {
4138
kind: 'label-fallback',
4239
};
4340

44-
/** Shared construction site for every runtime-ref resolution disclosure. */
41+
/** Shared construction site for every runtime-ref and native-ref preflight resolution disclosure. */
4542
export function buildRefResolution(
4643
ref: string,
4744
node: SnapshotAddressableNode,

‎packages/contracts/src/interaction-guarantees.ts‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -467,10 +467,11 @@ export const INTERACTION_DISPATCH_PATHS: Record<InteractionPathId, InteractionPa
467467
kind: 'runtime',
468468
via: 'packages/selectors/src/internal/resolve.ts#STALE_REF_HINT',
469469
},
470-
// An @ref names exactly one node by construction (same cell as runtime-ref).
470+
// The preflight's ref lookup goes through the same builder as runtime-ref, so it discloses
471+
// label-fallback when it recovers the node by label. A no-op preflight discloses exact.
471472
resolutionDisclosure: {
472473
kind: 'runtime',
473-
via: 'packages/command-runtime/src/commands/interaction/runtime/resolution-disclosure.ts#EXACT_REF_RESOLUTION',
474+
via: 'packages/command-runtime/src/commands/interaction/runtime/resolution-disclosure.ts#buildRefResolution',
474475
},
475476
outcomeObservation: {
476477
kind: 'waived',

‎packages/contracts/src/interaction.ts‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,9 @@ export type ResolutionDiagnosticEntry = {
6363
* its target, including each endpoint of a target-authored drag. Never ref-issuing.
6464
* `direct-ios`/`not-observed` = the XCTest fast path has no daemon tree to
6565
* report from; `ref`/`label-fallback` = a stale `@ref` recovered via
66-
* first-match label lookup, never exact ref provenance; `alternatives` holds
66+
* first-match label lookup, never exact ref provenance. On the native-ref
67+
* path the daemon preflight finds that label match, but the backend still
68+
* dispatches the original ref; `alternatives` holds
6769
* at most 5 losing candidates, winner excluded.
6870
*/
6971
export type ResolutionDisclosure =

‎test/integration/interaction-contract/native-ref.contract.test.ts‎

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ import assert from 'node:assert/strict';
33
import { test } from 'vitest';
44
import type { InteractionGuarantee } from '@agent-device/contracts/interaction-guarantees';
55
import type { SnapshotState } from '@agent-device/contracts/snapshot-model';
6-
import { scenarioName } from './coverage-manifest.ts';
6+
import { scenarioName, scenarioNames } from './coverage-manifest.ts';
77
import { NATIVE_REF_COVERAGE } from './native-ref.coverage.ts';
88
import {
99
closedDrawerSnapshot,
@@ -193,7 +193,7 @@ test(scenario('responseConstruction'), async () => {
193193
assert.ok(Array.isArray(result.preAction?.nodes));
194194
});
195195

196-
test(scenario('resolutionDisclosure'), async () => {
196+
test(scenarioNames(NATIVE_REF_COVERAGE, 'resolutionDisclosure')[0]!, async () => {
197197
const calls: string[] = [];
198198
const device = createNativeRefDevice(continueButtonSnapshot(), calls);
199199

@@ -202,3 +202,22 @@ test(scenario('resolutionDisclosure'), async () => {
202202
assert.equal(result.kind, 'ref');
203203
assert.deepEqual(result.resolution, { source: 'ref', phase: 'pre-action', kind: 'exact' });
204204
});
205+
206+
test(scenarioNames(NATIVE_REF_COVERAGE, 'resolutionDisclosure')[1]!, async () => {
207+
const calls: string[] = [];
208+
const device = createNativeRefDevice(continueButtonSnapshot(), calls);
209+
210+
// @e9 is not in the stored tree; the preflight recovers the Continue button by its trailing label.
211+
const result = await device.interactions.click(ref('@e9', { fallbackLabel: 'Continue' }), {
212+
session: 'default',
213+
});
214+
215+
assert.deepEqual(calls, ['@e9']);
216+
assert.equal(result.kind, 'ref');
217+
assert.equal(result.node?.label, 'Continue');
218+
assert.deepEqual(result.resolution, {
219+
source: 'ref',
220+
phase: 'pre-action',
221+
kind: 'label-fallback',
222+
});
223+
});

‎test/integration/interaction-contract/native-ref.coverage.ts‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,8 @@ export const NATIVE_REF_COVERAGE = definePathCoverage('native-ref', {
1818
// reason, hint), which is the shared taxonomy on this path.
1919
errorTaxonomy:
2020
'native-ref offscreen: preflight refuses an off-screen ref before the backend call',
21-
resolutionDisclosure:
21+
resolutionDisclosure: [
2222
'native-ref resolutionDisclosure: fast-path result discloses the exact ref-provenance shape',
23+
'native-ref resolutionDisclosure: preflight trailing-label recovery discloses label-fallback, never exact',
24+
],
2325
});

0 commit comments

Comments
 (0)