Skip to content

Commit 52a25cd

Browse files
committed
refactor: route selector structural stages into typed policy
#1649 landed the per-caller ambiguity matrix and deliberately left four structural columns out: occlusion, off-screen, hittable-ancestor promotion, and the poll budget were per-caller pipeline code, so declaring them would have been an unverifiable claim (nothing consumed them; flipping one left the suite green). This adds the missing half as a table with runners. `SELECTOR_PIPELINE_POLICIES` (src/core/selector-pipeline-policy.ts) gives each caller ONE row naming its ambiguity contract plus its four stages, and every stage is reached only through a runner that reads the row: - occlusion -> selectorPipelineCandidates (candidacy) and resolveSelectorPipelineTarget (refusal). Acting rows exclude covered nodes and refuse covered targets; `find` and the diagnosis probe keep them as candidates and refuse at the target; reads and `wait` ignore them. - promotion -> resolveSelectorPipelineTarget. The per-call-site `promoteToHittableAncestor: boolean` is gone: click/press/longpress name `promotedTarget`, fill/focus/scroll/drag endpoints and the native-ref preflight name `resolvedTarget`. `find`'s below-the-root variant is a declared value rather than a second local helper. - off-screen -> throwIfOffscreenInteractionTarget, which now takes the row and returns the node untouched (no iOS rescue round trip) for observation rows. - poll -> selectorPollBudget, which createWaitPolling derives its deadline and inter-poll delay from; the two wait loops carry a budget, every other row carries none and cannot be polled. Behavior is byte-identical. The acting refusal keeps its exact node, label and details in every branch (promotion declines to retarget away from a covered node, so the "both covered" case names the same node it always did), and `find` carries the occlusion verdict to the focus/type seam rather than raising it early, because find click/fill still delegate that refusal to the interaction leaf's own error shape. selector-pipeline-policy.test.ts drives EVERY row through EVERY runner, including the rows whose answer is "skip" — the half that used to be an absence of code, and an absence cannot fail. Each stage was proven red by flipping its cell (occlusion, promotion, off-screen, poll, plus the declare-only-what-is-enforced guard). The ADR 0011 occlusion/nonHittable `via` pointers for the runtime tree paths now name the runner that makes the decision, not the predicate it applies. Closes #1656; prework for #1739 (waves 4-5).
1 parent 057ab1c commit 52a25cd

17 files changed

Lines changed: 795 additions & 164 deletions

‎packages/contracts/src/interaction-guarantees.ts‎

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -150,9 +150,12 @@ const SHARED_RESPONSE_CONSTRUCTION: GuaranteeEnforcement = {
150150
// guard/observation implementations; only how the target is found
151151
// (disambiguation) and how failures are described (errorTaxonomy) differ.
152152
const RUNTIME_TREE_SHARED_GUARANTEES = {
153+
// #1656: the decision point is the pipeline runner, which reads the acting
154+
// row's occlusion stage; isSnapshotNodeInteractionBlocked stays the
155+
// predicate it applies (and the annotation contract it reads).
153156
occlusion: {
154157
kind: 'runtime',
155-
via: 'src/snapshot/snapshot-occlusion.ts#isSnapshotNodeInteractionBlocked',
158+
via: 'src/core/selector-pipeline-policy.ts#resolveSelectorPipelineTarget',
156159
},
157160
parentOwnedTouchPoint: {
158161
kind: 'runtime',
@@ -171,9 +174,11 @@ const RUNTIME_TREE_SHARED_GUARANTEES = {
171174
kind: 'runtime',
172175
via: 'src/commands/interaction/runtime/resolution.ts#throwIfOffscreenInteractionTarget',
173176
},
177+
// Promotion runs only for rows that declare it (#1656); the retarget itself
178+
// is still resolveActionableTouchResolution.
174179
nonHittable: {
175180
kind: 'runtime',
176-
via: 'src/core/interaction-targeting.ts#resolveActionableTouchResolution',
181+
via: 'src/core/selector-pipeline-policy.ts#resolveSelectorPipelineTarget',
177182
},
178183
responseConstruction: SHARED_RESPONSE_CONSTRUCTION,
179184
responseIdentity: {
@@ -249,7 +254,7 @@ export const INTERACTION_DISPATCH_PATHS: Record<InteractionPathId, InteractionPa
249254
},
250255
occlusion: {
251256
kind: 'runtime',
252-
via: 'src/snapshot/snapshot-occlusion.ts#isSnapshotNodeInteractionBlocked',
257+
via: 'src/core/selector-pipeline-policy.ts#resolveSelectorPipelineTarget',
253258
},
254259
parentOwnedTouchPoint: {
255260
kind: 'runtime',
@@ -370,7 +375,7 @@ export const INTERACTION_DISPATCH_PATHS: Record<InteractionPathId, InteractionPa
370375
},
371376
occlusion: {
372377
kind: 'runtime',
373-
via: 'src/snapshot/snapshot-occlusion.ts#isSnapshotNodeInteractionBlocked',
378+
via: 'src/core/selector-pipeline-policy.ts#resolveSelectorPipelineTarget',
374379
},
375380
parentOwnedTouchPoint: {
376381
kind: 'inapplicable',

‎packages/selectors/src/internal/resolution-policy.ts‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -28,11 +28,13 @@ import type { SelectorResolutionOptions } from './public-resolution-types.ts';
2828
* documented semantics fails a test.
2929
*
3030
* The surrounding pipeline stages — occlusion, the off-screen guard,
31-
* hittable-ancestor promotion, and the wait poll budget — still live in the
32-
* callers and are NOT declared here. An earlier revision listed them as
33-
* columns; nothing consumed them, so they were unverifiable claims that read
34-
* as truth while being free to drift (#1649 review). Routing them into typed
35-
* behavior is tracked in #1656.
31+
* hittable-ancestor promotion, and the wait poll budget — are declared in the
32+
* companion structural table, `src/core/selector-pipeline-policy.ts` (#1656),
33+
* whose rows each name one row of this matrix. They live there rather than
34+
* here because this package is deliberately blind to snapshot occlusion
35+
* annotations, backend visibility probes, and the wait clock; a column here
36+
* would be a claim nothing in this package could enforce, which is exactly the
37+
* unverifiable-column failure the #1649 review caught.
3638
*/
3739

3840
export type KnobBackedSelectorAmbiguity = 'disambiguate' | 'fail-closed' | 'first-match';

‎src/commands/interaction/runtime/__tests__/resolution-policy-parity.test.ts‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -166,8 +166,10 @@ test('rect-requiring rows skip rectless nodes; read and wait rows accept them',
166166
* The matrix may only declare what it can enforce (#1649 review). An earlier
167167
* revision carried occlusion / off-screen / promotion / poll columns that no
168168
* code consumed, so changing them left both behavior and the suite green —
169-
* an unverifiable claim reading as truth. This fails if such a field returns
170-
* without behavioral coverage.
169+
* an unverifiable claim reading as truth. Those four stages now live in the
170+
* structural table (`src/core/selector-pipeline-policy.ts`, #1656), where
171+
* runners consume them; this still fails if such a field reappears HERE, where
172+
* the selectors package has nothing to enforce it with.
171173
*/
172174
test('policy rows declare only the fields this matrix actually enforces', () => {
173175
for (const [name, policy] of Object.entries(SELECTOR_RESOLUTION_POLICIES)) {

‎src/commands/interaction/runtime/gestures.ts‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ import {
1313
singlePointerPlanEndpoints,
1414
} from '@agent-device/contracts/interaction';
1515
import { AppError } from '@agent-device/kernel/errors';
16+
import { SELECTOR_PIPELINE_POLICIES } from '../../../core/selector-pipeline-policy.ts';
1617
import type { Point, Rect, SnapshotNode } from '@agent-device/kernel/snapshot';
1718
import type { AgentDeviceRuntime, CommandContext } from '../../../runtime-contract.ts';
1819
import {
@@ -164,7 +165,7 @@ export const focusCommand: RuntimeCommand<FocusCommandOptions, FocusCommandResul
164165
const resolved = await resolveInteractionTarget(runtime, options, {
165166
action: 'focus',
166167
requireInteractive: true,
167-
promoteToHittableAncestor: false,
168+
pipeline: SELECTOR_PIPELINE_POLICIES.resolvedTarget,
168169
});
169170
if (!runtime.backend.focus) {
170171
throw new AppError('UNSUPPORTED_OPERATION', 'focus is not supported by this backend');
@@ -187,7 +188,7 @@ export const longPressCommand: RuntimeCommand<
187188
const resolved = await resolveInteractionTarget(runtime, options, {
188189
action: 'longPress',
189190
requireInteractive: true,
190-
promoteToHittableAncestor: true,
191+
pipeline: SELECTOR_PIPELINE_POLICIES.promotedTarget,
191192
captureEvidenceBaseline: observation.needsPreActionBaseline,
192193
expectedResolvedTarget: options.expectedResolvedTarget,
193194
});
@@ -277,7 +278,7 @@ async function resolveDragTarget(
277278
{
278279
action: 'drag',
279280
requireInteractive: false,
280-
promoteToHittableAncestor: false,
281+
pipeline: SELECTOR_PIPELINE_POLICIES.resolvedTarget,
281282
expectedResolvedTarget: options.expectedResolvedTargets?.[endpoint],
282283
replayTargetRole: endpoint,
283284
},
@@ -410,7 +411,7 @@ async function resolveScrollTarget(
410411
{
411412
action: 'scroll',
412413
requireInteractive: false,
413-
promoteToHittableAncestor: false,
414+
pipeline: SELECTOR_PIPELINE_POLICIES.resolvedTarget,
414415
},
415416
);
416417
}

‎src/commands/interaction/runtime/interactions.ts‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ import type {
77
ResolvedTarget,
88
} from '@agent-device/contracts/interaction';
99
import { AppError } from '@agent-device/kernel/errors';
10+
import { SELECTOR_PIPELINE_POLICIES } from '../../../core/selector-pipeline-policy.ts';
1011
import type { Point } from '@agent-device/kernel/snapshot';
1112
import type { AgentDeviceRuntime, CommandContext } from '../../../runtime-contract.ts';
1213
import { isFillableType } from '@agent-device/contracts/snapshot';
@@ -105,7 +106,7 @@ export const fillCommand: RuntimeCommand<FillCommandOptions, FillCommandResult>
105106
const resolved = await resolveInteractionTarget(runtime, options, {
106107
action: 'fill',
107108
requireInteractive: true,
108-
promoteToHittableAncestor: false,
109+
pipeline: SELECTOR_PIPELINE_POLICIES.resolvedTarget,
109110
captureEvidenceBaseline: observation.needsPreActionBaseline,
110111
expectedResolvedTarget: options.expectedResolvedTarget,
111112
preresolvedTarget: options.preresolvedTarget,
@@ -195,7 +196,7 @@ async function tapCommand(
195196
const resolved = await resolveInteractionTarget(runtime, options, {
196197
action,
197198
requireInteractive: true,
198-
promoteToHittableAncestor: true,
199+
pipeline: SELECTOR_PIPELINE_POLICIES.promotedTarget,
199200
captureEvidenceBaseline: observation.needsPreActionBaseline,
200201
expectedResolvedTarget: options.expectedResolvedTarget,
201202
preresolvedTarget: options.preresolvedTarget,

‎src/commands/interaction/runtime/resolution.test.ts‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ import {
99
} from './resolution.ts';
1010
import { resolveRecordedTarget } from '@agent-device/selectors';
1111
import { makeSnapshotState } from '../../../__tests__/test-utils/index.ts';
12+
import { SELECTOR_PIPELINE_POLICIES } from '../../../core/selector-pipeline-policy.ts';
1213
import type { Point } from '@agent-device/kernel/snapshot';
1314
import {
1415
clickRefE2,
@@ -566,6 +567,7 @@ test('throwIfOffscreenInteractionTarget: an on-screen node passes through unchan
566567
{ session: 'default' },
567568
nodes[1]!,
568569
nodes,
570+
SELECTOR_PIPELINE_POLICIES.promotedTarget,
569571
fakeOffscreenFailure(),
570572
);
571573

@@ -592,13 +594,46 @@ test('throwIfOffscreenInteractionTarget: off-screen + backend confirms -> return
592594
{ session: 'default' },
593595
nodes[1]!,
594596
nodes,
597+
SELECTOR_PIPELINE_POLICIES.promotedTarget,
595598
fakeOffscreenFailure(),
596599
);
597600

598601
assert.deepEqual(result.rect, { x: 30, y: 30, width: 40, height: 40 });
599602
assert.equal(result.index, nodes[1]!.index);
600603
});
601604

605+
test('throwIfOffscreenInteractionTarget: a row whose off-screen stage is ignore returns the node untouched', async () => {
606+
// #1656: reads and waits answer about the tree as captured, so the same
607+
// node the acting rows refuse below passes through here — and the iOS
608+
// rescue hook is never consulted, which is why it would throw if it were.
609+
const device = createInteractionDevice(makeSnapshotState([]), {
610+
confirmOffscreenTargetVisible: async () => {
611+
throw new Error('an observation row must not spend the rescue round trip');
612+
},
613+
});
614+
const nodes = makeSnapshotState([
615+
{ index: 0, depth: 0, type: 'Application', rect: { x: 0, y: 0, width: 400, height: 800 } },
616+
{
617+
index: 1,
618+
depth: 1,
619+
parentIndex: 0,
620+
type: 'Button',
621+
rect: { x: 20, y: 2000, width: 40, height: 40 },
622+
},
623+
]).nodes;
624+
625+
const result = await throwIfOffscreenInteractionTarget(
626+
device,
627+
{ session: 'default' },
628+
nodes[1]!,
629+
nodes,
630+
SELECTOR_PIPELINE_POLICIES.readUnique,
631+
fakeOffscreenFailure(),
632+
);
633+
634+
assert.equal(result, nodes[1]);
635+
});
636+
602637
test('throwIfOffscreenInteractionTarget: off-screen + no rescue -> throws with the supplied failure shape', async () => {
603638
const device = createInteractionDevice(makeSnapshotState([]));
604639
const nodes = makeSnapshotState([
@@ -619,6 +654,7 @@ test('throwIfOffscreenInteractionTarget: off-screen + no rescue -> throws with t
619654
{ session: 'default' },
620655
nodes[1]!,
621656
nodes,
657+
SELECTOR_PIPELINE_POLICIES.promotedTarget,
622658
fakeOffscreenFailure(),
623659
),
624660
/off-screen/,

0 commit comments

Comments
 (0)