Skip to content

Commit c506ddf

Browse files
authored
RFC: ADR 0011 — interaction guarantee contract (path × guarantee matrix) (#1080)
* docs+feat: ADR 0011 interaction guarantee contract, Layer-1 registry and gate Design for making interaction guarantees hold across every dispatch path (runtime selector/ref, direct iOS selector, native ref, coordinate, maestro fallback) instead of eroding at path boundaries one incident at a time — every interaction bug this week was a (path, guarantee) cell nobody was watching. Three layers (ADR 0011): declare the path x guarantee matrix as a typed registry whose completeness is a compile error; share one implementation per rule on both sides of the wire with golden fixture tables proving TS/Swift parity; prove every non-waived cell with contract scenarios generated from the registry. This lands Layer 1: the registry with an HONEST initial classification — ten cells are acknowledged gap waivers (direct-path disambiguation/ occlusion/nonHittable/responseFields/errorTaxonomy, native-ref guards, coordinate bounds) — plus the gate test that keeps entries truthful: referenced TS symbols must be exported, runner symbols must exist in the Swift sources, delegations must land on paths that actually enforce the guarantee, and the gap list is pinned so it can only change explicitly in a reviewed diff. * refactor: apply ADR 0011 design review - Frame Layer 1 as an honesty/completeness gate, not a truth gate: it proves every path declared a stance and referenced symbols exist; behavioral parity starts with the Layer-2/3 fixture and scenario work. - Split responseFields into responseConstruction (one shared response construction site — a single Layer-2 refactor) and responseIdentity (which identity fields a path can provide — per-path capability work); note the anticipated errorTaxonomy split (codes vs diagnostics). - Encode the hybrid gap-closure strategy: runner-side parity for geometry-local rules, delegation-on-error for semantic failures (with the explicit caveat that delegation-on-error is NOT success-path parity), and a shared runtime preflight for native-ref where a silent backend success means delegation never triggers. - Gap waivers now require a trackingIssue (gate-enforced URL); all 16 pinned gaps link the umbrella issue #1081. The honest reclassification grew the pin list from 10 to 16 — responseConstruction is a gap on every path including runtime ones, which is exactly the partial progress the coarser guarantee was hiding. - Align ADR wording with the code: parityTable is optional until Layer 3, required once a runner cell claims parity. * fix: address registry review — maestro disambiguation honesty, command-scoped verify 1. maestro-non-hittable-fallback/disambiguation was overclaimed: the guarantee is defined as visible-first/deepest/smallest ranking, but findElement only implements unique-or-ambiguous scanning. Reclassified as an intentional waiver (deliberate Maestro-semantics divergence), mirroring how the direct path keeps its success-path parity gap. 2. verifyEvidence was claimed path-wide on paths that dispatch longpress, which has no --verify. Cells can now be command-scoped via appliesTo (non-empty strict subset of the path's commands, gate-enforced), and the three affected cells scope to press/click/fill.
1 parent 07181ac commit c506ddf

3 files changed

Lines changed: 854 additions & 0 deletions

File tree

Lines changed: 265 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,265 @@
1+
# ADR 0011: Interaction Guarantee Contract (path × guarantee matrix)
2+
3+
## Status
4+
5+
Proposed
6+
7+
## Context
8+
9+
Interaction commands (`press`/`click`/`fill`/`longpress`, and by extension the
10+
read/wait surfaces built on the same resolution machinery) reach the device
11+
through several dispatch paths, each of which chose its latency-vs-semantics
12+
trade-offs independently:
13+
14+
| Path | What it is | Why it exists |
15+
| --- | --- | --- |
16+
| `runtime-selector` | daemon tree capture → `resolveSelectorChain` → guards → coordinate tap | full semantics |
17+
| `runtime-ref` | session snapshot → ref lookup → guards → coordinate tap | full semantics |
18+
| `direct-ios-selector` | selector sent to the XCTest runner, which queries and taps natively | saves a full snapshot round trip |
19+
| `native-ref` | `backend.tapTarget`/`fillTarget` for `click @ref` / `fill @ref` | saves resolution round trips |
20+
| `coordinate` | raw x/y tap | escape hatch; semantics intentionally minimal |
21+
| replay/maestro variants | `allowNonHittableCoordinateFallback`, replay-heal | third-party compat |
22+
23+
Over one week of fixes, every interaction bug we shipped or caught was the same
24+
shape — a guarantee enforced on one path and silently absent on a sibling:
25+
26+
- off-screen refusal existed for `@ref` targets but not selector targets, and
27+
not in the runner's native tap (Bluesky closed drawer: `Tapped (-161, 265)`
28+
reported as success; fixed in #1075);
29+
- `--verify` evidence attached on `press @ref` but was dropped by
30+
`fill @ref`'s hand-built response branch (#1064 review);
31+
- adb failure classification enriched thrown errors but not
32+
`allowFailure`-result errors, and wrapped `exec` but not semantic provider
33+
methods (#1067 review);
34+
- the user's wait budget bounded prepare/replay/snapshot request envelopes but
35+
not `wait` itself, and `wait` sat on the daemon-reset timeout path while
36+
`snapshot` — same failure mode — preserved the daemon (#1075).
37+
38+
The guarantees themselves are small and well-implemented. What is missing is
39+
any structure that *knows the full set of paths and the full set of
40+
guarantees* and can therefore notice an unfilled cell. Reviews catch cells
41+
only when a reviewer happens to hold both axes in their head; dogfooding
42+
catches them one production incident at a time.
43+
44+
## Decision
45+
46+
Make the path × guarantee matrix a first-class, machine-checked artifact, with
47+
three enforcement layers. Types enforce **completeness** of declarations,
48+
shared implementations prevent **drift**, and generated test coverage enforces
49+
**truth**.
50+
51+
### Layer 1 — Declare: the matrix as a typed registry with a gate
52+
53+
Layer 1 is an **honesty/completeness gate, not a truth gate**: it proves that
54+
every path has declared a stance on every guarantee and that the referenced
55+
implementations exist. It does not prove the declared behavior — behavioral
56+
parity only starts once the fixture tables (Layer 2) and contract scenarios
57+
(Layer 3) land. Landing Layer 1 alone still changes the failure mode: an
58+
unwatched cell becomes impossible; an unproven cell is at least a visible,
59+
owned claim.
60+
61+
`src/contracts/interaction-guarantees.ts` declares both axes and requires
62+
every cell to be classified:
63+
64+
```ts
65+
export const INTERACTION_GUARANTEES = [
66+
'disambiguation', // visible > deepest > smallest; ties fail
67+
'occlusion', // covered targets are refused
68+
'offscreen', // tap point (rect center) must lie in the root viewport
69+
'nonHittable', // promotion + targetHittable/hint annotation
70+
'responseConstruction', // one shared response construction site (Layer 2)
71+
'responseIdentity', // refLabel/selectorChain availability on this path
72+
'verifyEvidence', // --verify baseline + post-action digest
73+
'errorTaxonomy', // no-match/ambiguous/offscreen codes, messages, hints
74+
] as const;
75+
```
76+
77+
Cells may be **command-scoped** via `appliesTo` when a guarantee only exists
78+
for a subset of the path's commands — e.g. `verifyEvidence` applies to
79+
`press`/`click`/`fill` but not `longpress`, and claiming it path-wide would
80+
overstate coverage. The gate rejects `appliesTo` entries naming commands the
81+
path does not dispatch, and rejects redundant full-coverage lists.
82+
83+
`responseConstruction` and `responseIdentity` are deliberately separate: "use
84+
one shared construction site" and "which identity fields this path can
85+
provide" have different closure strategies (the former is a single Layer-2
86+
refactor; the latter is per-path capability work). `errorTaxonomy` is expected
87+
to split the same way later — stable codes/fallback classification vs rich
88+
selector diagnostics and hints — because direct runner paths can close codes
89+
long before full diagnostics.
90+
91+
```ts
92+
93+
export type GuaranteeEnforcement =
94+
| { kind: 'runtime'; via: string } // shared TS implementation (symbol name)
95+
| { kind: 'runner'; via: string; parityTable?: string } // Swift twin; parityTable optional until Layer 3, required once the cell claims parity
96+
| { kind: 'delegated'; to: InteractionPathId } // path defers (e.g. direct → runtime on ELEMENT_OFFSCREEN)
97+
| { kind: 'waived'; reason: string; trackingIssue?: string }; // explicit, reviewed waiver; gap waivers must carry a tracking issue
98+
99+
export const INTERACTION_DISPATCH_PATHS: Record<
100+
InteractionPathId,
101+
{
102+
description: string;
103+
commands: readonly string[];
104+
guarantees: Record<InteractionGuarantee, GuaranteeEnforcement>;
105+
}
106+
> = { /* every path, every cell */ };
107+
```
108+
109+
Because `guarantees` is a `Record` over the guarantee union, **tsc fails** the
110+
moment someone adds a guarantee without classifying it for every path, or adds
111+
a path without classifying every guarantee. A unit gate
112+
(`interaction-guarantees.test.ts`) additionally checks that:
113+
114+
- every `via` string resolves to a real exported symbol (declarations cannot
115+
rot into fiction);
116+
- every `parityTable` names an existing fixture file;
117+
- every `waived` reason is non-empty, and every `gap:` waiver carries a
118+
`trackingIssue` — waivers must be owned, not merely visible. One umbrella
119+
tracking issue with sub-issues split off as work is scheduled is
120+
sufficient; the gate enforces the link, not the granularity.
121+
122+
This is the same "make the gap declare itself" pattern already proven in this
123+
repo by `scripts/integration-progress-model.ts` (which caught the unclassified
124+
`--verify` flag on #1064) and the cross-command apple-leak guard from the
125+
platform consolidation (ADR 0009).
126+
127+
### Layer 2 — Share: one implementation per rule, on both sides of the wire
128+
129+
Each guarantee has exactly one home that all TS paths import (most already
130+
exist after #1075: `isNodeVisibleOnScreen`, `accumulateDisambiguationCandidate`,
131+
`isSnapshotNodeInteractionBlocked`, `describeResolvedNode`,
132+
`interactionResultExtra`, `reconcileNonHittableHintWithEvidence`). The
133+
registry's `via` fields point at them; fallow's duplication gate keeps
134+
re-implementations from creeping back.
135+
136+
Rules that must run **runner-side** (the direct iOS path cannot see the daemon
137+
tree) get pure-function Swift twins operating on plain geometry — no
138+
`XCUIElement` — e.g. `TapPointPolicy.isAllowed(elementFrame:windowFrame:)`.
139+
Parity is enforced by **golden fixture tables**: JSON files under
140+
`contracts/fixtures/` consumed by three test suites —
141+
142+
1. vitest asserts the TS rule over the table;
143+
2. the runner's Swift unit tests (already compiled in CI by "Swift Runner Unit
144+
Compile") assert the Swift twin over the same table;
145+
3. the provider harness's fake runner derives its behavior from the same
146+
table, so integration scenarios exercise the real contract rather than a
147+
hand-written approximation.
148+
149+
Drift between TS and Swift then turns CI red on whichever side changed,
150+
without needing a simulator.
151+
152+
For `responseFields`, one `buildInteractionResponseData(...)` becomes the only
153+
construction site for interaction response payloads (this deletes the class of
154+
bug where `fill @ref` rebuilt its response by hand and dropped `evidence`). A
155+
small guard test — repo-precedented — fails if an interaction handler contains
156+
a hand-rolled `responseData = {` literal.
157+
158+
### Layer 3 — Prove: a contract suite generated from the registry
159+
160+
`test/integration/interaction-contract/` holds table-driven scenarios: fixture
161+
tree × command × forced path. The fixture trees are the real shapes that found
162+
this week's bugs, kept permanently:
163+
164+
- closed drawer (all candidates off-screen) → `offscreen_selector`/`offscreen_ref`;
165+
- drawer item + visible twin (ambiguous on/off-screen) → visible candidate wins;
166+
- edge-grazing container (0.07 px viewport overlap, center off-screen) → still refused;
167+
- covered node → occlusion refusal;
168+
- non-hittable target ± `--verify` → hint present / suppressed by evidence;
169+
- stripped-root tree (no Application/Window) → safe-default visibility.
170+
171+
Path forcing is explicit (`AGENT_DEVICE_FORCE_INTERACTION_PATH=runtime|direct|native-ref`,
172+
test-only) so cases stay stable when path-selection heuristics change.
173+
174+
The gate closes the loop: it walks the Layer-1 registry and fails when any
175+
non-waived cell has no contract case tagged for it. Coverage of the matrix is
176+
therefore by construction, not by reviewer memory.
177+
178+
### Closing the gaps: a hybrid strategy, not one answer
179+
180+
The acknowledged gaps close by different mechanisms depending on what the
181+
guarantee needs:
182+
183+
- **Runner-side parity for cheap geometry-local rules** (offscreen /
184+
tappable-frame on the direct iOS path). These are pure frame math, provable
185+
with golden tables, and keep the fast path fast.
186+
- **Delegation-on-error for semantic and rich-runtime cases**
187+
(`ELEMENT_OFFSCREEN`, `AMBIGUOUS_MATCH`, `ELEMENT_NOT_FOUND`, non-hittable
188+
refusal): the fast path fails cheaply, and the runtime path supplies
189+
disambiguation and full diagnostics only when needed. **Delegation-on-error
190+
is not success-path parity**: it cannot catch the case where XCTest finds
191+
one hittable candidate that runtime rules would refuse or rank differently.
192+
Those cells stay `gap:` waivers until parity tables or contract scenarios
193+
prove the success path too.
194+
- **A shared runtime preflight for the native-ref path**: the ref came from a
195+
daemon snapshot, so the node is already in hand — check offscreen /
196+
occlusion / non-hittable against it *before* calling
197+
`tapTarget`/`fillTarget`. A backend fast path can silently "succeed", so
198+
delegation-on-error would never trigger there.
199+
200+
### Timeout policy joins the descriptor registry
201+
202+
The `wait` timeout bug existed because request-envelope budgets and
203+
on-timeout daemon policy lived in two hand-maintained lists in the client
204+
(`isExplicitTimeoutCommand`, `shouldResetDaemonAfterRequestTimeout`). Both are
205+
replaced by declarations on the command descriptors (ADR 0008 registry):
206+
207+
```ts
208+
timeoutPolicy: {
209+
budget: { source: 'flag' | 'positional-parser' | 'none'; parser?: string };
210+
onTimeout: 'preserve-daemon' | 'reset-daemon';
211+
}
212+
```
213+
214+
with a completeness gate over all public commands. Read-only polling commands
215+
declare `preserve-daemon`; the client derives the envelope from the declared
216+
budget source instead of special-casing command names.
217+
218+
## Consequences
219+
220+
- Adding a dispatch path or a guarantee becomes a *forced* whole-matrix
221+
decision: tsc will not compile an unclassified cell, and the gate will not
222+
pass an untested one. Silent erosion is structurally impossible; explicit
223+
waivers remain possible but visible and linkable to issues.
224+
- Fast paths keep their latency wins. Their divergence is priced and
225+
documented instead of discovered on-device.
226+
- The Swift/TS split stops being a parity blind spot: golden tables are the
227+
single source of truth for cross-language rules, and the fake runner stops
228+
being a second, drifting implementation.
229+
- Initial cost is mostly classification honesty: the first registry will
230+
contain `waived('gap: ...')` cells (e.g. the maestro non-hittable fallback
231+
intentionally waives `offscreen`), which is the point — the debt becomes a
232+
diffable list instead of folklore.
233+
- The registry doubles as documentation input for help/skill output ("what
234+
press guarantees"), which matters for small-model agents that only read the
235+
contract, never the code.
236+
237+
## Migration plan
238+
239+
Each step lands green and independently useful:
240+
241+
1. **Registry + gate** with an honest initial classification (waivers linked
242+
to issues). No behavior change.
243+
2. **Response builder consolidation** (`buildInteractionResponseData`) + the
244+
hand-rolled-literal guard test.
245+
3. **Golden fixture tables** + `TapPointPolicy` Swift extraction + the three
246+
consumers (vitest, XCTest, fake runner).
247+
4. **Contract scenario suite** + registry-driven coverage gate; port the
248+
Bluesky fixtures from #1075's tests into the permanent tables.
249+
5. **Descriptor timeout policy**; delete the two client-side command lists.
250+
251+
## Alternatives considered
252+
253+
- **Delete the fast paths** (single resolution spine): unacceptable — the
254+
direct iOS path saves a full snapshot round trip per interaction and the
255+
ref fast path is the backbone of replay throughput. The problem is not that
256+
fast paths exist; it is that their trade-offs were implicit.
257+
- **Typestate/branded types** ("cannot construct a response without proof the
258+
guards ran"): cannot reach across the wire into Swift, and the proof tokens
259+
go viral through every signature for marginal gain over the
260+
registry-plus-gate split. Types are used where they are strongest —
261+
completeness of the declaration — and tests where they are strongest —
262+
truth of the declaration.
263+
- **More integration tests without the registry**: this is the status quo
264+
plus effort. Without the matrix as code, nothing forces a new path to
265+
acquire the existing suite, which is exactly how this week's bugs happened.

0 commit comments

Comments
 (0)