|
| 1 | +# ADR 0011: Interaction Guarantee Contract (path × guarantee matrix) |
| 2 | + |
| 3 | +## Status |
| 4 | + |
| 5 | +Proposed |
| 6 | + |
| 7 | +## Context |
| 8 | + |
| 9 | +Interaction commands (`press`/`click`/`fill`/`longpress`, and by extension the |
| 10 | +read/wait surfaces built on the same resolution machinery) reach the device |
| 11 | +through several dispatch paths, each of which chose its latency-vs-semantics |
| 12 | +trade-offs independently: |
| 13 | + |
| 14 | +| Path | What it is | Why it exists | |
| 15 | +| --- | --- | --- | |
| 16 | +| `runtime-selector` | daemon tree capture → `resolveSelectorChain` → guards → coordinate tap | full semantics | |
| 17 | +| `runtime-ref` | session snapshot → ref lookup → guards → coordinate tap | full semantics | |
| 18 | +| `direct-ios-selector` | selector sent to the XCTest runner, which queries and taps natively | saves a full snapshot round trip | |
| 19 | +| `native-ref` | `backend.tapTarget`/`fillTarget` for `click @ref` / `fill @ref` | saves resolution round trips | |
| 20 | +| `coordinate` | raw x/y tap | escape hatch; semantics intentionally minimal | |
| 21 | +| replay/maestro variants | `allowNonHittableCoordinateFallback`, replay-heal | third-party compat | |
| 22 | + |
| 23 | +Over one week of fixes, every interaction bug we shipped or caught was the same |
| 24 | +shape — a guarantee enforced on one path and silently absent on a sibling: |
| 25 | + |
| 26 | +- off-screen refusal existed for `@ref` targets but not selector targets, and |
| 27 | + not in the runner's native tap (Bluesky closed drawer: `Tapped (-161, 265)` |
| 28 | + reported as success; fixed in #1075); |
| 29 | +- `--verify` evidence attached on `press @ref` but was dropped by |
| 30 | + `fill @ref`'s hand-built response branch (#1064 review); |
| 31 | +- adb failure classification enriched thrown errors but not |
| 32 | + `allowFailure`-result errors, and wrapped `exec` but not semantic provider |
| 33 | + methods (#1067 review); |
| 34 | +- the user's wait budget bounded prepare/replay/snapshot request envelopes but |
| 35 | + not `wait` itself, and `wait` sat on the daemon-reset timeout path while |
| 36 | + `snapshot` — same failure mode — preserved the daemon (#1075). |
| 37 | + |
| 38 | +The guarantees themselves are small and well-implemented. What is missing is |
| 39 | +any structure that *knows the full set of paths and the full set of |
| 40 | +guarantees* and can therefore notice an unfilled cell. Reviews catch cells |
| 41 | +only when a reviewer happens to hold both axes in their head; dogfooding |
| 42 | +catches them one production incident at a time. |
| 43 | + |
| 44 | +## Decision |
| 45 | + |
| 46 | +Make the path × guarantee matrix a first-class, machine-checked artifact, with |
| 47 | +three enforcement layers. Types enforce **completeness** of declarations, |
| 48 | +shared implementations prevent **drift**, and generated test coverage enforces |
| 49 | +**truth**. |
| 50 | + |
| 51 | +### Layer 1 — Declare: the matrix as a typed registry with a gate |
| 52 | + |
| 53 | +Layer 1 is an **honesty/completeness gate, not a truth gate**: it proves that |
| 54 | +every path has declared a stance on every guarantee and that the referenced |
| 55 | +implementations exist. It does not prove the declared behavior — behavioral |
| 56 | +parity only starts once the fixture tables (Layer 2) and contract scenarios |
| 57 | +(Layer 3) land. Landing Layer 1 alone still changes the failure mode: an |
| 58 | +unwatched cell becomes impossible; an unproven cell is at least a visible, |
| 59 | +owned claim. |
| 60 | + |
| 61 | +`src/contracts/interaction-guarantees.ts` declares both axes and requires |
| 62 | +every cell to be classified: |
| 63 | + |
| 64 | +```ts |
| 65 | +export const INTERACTION_GUARANTEES = [ |
| 66 | + 'disambiguation', // visible > deepest > smallest; ties fail |
| 67 | + 'occlusion', // covered targets are refused |
| 68 | + 'offscreen', // tap point (rect center) must lie in the root viewport |
| 69 | + 'nonHittable', // promotion + targetHittable/hint annotation |
| 70 | + 'responseConstruction', // one shared response construction site (Layer 2) |
| 71 | + 'responseIdentity', // refLabel/selectorChain availability on this path |
| 72 | + 'verifyEvidence', // --verify baseline + post-action digest |
| 73 | + 'errorTaxonomy', // no-match/ambiguous/offscreen codes, messages, hints |
| 74 | +] as const; |
| 75 | +``` |
| 76 | + |
| 77 | +Cells may be **command-scoped** via `appliesTo` when a guarantee only exists |
| 78 | +for a subset of the path's commands — e.g. `verifyEvidence` applies to |
| 79 | +`press`/`click`/`fill` but not `longpress`, and claiming it path-wide would |
| 80 | +overstate coverage. The gate rejects `appliesTo` entries naming commands the |
| 81 | +path does not dispatch, and rejects redundant full-coverage lists. |
| 82 | + |
| 83 | +`responseConstruction` and `responseIdentity` are deliberately separate: "use |
| 84 | +one shared construction site" and "which identity fields this path can |
| 85 | +provide" have different closure strategies (the former is a single Layer-2 |
| 86 | +refactor; the latter is per-path capability work). `errorTaxonomy` is expected |
| 87 | +to split the same way later — stable codes/fallback classification vs rich |
| 88 | +selector diagnostics and hints — because direct runner paths can close codes |
| 89 | +long before full diagnostics. |
| 90 | + |
| 91 | +```ts |
| 92 | + |
| 93 | +export type GuaranteeEnforcement = |
| 94 | + | { kind: 'runtime'; via: string } // shared TS implementation (symbol name) |
| 95 | + | { kind: 'runner'; via: string; parityTable?: string } // Swift twin; parityTable optional until Layer 3, required once the cell claims parity |
| 96 | + | { kind: 'delegated'; to: InteractionPathId } // path defers (e.g. direct → runtime on ELEMENT_OFFSCREEN) |
| 97 | + | { kind: 'waived'; reason: string; trackingIssue?: string }; // explicit, reviewed waiver; gap waivers must carry a tracking issue |
| 98 | + |
| 99 | +export const INTERACTION_DISPATCH_PATHS: Record< |
| 100 | + InteractionPathId, |
| 101 | + { |
| 102 | + description: string; |
| 103 | + commands: readonly string[]; |
| 104 | + guarantees: Record<InteractionGuarantee, GuaranteeEnforcement>; |
| 105 | + } |
| 106 | +> = { /* every path, every cell */ }; |
| 107 | +``` |
| 108 | + |
| 109 | +Because `guarantees` is a `Record` over the guarantee union, **tsc fails** the |
| 110 | +moment someone adds a guarantee without classifying it for every path, or adds |
| 111 | +a path without classifying every guarantee. A unit gate |
| 112 | +(`interaction-guarantees.test.ts`) additionally checks that: |
| 113 | + |
| 114 | +- every `via` string resolves to a real exported symbol (declarations cannot |
| 115 | + rot into fiction); |
| 116 | +- every `parityTable` names an existing fixture file; |
| 117 | +- every `waived` reason is non-empty, and every `gap:` waiver carries a |
| 118 | + `trackingIssue` — waivers must be owned, not merely visible. One umbrella |
| 119 | + tracking issue with sub-issues split off as work is scheduled is |
| 120 | + sufficient; the gate enforces the link, not the granularity. |
| 121 | + |
| 122 | +This is the same "make the gap declare itself" pattern already proven in this |
| 123 | +repo by `scripts/integration-progress-model.ts` (which caught the unclassified |
| 124 | +`--verify` flag on #1064) and the cross-command apple-leak guard from the |
| 125 | +platform consolidation (ADR 0009). |
| 126 | + |
| 127 | +### Layer 2 — Share: one implementation per rule, on both sides of the wire |
| 128 | + |
| 129 | +Each guarantee has exactly one home that all TS paths import (most already |
| 130 | +exist after #1075: `isNodeVisibleOnScreen`, `accumulateDisambiguationCandidate`, |
| 131 | +`isSnapshotNodeInteractionBlocked`, `describeResolvedNode`, |
| 132 | +`interactionResultExtra`, `reconcileNonHittableHintWithEvidence`). The |
| 133 | +registry's `via` fields point at them; fallow's duplication gate keeps |
| 134 | +re-implementations from creeping back. |
| 135 | + |
| 136 | +Rules that must run **runner-side** (the direct iOS path cannot see the daemon |
| 137 | +tree) get pure-function Swift twins operating on plain geometry — no |
| 138 | +`XCUIElement` — e.g. `TapPointPolicy.isAllowed(elementFrame:windowFrame:)`. |
| 139 | +Parity is enforced by **golden fixture tables**: JSON files under |
| 140 | +`contracts/fixtures/` consumed by three test suites — |
| 141 | + |
| 142 | +1. vitest asserts the TS rule over the table; |
| 143 | +2. the runner's Swift unit tests (already compiled in CI by "Swift Runner Unit |
| 144 | + Compile") assert the Swift twin over the same table; |
| 145 | +3. the provider harness's fake runner derives its behavior from the same |
| 146 | + table, so integration scenarios exercise the real contract rather than a |
| 147 | + hand-written approximation. |
| 148 | + |
| 149 | +Drift between TS and Swift then turns CI red on whichever side changed, |
| 150 | +without needing a simulator. |
| 151 | + |
| 152 | +For `responseFields`, one `buildInteractionResponseData(...)` becomes the only |
| 153 | +construction site for interaction response payloads (this deletes the class of |
| 154 | +bug where `fill @ref` rebuilt its response by hand and dropped `evidence`). A |
| 155 | +small guard test — repo-precedented — fails if an interaction handler contains |
| 156 | +a hand-rolled `responseData = {` literal. |
| 157 | + |
| 158 | +### Layer 3 — Prove: a contract suite generated from the registry |
| 159 | + |
| 160 | +`test/integration/interaction-contract/` holds table-driven scenarios: fixture |
| 161 | +tree × command × forced path. The fixture trees are the real shapes that found |
| 162 | +this week's bugs, kept permanently: |
| 163 | + |
| 164 | +- closed drawer (all candidates off-screen) → `offscreen_selector`/`offscreen_ref`; |
| 165 | +- drawer item + visible twin (ambiguous on/off-screen) → visible candidate wins; |
| 166 | +- edge-grazing container (0.07 px viewport overlap, center off-screen) → still refused; |
| 167 | +- covered node → occlusion refusal; |
| 168 | +- non-hittable target ± `--verify` → hint present / suppressed by evidence; |
| 169 | +- stripped-root tree (no Application/Window) → safe-default visibility. |
| 170 | + |
| 171 | +Path forcing is explicit (`AGENT_DEVICE_FORCE_INTERACTION_PATH=runtime|direct|native-ref`, |
| 172 | +test-only) so cases stay stable when path-selection heuristics change. |
| 173 | + |
| 174 | +The gate closes the loop: it walks the Layer-1 registry and fails when any |
| 175 | +non-waived cell has no contract case tagged for it. Coverage of the matrix is |
| 176 | +therefore by construction, not by reviewer memory. |
| 177 | + |
| 178 | +### Closing the gaps: a hybrid strategy, not one answer |
| 179 | + |
| 180 | +The acknowledged gaps close by different mechanisms depending on what the |
| 181 | +guarantee needs: |
| 182 | + |
| 183 | +- **Runner-side parity for cheap geometry-local rules** (offscreen / |
| 184 | + tappable-frame on the direct iOS path). These are pure frame math, provable |
| 185 | + with golden tables, and keep the fast path fast. |
| 186 | +- **Delegation-on-error for semantic and rich-runtime cases** |
| 187 | + (`ELEMENT_OFFSCREEN`, `AMBIGUOUS_MATCH`, `ELEMENT_NOT_FOUND`, non-hittable |
| 188 | + refusal): the fast path fails cheaply, and the runtime path supplies |
| 189 | + disambiguation and full diagnostics only when needed. **Delegation-on-error |
| 190 | + is not success-path parity**: it cannot catch the case where XCTest finds |
| 191 | + one hittable candidate that runtime rules would refuse or rank differently. |
| 192 | + Those cells stay `gap:` waivers until parity tables or contract scenarios |
| 193 | + prove the success path too. |
| 194 | +- **A shared runtime preflight for the native-ref path**: the ref came from a |
| 195 | + daemon snapshot, so the node is already in hand — check offscreen / |
| 196 | + occlusion / non-hittable against it *before* calling |
| 197 | + `tapTarget`/`fillTarget`. A backend fast path can silently "succeed", so |
| 198 | + delegation-on-error would never trigger there. |
| 199 | + |
| 200 | +### Timeout policy joins the descriptor registry |
| 201 | + |
| 202 | +The `wait` timeout bug existed because request-envelope budgets and |
| 203 | +on-timeout daemon policy lived in two hand-maintained lists in the client |
| 204 | +(`isExplicitTimeoutCommand`, `shouldResetDaemonAfterRequestTimeout`). Both are |
| 205 | +replaced by declarations on the command descriptors (ADR 0008 registry): |
| 206 | + |
| 207 | +```ts |
| 208 | +timeoutPolicy: { |
| 209 | + budget: { source: 'flag' | 'positional-parser' | 'none'; parser?: string }; |
| 210 | + onTimeout: 'preserve-daemon' | 'reset-daemon'; |
| 211 | +} |
| 212 | +``` |
| 213 | + |
| 214 | +with a completeness gate over all public commands. Read-only polling commands |
| 215 | +declare `preserve-daemon`; the client derives the envelope from the declared |
| 216 | +budget source instead of special-casing command names. |
| 217 | + |
| 218 | +## Consequences |
| 219 | + |
| 220 | +- Adding a dispatch path or a guarantee becomes a *forced* whole-matrix |
| 221 | + decision: tsc will not compile an unclassified cell, and the gate will not |
| 222 | + pass an untested one. Silent erosion is structurally impossible; explicit |
| 223 | + waivers remain possible but visible and linkable to issues. |
| 224 | +- Fast paths keep their latency wins. Their divergence is priced and |
| 225 | + documented instead of discovered on-device. |
| 226 | +- The Swift/TS split stops being a parity blind spot: golden tables are the |
| 227 | + single source of truth for cross-language rules, and the fake runner stops |
| 228 | + being a second, drifting implementation. |
| 229 | +- Initial cost is mostly classification honesty: the first registry will |
| 230 | + contain `waived('gap: ...')` cells (e.g. the maestro non-hittable fallback |
| 231 | + intentionally waives `offscreen`), which is the point — the debt becomes a |
| 232 | + diffable list instead of folklore. |
| 233 | +- The registry doubles as documentation input for help/skill output ("what |
| 234 | + press guarantees"), which matters for small-model agents that only read the |
| 235 | + contract, never the code. |
| 236 | + |
| 237 | +## Migration plan |
| 238 | + |
| 239 | +Each step lands green and independently useful: |
| 240 | + |
| 241 | +1. **Registry + gate** with an honest initial classification (waivers linked |
| 242 | + to issues). No behavior change. |
| 243 | +2. **Response builder consolidation** (`buildInteractionResponseData`) + the |
| 244 | + hand-rolled-literal guard test. |
| 245 | +3. **Golden fixture tables** + `TapPointPolicy` Swift extraction + the three |
| 246 | + consumers (vitest, XCTest, fake runner). |
| 247 | +4. **Contract scenario suite** + registry-driven coverage gate; port the |
| 248 | + Bluesky fixtures from #1075's tests into the permanent tables. |
| 249 | +5. **Descriptor timeout policy**; delete the two client-side command lists. |
| 250 | + |
| 251 | +## Alternatives considered |
| 252 | + |
| 253 | +- **Delete the fast paths** (single resolution spine): unacceptable — the |
| 254 | + direct iOS path saves a full snapshot round trip per interaction and the |
| 255 | + ref fast path is the backbone of replay throughput. The problem is not that |
| 256 | + fast paths exist; it is that their trade-offs were implicit. |
| 257 | +- **Typestate/branded types** ("cannot construct a response without proof the |
| 258 | + guards ran"): cannot reach across the wire into Swift, and the proof tokens |
| 259 | + go viral through every signature for marginal gain over the |
| 260 | + registry-plus-gate split. Types are used where they are strongest — |
| 261 | + completeness of the declaration — and tests where they are strongest — |
| 262 | + truth of the declaration. |
| 263 | +- **More integration tests without the registry**: this is the status quo |
| 264 | + plus effort. Without the matrix as code, nothing forces a new path to |
| 265 | + acquire the existing suite, which is exactly how this week's bugs happened. |
0 commit comments