Skip to content

Commit de65ecb

Browse files
committed
fix(replay): key the repair-segment exclusion on provenance, scope --record (#1271 review)
Addresses the maintainer review on #1303. P1 — the exclusion dropped PLANNED reads from the heal. It discriminated by command class, but the real discriminator is provenance. Replayed plan steps dispatch through the ordinary request path, so an authored get/is/find step hit the same recordIfSession -> exclusion path as an interactive read and never reached session.actions -- and the heal IS session.actions.slice(boundary). A repaired flow therefore replayed its authored `is visible` assertion and then silently dropped it from its own healed script: the heal quietly stops checking what it used to check, which for a 10x-QA-replay suite is the worst failure mode. Fix: an explicit provenance marker, not a heuristic. `internal.replayPlanStep` is stamped by invokeResolvedReplayAction -- the single point every plan step is dispatched, so it covers annotated and unannotated steps alike. `internal` is daemon-only (toDaemonRequest never copies it off the wire), so authored provenance cannot be spoofed; same channel as replayTargetGuard. The rule now lives once in isInteractiveObservation and both recording call sites consume it, so the mock fixture uses the production classifier instead of mirroring it. Planned observations survive automatically -- users never annotate their own .ad steps. --record is no longer a common flag: removed from COMMON_COMMAND_SUPPORTED_FLAG_KEYS, statically scoped via allowedFlags to snapshot/get/is, and validated dynamically for find (read-only allows; a mutating find click|fill|focus|type is INVALID_ARGS before any device work, sharing one isReadOnlyFindAction predicate with the read-only routing so the two cannot disagree). --no-record stays shared -- it applies to every recordable command. Removed from `open`, which is never observation-only. Rebased onto #1304 and dropped the four hand-rolled reader blocks. Split its helper rather than broadening it: noRecordInputFromFlags (all 13 readers) + observationRecordInputFromFlags (snapshot/get/is/find only). Two named helpers over one `allowRecord` policy arg -- the capability is then the helper's NAME, so a mutating reader physically cannot forward --record, whereas a policy arg would let a future mutating reader opt in by flipping a literal with no schema change. ADR-0012 decision 6 now states the provenance rule, not a command-class rule. The scenario gates the P1: its authored step is a distinguishable `is visible`, and it fails without the provenance check ("the authored 'is visible' step must survive the heal").
1 parent 4e9277a commit de65ecb

10 files changed

Lines changed: 208 additions & 71 deletions

File tree

‎docs/adr/0012-interactive-replay.md‎

Lines changed: 98 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -953,17 +953,38 @@ The tombstone itself expires after its bounded window, after which the key is fu
953953
> behavior the default, superseding stage 1's "opt out by hand" guidance with "opt in by hand" for the one
954954
> case that needs it.
955955
>
956-
> **1. Which read-only commands are excluded.** Exactly the enumerated set stage 1 already named:
957-
> `snapshot`, `get`, `is`, and a **read-only** `find` sub-action (`exists`, `wait`, `get_text`, `get_attrs`).
958-
> A **mutating** `find … click|fill|focus|type` is not in this set — it is a corrective action like `press`/
959-
> `fill` and always records. The top-level `wait` command is deliberately **not** in this set either: it is
960-
> flow timing/synchronisation (pacing the script), not observation, so it must keep recording unconditionally
961-
> even inside a repair segment — a healed script that silently dropped its own timing steps would replay
962-
> differently from what actually happened.
956+
> **1. Which observations are excluded — a PROVENANCE rule, not a command-class rule.** The exclusion
957+
> applies to an observation that is BOTH:
963958
>
964-
> **2. The corrective-read trap, and why blanket exclusion by command class is unsafe.** "Exclude every
965-
> read from a repair segment" is not a safe rule on its own: a divergence's **correction** can itself be a
966-
> read. In the wave-3 E3 drift, the diverged step **was** a `get`, so its corrective replacement is a
959+
> - **observation-only by command**: `snapshot`, `get`, `is`, or a **read-only** `find` sub-action
960+
> (`exists`, `wait`, `get_text`, `get_attrs`). A **mutating** `find … click|fill|focus|type` is not in
961+
> this set — it is a corrective action like `press`/`fill` and always records. The top-level `wait`
962+
> command is deliberately **not** in this set either: it is flow timing/synchronisation (pacing the
963+
> script), not observation, so it must keep recording unconditionally — a healed script that silently
964+
> dropped its own timing steps would replay differently from what actually happened; **AND**
965+
> - **dispatched OUT OF BAND**: typed by the agent mid-repair, NOT replayed from the `.ad` plan under
966+
> repair.
967+
>
968+
> The second clause is load-bearing and is the rule's real discriminator. Replayed plan steps dispatch
969+
> through the ordinary request path and land in `session.actions` like any other action, and the healed
970+
> script IS that slice (`buildOptimizedActions` over `session.actions.slice(saveScriptBoundary)`). So a
971+
> command-class-only exclusion would replay an authored `is visible` assertion and then **silently drop it
972+
> from its own healed script** — the repaired flow would quietly stop checking what the original checked,
973+
> which for a deterministic QA suite is the worst possible failure mode. **Planned prefix/suffix
974+
> observations must survive automatically; a user must never have to annotate authored `.ad` steps with
975+
> `--record` to keep them.** An authored observation and an interactive diagnostic read are the same
976+
> command, so only provenance separates them.
977+
>
978+
> Provenance travels as `internal.replayPlanStep`, stamped by `invokeResolvedReplayAction`
979+
> (`src/daemon/handlers/session-replay-action-runtime.ts`) — the single point every plan step is
980+
> dispatched, so the marker covers annotated and unannotated steps alike. `internal` is a daemon-only
981+
> channel (`toDaemonRequest`, `src/daemon/server/http-server.ts`, never copies it off the wire), so
982+
> authored provenance cannot be spoofed by a client; this is the same channel `replayTargetGuard` and
983+
> `findResolvedTarget` already use. The rule is written once, in `isInteractiveObservation`
984+
> (`src/daemon/session-action-recorder.ts`), and both recording call sites consume it.
985+
>
986+
> **2. The corrective-read trap: why even out-of-band reads need an opt-in.** Excluding every out-of-band
987+
> read is still not safe on its own: a divergence's **correction** can itself be a read. In the wave-3 E3 drift, the diverged step **was** a `get`, so its corrective replacement is a
967988
> recorded `get` that **must** land in the heal — a blanket read-exclusion would silently drop the very
968989
> step being repaired, producing an incomplete heal with no signal anything was missing (the same class of
969990
> silent-failure mode decision 3 exists to close for target-binding, and decision 6 R4 exists to close for
@@ -975,36 +996,57 @@ The tombstone itself expires after its bounded window, after which the key is fu
975996
> opt-in per action**: `--record` (below) marks the ONE read that must count, independent of its position in
976997
> the repair segment.
977998
>
978-
> **3. Explicit recording remains possible.** `--record` is a new, standalone flag — distinct from
979-
> `saveScript`, and the mirror image of `--no-record` — that forces the action it is passed on into the heal
980-
> even though its command is observation-only. It is a no-op outside a repair-armed session (there is no
981-
> default exclusion to override) and a no-op on any non-observation-only command (those already always
982-
> record). `--record` and `--no-record` express opposite intents for the same action and are **mutually
983-
> exclusive**: passing both is rejected as `INVALID_ARGS` before the command runs, uniformly for every
984-
> surface, rather than letting one silently win.
999+
> **3. Explicit recording remains possible.** `--record` forces an out-of-band observation into the heal
1000+
> even though its command is observation-only. It exists for exactly one job: an interactive corrective
1001+
> read the agent is deliberately inserting into the healed script. It is a no-op outside a repair-armed
1002+
> session (there is no default exclusion to override), and it is **not** needed for authored plan steps —
1003+
> those survive on provenance alone (rule 1).
1004+
>
1005+
> `--record` is **not a common flag**, unlike `--no-record`. `--no-record` applies to every recordable
1006+
> command, mutations included, so it stays universal
1007+
> (`COMMON_COMMAND_SUPPORTED_FLAG_KEYS`). `--record` only means something for a command the exclusion can
1008+
> drop, so it is scoped **statically** to `snapshot`, `get`, and `is` through each command schema's
1009+
> `allowedFlags` — accepting it on `press`/`fill`/`click` would be misleading, and the grammar rejects
1010+
> unsupported uses before projection. `find` is the one command whose observe-vs-mutate split is a
1011+
> POSITIONAL rather than the command name, so it cannot be settled statically: it allows `--record`
1012+
> in the grammar and validates **dynamically** in the daemon (`handleFindCommands`), rejecting
1013+
> `--record` on a mutating `find … click|fill|focus|type` with `INVALID_ARGS` before any device work. Both
1014+
> halves read one shared predicate (`isReadOnlyFindAction`, `src/selectors/find.ts`), so the routing and
1015+
> the validation can never disagree about which sub-actions observe.
1016+
>
1017+
> `--record` and `--no-record` express opposite intents for the same action and are **mutually exclusive**:
1018+
> passing both is rejected as `INVALID_ARGS` before the command runs, uniformly for every surface, rather
1019+
> than letting one silently win.
9851020
>
9861021
> **4. How the rule projects across CLI/Node/MCP surfaces.** The exclusion is enforced at a single
9871022
> daemon-side choke point — `isExcludedRepairSegmentObservation` in `recordActionEntry`
9881023
> (`src/daemon/session-action-recorder.ts`), gated on `session.saveScriptBoundary !== undefined` (set ONLY
9891024
> by a repair-armed `replay --save-script`; an ordinary, non-repair `open --save-script`/`close --save-script`
9901025
> authoring recording never sets it — see decision 6's "Scope" note above) and on the entry's
991-
> `observationOnly` marker (set by the caller — `get`/`is`/a read-only `find` in
992-
> `src/daemon/selector-recording.ts`, `snapshot` in `src/daemon/snapshot-runtime.ts` — never inferred from
993-
> `command`/`positionals`, since a mutating `find … click` shares the literal string `command: 'find'` with
994-
> its read-only sibling). Every surface reaches this same function because every surface's request reaches
995-
> the same daemon: the CLI parses `--record`/`--no-record` into `CommandFlags` (`src/contracts/cli-flags.ts`,
996-
> `src/commands/cli-grammar/flag-definitions-action.ts`, `src/commands/cli-grammar/flag-groups.ts`'s
997-
> `COMMON_COMMAND_SUPPORTED_FLAG_KEYS` — so both flags are syntactically valid on every command, exactly like
998-
> `--no-record` already was); the Node client threads the same-named option through
999-
> `buildRequestFlags`/`buildFlags` (`src/commands/command-flags.ts`) from `InternalRequestOptions` and
1000-
> `AppOpenOptions` (`src/client/client-types.ts`) and from each typed per-command Options type
1001-
> (`GetOptions`/`IsOptions`/`FindOptions`/`CaptureSnapshotOptions`); and the MCP tool schema exposes `record`/
1002-
> `noRecord` fields on the `get`/`is`/`find`/`snapshot` tools (`src/commands/interaction/metadata.ts`,
1003-
> `src/commands/capture/snapshot.ts`) that flow through the same `CommandMetadata.inputSchema` +
1004-
> `readMetadataCommandFlags` path every other typed field already uses. `management/app.ts` mirrors both
1005-
> flags onto `open` for symmetry with `--no-record`'s existing presence there, though neither has an effect
1006-
> on a command that is never observation-only. No surface has a separate recording contract; all of them
1007-
> hit the identical daemon-side default.
1026+
> `interactiveObservation` marker, which both recording call sites
1027+
> (`src/daemon/selector-recording.ts` for `get`/`is`/read-only `find`, `src/daemon/snapshot-runtime.ts` for
1028+
> `snapshot`) compute from the shared `isInteractiveObservation` predicate rather than each re-deriving it.
1029+
> Every surface reaches this same function because every surface's request reaches the same daemon.
1030+
>
1031+
> The flags project asymmetrically, matching their different scopes:
1032+
>
1033+
> - `--no-record` stays a common flag: its key is in `COMMON_COMMAND_SUPPORTED_FLAG_KEYS`, and every
1034+
> recordable reader forwards it through the shared `noRecordInputFromFlags` helper
1035+
> (`src/commands/cli-grammar/common.ts`, per #1304).
1036+
> - `--record` is scoped: its key is NOT common, it appears in `allowedFlags` only for `snapshot`/`get`/`is`
1037+
> (plus `find`, which validates dynamically per rule 3), and only those readers spread
1038+
> `observationRecordInputFromFlags`. Two named helpers rather than one helper with an `allowRecord`
1039+
> policy argument: the capability is then the helper's NAME, so a mutating reader physically cannot
1040+
> forward `--record`, whereas a policy argument would let a future mutating reader opt in by flipping a
1041+
> literal with no schema change — the fail-open the split exists to prevent.
1042+
>
1043+
> From there both flags ride the ordinary typed path: the Node client threads them through
1044+
> `buildRequestFlags`/`buildFlags` (`src/commands/command-flags.ts`) from `InternalRequestOptions` and each
1045+
> typed per-command Options type (`GetOptions`/`IsOptions`/`FindOptions`/`CaptureSnapshotOptions`), and the
1046+
> MCP tool schema exposes `record`/`noRecord` on the `get`/`is`/`find`/`snapshot` tools
1047+
> (`src/commands/interaction/metadata.ts`, `src/commands/capture/snapshot.ts`) through the same
1048+
> `CommandMetadata.inputSchema` + `readMetadataCommandFlags` path every other typed field already uses. No
1049+
> surface has a separate recording contract; all of them hit the identical daemon-side default.
10081050
>
10091051
> **5. The fail-loud guard is the EXISTING resume watermark, not new state.** Because the exclusion happens
10101052
> at the same point `--no-record` is enforced — before an action ever reaches `session.actions` — an excluded
@@ -1019,13 +1061,16 @@ The tombstone itself expires after its bounded window, after which the key is fu
10191061
> action was recorded in this repair segment; press the correct control via a blessed `@ref` … — or, if your
10201062
> corrective action was a read, re-run it with `--record` … — before resuming."
10211063
>
1022-
> **Regression coverage** proves: a diagnostic read inside an armed repair segment is omitted from the
1023-
> healed script by default; a corrective action remains for both a mutating press/fill (unaffected — never
1024-
> observation-only) and a `--record`ed read (the diverged-step-was-a-`get` case); the resulting healed script
1025-
> replays cleanly in a fresh session; the empty-segment guard refuses a blind resume with the actionable
1026-
> `--record` hint; and ordinary, non-repair `open --save-script` authoring recording is completely unchanged
1027-
> — a read in a fresh authoring session still records with no flag needed, because `session.saveScriptBoundary`
1028-
> is never set there.
1064+
> **Regression coverage** proves: an out-of-band diagnostic read inside an armed repair segment is omitted
1065+
> from the healed script by default; an **authored** `is visible` plan step in the same repair, carrying no
1066+
> `--record`, **survives** into the heal (the provenance rule — this fails if the exclusion is
1067+
> command-class-only); a corrective action remains for both a mutating press/fill (unaffected — never
1068+
> observation-only) and a `--record`ed read (the diverged-step-was-a-`get` case); the resulting healed
1069+
> script replays cleanly in a fresh session; `--record` on a mutating `find … click` is refused with
1070+
> `INVALID_ARGS`; the empty-segment guard refuses a blind resume with the actionable `--record` hint; and
1071+
> ordinary, non-repair `open --save-script` authoring recording is completely unchanged — a read in a fresh
1072+
> authoring session still records with no flag needed, because `session.saveScriptBoundary` is never set
1073+
> there.
10291074
10301075
## Consequences
10311076

@@ -1138,15 +1183,19 @@ each states its dependencies explicitly.
11381183
pre-existing target — complete or partial, default or explicit path alike, and uniformly for an
11391184
ordinary non-repair recording's target too, since the publish primitive is shared — never overwriting
11401185
one; see "Scope" under Decision 6 above).
1141-
10. **Repair-segment default exclusion of observation-only commands, and the `--record` opt-in**
1186+
10. **Repair-segment default exclusion of out-of-band observations, and the `--record` opt-in**
11421187
(decision 6 amendment) — **SHIPPED (#1271 stage 2)**; depends on 9 (`session.saveScriptBoundary`/
1143-
`repairSessionHeld` are the signals the exclusion and its guidance key off). Adds: default exclusion of
1144-
`snapshot`/`get`/`is`/a read-only `find` from `session.actions` while repair-armed
1145-
(`isExcludedRepairSegmentObservation`, `session-action-recorder.ts`); the `--record` opt-in, plumbed
1146-
across CLI/Node client/MCP identically to `--no-record`, and rejected together with it as
1147-
`INVALID_ARGS`; and an updated `describeUnperformedRecordAndHeal` message naming `--record` for the
1148-
corrective-read case. Stage 1 (#1287) shipped interim `--no-record` guidance only, gated on this same
1149-
`repairSessionHeld` signal.
1188+
`repairSessionHeld` are the signals the exclusion and its guidance key off) and on #1304 (the shared
1189+
reader helper `--record`'s scoped sibling is split from). Adds: the `internal.replayPlanStep`
1190+
provenance marker stamped by `invokeResolvedReplayAction` (`session-replay-action-runtime.ts`);
1191+
default exclusion of OUT-OF-BAND `snapshot`/`get`/`is`/read-only `find` from `session.actions` while
1192+
repair-armed, keyed off that provenance via `isInteractiveObservation`/
1193+
`isExcludedRepairSegmentObservation` (`session-action-recorder.ts`), so authored plan steps survive
1194+
their own heal automatically; the `--record` opt-in, statically scoped to `snapshot`/`get`/`is` and
1195+
dynamically validated for `find`, rejected together with `--no-record` as `INVALID_ARGS`; and an
1196+
updated `describeUnperformedRecordAndHeal` message naming `--record` for the corrective-read case.
1197+
Stage 1 (#1287) shipped interim `--no-record` guidance only, gated on this same `repairSessionHeld`
1198+
signal.
11501199

11511200
## Migration progress
11521201

0 commit comments

Comments
 (0)