Skip to content

Commit e1f2fed

Browse files
committed
fix(replay): key the repair-segment exclusion on provenance, scope --record (#1271 review)
Addresses the maintainer review on #1303. P1 — the exclusion dropped PLANNED reads from the heal. It discriminated by command class, but the real discriminator is provenance. Replayed plan steps dispatch through the ordinary request path, so an authored get/is/find step hit the same recordIfSession -> exclusion path as an interactive read and never reached session.actions -- and the heal IS session.actions.slice(boundary). A repaired flow therefore replayed its authored `is visible` assertion and then silently dropped it from its own healed script: the heal quietly stops checking what it used to check, which for a 10x-QA-replay suite is the worst failure mode. Fix: an explicit provenance marker, not a heuristic. `internal.replayPlanStep` is stamped by invokeResolvedReplayAction -- the single point every plan step is dispatched, so it covers annotated and unannotated steps alike. `internal` is daemon-only (toDaemonRequest never copies it off the wire), so authored provenance cannot be spoofed; same channel as replayTargetGuard. The rule now lives once in isInteractiveObservation and both recording call sites consume it, so the mock fixture uses the production classifier instead of mirroring it. Planned observations survive automatically -- users never annotate their own .ad steps. --record is no longer a common flag: removed from COMMON_COMMAND_SUPPORTED_FLAG_KEYS, statically scoped via allowedFlags to snapshot/get/is, and validated dynamically for find (read-only allows; a mutating find click|fill|focus|type is INVALID_ARGS before any device work, sharing one isReadOnlyFindAction predicate with the read-only routing so the two cannot disagree). --no-record stays shared -- it applies to every recordable command. Removed from `open`, which is never observation-only. Rebased onto #1304 and dropped the four hand-rolled reader blocks. Split its helper rather than broadening it: noRecordInputFromFlags (all 13 readers) + observationRecordInputFromFlags (snapshot/get/is/find only). Two named helpers over one `allowRecord` policy arg -- the capability is then the helper's NAME, so a mutating reader physically cannot forward --record, whereas a policy arg would let a future mutating reader opt in by flipping a literal with no schema change. ADR-0012 decision 6 now states the provenance rule, not a command-class rule. The scenario gates the P1: its authored step is a distinguishable `is visible`, and it fails without the provenance check ("the authored 'is visible' step must survive the heal").
1 parent 78126f5 commit e1f2fed

10 files changed

Lines changed: 208 additions & 71 deletions

File tree

‎docs/adr/0012-interactive-replay.md‎

Lines changed: 98 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -993,17 +993,38 @@ The tombstone itself expires after its bounded window, after which the key is fu
993993
> behavior the default, superseding stage 1's "opt out by hand" guidance with "opt in by hand" for the one
994994
> case that needs it.
995995
>
996-
> **1. Which read-only commands are excluded.** Exactly the enumerated set stage 1 already named:
997-
> `snapshot`, `get`, `is`, and a **read-only** `find` sub-action (`exists`, `wait`, `get_text`, `get_attrs`).
998-
> A **mutating** `find … click|fill|focus|type` is not in this set — it is a corrective action like `press`/
999-
> `fill` and always records. The top-level `wait` command is deliberately **not** in this set either: it is
1000-
> flow timing/synchronisation (pacing the script), not observation, so it must keep recording unconditionally
1001-
> even inside a repair segment — a healed script that silently dropped its own timing steps would replay
1002-
> differently from what actually happened.
996+
> **1. Which observations are excluded — a PROVENANCE rule, not a command-class rule.** The exclusion
997+
> applies to an observation that is BOTH:
1003998
>
1004-
> **2. The corrective-read trap, and why blanket exclusion by command class is unsafe.** "Exclude every
1005-
> read from a repair segment" is not a safe rule on its own: a divergence's **correction** can itself be a
1006-
> read. In the wave-3 E3 drift, the diverged step **was** a `get`, so its corrective replacement is a
999+
> - **observation-only by command**: `snapshot`, `get`, `is`, or a **read-only** `find` sub-action
1000+
> (`exists`, `wait`, `get_text`, `get_attrs`). A **mutating** `find … click|fill|focus|type` is not in
1001+
> this set — it is a corrective action like `press`/`fill` and always records. The top-level `wait`
1002+
> command is deliberately **not** in this set either: it is flow timing/synchronisation (pacing the
1003+
> script), not observation, so it must keep recording unconditionally — a healed script that silently
1004+
> dropped its own timing steps would replay differently from what actually happened; **AND**
1005+
> - **dispatched OUT OF BAND**: typed by the agent mid-repair, NOT replayed from the `.ad` plan under
1006+
> repair.
1007+
>
1008+
> The second clause is load-bearing and is the rule's real discriminator. Replayed plan steps dispatch
1009+
> through the ordinary request path and land in `session.actions` like any other action, and the healed
1010+
> script IS that slice (`buildOptimizedActions` over `session.actions.slice(saveScriptBoundary)`). So a
1011+
> command-class-only exclusion would replay an authored `is visible` assertion and then **silently drop it
1012+
> from its own healed script** — the repaired flow would quietly stop checking what the original checked,
1013+
> which for a deterministic QA suite is the worst possible failure mode. **Planned prefix/suffix
1014+
> observations must survive automatically; a user must never have to annotate authored `.ad` steps with
1015+
> `--record` to keep them.** An authored observation and an interactive diagnostic read are the same
1016+
> command, so only provenance separates them.
1017+
>
1018+
> Provenance travels as `internal.replayPlanStep`, stamped by `invokeResolvedReplayAction`
1019+
> (`src/daemon/handlers/session-replay-action-runtime.ts`) — the single point every plan step is
1020+
> dispatched, so the marker covers annotated and unannotated steps alike. `internal` is a daemon-only
1021+
> channel (`toDaemonRequest`, `src/daemon/server/http-server.ts`, never copies it off the wire), so
1022+
> authored provenance cannot be spoofed by a client; this is the same channel `replayTargetGuard` and
1023+
> `findResolvedTarget` already use. The rule is written once, in `isInteractiveObservation`
1024+
> (`src/daemon/session-action-recorder.ts`), and both recording call sites consume it.
1025+
>
1026+
> **2. The corrective-read trap: why even out-of-band reads need an opt-in.** Excluding every out-of-band
1027+
> read is still not safe on its own: a divergence's **correction** can itself be a read. In the wave-3 E3 drift, the diverged step **was** a `get`, so its corrective replacement is a
10071028
> recorded `get` that **must** land in the heal — a blanket read-exclusion would silently drop the very
10081029
> step being repaired, producing an incomplete heal with no signal anything was missing (the same class of
10091030
> silent-failure mode decision 3 exists to close for target-binding, and decision 6 R4 exists to close for
@@ -1015,36 +1036,57 @@ The tombstone itself expires after its bounded window, after which the key is fu
10151036
> opt-in per action**: `--record` (below) marks the ONE read that must count, independent of its position in
10161037
> the repair segment.
10171038
>
1018-
> **3. Explicit recording remains possible.** `--record` is a new, standalone flag — distinct from
1019-
> `saveScript`, and the mirror image of `--no-record` — that forces the action it is passed on into the heal
1020-
> even though its command is observation-only. It is a no-op outside a repair-armed session (there is no
1021-
> default exclusion to override) and a no-op on any non-observation-only command (those already always
1022-
> record). `--record` and `--no-record` express opposite intents for the same action and are **mutually
1023-
> exclusive**: passing both is rejected as `INVALID_ARGS` before the command runs, uniformly for every
1024-
> surface, rather than letting one silently win.
1039+
> **3. Explicit recording remains possible.** `--record` forces an out-of-band observation into the heal
1040+
> even though its command is observation-only. It exists for exactly one job: an interactive corrective
1041+
> read the agent is deliberately inserting into the healed script. It is a no-op outside a repair-armed
1042+
> session (there is no default exclusion to override), and it is **not** needed for authored plan steps —
1043+
> those survive on provenance alone (rule 1).
1044+
>
1045+
> `--record` is **not a common flag**, unlike `--no-record`. `--no-record` applies to every recordable
1046+
> command, mutations included, so it stays universal
1047+
> (`COMMON_COMMAND_SUPPORTED_FLAG_KEYS`). `--record` only means something for a command the exclusion can
1048+
> drop, so it is scoped **statically** to `snapshot`, `get`, and `is` through each command schema's
1049+
> `allowedFlags` — accepting it on `press`/`fill`/`click` would be misleading, and the grammar rejects
1050+
> unsupported uses before projection. `find` is the one command whose observe-vs-mutate split is a
1051+
> POSITIONAL rather than the command name, so it cannot be settled statically: it allows `--record`
1052+
> in the grammar and validates **dynamically** in the daemon (`handleFindCommands`), rejecting
1053+
> `--record` on a mutating `find … click|fill|focus|type` with `INVALID_ARGS` before any device work. Both
1054+
> halves read one shared predicate (`isReadOnlyFindAction`, `src/selectors/find.ts`), so the routing and
1055+
> the validation can never disagree about which sub-actions observe.
1056+
>
1057+
> `--record` and `--no-record` express opposite intents for the same action and are **mutually exclusive**:
1058+
> passing both is rejected as `INVALID_ARGS` before the command runs, uniformly for every surface, rather
1059+
> than letting one silently win.
10251060
>
10261061
> **4. How the rule projects across CLI/Node/MCP surfaces.** The exclusion is enforced at a single
10271062
> daemon-side choke point — `isExcludedRepairSegmentObservation` in `recordActionEntry`
10281063
> (`src/daemon/session-action-recorder.ts`), gated on `session.saveScriptBoundary !== undefined` (set ONLY
10291064
> by a repair-armed `replay --save-script`; an ordinary, non-repair `open --save-script`/`close --save-script`
10301065
> authoring recording never sets it — see decision 6's "Scope" note above) and on the entry's
1031-
> `observationOnly` marker (set by the caller — `get`/`is`/a read-only `find` in
1032-
> `src/daemon/selector-recording.ts`, `snapshot` in `src/daemon/snapshot-runtime.ts` — never inferred from
1033-
> `command`/`positionals`, since a mutating `find … click` shares the literal string `command: 'find'` with
1034-
> its read-only sibling). Every surface reaches this same function because every surface's request reaches
1035-
> the same daemon: the CLI parses `--record`/`--no-record` into `CommandFlags` (`src/contracts/cli-flags.ts`,
1036-
> `src/commands/cli-grammar/flag-definitions-action.ts`, `src/commands/cli-grammar/flag-groups.ts`'s
1037-
> `COMMON_COMMAND_SUPPORTED_FLAG_KEYS` — so both flags are syntactically valid on every command, exactly like
1038-
> `--no-record` already was); the Node client threads the same-named option through
1039-
> `buildRequestFlags`/`buildFlags` (`src/commands/command-flags.ts`) from `InternalRequestOptions` and
1040-
> `AppOpenOptions` (`src/client/client-types.ts`) and from each typed per-command Options type
1041-
> (`GetOptions`/`IsOptions`/`FindOptions`/`CaptureSnapshotOptions`); and the MCP tool schema exposes `record`/
1042-
> `noRecord` fields on the `get`/`is`/`find`/`snapshot` tools (`src/commands/interaction/metadata.ts`,
1043-
> `src/commands/capture/snapshot.ts`) that flow through the same `CommandMetadata.inputSchema` +
1044-
> `readMetadataCommandFlags` path every other typed field already uses. `management/app.ts` mirrors both
1045-
> flags onto `open` for symmetry with `--no-record`'s existing presence there, though neither has an effect
1046-
> on a command that is never observation-only. No surface has a separate recording contract; all of them
1047-
> hit the identical daemon-side default.
1066+
> `interactiveObservation` marker, which both recording call sites
1067+
> (`src/daemon/selector-recording.ts` for `get`/`is`/read-only `find`, `src/daemon/snapshot-runtime.ts` for
1068+
> `snapshot`) compute from the shared `isInteractiveObservation` predicate rather than each re-deriving it.
1069+
> Every surface reaches this same function because every surface's request reaches the same daemon.
1070+
>
1071+
> The flags project asymmetrically, matching their different scopes:
1072+
>
1073+
> - `--no-record` stays a common flag: its key is in `COMMON_COMMAND_SUPPORTED_FLAG_KEYS`, and every
1074+
> recordable reader forwards it through the shared `noRecordInputFromFlags` helper
1075+
> (`src/commands/cli-grammar/common.ts`, per #1304).
1076+
> - `--record` is scoped: its key is NOT common, it appears in `allowedFlags` only for `snapshot`/`get`/`is`
1077+
> (plus `find`, which validates dynamically per rule 3), and only those readers spread
1078+
> `observationRecordInputFromFlags`. Two named helpers rather than one helper with an `allowRecord`
1079+
> policy argument: the capability is then the helper's NAME, so a mutating reader physically cannot
1080+
> forward `--record`, whereas a policy argument would let a future mutating reader opt in by flipping a
1081+
> literal with no schema change — the fail-open the split exists to prevent.
1082+
>
1083+
> From there both flags ride the ordinary typed path: the Node client threads them through
1084+
> `buildRequestFlags`/`buildFlags` (`src/commands/command-flags.ts`) from `InternalRequestOptions` and each
1085+
> typed per-command Options type (`GetOptions`/`IsOptions`/`FindOptions`/`CaptureSnapshotOptions`), and the
1086+
> MCP tool schema exposes `record`/`noRecord` on the `get`/`is`/`find`/`snapshot` tools
1087+
> (`src/commands/interaction/metadata.ts`, `src/commands/capture/snapshot.ts`) through the same
1088+
> `CommandMetadata.inputSchema` + `readMetadataCommandFlags` path every other typed field already uses. No
1089+
> surface has a separate recording contract; all of them hit the identical daemon-side default.
10481090
>
10491091
> **5. The fail-loud guard is the EXISTING resume watermark, not new state.** Because the exclusion happens
10501092
> at the same point `--no-record` is enforced — before an action ever reaches `session.actions` — an excluded
@@ -1059,13 +1101,16 @@ The tombstone itself expires after its bounded window, after which the key is fu
10591101
> action was recorded in this repair segment; press the correct control via a blessed `@ref` … — or, if your
10601102
> corrective action was a read, re-run it with `--record` … — before resuming."
10611103
>
1062-
> **Regression coverage** proves: a diagnostic read inside an armed repair segment is omitted from the
1063-
> healed script by default; a corrective action remains for both a mutating press/fill (unaffected — never
1064-
> observation-only) and a `--record`ed read (the diverged-step-was-a-`get` case); the resulting healed script
1065-
> replays cleanly in a fresh session; the empty-segment guard refuses a blind resume with the actionable
1066-
> `--record` hint; and ordinary, non-repair `open --save-script` authoring recording is completely unchanged
1067-
> — a read in a fresh authoring session still records with no flag needed, because `session.saveScriptBoundary`
1068-
> is never set there.
1104+
> **Regression coverage** proves: an out-of-band diagnostic read inside an armed repair segment is omitted
1105+
> from the healed script by default; an **authored** `is visible` plan step in the same repair, carrying no
1106+
> `--record`, **survives** into the heal (the provenance rule — this fails if the exclusion is
1107+
> command-class-only); a corrective action remains for both a mutating press/fill (unaffected — never
1108+
> observation-only) and a `--record`ed read (the diverged-step-was-a-`get` case); the resulting healed
1109+
> script replays cleanly in a fresh session; `--record` on a mutating `find … click` is refused with
1110+
> `INVALID_ARGS`; the empty-segment guard refuses a blind resume with the actionable `--record` hint; and
1111+
> ordinary, non-repair `open --save-script` authoring recording is completely unchanged — a read in a fresh
1112+
> authoring session still records with no flag needed, because `session.saveScriptBoundary` is never set
1113+
> there.
10691114
10701115
## Consequences
10711116

@@ -1178,15 +1223,19 @@ each states its dependencies explicitly.
11781223
pre-existing target — complete or partial, default or explicit path alike, and uniformly for an
11791224
ordinary non-repair recording's target too, since the publish primitive is shared — never overwriting
11801225
one; see "Scope" under Decision 6 above).
1181-
10. **Repair-segment default exclusion of observation-only commands, and the `--record` opt-in**
1226+
10. **Repair-segment default exclusion of out-of-band observations, and the `--record` opt-in**
11821227
(decision 6 amendment) — **SHIPPED (#1271 stage 2)**; depends on 9 (`session.saveScriptBoundary`/
1183-
`repairSessionHeld` are the signals the exclusion and its guidance key off). Adds: default exclusion of
1184-
`snapshot`/`get`/`is`/a read-only `find` from `session.actions` while repair-armed
1185-
(`isExcludedRepairSegmentObservation`, `session-action-recorder.ts`); the `--record` opt-in, plumbed
1186-
across CLI/Node client/MCP identically to `--no-record`, and rejected together with it as
1187-
`INVALID_ARGS`; and an updated `describeUnperformedRecordAndHeal` message naming `--record` for the
1188-
corrective-read case. Stage 1 (#1287) shipped interim `--no-record` guidance only, gated on this same
1189-
`repairSessionHeld` signal.
1228+
`repairSessionHeld` are the signals the exclusion and its guidance key off) and on #1304 (the shared
1229+
reader helper `--record`'s scoped sibling is split from). Adds: the `internal.replayPlanStep`
1230+
provenance marker stamped by `invokeResolvedReplayAction` (`session-replay-action-runtime.ts`);
1231+
default exclusion of OUT-OF-BAND `snapshot`/`get`/`is`/read-only `find` from `session.actions` while
1232+
repair-armed, keyed off that provenance via `isInteractiveObservation`/
1233+
`isExcludedRepairSegmentObservation` (`session-action-recorder.ts`), so authored plan steps survive
1234+
their own heal automatically; the `--record` opt-in, statically scoped to `snapshot`/`get`/`is` and
1235+
dynamically validated for `find`, rejected together with `--no-record` as `INVALID_ARGS`; and an
1236+
updated `describeUnperformedRecordAndHeal` message naming `--record` for the corrective-read case.
1237+
Stage 1 (#1287) shipped interim `--no-record` guidance only, gated on this same `repairSessionHeld`
1238+
signal.
11901239

11911240
## Migration progress
11921241

0 commit comments

Comments
 (0)