You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit e1f2fed
Browse filesBrowse the repository at this point in the historyBrowse files
fix(replay): key the repair-segment exclusion on provenance, scope --record (#1271 review)
Addresses the maintainer review on #1303.
P1 — the exclusion dropped PLANNED reads from the heal. It discriminated by
command class, but the real discriminator is provenance. Replayed plan steps
dispatch through the ordinary request path, so an authored get/is/find step hit
the same recordIfSession -> exclusion path as an interactive read and never
reached session.actions -- and the heal IS session.actions.slice(boundary). A
repaired flow therefore replayed its authored `is visible` assertion and then
silently dropped it from its own healed script: the heal quietly stops checking
what it used to check, which for a 10x-QA-replay suite is the worst failure
mode.
Fix: an explicit provenance marker, not a heuristic. `internal.replayPlanStep`
is stamped by invokeResolvedReplayAction -- the single point every plan step is
dispatched, so it covers annotated and unannotated steps alike. `internal` is
daemon-only (toDaemonRequest never copies it off the wire), so authored
provenance cannot be spoofed; same channel as replayTargetGuard. The rule now
lives once in isInteractiveObservation and both recording call sites consume it,
so the mock fixture uses the production classifier instead of mirroring it.
Planned observations survive automatically -- users never annotate their own .ad
steps.
--record is no longer a common flag: removed from
COMMON_COMMAND_SUPPORTED_FLAG_KEYS, statically scoped via allowedFlags to
snapshot/get/is, and validated dynamically for find (read-only allows; a
mutating find click|fill|focus|type is INVALID_ARGS before any device work,
sharing one isReadOnlyFindAction predicate with the read-only routing so the two
cannot disagree). --no-record stays shared -- it applies to every recordable
command. Removed from `open`, which is never observation-only.
Rebased onto #1304 and dropped the four hand-rolled reader blocks. Split its
helper rather than broadening it: noRecordInputFromFlags (all 13 readers) +
observationRecordInputFromFlags (snapshot/get/is/find only). Two named helpers
over one `allowRecord` policy arg -- the capability is then the helper's NAME, so
a mutating reader physically cannot forward --record, whereas a policy arg would
let a future mutating reader opt in by flipping a literal with no schema change.
ADR-0012 decision 6 now states the provenance rule, not a command-class rule.
The scenario gates the P1: its authored step is a distinguishable `is visible`,
and it fails without the provenance check ("the authored 'is visible' step must
survive the heal").
Copy file name to clipboardExpand all lines: docs/adr/0012-interactive-replay.md
+98-49Lines changed: 98 additions & 49 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -993,17 +993,38 @@ The tombstone itself expires after its bounded window, after which the key is fu
993
993
> behavior the default, superseding stage 1's "opt out by hand" guidance with "opt in by hand" for the one
994
994
> case that needs it.
995
995
>
996
-
> **1. Which read-only commands are excluded.** Exactly the enumerated set stage 1 already named:
997
-
> `snapshot`, `get`, `is`, and a **read-only**`find` sub-action (`exists`, `wait`, `get_text`, `get_attrs`).
998
-
> A **mutating**`find … click|fill|focus|type` is not in this set — it is a corrective action like `press`/
999
-
> `fill` and always records. The top-level `wait` command is deliberately **not** in this set either: it is
1000
-
> flow timing/synchronisation (pacing the script), not observation, so it must keep recording unconditionally
1001
-
> even inside a repair segment — a healed script that silently dropped its own timing steps would replay
1002
-
> differently from what actually happened.
996
+
> **1. Which observations are excluded — a PROVENANCE rule, not a command-class rule.** The exclusion
997
+
> applies to an observation that is BOTH:
1003
998
>
1004
-
> **2. The corrective-read trap, and why blanket exclusion by command class is unsafe.** "Exclude every
1005
-
> read from a repair segment" is not a safe rule on its own: a divergence's **correction** can itself be a
1006
-
> read. In the wave-3 E3 drift, the diverged step **was** a `get`, so its corrective replacement is a
999
+
> -**observation-only by command**: `snapshot`, `get`, `is`, or a **read-only**`find` sub-action
1000
+
> (`exists`, `wait`, `get_text`, `get_attrs`). A **mutating**`find … click|fill|focus|type` is not in
1001
+
> this set — it is a corrective action like `press`/`fill` and always records. The top-level `wait`
1002
+
> command is deliberately **not** in this set either: it is flow timing/synchronisation (pacing the
1003
+
> script), not observation, so it must keep recording unconditionally — a healed script that silently
1004
+
> dropped its own timing steps would replay differently from what actually happened; **AND**
1005
+
> -**dispatched OUT OF BAND**: typed by the agent mid-repair, NOT replayed from the `.ad` plan under
1006
+
> repair.
1007
+
>
1008
+
> The second clause is load-bearing and is the rule's real discriminator. Replayed plan steps dispatch
1009
+
> through the ordinary request path and land in `session.actions` like any other action, and the healed
1010
+
> script IS that slice (`buildOptimizedActions` over `session.actions.slice(saveScriptBoundary)`). So a
1011
+
> command-class-only exclusion would replay an authored `is visible` assertion and then **silently drop it
1012
+
> from its own healed script** — the repaired flow would quietly stop checking what the original checked,
1013
+
> which for a deterministic QA suite is the worst possible failure mode. **Planned prefix/suffix
1014
+
> observations must survive automatically; a user must never have to annotate authored `.ad` steps with
1015
+
> `--record` to keep them.** An authored observation and an interactive diagnostic read are the same
1016
+
> command, so only provenance separates them.
1017
+
>
1018
+
> Provenance travels as `internal.replayPlanStep`, stamped by `invokeResolvedReplayAction`
1019
+
> (`src/daemon/handlers/session-replay-action-runtime.ts`) — the single point every plan step is
1020
+
> dispatched, so the marker covers annotated and unannotated steps alike. `internal` is a daemon-only
1021
+
> channel (`toDaemonRequest`, `src/daemon/server/http-server.ts`, never copies it off the wire), so
1022
+
> authored provenance cannot be spoofed by a client; this is the same channel `replayTargetGuard` and
1023
+
> `findResolvedTarget` already use. The rule is written once, in `isInteractiveObservation`
1024
+
> (`src/daemon/session-action-recorder.ts`), and both recording call sites consume it.
1025
+
>
1026
+
> **2. The corrective-read trap: why even out-of-band reads need an opt-in.** Excluding every out-of-band
1027
+
> read is still not safe on its own: a divergence's **correction** can itself be a read. In the wave-3 E3 drift, the diverged step **was** a `get`, so its corrective replacement is a
1007
1028
> recorded `get` that **must** land in the heal — a blanket read-exclusion would silently drop the very
1008
1029
> step being repaired, producing an incomplete heal with no signal anything was missing (the same class of
1009
1030
> silent-failure mode decision 3 exists to close for target-binding, and decision 6 R4 exists to close for
@@ -1015,36 +1036,57 @@ The tombstone itself expires after its bounded window, after which the key is fu
1015
1036
> opt-in per action**: `--record` (below) marks the ONE read that must count, independent of its position in
1016
1037
> the repair segment.
1017
1038
>
1018
-
> **3. Explicit recording remains possible.**`--record` is a new, standalone flag — distinct from
1019
-
> `saveScript`, and the mirror image of `--no-record` — that forces the action it is passed on into the heal
1020
-
> even though its command is observation-only. It is a no-op outside a repair-armed session (there is no
1021
-
> default exclusion to override) and a no-op on any non-observation-only command (those already always
1022
-
> record). `--record` and `--no-record` express opposite intents for the same action and are **mutually
1023
-
> exclusive**: passing both is rejected as `INVALID_ARGS` before the command runs, uniformly for every
1024
-
> surface, rather than letting one silently win.
1039
+
> **3. Explicit recording remains possible.**`--record` forces an out-of-band observation into the heal
1040
+
> even though its command is observation-only. It exists for exactly one job: an interactive corrective
1041
+
> read the agent is deliberately inserting into the healed script. It is a no-op outside a repair-armed
1042
+
> session (there is no default exclusion to override), and it is **not** needed for authored plan steps —
1043
+
> those survive on provenance alone (rule 1).
1044
+
>
1045
+
> `--record` is **not a common flag**, unlike `--no-record`. `--no-record` applies to every recordable
1046
+
> command, mutations included, so it stays universal
1047
+
> (`COMMON_COMMAND_SUPPORTED_FLAG_KEYS`). `--record` only means something for a command the exclusion can
1048
+
> drop, so it is scoped **statically** to `snapshot`, `get`, and `is` through each command schema's
1049
+
> `allowedFlags` — accepting it on `press`/`fill`/`click` would be misleading, and the grammar rejects
1050
+
> unsupported uses before projection. `find` is the one command whose observe-vs-mutate split is a
1051
+
> POSITIONAL rather than the command name, so it cannot be settled statically: it allows `--record`
1052
+
> in the grammar and validates **dynamically** in the daemon (`handleFindCommands`), rejecting
1053
+
> `--record` on a mutating `find … click|fill|focus|type` with `INVALID_ARGS` before any device work. Both
1054
+
> halves read one shared predicate (`isReadOnlyFindAction`, `src/selectors/find.ts`), so the routing and
1055
+
> the validation can never disagree about which sub-actions observe.
1056
+
>
1057
+
> `--record` and `--no-record` express opposite intents for the same action and are **mutually exclusive**:
1058
+
> passing both is rejected as `INVALID_ARGS` before the command runs, uniformly for every surface, rather
1059
+
> than letting one silently win.
1025
1060
>
1026
1061
> **4. How the rule projects across CLI/Node/MCP surfaces.** The exclusion is enforced at a single
1027
1062
> daemon-side choke point — `isExcludedRepairSegmentObservation` in `recordActionEntry`
1028
1063
> (`src/daemon/session-action-recorder.ts`), gated on `session.saveScriptBoundary !== undefined` (set ONLY
1029
1064
> by a repair-armed `replay --save-script`; an ordinary, non-repair `open --save-script`/`close --save-script`
1030
1065
> authoring recording never sets it — see decision 6's "Scope" note above) and on the entry's
1031
-
> `observationOnly` marker (set by the caller — `get`/`is`/a read-only `find` in
1032
-
> `src/daemon/selector-recording.ts`, `snapshot` in `src/daemon/snapshot-runtime.ts` — never inferred from
1033
-
> `command`/`positionals`, since a mutating `find … click` shares the literal string `command: 'find'` with
1034
-
> its read-only sibling). Every surface reaches this same function because every surface's request reaches
1035
-
> the same daemon: the CLI parses `--record`/`--no-record` into `CommandFlags` (`src/contracts/cli-flags.ts`,
0 commit comments