You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit de65ecb
Browse filesBrowse the repository at this point in the historyBrowse files
fix(replay): key the repair-segment exclusion on provenance, scope --record (#1271 review)
Addresses the maintainer review on #1303.
P1 — the exclusion dropped PLANNED reads from the heal. It discriminated by
command class, but the real discriminator is provenance. Replayed plan steps
dispatch through the ordinary request path, so an authored get/is/find step hit
the same recordIfSession -> exclusion path as an interactive read and never
reached session.actions -- and the heal IS session.actions.slice(boundary). A
repaired flow therefore replayed its authored `is visible` assertion and then
silently dropped it from its own healed script: the heal quietly stops checking
what it used to check, which for a 10x-QA-replay suite is the worst failure
mode.
Fix: an explicit provenance marker, not a heuristic. `internal.replayPlanStep`
is stamped by invokeResolvedReplayAction -- the single point every plan step is
dispatched, so it covers annotated and unannotated steps alike. `internal` is
daemon-only (toDaemonRequest never copies it off the wire), so authored
provenance cannot be spoofed; same channel as replayTargetGuard. The rule now
lives once in isInteractiveObservation and both recording call sites consume it,
so the mock fixture uses the production classifier instead of mirroring it.
Planned observations survive automatically -- users never annotate their own .ad
steps.
--record is no longer a common flag: removed from
COMMON_COMMAND_SUPPORTED_FLAG_KEYS, statically scoped via allowedFlags to
snapshot/get/is, and validated dynamically for find (read-only allows; a
mutating find click|fill|focus|type is INVALID_ARGS before any device work,
sharing one isReadOnlyFindAction predicate with the read-only routing so the two
cannot disagree). --no-record stays shared -- it applies to every recordable
command. Removed from `open`, which is never observation-only.
Rebased onto #1304 and dropped the four hand-rolled reader blocks. Split its
helper rather than broadening it: noRecordInputFromFlags (all 13 readers) +
observationRecordInputFromFlags (snapshot/get/is/find only). Two named helpers
over one `allowRecord` policy arg -- the capability is then the helper's NAME, so
a mutating reader physically cannot forward --record, whereas a policy arg would
let a future mutating reader opt in by flipping a literal with no schema change.
ADR-0012 decision 6 now states the provenance rule, not a command-class rule.
The scenario gates the P1: its authored step is a distinguishable `is visible`,
and it fails without the provenance check ("the authored 'is visible' step must
survive the heal").
Copy file name to clipboardExpand all lines: docs/adr/0012-interactive-replay.md
+98-49Lines changed: 98 additions & 49 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -953,17 +953,38 @@ The tombstone itself expires after its bounded window, after which the key is fu
953
953
> behavior the default, superseding stage 1's "opt out by hand" guidance with "opt in by hand" for the one
954
954
> case that needs it.
955
955
>
956
-
> **1. Which read-only commands are excluded.** Exactly the enumerated set stage 1 already named:
957
-
> `snapshot`, `get`, `is`, and a **read-only**`find` sub-action (`exists`, `wait`, `get_text`, `get_attrs`).
958
-
> A **mutating**`find … click|fill|focus|type` is not in this set — it is a corrective action like `press`/
959
-
> `fill` and always records. The top-level `wait` command is deliberately **not** in this set either: it is
960
-
> flow timing/synchronisation (pacing the script), not observation, so it must keep recording unconditionally
961
-
> even inside a repair segment — a healed script that silently dropped its own timing steps would replay
962
-
> differently from what actually happened.
956
+
> **1. Which observations are excluded — a PROVENANCE rule, not a command-class rule.** The exclusion
957
+
> applies to an observation that is BOTH:
963
958
>
964
-
> **2. The corrective-read trap, and why blanket exclusion by command class is unsafe.** "Exclude every
965
-
> read from a repair segment" is not a safe rule on its own: a divergence's **correction** can itself be a
966
-
> read. In the wave-3 E3 drift, the diverged step **was** a `get`, so its corrective replacement is a
959
+
> -**observation-only by command**: `snapshot`, `get`, `is`, or a **read-only**`find` sub-action
960
+
> (`exists`, `wait`, `get_text`, `get_attrs`). A **mutating**`find … click|fill|focus|type` is not in
961
+
> this set — it is a corrective action like `press`/`fill` and always records. The top-level `wait`
962
+
> command is deliberately **not** in this set either: it is flow timing/synchronisation (pacing the
963
+
> script), not observation, so it must keep recording unconditionally — a healed script that silently
964
+
> dropped its own timing steps would replay differently from what actually happened; **AND**
965
+
> -**dispatched OUT OF BAND**: typed by the agent mid-repair, NOT replayed from the `.ad` plan under
966
+
> repair.
967
+
>
968
+
> The second clause is load-bearing and is the rule's real discriminator. Replayed plan steps dispatch
969
+
> through the ordinary request path and land in `session.actions` like any other action, and the healed
970
+
> script IS that slice (`buildOptimizedActions` over `session.actions.slice(saveScriptBoundary)`). So a
971
+
> command-class-only exclusion would replay an authored `is visible` assertion and then **silently drop it
972
+
> from its own healed script** — the repaired flow would quietly stop checking what the original checked,
973
+
> which for a deterministic QA suite is the worst possible failure mode. **Planned prefix/suffix
974
+
> observations must survive automatically; a user must never have to annotate authored `.ad` steps with
975
+
> `--record` to keep them.** An authored observation and an interactive diagnostic read are the same
976
+
> command, so only provenance separates them.
977
+
>
978
+
> Provenance travels as `internal.replayPlanStep`, stamped by `invokeResolvedReplayAction`
979
+
> (`src/daemon/handlers/session-replay-action-runtime.ts`) — the single point every plan step is
980
+
> dispatched, so the marker covers annotated and unannotated steps alike. `internal` is a daemon-only
981
+
> channel (`toDaemonRequest`, `src/daemon/server/http-server.ts`, never copies it off the wire), so
982
+
> authored provenance cannot be spoofed by a client; this is the same channel `replayTargetGuard` and
983
+
> `findResolvedTarget` already use. The rule is written once, in `isInteractiveObservation`
984
+
> (`src/daemon/session-action-recorder.ts`), and both recording call sites consume it.
985
+
>
986
+
> **2. The corrective-read trap: why even out-of-band reads need an opt-in.** Excluding every out-of-band
987
+
> read is still not safe on its own: a divergence's **correction** can itself be a read. In the wave-3 E3 drift, the diverged step **was** a `get`, so its corrective replacement is a
967
988
> recorded `get` that **must** land in the heal — a blanket read-exclusion would silently drop the very
968
989
> step being repaired, producing an incomplete heal with no signal anything was missing (the same class of
969
990
> silent-failure mode decision 3 exists to close for target-binding, and decision 6 R4 exists to close for
@@ -975,36 +996,57 @@ The tombstone itself expires after its bounded window, after which the key is fu
975
996
> opt-in per action**: `--record` (below) marks the ONE read that must count, independent of its position in
976
997
> the repair segment.
977
998
>
978
-
> **3. Explicit recording remains possible.**`--record` is a new, standalone flag — distinct from
979
-
> `saveScript`, and the mirror image of `--no-record` — that forces the action it is passed on into the heal
980
-
> even though its command is observation-only. It is a no-op outside a repair-armed session (there is no
981
-
> default exclusion to override) and a no-op on any non-observation-only command (those already always
982
-
> record). `--record` and `--no-record` express opposite intents for the same action and are **mutually
983
-
> exclusive**: passing both is rejected as `INVALID_ARGS` before the command runs, uniformly for every
984
-
> surface, rather than letting one silently win.
999
+
> **3. Explicit recording remains possible.**`--record` forces an out-of-band observation into the heal
1000
+
> even though its command is observation-only. It exists for exactly one job: an interactive corrective
1001
+
> read the agent is deliberately inserting into the healed script. It is a no-op outside a repair-armed
1002
+
> session (there is no default exclusion to override), and it is **not** needed for authored plan steps —
1003
+
> those survive on provenance alone (rule 1).
1004
+
>
1005
+
> `--record` is **not a common flag**, unlike `--no-record`. `--no-record` applies to every recordable
1006
+
> command, mutations included, so it stays universal
1007
+
> (`COMMON_COMMAND_SUPPORTED_FLAG_KEYS`). `--record` only means something for a command the exclusion can
1008
+
> drop, so it is scoped **statically** to `snapshot`, `get`, and `is` through each command schema's
1009
+
> `allowedFlags` — accepting it on `press`/`fill`/`click` would be misleading, and the grammar rejects
1010
+
> unsupported uses before projection. `find` is the one command whose observe-vs-mutate split is a
1011
+
> POSITIONAL rather than the command name, so it cannot be settled statically: it allows `--record`
1012
+
> in the grammar and validates **dynamically** in the daemon (`handleFindCommands`), rejecting
1013
+
> `--record` on a mutating `find … click|fill|focus|type` with `INVALID_ARGS` before any device work. Both
1014
+
> halves read one shared predicate (`isReadOnlyFindAction`, `src/selectors/find.ts`), so the routing and
1015
+
> the validation can never disagree about which sub-actions observe.
1016
+
>
1017
+
> `--record` and `--no-record` express opposite intents for the same action and are **mutually exclusive**:
1018
+
> passing both is rejected as `INVALID_ARGS` before the command runs, uniformly for every surface, rather
1019
+
> than letting one silently win.
985
1020
>
986
1021
> **4. How the rule projects across CLI/Node/MCP surfaces.** The exclusion is enforced at a single
987
1022
> daemon-side choke point — `isExcludedRepairSegmentObservation` in `recordActionEntry`
988
1023
> (`src/daemon/session-action-recorder.ts`), gated on `session.saveScriptBoundary !== undefined` (set ONLY
989
1024
> by a repair-armed `replay --save-script`; an ordinary, non-repair `open --save-script`/`close --save-script`
990
1025
> authoring recording never sets it — see decision 6's "Scope" note above) and on the entry's
991
-
> `observationOnly` marker (set by the caller — `get`/`is`/a read-only `find` in
992
-
> `src/daemon/selector-recording.ts`, `snapshot` in `src/daemon/snapshot-runtime.ts` — never inferred from
993
-
> `command`/`positionals`, since a mutating `find … click` shares the literal string `command: 'find'` with
994
-
> its read-only sibling). Every surface reaches this same function because every surface's request reaches
995
-
> the same daemon: the CLI parses `--record`/`--no-record` into `CommandFlags` (`src/contracts/cli-flags.ts`,
0 commit comments