Skip to content

Commit ea527a8

Browse files
committed
Merge remote-tracking branch 'origin/main' into claude/issue-1385-90ab77
* origin/main: docs(adr): rules-first ADR restructure + ADR 0017 proposal (unified event journal) (#1399) feat: add first-class Vega VVD TV support (#1396) fix(replay): preserve cwd scope for opened sessions (#1401) docs: restructure AGENTS.md and CONTEXT.md for progressive disclosure (#1402) fix(cli): compact stale device status (#1388) feat: add WebView accessibility lab (#1397) feat: parameterize sensitive recorded inputs (#1369) fix(daemon): keep an active replay session's daemon alive over the CLI path (#1390) # Conflicts: # docs/adr/0012-interactive-replay.md
2 parents c24611e + 152894c commit ea527a8

152 files changed

Lines changed: 6163 additions & 1059 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎AGENTS.md‎

Lines changed: 218 additions & 300 deletions
Large diffs are not rendered by default.

‎CONTEXT.md‎

Lines changed: 247 additions & 105 deletions
Large diffs are not rendered by default.

‎README.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@
1515

1616
Let your coding agent verify its changes in the running app.
1717

18-
`agent-device` lets coding agents inspect, control, and verify apps on iOS, Android, TV, web, macOS, and Linux. Agents can read token-efficient accessibility snapshots, find elements by ref or selector, run device actions, and save evidence for review.
18+
`agent-device` lets coding agents inspect, control, and verify apps on iOS, Android, tvOS, Android TV, Amazon Vega OS TV through the Vega Virtual Device (VVD), web, macOS, and Linux. Agents can read token-efficient accessibility snapshots where supported, find elements by ref or selector, run device actions, and save evidence for review. Initial Vega OS support is VVD-only and covers discovery, app lifecycle, and complete TV-remote control; physical Fire TV, capture, and selector backends remain unsupported.
1919

2020
Your coding agent or QA tool reads each result and chooses the next command. `agent-device` runs the command and saves evidence when asked.
2121

@@ -30,6 +30,7 @@ npm install -g agent-device@latest
3030
agent-device doctor
3131
agent-device --version
3232
agent-device help workflow
33+
agent-device help tv
3334
```
3435

3536
Run `agent-device doctor` yourself before handing the CLI to an agent. The installed CLI help defines current behavior. `agent-device help workflow` links to guides for debugging, replay, React Native profiling, and other tasks.
@@ -108,7 +109,7 @@ See [Commands](https://oss.callstack.com/agent-device/docs/commands) for the com
108109

109110
## How it works
110111

111-
`agent-device` keeps device state in sessions. It sends commands to XCTest on iOS and tvOS, ADB and the snapshot helper on Android, a local helper on macOS, and AT-SPI on Linux.
112+
`agent-device` keeps device state in sessions. It sends commands to XCTest on iOS and tvOS, ADB and the snapshot helper on Android, Vega CLI/VDA on the Vega Virtual Device, a local helper on macOS, and AT-SPI on Linux.
112113

113114
Node.js apps can use the typed client or public subpaths. `agent-device/android-adb` provides the Android ADB provider interface, helpers for logcat, the clipboard, the keyboard, and apps, and port reverse management.
114115

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Snapshot.swift‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -57,21 +57,23 @@ extension RunnerTests {
5757
case .collectionView: return "CollectionView"
5858
case .table: return "Table"
5959
case .scrollView: return "ScrollView"
60+
case .toolbar: return "Toolbar"
6061
case .searchField: return "SearchField"
6162
case .segmentedControl: return "SegmentedControl"
6263
case .stepper: return "Stepper"
6364
case .picker: return "Picker"
65+
case .activityIndicator: return "ActivityIndicator"
66+
case .progressIndicator: return "ProgressIndicator"
6467
case .checkBox: return "CheckBox"
6568
case .menuItem: return "MenuItem"
69+
case .webView: return "WebView"
6670
case .other: return "Other"
6771
default:
6872
switch type.rawValue {
6973
case 19:
7074
return "Keyboard"
7175
case 20:
7276
return "Key"
73-
case 24:
74-
return "SearchField"
7577
default:
7678
return "Element(\(type.rawValue))"
7779
}

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests.swift‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -131,7 +131,8 @@ final class RunnerTests: XCTestCase {
131131
.tabBar,
132132
.textField,
133133
.secureTextField,
134-
.textView
134+
.textView,
135+
.webView
135136
]
136137
// Keep blocker actions narrow to avoid false positives from generic hittable containers.
137138
let actionableTypes: Set<XCUIElement.ElementType> = [

‎docs/adr/0012-interactive-replay.md‎

Lines changed: 214 additions & 303 deletions
Large diffs are not rendered by default.

‎docs/adr/0014-session-ref-frame-lifetime.md‎

Lines changed: 33 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,37 @@
22

33
## Status
44

5-
Accepted
5+
Accepted; implemented and enforced on `main` across all platforms. Fresh live evidence exercised
6+
every production seam except Android blocking-dialog recovery, which shipped as a documented,
7+
accepted gap: no deterministic repro exists to raise the system dialog on demand, and the seam's
8+
transition and abort logic are pinned by fixture regressions
9+
(`android-system-dialog-ref-frame.test.ts`, `interaction-android-recovery-abort.test.ts`). The
10+
staged migration plan lives in this file's git history; PR #1241 was an earlier compatible
11+
transitional fix, not this architecture migration.
12+
13+
## Rules at a glance
14+
15+
Normative summary; the binding contracts and edge cases are in [Decision](#decision), and the
16+
ref-frame vocabulary is promoted into `CONTEXT.md`.
17+
18+
- A session owns at most one **ref frame** — the authorization namespace for mutation refs (epoch
19+
exposed as `refsGeneration`, immutable source tree, `active`/`expired` state, `all` or bounded
20+
issuance scope) — owned by `src/daemon/ref-frame.ts` and kept separate from the latest
21+
operational observation (`session.snapshot`).
22+
- A complete snapshot activates an `all` frame; `find`, settled diffs, and replay divergence
23+
screens activate a bounded partial frame that supersedes the prior one; internal read-only
24+
captures never activate, reindex, or expire a frame.
25+
- Every mutating leaf expires the frame at the side-effect seam — after all pre-action guards,
26+
immediately before the device op — with no success-only rollback; a post-dispatch failure still
27+
leaves it expired.
28+
- Mutation admission requires an active frame whose epoch and issuance scope authorize the ref;
29+
rejections carry ordered typed reasons `ref_frame_expired` -> `ref_generation_mismatch` ->
30+
`plain_ref_requires_complete_frame` -> `ref_not_issued`.
31+
- Read-only ref consumers stay fail-open with a structured staleness warning only while the frame
32+
retains the ref's evidence; missing evidence fails rather than resolving by positional
33+
coincidence against a newer observation.
34+
- Every daemon command declares a `RefFrameEffect` (`preserve` | `may-invalidate` | `delegated`) in
35+
its descriptor daemon facet; the registry classification is completeness-gated.
636

737
## Context
838

@@ -36,7 +66,7 @@ remain compatible with selector-based replay, and add no automatic capture or pe
3666
## Decision
3767

3868
The terms introduced below are now the implemented model; the ref-frame vocabulary is promoted into
39-
`CONTEXT.md`. The superseded coarse `snapshotRefsStale` marker has been removed (migration step 8):
69+
`CONTEXT.md`. The superseded coarse `snapshotRefsStale` marker has been removed:
4070
read-only ref staleness is now derived from frame state — a plain ref warns once the frame has expired,
4171
and a read-only capture no longer marks refs stale because it does not expire the frame.
4272

@@ -64,7 +94,7 @@ the evidence tree, bounds partial authority. The frame and latest observation sh
6494
data when they originate from the same capture; neither transition deep-copies the tree.
6595

6696
The existing `snapshotGeneration` implementation evolves behind one ref-frame module; the
67-
`snapshotRefsStale` marker it originally paired with has since been removed (migration step 8). The
97+
`snapshotRefsStale` marker it originally paired with has since been removed. The
6898
public name `refsGeneration` and the `@e12~s42` grammar remain unchanged for wire compatibility.
6999

70100
### Frame transitions
@@ -418,50 +448,6 @@ the stale ref is rejected before dispatch, and a fresh observation restores usab
418448
exercised remains disabled or is recorded as an explicit release blocker. Fixture-backed tests and
419449
registry claims are necessary but do not substitute for this live evidence.
420450

421-
## Migration
422-
423-
Each step lands green and independently useful:
424-
425-
1. introduce the bounded ref-frame module and separate operational observations without changing
426-
enforcement;
427-
2. add the complete daemon descriptor classification and gate, including request-sensitive resolvers;
428-
3. route every leaf side effect, fallback, retry, and readiness recovery through the idempotent
429-
pre-side-effect transition;
430-
4. correct complete/partial publication, bounded scoped-snapshot lineage, MCP pin retention, pinned
431-
partial CLI text, and JSON/Node.js response-level generation handling;
432-
5. decouple Android freshness capture from positional ref authorization;
433-
6. add sequence, batch, replay, series, failure-boundary, and cross-platform provider contracts;
434-
7. enable fail-closed mutation enforcement per platform only after its paths have evidence, ending in
435-
one uniform policy; and
436-
8. update help, changelog, ADR 0012's proposed-amendment note and tests, and replay/batch guidance for
437-
refresh-between-mutations, conservative fused no-op invalidation, and reads without retained
438-
evidence; promote the implemented vocabulary into `CONTEXT.md`; then remove the superseded coarse
439-
stale marker.
440-
441-
Implementation status: all migration steps have landed — the ref-frame module and observation split
442-
(1), the complete daemon classification and gate (2), the pre-side-effect seam at every leaf (3),
443-
correct complete/partial publication with bounded scope, MCP pin retention, and pinned partial CLI text
444-
(4), Android freshness decoupled from positional ref authorization (5), the cross-platform contract and
445-
provider evidence (6), fail-closed admission enforcement across platforms with typed reasons (7), and
446-
the docs/vocabulary promotion plus removal of the superseded coarse `snapshotRefsStale` marker (8).
447-
Fresh live evidence has exercised nearly every production seam — Apple runtime-ref, direct/native
448-
selector, generation-pin, generic, and lifecycle paths; Android helper freshness (including proven
449-
non-retarget) and Android existing-session relaunch; and a real provider-backed interaction plus
450-
provider-backed lifecycle operation (AWS Device Farm, `backend: webdriver`: a fresh ref succeeded, an
451-
immediate stale ref was rejected before dispatch with the shared typed fields, an `open --relaunch`
452-
lifecycle mutation expired the frame, and a fresh observation restored authorization). ONE seam —
453-
Android blocking-dialog recovery — was NOT live-exercised: the repo harness exposes no deterministic
454-
app-owned ANR trigger, and no reproducible control exists to raise the system dialog on demand. The
455-
team accepted shipping without a live run for it: its transition and abort logic are covered by fixture
456-
regressions (`android-system-dialog-ref-frame.test.ts` proves recovery expires the frame before its
457-
tap; `interaction-android-recovery-abort.test.ts` proves an outstanding ref action then aborts with the
458-
shared `ref_frame_expired` rejection and no dispatch), the seam is enforced in code identically to the
459-
verified paths, and it is recorded here as a documented, accepted evidence gap rather than an open
460-
release blocker. Every other enabled seam is proven on hardware.
461-
462-
PR #1241 landed independently as a compatible transitional fix. It rejects a known iOS stale-marker
463-
case before this full lifecycle is implemented; it does not own the architecture migration.
464-
465451
## Alternatives considered
466452

467453
- **Stable refs across trees:** rejected because positional accessibility snapshots have no durable

‎docs/adr/0016-active-session-script-publication.md‎

Lines changed: 53 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,31 @@
22

33
## Status
44

5-
Accepted
5+
Accepted; implemented on `main` (`src/daemon/handlers/session-script-publication.ts`,
6+
`src/daemon/session-script-active-publication.ts`), including the #1349 identity-level
7+
destination-guard amendment.
8+
9+
## Rules at a glance
10+
11+
Normative summary; the binding contracts and refusal cases are in [Decision](#decision).
12+
13+
- `session save-script [path] [--force]` publishes the current ordinary script recording without
14+
closing the app or the session; the session stays live at the destination.
15+
- Recording lifecycle: **ARMED** (only by a new session's initial `open --save-script`) ->
16+
**ABORTED** (a second successful plain `open`) or **PUBLISHED** (successful save). ABORTED and
17+
PUBLISHED are terminal until the session is destroyed; a filesystem/target failure leaves the
18+
recording ARMED and retryable.
19+
- Publication requires exactly one initial recorded `open`, a portable selector-`wait` destination
20+
guard after the last descriptor-classified mutating action — since #1349 the guard's `target-v1`
21+
annotation must be `verification: "verified"` — and no unresolved session-local `@ref`. Every
22+
refusal happens before filesystem work and names its recovery.
23+
- Repair transactions (ADR 0012) are disjoint: a session with `saveScriptBoundary` set refuses this
24+
action without changing repair state.
25+
- `ReplayCommandResult.sessionActive` (from the daemon's session store, never script re-parsing) is
26+
what lets a one-shot client keep the daemon alive on a close-less handoff.
27+
- Sensitive `fill` inputs must be recorded as placeholders via `fill --record-as <VAR>`
28+
(ADR 0017, shipped for #1348); unparameterized `fill`/`type` values persist literally into the
29+
artifact, so a secret entered without `--record-as` is published.
630

731
## Context
832

@@ -119,19 +143,27 @@ enough for this boundary.
119143
On consumption, a script without `close` preserves the existing replay behavior: the named session stays
120144
active and the successful `ReplayCommandResult` returns its `session` id. The caller binds subsequent
121145
commands to that returned id. Replay reports success only after the destination guard completes; the
122-
absence of `close` changes neither action dispatch nor the success response shape.
146+
absence of `close` changes neither action dispatch nor when replay reports success.
147+
148+
**Response shape (issue #1384).** `ReplayCommandResult` carries a required `sessionActive: boolean`,
149+
computed from whether `session` still exists in the daemon's own store when the response is built — never
150+
by re-parsing the script for `close`. This is what lets the real CLI/IPC client (not just the in-process
151+
handler) actually honor "the session stays active": without an explicit signal in the response, the
152+
client's one-shot `replay`/`test` teardown had no way to distinguish a still-active handoff from a
153+
finished one, and tore the owning daemon down regardless (`src/daemon/client/daemon-client-lifecycle.ts`).
154+
`sessionActive` is `true` for every close-less run (including a `--from` resume) and `false` once the
155+
script's terminal `close` — or a repair-armed run's deferred equivalent — has executed.
123156

124157
### Sensitive inputs
125158

126-
Executable `.ad` artifacts serialize action inputs literally. A recorded `fill` therefore writes its
127-
text to the published file; diagnostic and event-log redaction cannot protect an input that the replay
128-
engine must later execute. V1 does not claim secret-bearing login flows are safe to publish.
159+
Executable `.ad` artifacts serialize ordinary action inputs literally; diagnostic and event-log
160+
redaction cannot protect an input that replay must later execute. ADR 0017 resolves #1348 for explicit
161+
sensitive fills: `fill ... --record-as PASSWORD` sends the live text to the app while recording only
162+
`${PASSWORD}`. Replay receives the value through `AD_VAR_PASSWORD` or `--env PASSWORD=<value>`.
129163

130-
Native `.ad` replay already supports late-bound `${VAR}` values, but ordinary recording cannot yet
131-
execute with a real value while publishing only its placeholder. That safe-authoring capability is
132-
tracked in [#1348](https://github.com/callstack/agent-device/issues/1348). Until it ships, authors must not
133-
record a journey that enters a secret: use pre-authenticated test state or deliberately non-secret fixture
134-
credentials that are safe to persist. CLI help must state this warning next to the authoring workflow.
164+
This is opt-in and fill-only. Unparameterized fill/type inputs remain literal artifact content, so
165+
authors must use ADR 0017 for each sensitive fill and avoid secret-bearing `type` steps. CLI help states
166+
both the safe workflow and the remaining literal-input warning next to publication guidance.
135167

136168
### Artifact contract
137169

@@ -179,8 +211,15 @@ executing that script, not the artifact being saved.
179211
bootstrap; authoring resumes only in a fresh session.
180212
- Intermediate lifecycle-free fragments, entry guards, include semantics, composed digests, and shared
181213
fragment pinning remain entirely under #1336.
182-
- Secret-bearing authoring remains unsafe until #1348; the initial workflow is limited to journeys that
183-
do not enter secrets. Arbitrary history ranges remain out of scope.
214+
- Secret-bearing fill authoring uses ADR 0017's explicit `--record-as <VAR>` contract. Unparameterized
215+
fill/type inputs remain literal script content. Arbitrary history ranges remain out of scope.
216+
- On consumption (issue #1384), a `replay` whose script has no terminal `close` leaves a live daemon and
217+
app session behind — including the ordinary one-shot `replay` invocation's own owned/ephemeral daemon,
218+
which the client keeps alive and reports a `--state-dir` address hint for instead of tearing down. An
219+
unattended close-less replay (e.g. in CI) therefore leaks a session exactly like one opened
220+
interactively: bounded by ordinary idle-reap or an explicit `close`, never by the one-shot command's own
221+
process lifetime. This is the intended shape of "the caller owns close," not an accident, and recorded
222+
test flows already end with `close` so `test` runs are unaffected.
184223

185224
## Alternatives Considered
186225

@@ -229,7 +268,7 @@ executing that script, not the artifact being saved.
229268
request-sensitive read-only/mutating subcommands, and destination-guard ordering consumes only that
230269
trait.
231270
- Repair-armed sessions refuse this action without changing repair state.
232-
- CLI help warns that literal `fill` inputs are persisted and tells authors not to record secret-bearing
233-
journeys until #1348's parameterized-input mechanism is available.
271+
- CLI help documents ADR 0017's `--record-as` workflow and warns that unparameterized fill/type inputs
272+
remain literal script content.
234273
- Provider-backed integration scenarios cover the public daemon route, and live iOS and Android runs
235274
prove the saved artifact and post-save session behavior on real backends.

0 commit comments

Comments
 (0)