Skip to content

Commit e0dba23

Browse files
committed
Merge branch 'main' into claude/command-registry-refactor-3fcbb5
#2713 landed on main with two commits beyond the head this branch had merged; its own files take main's version. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_014x96iYiFTbAwD4YEKU3PQ1
2 parents 3c6a06a + c442cf6 commit e0dba23

92 files changed

Lines changed: 4346 additions & 1606 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.device-evidence/CHECKLIST.md‎

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
# #2682 physical-device checklist — `thymikee-iphone` (coordinator-owned)
2+
3+
Simulator evidence is recorded in `DECISION.md`. This is the same scenario on the physical lane.
4+
Use a purpose-specific session name; close it afterwards. Do not reuse another lane's session.
5+
6+
```sh
7+
UDID=<thymikee-iphone udid from: agent-device devices --platform ios>
8+
S="--platform ios --udid $UDID --session apex2682-physical"
9+
bin/agent-device.mjs open com.callstack.agentdevicelab $S --foreground
10+
```
11+
12+
The fixture must show its own surface (Metro/dev-client per `examples/test-app/README.md`); an
13+
already-installed bundle id is not evidence.
14+
15+
## Sequence
16+
17+
1. `bin/agent-device.mjs screenshot --out /tmp/2682-1-app.png $S` → fixture app.
18+
2. Hand off inside the app (prefer a real in-app handoff; the AccessorySetupKit/permission or an
19+
external link surface both qualify). Fallback that exercises the identical runner path:
20+
`xcrun devicectl handle URL`/a user tap that opens Safari or Settings.
21+
3. `bin/agent-device.mjs screenshot --out /tmp/2682-2-foreign.png $S`
22+
4. `bin/agent-device.mjs snapshot -i $S | tee /tmp/2682-3-snapshot.txt`
23+
5. `bin/agent-device.mjs snapshot -i $S` (again, app already foreground)
24+
6. `bin/agent-device.mjs close $S`
25+
26+
## Expected BEFORE (`main`, no disclosure)
27+
28+
- Step 3 shows the other app; step 4 shows the fixture app's tree.
29+
- Nothing in either output says why. The only record is in the session's `runner.log`:
30+
`AGENT_DEVICE_RUNNER_ACTIVATE bundle=com.callstack.agentdevicelab state=<2|3> reason=<reason>`.
31+
- A run where step 3 still shows the fixture app is not evidence — the handoff must be visible.
32+
33+
## Expected AFTER (PR1 head)
34+
35+
- Step 4 carries, as a warning line and as `data.targetActivation` from PR2:
36+
`The session app was not foreground when this command arrived (prior state runningBackground), so
37+
the runner activated it before answering (reason
38+
<stale_target|bundle_changed|missing_after_wait|interaction_foreground_guard>). Any capture taken
39+
earlier in this session described the only app other than the session app with an active
40+
accessibility session (pid <N>), not the session app. Re-capture now that the session app answers,
41+
or drive the other app in its own session.`
42+
- Step 5 carries no disclosure — the fact belongs to the command that activated.
43+
- `<N>` is a **liveness** claim and the sentence is written to claim no more: `otherActiveApplicationPid`
44+
is present only when exactly one application other than the session app held an active accessibility
45+
session. `activeApplications` exposes no ordering, so nothing here proves which app owned the screen.
46+
A pid that is not an AX-active application of the session is a bug; an absent pid is not.
47+
48+
## Proving the disclosure comes from activation, not a later state read
49+
50+
`priorState` is read at `RunnerTests+Lifecycle.swift` inside `activateTarget` **before**
51+
`XCUIApplication.activate()` runs, and the fact is stamped only in the branch where `activate()` is
52+
actually called. Assert the pair on the same device run:
53+
54+
- `runner.log`: `AGENT_DEVICE_RUNNER_ACTIVATE_FACT bundle=... reason=... priorState=2
55+
otherActiveApplicationPid=<N>` — the same line's `priorState` is what the response must carry;
56+
`state=` on the preceding `AGENT_DEVICE_RUNNER_ACTIVATE` must be non-foreground.
57+
- The response can therefore never report `priorState: runningForeground`. Both directions are pinned
58+
in the runner unit lane (`UnitTests/RunnerTests+LifecycleCacheTests.swift`): the already-foreground
59+
call asserts `pendingTargetActivation == nil` and the backgrounded call asserts the stamped
60+
`priorState` equals `XCUIApplication.State.runningBackground.rawValue`.
61+
- Correlate `<N>` with the app from step 3: `xcrun devicectl list processes --device $UDID` (or the
62+
physical-lane equivalent of the simulator's `ps`) should find that pid alive. Confirming it is
63+
AX-active is what the runner probed; confirming it *owned the screen* is not what this fact claims.
64+
On a physical device `proc_pidpath`-style resolution is deliberately NOT attempted.
65+
66+
## Still to prove on this lane (PR1 risk list)
67+
68+
- Whether `activeApplications` reports more than the session app while a foreign app is foreground
69+
on real hardware (decides how often `otherActiveApplicationPid` is present at all).
70+
- That screenshot stays lifecycle, #2438 in-place system-host serving still applies (Apple Pay sheet
71+
from **Automation lab → Open Apple Pay**), and ADR 0005 `targetReset` on external relaunch is
72+
untouched.
73+
74+
## Not covered by the disclosure shipped in PR2
75+
76+
`press <x> <y>` and `press @ref` answered from a live ref frame consume no capture, so they carry no
77+
disclosure even though the runner may have re-activated the app to serve them. Treat a silent
78+
interaction as "unknown", never as "no repair". Follow-up: callstack/agent-device#2694.
79+
80+
## PR2 simulator lane — daemon disclosure (apex-2682-proto, iOS 26.2)
81+
82+
Driven with `bin/agent-device.mjs` after `pnpm build && pnpm clean:daemon`, session `apex2682-pr3`,
83+
off-app handoff via `xcrun simctl openurl <UDID> https://example.com` before each command. Every
84+
command below re-activated the session app, and each answered with `targetActivation` plus the
85+
appended warning:
86+
87+
| Command | Result |
88+
| --- | --- |
89+
| `snapshot -i` | warning first in `warnings`, ahead of the AX-backend warning; `targetActivation` carried |
90+
| `snapshot -i --level digest` | `targetActivation` survives the digest projection |
91+
| `find "Tab Bar" --first` | disclosed on the matched response |
92+
| `press 'label="INFO"'` | disclosed on the interaction response |
93+
| `wait 'label="INFO"' 8000` | disclosed on the satisfied response |
94+
95+
`otherActiveApplicationPid` on the simulator was the Safari process, consistent across commands (pid 33878).
96+
97+
### Proof the review asked for on this lane
98+
99+
The false-attribution class is covered by unit tests rather than a device run, because the device
100+
cannot show a cache hit on command: `src/daemon/__tests__/capture-disclosure-target-activation.test.ts`
101+
pins that a cache-reused tree discloses nothing, and
102+
`src/daemon/interaction/internal/__tests__/interaction-target-activation-disclosure.test.ts` pins that
103+
a press which consumed no capture is not disclosed against an older tree.
104+
105+
## Why the live activation lane is manual
106+
107+
`test/integration/smoke-ios-target-activation.test.ts` is env-gated and deliberately absent from the
108+
`ios.yml` / `replays-manual.yml` test lists. On GitHub-hosted simulators the runner reads its own
109+
target as `.runningForeground` while a foreign app is provably on screen (uploaded artifact of
110+
https://github.com/callstack/agent-device/actions/runs/35460535722/job/105943468492: rendered app
111+
screenshot, Settings screenshot, 31 captures in ~450 ms each, no repair). The runner is then correct
112+
to activate nothing, so a CI assertion on the disclosure measures `XCUIApplication.state` on that
113+
host rather than this feature. Tracked as callstack/agent-device#2696.
114+
115+
Manual evidence stands in for it, and both are reproducible:
116+
- local simulator (`apex-2682-proto`), first capture after the handoff: the shared sentence in
117+
`data.warnings`, printed under the snapshot node count and naming a non-foreground prior state and the
118+
pid probe — `The session app was not foreground when this command
119+
arrived (prior state runningBackground), so the runner activated it before answering (reason
120+
stale_target). Any capture taken earlier in this session described the only app other than the
121+
session app with an active accessibility session (pid 33878), not the session app. Re-capture now
122+
that the session app answers, or drive the other app in its own session.` — and, on this head, the
123+
typed fact beside it as `data.targetActivation`:
124+
`{"reason":"stale_target","priorState":"runningBackground","otherActiveApplicationPid":33878}`.
125+
PR1 alone carries the sentence only; the typed field is this PR's daemon seam. Either way the
126+
following capture reports neither.
127+
- the physical device sequence in this file, which the coordinator runs.

‎.device-evidence/DECISION.md‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# #2682 — target activation disclosure: settled policy
2+
3+
Prototype branch `apex/2682-proto` (throwaway, not shipped). Lane: dedicated iPhone 17 Pro
4+
simulator `apex-2682-proto` (`C2748D97-B92D-4B3B-9C65-AF86494AD904`), iOS 26.2, fixture
5+
`com.callstack.agentdevicelab`. Physical `thymikee-iphone` untouched.
6+
7+
## Policy
8+
9+
**Ship (a): the runner keeps re-activating and discloses it. The (a) prior is CONFIRMED.**
10+
11+
Reproduced on the simulator: an off-app handoff (`xcrun simctl openurl <udid>
12+
https://example.com` — the LaunchServices path an in-app external link takes) put MobileSafari in
13+
the foreground. `screenshot` returned Safari's `Example Domain` page, status bar reading
14+
"◀ Agent Device Tes…". The next `snapshot -i` returned `@e1 [application] "Agent Device Tester" …
15+
@e13 [button] "Home"` — the session app — and said nothing about why. The only record anywhere was
16+
`AGENT_DEVICE_RUNNER_ACTIVATE … state=2 reason=bundle_changed` in `runner.log`. The agent's own two
17+
observations contradicted each other silently.
18+
19+
Latency is not the reason to prefer (a) — disclosure is free there. The state is already read before
20+
`activate()` runs (`RunnerTests+Lifecycle.swift:205`, for the existing NSLog), so the emit sits
21+
inside the branch that already pays `activate()` (measured 36–48 ms). The happy path never enters
22+
that branch: five already-foreground `snapshot -i` runs measured 221–240 ms against 245–262 ms
23+
before the change, and carry no field. (b) would break the screenshot→snapshot flow callers already
24+
use and needs per-caller recovery; (a) preserves behavior and PR1 ships alone.
25+
26+
## Foreground identity: ship `priorState` + `otherActiveApplicationPid`, not a foreground owner
27+
28+
`XCAXClient_iOS` on this runtime answers `activeApplications` and `systemApplication` only;
29+
`frontmostApplication` and `focusedApplication` do not exist there. Each element exposes
30+
`processIdentifier` and nothing else — no `bundleIdentifier`, `executablePath`, `active`, or `state`.
31+
The bundle id is reachable only by escalating: `proc_pidpath(pid)` (no iOS SDK header; symbol
32+
declared by hand) → `NSBundle` → `com.apple.mobilesafari`, cross-checked against host `ps` for the
33+
same pid. That is a new private surface with unknown physical-device behavior, and
34+
`activeApplications` has no proven ordering to identify which element is foreground. So PR1 states a
35+
pid, and only when exactly one foreign application is AX-active; it never guesses a bundle id.
36+
37+
Adversarial review (PR1/PR2) caught the naming over-claiming what that probe proves: with no ordering
38+
in `activeApplications`, the pid establishes only that exactly one other application held an active
39+
accessibility session at that moment — a liveness claim, not a foreground owner. The field is therefore
40+
`otherActiveApplicationPid` and the sentence says the same thing the value proves. `priorState` is
41+
unaffected: it is `XCUIApplication.state` read before `activate()` ran, which is a real fact about the
42+
session app.
43+
44+
## Disclosure as it lands (same simulator, PR1 head)
45+
46+
```
47+
The session app was not foreground when this command arrived (prior state runningBackground), so
48+
the runner activated it before answering (reason stale_target). Any capture taken earlier in this
49+
session described the only app other than the session app with an active accessibility session
50+
(pid 33878), not the session app. Re-capture now that the session app answers, or drive the other
51+
app in its own session.
52+
```
53+
54+
pid 33878 resolved to `…/MobileSafari.app/MobileSafari` by host `ps`. The immediately following
55+
`snapshot -i`, with the app already foreground, carries no disclosure — the fact belongs to the
56+
command that paid for it. `test/integration/smoke-ios-target-activation.test.ts` runs the whole
57+
sequence as an env-gated live lane and passes on this simulator.

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerAXSnapshotBridge.h‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -118,6 +118,10 @@ FOUNDATION_EXPORT NSString *const RunnerAXSnapshotCustomActionsBlockedKey;
118118

119119
+ (NSInteger)processIdentifierForApplication:(XCUIApplication *)application;
120120

121+
/// Pids of the applications the AX client currently reports as active. Same private accessor the
122+
/// snapshot target match uses; pids only — the client resolves no bundle id for an arbitrary app.
123+
+ (NSArray<NSNumber *> *)activeApplicationProcessIdentifiers;
124+
121125
/// The frontier-extension loop, exposed for the runner unit bundle: a frontier
122126
/// whose element vanished or whose re-rooted request fails must count as
123127
/// missed (see the deepExtension keys above) — an all-miss extension reporting

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerAXSnapshotBridge.m‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -707,6 +707,26 @@ + (NSInteger)processIdentifierForApplication:(XCUIApplication *)application
707707
return [self integerFrom:application selectorName:@"processID"];
708708
}
709709

710+
+ (NSArray<NSNumber *> *)activeApplicationProcessIdentifiers
711+
{
712+
id axClient = [self accessibilityClient];
713+
if (nil == axClient) {
714+
return @[];
715+
}
716+
id activeApplications = [self objectFrom:axClient selectorName:@"activeApplications"];
717+
if (![activeApplications isKindOfClass:NSArray.class]) {
718+
return @[];
719+
}
720+
NSMutableArray<NSNumber *> *processIdentifiers = [NSMutableArray array];
721+
for (id candidate in (NSArray *)activeApplications) {
722+
NSInteger processIdentifier = [self integerFrom:candidate selectorName:@"processIdentifier"];
723+
if (processIdentifier > 0) {
724+
[processIdentifiers addObject:@(processIdentifier)];
725+
}
726+
}
727+
return processIdentifiers;
728+
}
729+
710730
+ (id)accessibilityApplicationForApplication:(XCUIApplication *)application axClient:(id)axClient
711731
{
712732
NSInteger targetProcessID = [self integerFrom:application selectorName:@"processID"];

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandExecution.swift‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -949,11 +949,18 @@ extension RunnerTests {
949949

950950
func executeAccepted(command: Command) throws -> Response {
951951
commandJournal.start(command: command)
952+
pendingTargetActivation = nil
952953
do {
953954
let response = try executeDispatched(command: command)
954955
commandJournal.finish(command: command, response: response)
955-
return response
956+
guard let fact = pendingTargetActivation else { return response }
957+
// Stamped after `finish`, like the uptime anchor: a journal-replayed result carries no
958+
// activation fact, because the command that paid for it is the one being replayed, not one
959+
// that just repaired foreground (#2682).
960+
pendingTargetActivation = nil
961+
return response.stampingTargetActivation(fact)
956962
} catch {
963+
pendingTargetActivation = nil
957964
commandJournal.fail(command: command, error: error)
958965
throw error
959966
}

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Lifecycle.swift‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -200,6 +200,20 @@ extension RunnerTests {
200200
return true
201201
}
202202

203+
/// The pid of the one other application holding an active accessibility session, or nil unless
204+
/// exactly one exists. What this proves is that liveness claim and nothing more: the private AX
205+
/// client exposes no ordering of `activeApplications`, so this is NOT a foreground owner — it is
206+
/// the only other process that could have been on screen while the session app sat out of the
207+
/// foreground. The client resolves pids only, answering no bundle id for an arbitrary app, so
208+
/// anything other than exactly one foreign pid stays unstated rather than guessed (#2682).
209+
func soleOtherActiveApplicationPid(excluding sessionPid: Int?) -> Int? {
210+
let pids = RunnerAXSnapshotBridge.activeApplicationProcessIdentifiers().compactMap {
211+
($0 as? NSNumber)?.intValue
212+
}
213+
let foreign = Set(pids.filter { $0 > 0 && $0 != sessionPid })
214+
return foreign.count == 1 ? foreign.first : nil
215+
}
216+
203217
func activateTarget(bundleId: String, reason: String) -> XCUIApplication {
204218
let target = XCUIApplication(bundleIdentifier: bundleId)
205219
let initialState = target.state
@@ -216,7 +230,22 @@ extension RunnerTests {
216230
bundleId
217231
)
218232
} else {
233+
// Read the other app's pid before activating: after `activate()` that app is gone from the
234+
// active set, so the fact would describe the repair instead of the state it repaired (#2682).
235+
let otherActiveApplicationPid = soleOtherActiveApplicationPid(excluding: Self.processIdentifier(of: target))
219236
target.activate()
237+
pendingTargetActivation = TargetActivationFactPayload(
238+
reason: reason,
239+
priorState: Int(initialState.rawValue),
240+
otherActiveApplicationPid: otherActiveApplicationPid
241+
)
242+
NSLog(
243+
"AGENT_DEVICE_RUNNER_ACTIVATE_FACT bundle=%@ reason=%@ priorState=%d otherActiveApplicationPid=%@",
244+
bundleId,
245+
reason,
246+
initialState.rawValue,
247+
otherActiveApplicationPid.map(String.init) ?? "-"
248+
)
220249
}
221250
currentApp = target
222251
currentBundleId = bundleId

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Models.swift‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -248,6 +248,28 @@ extension Response {
248248
payload.runnerMainThreadBusy = value
249249
return Response(ok: ok, data: payload, error: error)
250250
}
251+
252+
/// The serving command had to bring the bound app back to the foreground to answer at all.
253+
/// Stamped on the response of that command, never on a later one (#2682). Only successful
254+
/// responses carry it: a refusal is already the disclosure of a command that did not run.
255+
func stampingTargetActivation(_ value: TargetActivationFactPayload) -> Response {
256+
guard ok else { return self }
257+
var payload = data ?? DataPayload()
258+
payload.targetActivation = value
259+
return Response(ok: ok, data: payload, error: error)
260+
}
261+
}
262+
263+
/// Foreground repair the runner performed while serving one command (#2682). `priorState` is the
264+
/// bound app's `XCApplicationState` raw value read BEFORE `XCUIApplication.activate()` ran, so the
265+
/// fact describes what was repaired rather than what the repair produced. `otherActiveApplicationPid`
266+
/// names the only other application holding an active accessibility session when exactly one existed
267+
/// — a liveness claim, not a foreground owner, since the private AX client exposes no ordering of
268+
/// `activeApplications`, resolves no bundle id for an arbitrary app, and reports only pids.
269+
struct TargetActivationFactPayload: Codable {
270+
let reason: String
271+
let priorState: Int
272+
let otherActiveApplicationPid: Int?
251273
}
252274

253275
struct DataPayload: Codable {
@@ -307,6 +329,7 @@ struct DataPayload: Codable {
307329
var completedSteps: Int?
308330
var failedStepIndex: Int?
309331
var sequenceResults: [SequenceStepResult]?
332+
var targetActivation: TargetActivationFactPayload?
310333
}
311334

312335
/// `kind` mirrors the TS `SnapshotKeyboardBandFact`: "visible" carries `frame`, "unmeasurable"

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests.swift‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,9 @@ final class RunnerTests: XCTestCase {
5555
var currentApp: XCUIApplication?
5656
var currentBundleId: String?
5757
var currentAppProcessIdentifier: Int?
58+
// Set while serving a command that had to re-activate the bound app, and stamped onto that
59+
// command's response before it leaves the execution queue (#2682).
60+
var pendingTargetActivation: TargetActivationFactPayload?
5861
// iOS does not reliably expose hasKeyboardFocus for a bare type request, especially when
5962
// hardware-keyboard input hides the software keyboard. A successful tap on a concrete text
6063
// input is a scoped witness for the immediately-following bare type; lifecycle and non-text

0 commit comments

Comments
 (0)