Skip to content

Commit 4d5422f

Browse files
thymikeeclaude
thymikee
and
claude
committed
fix(ios): keep the bridge while a surface host only lingers
The simulator route sent every capture to XCTest while any registered system surface host process ran for the device. Password AutoFill at sign-in leaves SafariViewService running for the rest of the session, so after logging in to Bluesky every capture took the runner: 3-8s per capture on its feed against about 200ms on the host AX bridge. When a host process is running, ask the live runner for every registered host's XCUIApplication.state, the predicate the runner uses to serve a surface in place. Keep the bridge only when every host reads off screen; a foreground host, an unreadable state or no ready runner keeps the runner route. The read is fenced to the ready runner session, so it never starts or replaces one. Co-Authored-By: Claude Opus 5.5 <[email protected]>
1 parent 86637b2 commit 4d5422f

5 files changed

Lines changed: 251 additions & 2 deletions

File tree

‎docs/adr/0004-ios-snapshot-backend-strategy.md‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -469,8 +469,15 @@ The runner never adopts the host as the cached session target.
469469
The session binding stays on the app.
470470
Thus, after the surface is gone, the next command resolves back to the app.
471471
On the Simulator, a cheap, device-scoped host-side probe checks for a registered host process that runs for the device.
472-
If such a host is running, the probe routes the capture to the runner instead of the bridge.
472+
If such a host is running, the route asks the live runner for the `XCUIApplication.state` of every registered host.
473+
This is the same foreground predicate that the runner uses to serve a surface in place.
474+
If every host reports a state other than foreground, the capture stays on the bridge.
475+
A host process outlives the surface it presented.
476+
For example, password AutoFill at sign-in leaves `SafariViewService` running for the rest of the session.
477+
Otherwise, the capture goes to the runner: a foreground host, an unreadable state, or no ready runner.
478+
The read never starts or replaces a runner.
473479
If no host is running, the bridge fast path is unchanged.
480+
Rejected: remembering a lingering verdict per host process. One process can present again.
474481

475482
A presented surface also takes precedence over an explicitly requested bundle id.
476483
The runner checks for a presented host before it resolves or activates `command.appBundleId`.

‎packages/platform-apple/src/snapshot-route.test.ts‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,46 @@ test.for([presentSurface, 'unknown'] as const)(
9090
},
9191
);
9292

93+
// A host process outlives what it presented: password AutoFill at sign-in leaves SafariViewService
94+
// running for the rest of the session. Only the runner's foreground read of every registered host may
95+
// keep such a capture on the bridge; any other answer keeps the runner route.
96+
test.for([
97+
{ foreground: 'background', bridge: true },
98+
{ foreground: 'foreground', bridge: false },
99+
{ foreground: 'unknown', bridge: false },
100+
] as const)(
101+
'a running surface host the runner reads as $foreground keeps the bridge: $bridge',
102+
async ({ foreground, bridge }) => {
103+
const source = sourceReturning(bridgeAcquisition());
104+
const fallback = vi.fn(async () => runnerResult());
105+
const systemSurfaceForeground = vi.fn(async () => foreground);
106+
const presentIosAcquisition = vi.fn(async () => ({
107+
backend: 'xctest' as const,
108+
producer: 'simulator-ax-bridge' as const,
109+
nodes: [{ index: 0, type: 'Application' }],
110+
}));
111+
const host = {
112+
...platformRuntimeHostFixture(),
113+
snapshot: { captureSurface: vi.fn(), presentIosAcquisition },
114+
};
115+
const route = createAppleSnapshotRoute(host, {
116+
source,
117+
resolveTarget: vi.fn(async () => target),
118+
systemSurfacePresent: async () => presentSurface,
119+
systemSurfaceForeground,
120+
});
121+
122+
await route.capture(ios, { ...input, execution: { requestId: 'req-1' } }, signal(), fallback);
123+
124+
expect(systemSurfaceForeground).toHaveBeenCalledWith(
125+
ios,
126+
expect.objectContaining({ requestId: 'req-1' }),
127+
);
128+
expect(source.acquire).toHaveBeenCalledTimes(bridge ? 1 : 0);
129+
expect(fallback).toHaveBeenCalledTimes(bridge ? 0 : 1);
130+
},
131+
);
132+
93133
// The structural guarantee every comparison site downstream relies on (#2438): a surface capture is
94134
// lineaged to the HOST, not to the app (`target.targetId` here), so its comparison key cannot equal
95135
// an app capture's and no comparison site needs its own surface check.

‎packages/platform-apple/src/snapshot-route.ts‎

Lines changed: 33 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,10 @@ import {
3333
type SimulatorSnapshotTarget,
3434
type SimulatorSnapshotTargetResolver,
3535
} from './snapshot-target.ts';
36+
import {
37+
createSystemSurfaceForegroundProbe,
38+
type SystemSurfaceForegroundProbe,
39+
} from './system-surface-foreground.ts';
3640
import {
3741
createSystemSurfacePresenceProbe,
3842
type SystemSurfacePresenceProbe,
@@ -82,11 +86,14 @@ export function createAppleSnapshotRoute(
8286
source?: SimulatorSnapshotSource;
8387
resolveTarget?: SimulatorSnapshotTargetResolver;
8488
systemSurfacePresent?: SystemSurfacePresenceProbe;
89+
systemSurfaceForeground?: SystemSurfaceForegroundProbe;
8590
}> = {},
8691
): AppleSnapshotRoute {
8792
const source = options.source ?? createSimulatorSnapshotSource(host.hostEnvironment);
8893
const resolveTarget = options.resolveTarget ?? createSimulatorSnapshotTargetResolver();
8994
const systemSurfacePresent = options.systemSurfacePresent ?? createSystemSurfacePresenceProbe();
95+
const systemSurfaceForeground =
96+
options.systemSurfaceForeground ?? createSystemSurfaceForegroundProbe();
9097
const disabledGenerations = new Set<string>();
9198
const latestGeneration = new Map<string, string>();
9299
/**
@@ -133,7 +140,10 @@ export function createAppleSnapshotRoute(
133140
[unknownGenerationResidue()],
134141
);
135142
}
136-
if (surfacePresence !== 'absent') {
143+
if (
144+
surfacePresence !== 'absent' &&
145+
!(await isEverySurfaceHostInBackground(device, input, signal, systemSurfaceForeground))
146+
) {
137147
return await runSurfaceFallback(device, input, fallback, surfacePresence.host.bundleId);
138148
}
139149
let target: SimulatorSnapshotTarget;
@@ -222,6 +232,28 @@ export function createAppleSnapshotRoute(
222232
});
223233
}
224234

235+
/**
236+
* Whether the live runner reports every registered host off screen, which is what lets a capture
237+
* whose host process is only lingering keep the bridge. Only a positive read decides: anything the
238+
* probe could not establish keeps the runner route that serves a presented surface.
239+
*/
240+
async function isEverySurfaceHostInBackground(
241+
device: DeviceInfo,
242+
input: CaptureSnapshotInput,
243+
signal: AbortSignal,
244+
probe: SystemSurfaceForegroundProbe,
245+
): Promise<boolean> {
246+
const requestId = input.execution?.requestId;
247+
const foreground = await probe(device, { signal, ...(requestId ? { requestId } : {}) });
248+
if (foreground !== 'background') return false;
249+
emitDiagnostic({
250+
level: 'debug',
251+
phase: 'ios_snapshot_route_surface_host_background',
252+
data: { deviceId: device.id },
253+
});
254+
return true;
255+
}
256+
225257
/**
226258
* A discovery still in flight is not a failure while no runner can answer instead. The XCTest
227259
* fallback would first wait for a runner start, and #2198 keeps observation off that wait, so the

‎packages/platform-apple/src/system-surface-foreground.test.ts‎

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
import { beforeEach, expect, test, vi } from 'vitest';
2+
import type { DeviceInfo } from '@agent-device/kernel/device';
3+
import { IOS_SYSTEM_SURFACE_HOSTS } from '@agent-device/contracts/ios-system-surface';
4+
5+
const runner = vi.hoisted(() => ({
6+
readRunnerSessionLiveness: vi.fn(),
7+
runAppleRunnerCommand: vi.fn(),
8+
}));
9+
vi.mock('./core/runner-client.ts', () => runner);
10+
11+
import { createSystemSurfaceForegroundProbe } from './system-surface-foreground.ts';
12+
13+
const ios = {
14+
platform: 'apple',
15+
appleOs: 'ios',
16+
id: 'ios-1',
17+
name: 'iPhone',
18+
kind: 'simulator',
19+
target: 'mobile',
20+
booted: true,
21+
} as const satisfies DeviceInfo;
22+
23+
const [safari, passbook] = IOS_SYSTEM_SURFACE_HOSTS.map((host) => host.bundleId);
24+
25+
beforeEach(() => {
26+
runner.readRunnerSessionLiveness.mockReset();
27+
runner.runAppleRunnerCommand.mockReset();
28+
runner.readRunnerSessionLiveness.mockReturnValue({ sessionId: 'runner-1', liveness: 'ready' });
29+
});
30+
31+
function statesByHost(states: Record<string, unknown>) {
32+
runner.runAppleRunnerCommand.mockImplementation(async (_device, command) => ({
33+
applicationState: states[command.appBundleId],
34+
}));
35+
}
36+
37+
function probe() {
38+
return createSystemSurfaceForegroundProbe()(ios, {
39+
signal: new AbortController().signal,
40+
requestId: 'req-1',
41+
});
42+
}
43+
44+
test('every registered host off screen is background, read only from the ready runner session', async () => {
45+
statesByHost({ [safari!]: 'runningBackgroundSuspended', [passbook!]: 'notRunning' });
46+
47+
await expect(probe()).resolves.toBe('background');
48+
expect(
49+
runner.runAppleRunnerCommand.mock.calls.map(([, command, options]) => [command, options]),
50+
).toEqual(
51+
[safari, passbook].map((bundleId) => [
52+
{ command: 'appState', appBundleId: bundleId },
53+
expect.objectContaining({ expectedRunnerSessionId: 'runner-1', requestId: 'req-1' }),
54+
]),
55+
);
56+
});
57+
58+
test('one background host does not hide another one presenting', async () => {
59+
statesByHost({ [safari!]: 'runningBackground', [passbook!]: 'runningForeground' });
60+
61+
await expect(probe()).resolves.toBe('foreground');
62+
});
63+
64+
test.for([
65+
['no runner session', null],
66+
['a runner still starting', { sessionId: 'runner-1', liveness: 'starting' }],
67+
] as const)('%s is unknown and sends no command', async ([, liveness]) => {
68+
runner.readRunnerSessionLiveness.mockReturnValue(liveness);
69+
70+
await expect(probe()).resolves.toBe('unknown');
71+
expect(runner.runAppleRunnerCommand).not.toHaveBeenCalled();
72+
});
73+
74+
test.for([
75+
['an unknown state', 'unknown'],
76+
['an undeclared state', 'suspendedSomehow'],
77+
['no state', undefined],
78+
] as const)('%s is unknown', async ([, state]) => {
79+
statesByHost({ [safari!]: state, [passbook!]: 'notRunning' });
80+
81+
await expect(probe()).resolves.toBe('unknown');
82+
});
83+
84+
test('a failed read is unknown, and a cancelled request still cancels', async () => {
85+
runner.runAppleRunnerCommand.mockRejectedValue(new Error('runner unreachable'));
86+
await expect(probe()).resolves.toBe('unknown');
87+
88+
const controller = new AbortController();
89+
controller.abort(new Error('request cancelled'));
90+
await expect(
91+
createSystemSurfaceForegroundProbe()(ios, { signal: controller.signal }),
92+
).rejects.toThrow('request cancelled');
93+
});

‎packages/platform-apple/src/system-surface-foreground.ts‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
import { IOS_SYSTEM_SURFACE_HOSTS } from '@agent-device/contracts/ios-system-surface';
2+
import type { DeviceInfo } from '@agent-device/kernel/device';
3+
import { isAppleApplicationState } from '@agent-device/kernel/snapshot';
4+
import { readRunnerSessionLiveness, runAppleRunnerCommand } from './core/runner-client.ts';
5+
6+
/**
7+
* Whether any registered iOS system surface host is on screen, asked of the device's live runner
8+
* (#2438). The host-side process probe only proves a host process exists, and a host outlives what
9+
* it presented: password AutoFill at sign-in leaves `SafariViewService` running for the rest of the
10+
* session, which sent every later capture of the app to XCTest. On a deep tree such as Bluesky's feed
11+
* that capture costs 3-8s, where the host AX bridge answers in about 200ms.
12+
*
13+
* `background` is the only answer that may send a capture to the bridge, and it needs every
14+
* registered host to report a state other than foreground: one host lingering in the background says
15+
* nothing about another presenting. Anything else is `foreground` or `unknown`, and both keep the
16+
* runner route, which serves the surface while it is foreground and otherwise the app.
17+
*/
18+
export type SystemSurfaceForeground = 'foreground' | 'background' | 'unknown';
19+
20+
export type SystemSurfaceForegroundProbe = (
21+
device: DeviceInfo,
22+
input: Readonly<{ signal: AbortSignal; requestId?: string }>,
23+
) => Promise<SystemSurfaceForeground>;
24+
25+
/**
26+
* Bounds each read below the read-only resend window, so a runner busy draining abandoned XCTest
27+
* work costs this capture at most the bound and then the runner route it would have taken anyway.
28+
*/
29+
const HOST_STATE_READ_TIMEOUT_MS = 1_000;
30+
31+
const BACKGROUND_STATES: ReadonlySet<string> = new Set([
32+
'notRunning',
33+
'runningBackground',
34+
'runningBackgroundSuspended',
35+
]);
36+
37+
export function createSystemSurfaceForegroundProbe(): SystemSurfaceForegroundProbe {
38+
return async (device, input) => {
39+
const session = readRunnerSessionLiveness(device.id);
40+
// The read never starts a runner: a capture with no ready runner keeps the runner route, which
41+
// starts one for the capture itself, as it did before this probe existed.
42+
if (!session || session.liveness !== 'ready') return 'unknown';
43+
for (const host of IOS_SYSTEM_SURFACE_HOSTS) {
44+
const state = await readHostState(device, host.bundleId, session.sessionId, input);
45+
if (state === 'runningForeground') return 'foreground';
46+
if (state === undefined || !BACKGROUND_STATES.has(state)) return 'unknown';
47+
}
48+
return 'background';
49+
};
50+
}
51+
52+
async function readHostState(
53+
device: DeviceInfo,
54+
bundleId: string,
55+
runnerSessionId: string,
56+
input: Readonly<{ signal: AbortSignal; requestId?: string }>,
57+
): Promise<string | undefined> {
58+
try {
59+
// `appState` brings no app forward and binds nothing: it reads the named bundle's
60+
// `XCUIApplication.state`, the predicate the runner itself uses to serve a surface in place.
61+
// The expected session id refuses a replacement runner, so the read cannot start or restart one.
62+
const result = await runAppleRunnerCommand(
63+
device,
64+
{ command: 'appState', appBundleId: bundleId },
65+
{
66+
signal: AbortSignal.any([input.signal, AbortSignal.timeout(HOST_STATE_READ_TIMEOUT_MS)]),
67+
expectedRunnerSessionId: runnerSessionId,
68+
...(input.requestId ? { requestId: input.requestId } : {}),
69+
},
70+
);
71+
const state = result.applicationState;
72+
return isAppleApplicationState(state) ? state : undefined;
73+
} catch {
74+
input.signal.throwIfAborted();
75+
return undefined;
76+
}
77+
}

0 commit comments

Comments
 (0)