Skip to content

Commit 57677e0

Browse files
thymikeeclaude
andcommitted
perf(ios): check surface hosts inside the snapshot bridge, not with pgrep
A warm Simulator snapshot served by the host AX bridge has no ready runner to read the surface hosts' foreground state, so every capture ran the host-side presence probe first: two `pgrep -f` scans and their `ps` reads, about 45ms of a 75ms snapshot (#2751). When no runner can answer, the route now names the registered host executables in the bridge request. The guest reads the process table in place with sysctl, before and after the capture, and refuses with `system-surface-host-present` while a host process carries this Simulator's SIMULATOR_UDID, or `system-surface-host-unverified` when it cannot place one. The route sends both to the runner, as the probe's `present` and `unknown` did, and they never retire the app generation. The host-side probe is removed. `snapshotDiagnostics.stats.backends` now counts the producer that served each capture, so bridge captures read `simulator-ax-bridge` instead of the shared `xctest` channel. Fixes #2751 Co-Authored-By: Claude Opus 5.5 <[email protected]>
1 parent 7b3f63f commit 57677e0

35 files changed

Lines changed: 1151 additions & 630 deletions

‎apple/snapshot-bridge/README.md‎

Lines changed: 22 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,11 @@ UTF-8 JSON object
1717
```
1818

1919
The host owns all target identity, bounds, deadlines, and lifecycle decisions.
20-
The guest returns only a bounded raw tree, the target pid, truncation, and
21-
protocol/source versions. It does not expose an HTTP route or a public CLI
22-
surface.
20+
A successful response carries only a bounded raw tree, the target pid,
21+
truncation, and protocol/source versions. A refusal carries the target pid, a
22+
typed error kind and code, and, for a running surface host, the
23+
`surfaceHostExecutable` it found. It does not expose an HTTP route or a public
24+
CLI surface.
2325

2426
The private API is intentionally pinned to the idb v1.5.2-compatible shape.
2527
See `LICENSE.idb` for attribution.
@@ -50,6 +52,23 @@ not replace the native primary owner. An ownership refusal is evidence about
5052
the screen, so the route sends only that capture to XCTest and keeps the app
5153
generation on the bridge.
5254

55+
## System surface hosts
56+
57+
A registered system surface host, such as `SafariViewService` for a web sign-in
58+
sheet, presents out of the app's process. While its sheet is up, the app stays
59+
the primary owner, so the ownership check passes over the occluded app tree.
60+
When a request names `surfaceHostExecutables` and `simulatorUdid`, the reader
61+
reads the process table in place before acquisition, and again after an
62+
acquisition that returned a tree. If a process of a named executable carries
63+
this Simulator's `SIMULATOR_UDID` and its executable path ends with the named
64+
executable, it returns the typed refusal
65+
`system-surface-host-present` with `surfaceHostExecutable`, without the app
66+
tree. If it cannot read the process table or place a matching process, it
67+
returns `system-surface-host-unverified`. The host sends that capture to
68+
XCTest and keeps the app generation on the bridge, because both refusals are
69+
evidence about the screen. The host omits the check when a ready runner already
70+
reported every host off screen.
71+
5372
## Window placement
5473

5574
The snapshot API reports the frames of an app that runs in an iPadOS window

‎apple/snapshot-bridge/SnapshotBridge.m‎

Lines changed: 76 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
*/
55

66
#import "SnapshotBridgeRuntime.h"
7+
#import "SnapshotBridgeSurfaceHosts.h"
78

89
#import <Foundation/Foundation.h>
910

@@ -52,6 +53,65 @@ static BOOL validBoundInteger(id value, NSUInteger minimum, NSUInteger maximum,
5253
return YES;
5354
}
5455

56+
/*
57+
* A registered iOS system surface host (a web sign-in or Apple Pay sheet) presents out of the app's
58+
* process, and while it is up the app stays the primary accessibility owner, so this reader would
59+
* serve the occluded app tree as if healthy (#2438). When the host asks, the reader refuses the
60+
* capture while a host process runs for this Simulator. It reads the process table in place: a
61+
* spawned `pgrep` and `ps` cost each capture about 45ms (#2751).
62+
*/
63+
static NSString *const kSurfaceHostExecutablesKey = @"surfaceHostExecutables";
64+
static NSString *const kSimulatorUdidKey = @"simulatorUdid";
65+
static NSString *const kSurfaceHostExecutableKey = @"surfaceHostExecutable";
66+
static const NSUInteger kMaximumSurfaceHosts = 8;
67+
static const NSUInteger kMaximumSurfaceHostExecutableLength = 256;
68+
static const NSUInteger kMaximumSimulatorUdidLength = 64;
69+
70+
/* The refusal for a capture a running surface host may cover, or nil when none runs. */
71+
static NSDictionary *_Nullable captureRefusalForSurfaceHosts(NSArray<NSString *> *executables,
72+
NSString *simulatorUdid,
73+
NSString *requestId)
74+
{
75+
BOOL unverified = NO;
76+
NSString *running = runningSurfaceHost(executables, simulatorUdid, &unverified);
77+
if (running) {
78+
NSMutableDictionary *refusal = [failureResponse(requestId, @"unsupported", @"system-surface-host-present",
79+
@"a system surface host is running for this Simulator") mutableCopy];
80+
refusal[kSurfaceHostExecutableKey] = running;
81+
return refusal;
82+
}
83+
if (unverified) {
84+
return failureResponse(requestId, @"unsupported", @"system-surface-host-unverified",
85+
@"a system surface host process could not be placed on this Simulator");
86+
}
87+
return nil;
88+
}
89+
90+
/* The requested surface host check: nil when the request asks for none, NSNull when malformed. */
91+
static id surfaceHostExecutables(NSDictionary *request)
92+
{
93+
id value = request[kSurfaceHostExecutablesKey];
94+
if (value == nil) return nil;
95+
if (![value isKindOfClass:NSArray.class] || [value count] == 0 || [value count] > kMaximumSurfaceHosts) {
96+
return NSNull.null;
97+
}
98+
for (id executable in value) {
99+
if (![executable isKindOfClass:NSString.class] || [executable length] == 0 ||
100+
[executable length] > kMaximumSurfaceHostExecutableLength) {
101+
return NSNull.null;
102+
}
103+
}
104+
return value;
105+
}
106+
107+
static NSDictionary *withTargetIdentity(NSDictionary *response, NSNumber *pid, NSString *generation)
108+
{
109+
NSMutableDictionary *annotated = [response mutableCopy];
110+
annotated[@"pid"] = pid;
111+
annotated[@"generation"] = generation;
112+
return annotated;
113+
}
114+
55115
static NSDictionary *handleRequest(NSDictionary *request)
56116
{
57117
NSString *requestId = [request[kRequestIdKey] isKindOfClass:NSString.class] ? request[kRequestIdKey] : @"";
@@ -90,6 +150,16 @@ static BOOL validBoundInteger(id value, NSUInteger minimum, NSUInteger maximum,
90150
!validBoundInteger(request[@"nativeLevelsHint"], 1, kMaximumDepth + 1, &nativeLevelsHint)) {
91151
return failureResponse(requestId, @"bad_request", @"bounds-invalid", @"nativeLevelsHint is outside the bridge depth limits");
92152
}
153+
id surfaceHosts = surfaceHostExecutables(request);
154+
NSString *simulatorUdid = [request[kSimulatorUdidKey] isKindOfClass:NSString.class] ? request[kSimulatorUdidKey] : @"";
155+
if (surfaceHosts == NSNull.null ||
156+
(surfaceHosts && (simulatorUdid.length == 0 || simulatorUdid.length > kMaximumSimulatorUdidLength))) {
157+
return failureResponse(requestId, @"bad_request", @"surface-hosts-invalid", @"surfaceHostExecutables needs bounded executable names and a simulatorUdid");
158+
}
159+
if (surfaceHosts) {
160+
NSDictionary *refusal = captureRefusalForSurfaceHosts(surfaceHosts, simulatorUdid, requestId);
161+
if (refusal) return withTargetIdentity(refusal, pidValue, generation);
162+
}
93163

94164
NSString *setupError = nil;
95165
BridgeRuntime *runtime = sharedRuntime(&setupError);
@@ -108,13 +178,13 @@ static BOOL validBoundInteger(id value, NSUInteger minimum, NSUInteger maximum,
108178
generation:generation
109179
maxDurationMs:maxDurationMs
110180
error:&error];
111-
if (response) return response;
112-
if (error) {
113-
NSMutableDictionary *annotated = [error mutableCopy];
114-
annotated[@"pid"] = pidValue;
115-
annotated[@"generation"] = generation;
116-
return annotated;
181+
if (response && surfaceHosts) {
182+
// A sheet presented during the capture covers what it read.
183+
NSDictionary *refusal = captureRefusalForSurfaceHosts(surfaceHosts, simulatorUdid, requestId);
184+
if (refusal) return withTargetIdentity(refusal, pidValue, generation);
117185
}
186+
if (response) return response;
187+
if (error) return withTargetIdentity(error, pidValue, generation);
118188
return failureResponse(requestId, @"reader_unavailable", @"empty-response", @"AX bridge returned no response");
119189
}
120190

‎apple/snapshot-bridge/SnapshotBridgeRuntime.m‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@
1818
NSString *const kProtocolVersionKey = @"protocolVersion";
1919
NSString *const kSourceVersionKey = @"sourceVersion";
2020
NSString *const kRequestIdKey = @"requestId";
21-
NSString *const kSourceVersion = @"agent-device-simulator-ax-v1.9.0";
21+
NSString *const kSourceVersion = @"agent-device-simulator-ax-v1.10.0";
2222
const NSUInteger kProtocolVersion = 1;
2323
const uint32_t kMaximumFrameBytes = 16 * 1024 * 1024;
2424
const NSUInteger kMaximumDepth = 128;
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
#import <Foundation/Foundation.h>
2+
3+
NS_ASSUME_NONNULL_BEGIN
4+
5+
/*
6+
* The first of `executables` with a process running for the Simulator `simulatorUdid`, or nil.
7+
* When it returns nil, `unverified` reports that the scan proved nothing: the process table could
8+
* not be read, or a live candidate process could not be placed on a Simulator. A candidate that
9+
* exited, gone or a zombie, presents nothing and is skipped.
10+
*/
11+
NSString *_Nullable runningSurfaceHost(NSArray<NSString *> *executables,
12+
NSString *simulatorUdid,
13+
BOOL *unverified);
14+
15+
NS_ASSUME_NONNULL_END
Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
/*
2+
* The process-table scan behind the reader's system surface host check. It runs in the reader's own
3+
* process, so the check spawns nothing.
4+
*/
5+
6+
#import "SnapshotBridgeSurfaceHosts.h"
7+
8+
#import <errno.h>
9+
#import <stdlib.h>
10+
#import <string.h>
11+
#import <sys/sysctl.h>
12+
13+
typedef NS_ENUM(NSInteger, SurfaceHostScan) {
14+
SurfaceHostAbsent,
15+
SurfaceHostPresent,
16+
SurfaceHostUnverified,
17+
};
18+
19+
static const char kSimulatorUdidPrefix[] = "SIMULATOR_UDID=";
20+
21+
/*
22+
* Whether a process has exited: gone from the process table, or a zombie waiting to be reaped. The
23+
* kernel keeps no arguments for either, and neither presents anything, so a host that exited
24+
* between the table read and its argument read is absent, not unplaceable.
25+
*/
26+
static BOOL processExited(pid_t pid)
27+
{
28+
int mib[4] = {CTL_KERN, KERN_PROC, KERN_PROC_PID, pid};
29+
struct kinfo_proc process;
30+
size_t size = sizeof(process);
31+
if (sysctl(mib, 4, &process, &size, NULL, 0) != 0) return errno == ESRCH;
32+
return size == 0 || process.kp_proc.p_stat == SZOMB;
33+
}
34+
35+
/*
36+
* Whether one process runs this host's binary for this Simulator. KERN_PROCARGS2 holds argc, the
37+
* executable path, then argv and envp as NUL-terminated strings. Simulator processes carry
38+
* SIMULATOR_UDID, so only a scope naming another device is a negative. A live process whose
39+
* arguments cannot be read, or whose environment carries no scope, proves nothing.
40+
*/
41+
static SurfaceHostScan scanProcess(pid_t pid, const char *executable, const char *udidAssignment)
42+
{
43+
int argumentMaximum = 0;
44+
size_t size = sizeof(argumentMaximum);
45+
int argumentMaximumMib[2] = {CTL_KERN, KERN_ARGMAX};
46+
if (sysctl(argumentMaximumMib, 2, &argumentMaximum, &size, NULL, 0) != 0 || argumentMaximum <= 0) {
47+
return SurfaceHostUnverified;
48+
}
49+
char *buffer = malloc((size_t)argumentMaximum);
50+
if (!buffer) return SurfaceHostUnverified;
51+
int mib[3] = {CTL_KERN, KERN_PROCARGS2, pid};
52+
size = (size_t)argumentMaximum;
53+
if (sysctl(mib, 3, buffer, &size, NULL, 0) != 0 || size <= sizeof(int)) {
54+
free(buffer);
55+
return processExited(pid) ? SurfaceHostAbsent : SurfaceHostUnverified;
56+
}
57+
int argc = 0;
58+
memcpy(&argc, buffer, sizeof(argc));
59+
if (argc < 0) {
60+
free(buffer);
61+
return SurfaceHostUnverified;
62+
}
63+
const char *cursor = buffer + sizeof(int);
64+
const char *end = buffer + size;
65+
size_t pathLength = strnlen(cursor, (size_t)(end - cursor));
66+
if (cursor + pathLength >= end) {
67+
free(buffer);
68+
return SurfaceHostUnverified;
69+
}
70+
// The fragment must end the executable path at a path component, so a sibling such as
71+
// `Name.app/NameHelper` or `OtherName.app/Name` is not the host.
72+
size_t fragmentLength = strlen(executable);
73+
const char *suffix = pathLength >= fragmentLength ? cursor + pathLength - fragmentLength : NULL;
74+
if (!suffix || strcmp(suffix, executable) != 0 || (suffix > cursor && suffix[-1] != '/')) {
75+
free(buffer);
76+
return SurfaceHostAbsent;
77+
}
78+
cursor += pathLength;
79+
while (cursor < end && *cursor == '\0') cursor += 1;
80+
for (int skipped = 0; skipped < argc && cursor < end; skipped += 1) {
81+
cursor += strnlen(cursor, (size_t)(end - cursor)) + 1;
82+
}
83+
if (cursor > end) cursor = end;
84+
// The kernel may pad argv before the environment.
85+
while (cursor < end && *cursor == '\0') cursor += 1;
86+
// A matching scope anywhere in the environment places the process here. Another scope proves
87+
// absence only when the walk reached the environment's end, so an entry order or a truncated
88+
// block cannot hide this Simulator's scope.
89+
BOOL sawOtherScope = NO;
90+
BOOL walkedToEnd = NO;
91+
while (cursor < end) {
92+
if (*cursor == '\0') {
93+
walkedToEnd = YES;
94+
break;
95+
}
96+
size_t length = strnlen(cursor, (size_t)(end - cursor));
97+
if (cursor + length >= end) break;
98+
if (strncmp(cursor, kSimulatorUdidPrefix, sizeof(kSimulatorUdidPrefix) - 1) == 0) {
99+
if (strcmp(cursor, udidAssignment) == 0) {
100+
free(buffer);
101+
return SurfaceHostPresent;
102+
}
103+
sawOtherScope = YES;
104+
}
105+
cursor += length + 1;
106+
}
107+
// The last entry's terminator can be the block's last byte.
108+
if (cursor == end) walkedToEnd = YES;
109+
free(buffer);
110+
return sawOtherScope && walkedToEnd ? SurfaceHostAbsent : SurfaceHostUnverified;
111+
}
112+
113+
/*
114+
* The first registered host running for this Simulator, or nil. The kernel keeps the executable's
115+
* basename cut to MAXCOMLEN as the process name, so the name selects the few candidates whose
116+
* arguments are read. `unverified` reports a candidate the scan could not place.
117+
*/
118+
NSString *_Nullable runningSurfaceHost(NSArray<NSString *> *executables,
119+
NSString *simulatorUdid,
120+
BOOL *unverified)
121+
{
122+
*unverified = NO;
123+
int mib[3] = {CTL_KERN, KERN_PROC, KERN_PROC_ALL};
124+
struct kinfo_proc *processes = NULL;
125+
size_t length = 0;
126+
for (int attempt = 0; attempt < 3 && !processes; attempt += 1) {
127+
if (sysctl(mib, 3, NULL, &length, NULL, 0) != 0) break;
128+
// Room for processes that start between the size read and the copy.
129+
length += 32 * sizeof(struct kinfo_proc);
130+
processes = malloc(length);
131+
if (!processes) break;
132+
if (sysctl(mib, 3, processes, &length, NULL, 0) == 0) break;
133+
const int error = errno;
134+
free(processes);
135+
processes = NULL;
136+
if (error != ENOMEM) break;
137+
}
138+
if (!processes) {
139+
*unverified = YES;
140+
return nil;
141+
}
142+
NSString *udidAssignment = [NSString stringWithFormat:@"%s%@", kSimulatorUdidPrefix, simulatorUdid];
143+
NSString *running = nil;
144+
size_t count = length / sizeof(struct kinfo_proc);
145+
for (NSString *executable in executables) {
146+
const char *processName = executable.lastPathComponent.UTF8String;
147+
for (size_t index = 0; index < count && !running; index += 1) {
148+
if (strncmp(processes[index].kp_proc.p_comm, processName, MAXCOMLEN) != 0) continue;
149+
if (processes[index].kp_proc.p_stat == SZOMB) continue;
150+
SurfaceHostScan scan = scanProcess(processes[index].kp_proc.p_pid, executable.UTF8String,
151+
udidAssignment.UTF8String);
152+
if (scan == SurfaceHostPresent) running = executable;
153+
if (scan == SurfaceHostUnverified) *unverified = YES;
154+
}
155+
if (running) break;
156+
}
157+
free(processes);
158+
return running;
159+
}

‎contracts/fixtures/ios-system-surface-hosts.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{
2-
"description": "iOS out-of-process system surfaces that agent-device observes and drives IN PLACE, never by activation: activating or relaunching such a host destroys what it presents (issue #2438; rationale in docs/adr/0004). Each host's `note` says what it hosts and why it is registered. Source of truth shared by the TypeScript registry (packages/contracts/src/ios-system-surface.ts) and the Swift runner registry (RunnerSystemSurfaceHostPolicy.swift); a change here must keep both parity tests green. `processExecutable` is the simulator app-binary path fragment the TypeScript host-side presence probe matches with `pgrep -f`, confirming device scope from the matched process's environment; the Swift runner detects the host by bundle id via XCUIApplication.state and ignores it.",
2+
"description": "iOS out-of-process system surfaces that agent-device observes and drives IN PLACE, never by activation: activating or relaunching such a host destroys what it presents (issue #2438; rationale in docs/adr/0004). Each host's `note` says what it hosts and why it is registered. Source of truth shared by the TypeScript registry (packages/contracts/src/ios-system-surface.ts) and the Swift runner registry (RunnerSystemSurfaceHostPolicy.swift); a change here must keep both parity tests green. `processExecutable` is the simulator app-binary path fragment the snapshot bridge guest matches against the end of each process's executable path, at a path component, confirming device scope from the matched process's environment; the Swift runner detects the host by bundle id via XCUIApplication.state and ignores it.",
33
"hosts": [
44
{
55
"bundleId": "com.apple.SafariViewService",

‎docs/adr/0004-ios-snapshot-backend-strategy.md‎

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -494,12 +494,18 @@ A host process outlives the surface it presented.
494494
For example, password AutoFill at sign-in leaves `SafariViewService` running for the rest of the session.
495495
If a host reports foreground, the capture goes to the runner.
496496
The read never starts or replaces a runner.
497-
If no runner is ready or the read fails, a device-scoped host-side probe checks for a registered host process that runs for the device.
498-
If such a process is running, the capture goes to the runner.
497+
If no runner is ready or the read fails, the bridge request names every registered host executable.
498+
Before the capture, and again after a capture that returned a tree, the bridge guest reads the process table in place for a host process that carries this device's `SIMULATOR_UDID`.
499+
If such a process is running, the guest refuses with `system-surface-host-present`, and the capture goes to the runner.
499500
If no such process is running, the capture stays on the bridge.
500-
If the probe cannot answer, the capture goes to the runner with the reason `system-surface-probe-unavailable`.
501-
The runner read comes first because the probe spawns processes on every capture.
501+
If the guest cannot read the process table or place a live host process, it refuses with `system-surface-host-unverified`, and the capture goes to the runner with that reason.
502+
A host process that exited, whether gone or a zombie, is absent: it presents nothing, and an unverified answer would keep every warm capture on the Mac off the bridge for as long as it lingered.
503+
The route resolved the app generation before the bridge refused, so that runner capture keeps the generation; the runner's own surface stamp still identifies a sheet it served.
504+
These refusals describe the screen, so they never retire the app generation.
505+
The check spawns no process, so a warm bridge capture costs no host process probe.
502506
Rejected: remembering a lingering verdict per host process. One process can present again.
507+
Rejected: caching an absent verdict until the next action. A sheet presents after the action that requests it, so the first poll would cache absence before the host starts.
508+
Rejected: a host-side `pgrep` and `ps` probe before each capture. It cost each warm capture about 45 ms (#2751).
503509

504510
A presented surface also takes precedence over an explicitly requested bundle id.
505511
The runner checks for a presented host before it resolves or activates `command.appBundleId`.

‎packages/contracts/src/ios-system-surface.ts‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -22,9 +22,10 @@ export type IosSystemSurfaceHost = Readonly<{
2222
bundleId: string;
2323
kind: IosSystemSurfaceKind;
2424
/**
25-
* Simulator app-binary path fragment the host-side presence probe matches with `pgrep -f`. Only
26-
* a matched pid's environment is then read, to confirm the process belongs to the requested
27-
* device. The Swift runner detects the host by bundle id (`XCUIApplication.state`) instead.
25+
* Simulator app-binary path suffix the snapshot bridge guest matches against the end of each
26+
* process's executable path, at a path component. Only a matched process's environment is then read, to confirm the process
27+
* belongs to the requested device. The Swift runner detects the host by bundle id
28+
* (`XCUIApplication.state`) instead.
2829
*/
2930
processExecutable: string;
3031
}>;

0 commit comments

Comments
 (0)