Skip to content

Commit 7587364

Browse files
committed
fix(apple): name an Xcode too old for its host macOS as a runner startup failure
An Xcode older than the macOS it sits on cannot load its own DTServiceHub: AMFI refuses its version-restriction entitlement, so every `xcodebuild test-without-building` writes "DTServiceHubClient failed to bless service hub" into runner.log and the runner never serves a command. Callers saw only `Runner did not accept connection`, ordered by a classifier that had no name for this cause, so the request restarted the session and rebuilt a binary this macOS refuses to load (#3449). Classify the bless refusal from the launch output into the typed reason `simulator_service_hub_unavailable`, terminal on every recovery axis, with a hint to select a supporting Xcode on the daemon host (`xcode-select -s` or DEVELOPER_DIR). Both connect-stage failures now read the launch log through one classifier, so the message, reason, hint and logPath cannot disagree.
1 parent f633525 commit 7587364

8 files changed

Lines changed: 370 additions & 43 deletions

‎packages/platform-apple/src/runner/__tests__/runner-error-classification.test.ts‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -250,6 +250,39 @@ test('only a runner that never accepted a connection indicts the cached artifact
250250
assert.equal(shouldRebuildCachedRunnerArtifact(commandFailed('fetch failed')), false);
251251
});
252252

253+
test('a simulator service this macOS refuses to load is terminal on every recovery axis', () => {
254+
// The reason the launch paths publish is the whole fact: this Xcode fails every launch on this
255+
// macOS, so no restart, resend, replay, or clean rebuild may be ordered for it (#3449).
256+
const blessRefusal = commandFailed('Runner did not accept connection', {
257+
reason: 'simulator_service_hub_unavailable',
258+
runnerConnectFailureReason: 'runner_connect_refused',
259+
dispatched: 'no',
260+
});
261+
assert.equal(shouldRetryRunnerConnectError(blessRefusal), false);
262+
assert.equal(isRetryableRunnerError(blessRefusal), false);
263+
assert.equal(shouldRestartRunnerBeforeCommandSend(blessRefusal, SNAPSHOT), false);
264+
assert.equal(shouldRebuildCachedRunnerArtifact(blessRefusal), false);
265+
// The preflight's restart marker cannot outvote it either: the preflight would boot the same
266+
// runner that just failed to load its service hub.
267+
assert.equal(
268+
shouldRestartRunnerAfterReadinessPreflight(
269+
commandFailed('Runner readiness refused', {
270+
reason: 'simulator_service_hub_unavailable',
271+
runnerReadinessPreflightFailed: true,
272+
}),
273+
),
274+
false,
275+
);
276+
});
277+
278+
test('a refused connection still restarts and rebuilds without the service-hub reason', () => {
279+
// The closest negative: the same connect-shaped failure a host with a supported Xcode produces.
280+
const refused = unwrittenConnectRefusal();
281+
assert.equal(shouldRetryRunnerConnectError(refused), true);
282+
assert.equal(shouldRestartRunnerBeforeCommandSend(refused, TAP), true);
283+
assert.equal(shouldRebuildCachedRunnerArtifact(refused), true);
284+
});
285+
253286
test('a device that is busy connecting is a terminal connect verdict', () => {
254287
assert.equal(
255288
shouldRetryRunnerConnectError(commandFailed('Device is busy (Connecting to Simulator)')),

‎packages/platform-apple/src/runner/__tests__/runner-startup-failure-fixtures.ts‎

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,10 @@ import { IOS_SIMULATOR } from './device-fixtures.ts';
1515
*
1616
* - `captured` — `output` was pasted from a run, and `command` plus `xcodeVersion` (from
1717
* `xcodebuild -version`) were recorded with it by `.device-evidence/CHECKLIST-runner-failures.md`.
18+
* - `reported-log-excerpt` — `output` was pasted from a `runner.log` someone else ran, and the
19+
* versions in `xcodeVersion` are the ones that reporter stated. No `command` and no checklist row:
20+
* the host that produced it is not reachable from here, which is also why no run on this machine
21+
* can promote such an entry to `captured`.
1822
* - `shipped-sniff-trigger` — the substrings a rule matches are the ones shipped in
1923
* `resolveSigningFailureHint` before #2680, which is evidence xcodebuild can emit them. The
2024
* sentence around them is ours, so `command` and `xcodeVersion` stay unrecorded.
@@ -44,6 +48,7 @@ import { IOS_SIMULATOR } from './device-fixtures.ts';
4448

4549
export type RunnerStartupFailureSite =
4650
| 'build-for-testing'
51+
| 'launch-test-without-building'
4752
| 'host-dev-tools-security'
4853
| 'device-readiness';
4954

@@ -109,7 +114,7 @@ export type RunnerStartupFailureFixture = Readonly<{
109114
command?: string;
110115
/** `xcodebuild -version` recorded from that run, or `unobserved`. */
111116
xcodeVersion: string;
112-
provenance: 'captured' | 'shipped-sniff-trigger' | 'invented-shape';
117+
provenance: 'captured' | 'reported-log-excerpt' | 'shipped-sniff-trigger' | 'invented-shape';
113118
/** The tool's own stdout/stderr. */
114119
output: string;
115120
/** The argv the exec reported, which is never evidence of a cause (#2680). */
@@ -390,6 +395,19 @@ export const RUNNER_STARTUP_FAILURE_FIXTURES: readonly RunnerStartupFailureFixtu
390395
deviceReport: { developerMode: 'enabled', developerDiskImage: 'unavailable' },
391396
note: 'The decisive pairing, and the one #2682 used to answer with Developer Mode advice: the toggle is on and only the image is down. It is NOT a pre-build refusal (#2683 review): iOS 17+ mounts the image on demand during build and launch, so this report has to survive to a failure — which is what `unclassified-build-on-device-with-image-down` records. The enabled half is captured on some devices; a device waiting on device support has not been captured.',
392397
},
398+
{
399+
id: 'simulator-service-hub-bless-refused',
400+
reason: 'simulator_service_hub_unavailable',
401+
site: 'launch-test-without-building',
402+
xcodeVersion: 'Xcode 26.6 / Build version 17F113',
403+
provenance: 'reported-log-excerpt',
404+
output:
405+
'2026-10-15 10:23:41.112 xcodebuild[40217:166902] [MT] iOSSimulator: <DVTiOSDevice (0x1234), iPhone, 26.5 (23F770a), abcdef0123456789abcdef0123456789abcdef01> unable to connect to "com.apple.instruments.deviceservice.lockdown"\n' +
406+
'(Error Domain=DVTiPhoneSimulatorErrorDomain Code=-1 "DTServiceHubClient failed to bless service hub for simulator iPhone (abcdef0123456789abcdef0123456789abcdef01)" UserInfo={NSLocalizedDescription=DTServiceHubClient failed to bless service hub for simulator iPhone (abcdef0123456789abcdef0123456789abcdef01), NSUnderlyingError=0x1234 {Error Domain=DTServiceHubClient Code=1 "\'localDeviceConnectionWithError:\' failed but didn\'t provide an error."}})\n' +
407+
'** TEST EXECUTE FAILED **\n' +
408+
'** BUILD INTERRUPTED **\n',
409+
note: 'Reported on #3449 from two hosts (macOS 27.2 with Xcode 26.6, macOS 27.2 with Xcode 26.5), pasted from `runner.log`. The issue quotes the bless line and the `DTServiceHubClient` line with their middles elided; the device description, the pids and pointers, and the repeated `NSLocalizedDescription` are filled in with placeholders in the shape Xcode writes, and `** BUILD INTERRUPTED **` is the line the report says follows. The row keys on the two phrases it quotes verbatim. Not inducible here: it needs an Xcode older than the host macOS supports, and the AMFI refusal behind it is in the kernel log (`com.apple.DTServiceHub disallowed without correct com.apple.private.amfi.version-restriction entitlement version`).',
410+
},
393411
{
394412
id: 'devtools-security-disabled',
395413
reason: 'devtools_security_developer_mode_disabled',
Lines changed: 172 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
import assert from 'node:assert/strict';
2+
import fs from 'node:fs';
3+
import path from 'node:path';
4+
import { beforeEach, test, vi } from 'vitest';
5+
import { AppError, normalizeError } from '@agent-device/kernel/errors';
6+
import { resetAllProcessMemosForTests } from '@agent-device/kernel/ttl-memo';
7+
import type { ExecBackgroundResult } from '@agent-device/host-kit/command';
8+
import { buildRunnerEarlyExitError, waitForRunner } from '../runner-startup-transport.ts';
9+
import { readRunnerLogTail } from '../runner-io.ts';
10+
import { RunnerCommandAccounting, type RunnerSession } from '../runner-session-types.ts';
11+
import {
12+
isRetryableRunnerError,
13+
shouldRebuildCachedRunnerArtifact,
14+
shouldRestartRunnerAfterReadinessPreflight,
15+
shouldRestartRunnerBeforeCommandSend,
16+
shouldRetryRunnerConnectError,
17+
} from '../runner-error-classification.ts';
18+
import { RUNNER_ENDPOINT_FIXTURE, iosSimulator } from './runner-transport.fixtures.ts';
19+
import { buildFixtureById } from './runner-startup-failure-fixtures.ts';
20+
import { mkdtempForTestSync } from './tmp-dir.ts';
21+
22+
beforeEach(resetAllProcessMemosForTests);
23+
24+
/**
25+
* An Xcode older than the macOS it sits on cannot load its own `DTServiceHub`: AMFI refuses its
26+
* version-restriction entitlement (#3449). Every `xcodebuild test-without-building` then writes the
27+
* bless refusal into `runner.log` while the runner never serves a command, and the caller only ever
28+
* saw `Runner did not accept connection` on a runner the request had just restarted. The cause is
29+
* readable nowhere but that log, so both connect-stage failures have to read it, name it, and refuse
30+
* the restart and resend they would otherwise order. These cases drive the real producers.
31+
*/
32+
33+
const BLESS_LOG = buildFixtureById('simulator-service-hub-bless-refused').output;
34+
35+
function sessionWithLaunchLog(log: string, exitCode: number | null): RunnerSession {
36+
const runnerLogPath = path.join(mkdtempForTestSync('runner-service-hub-'), 'runner.log');
37+
fs.writeFileSync(runnerLogPath, log);
38+
return {
39+
sessionId: 'service-hub-session',
40+
device: iosSimulator,
41+
deviceId: iosSimulator.id,
42+
endpoint: RUNNER_ENDPOINT_FIXTURE,
43+
xctestrunPath: '/tmp/runner.xctestrun',
44+
jsonPath: '/tmp/runner.json',
45+
runnerLogPath,
46+
readLogTail: (maxBytes) =>
47+
readRunnerLogTail({ logPath: runnerLogPath, startOffset: 0 }, maxBytes),
48+
testPromise:
49+
exitCode === null
50+
? new Promise<never>(() => {})
51+
: Promise.resolve({ exitCode, stdout: '', stderr: '' }),
52+
child: { pid: 4242, exitCode } as ExecBackgroundResult['child'],
53+
state: 'starting',
54+
commandCharges: new RunnerCommandAccounting(),
55+
};
56+
}
57+
58+
/** The device states a simulator startup carries; the bless cause must outrank them (#2683). */
59+
const SIMULATOR_WITH_IMAGE_DOWN = {
60+
developerMode: 'enabled' as const,
61+
developerDiskImage: 'unavailable' as const,
62+
developerDiskImageHint: 'Unlock the iPhone so it can mount the developer disk image.',
63+
};
64+
65+
test('a launch that cannot bless the simulator service names the Xcode cause and its fix', async () => {
66+
const session = {
67+
...sessionWithLaunchLog(BLESS_LOG, 65),
68+
startupDeviceStates: SIMULATOR_WITH_IMAGE_DOWN,
69+
};
70+
71+
const error = await buildRunnerEarlyExitError({ session, port: 8100 });
72+
const envelope = normalizeError(error, { diagnosticId: 'diag-service-hub' });
73+
74+
assert.equal(envelope.code, 'COMMAND_FAILED');
75+
assert.equal(
76+
envelope.message,
77+
'Runner did not accept connection (xcodebuild exited early): this Xcode cannot start its simulator service on this macOS',
78+
);
79+
assert.equal(envelope.details?.reason, 'simulator_service_hub_unavailable');
80+
// The fix is the daemon host's Xcode selection, so the hint has to name both levers for it.
81+
assert.match(String(envelope.hint), /xcode-select -s/);
82+
assert.match(String(envelope.hint), /DEVELOPER_DIR/);
83+
// The bless line is in the runner's log, and that is the file the caller is sent to.
84+
assert.equal(envelope.logPath, session.runnerLogPath);
85+
assert.equal(envelope.diagnosticId, 'diag-service-hub');
86+
// A cause this log proved outranks a device state read before the launch; the state still travels.
87+
assert.equal(envelope.details?.developerDiskImage, 'unavailable');
88+
});
89+
90+
test('a still-alive xcodebuild whose log carries the bless refusal fails the connect stage by name and stays terminal', async () => {
91+
vi.stubGlobal(
92+
'fetch',
93+
vi.fn(async () => {
94+
throw Object.assign(new Error('connect ECONNREFUSED 127.0.0.1:8100'), {
95+
code: 'ECONNREFUSED',
96+
});
97+
}),
98+
);
99+
const session = sessionWithLaunchLog(BLESS_LOG, null);
100+
101+
const error = await waitForRunner(
102+
iosSimulator,
103+
RUNNER_ENDPOINT_FIXTURE,
104+
{ command: 'tap', x: 200, y: 300 },
105+
undefined,
106+
300,
107+
session,
108+
).catch((error: unknown) => error as AppError);
109+
110+
assert.ok(error instanceof AppError);
111+
assert.equal(
112+
error.message,
113+
'Runner did not accept connection: this Xcode cannot start its simulator service on this macOS',
114+
);
115+
assert.equal(error.details?.reason, 'simulator_service_hub_unavailable');
116+
assert.match(String(error.details?.hint), /xcode-select -s/);
117+
assert.equal(error.details?.logPath, session.runnerLogPath);
118+
// Nothing was written before the loop gave up, so the disclosure the refusal earns still holds.
119+
assert.equal(error.details?.dispatched, 'no');
120+
// Before the cause had a name, this same failure ordered a session restart, a resend, and a clean
121+
// rebuild of the very binary this macOS refuses to load (#3449). None of that may survive.
122+
assert.equal(shouldRetryRunnerConnectError(error), false);
123+
assert.equal(isRetryableRunnerError(error), false);
124+
assert.equal(shouldRestartRunnerBeforeCommandSend(error, { command: 'tap' }), false);
125+
assert.equal(shouldRebuildCachedRunnerArtifact(error), false);
126+
});
127+
128+
test('a still-alive xcodebuild whose log names no service-hub cause keeps the ordinary connect failure', async () => {
129+
vi.stubGlobal(
130+
'fetch',
131+
vi.fn(async () => {
132+
throw Object.assign(new Error('connect ECONNREFUSED 127.0.0.1:8100'), {
133+
code: 'ECONNREFUSED',
134+
});
135+
}),
136+
);
137+
138+
await assert.rejects(
139+
() =>
140+
waitForRunner(
141+
iosSimulator,
142+
RUNNER_ENDPOINT_FIXTURE,
143+
{ command: 'snapshot' },
144+
undefined,
145+
300,
146+
sessionWithLaunchLog(
147+
'2026-10-15 10:23:41.112 xcodebuild[9:1] [MT] iOSSimulator: waiting for the simulator to boot\n',
148+
null,
149+
),
150+
),
151+
(error: unknown) => {
152+
const appError = error as AppError;
153+
assert.equal(appError.message, 'Runner did not accept connection');
154+
assert.equal(appError.details?.reason, 'IOS_RUNNER_CONNECT_TIMEOUT');
155+
return true;
156+
},
157+
);
158+
});
159+
160+
test('bless wording without the typed reason cannot stop the connect retry or the resend', () => {
161+
// The recovery rows key on the reason the launch paths publish, so foreign text carrying Xcode's
162+
// own sentence — in a message or a captured log — cannot by itself mark a failure terminal.
163+
const untyped = new AppError(
164+
'COMMAND_FAILED',
165+
'Runner did not accept connection: DTServiceHubClient failed to bless service hub for simulator iPhone (UDID)',
166+
{ stderr: BLESS_LOG },
167+
);
168+
169+
// The connect loop keeps waiting on it, exactly as it does on any unclassified connect failure.
170+
assert.equal(shouldRetryRunnerConnectError(untyped), true);
171+
assert.equal(shouldRestartRunnerAfterReadinessPreflight(untyped), false);
172+
});

‎packages/platform-apple/src/runner/__tests__/runner-xctestrun-startup-failure-reasons.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,7 @@ const HINT_FOR_REASON: Record<RunnerStartupFailureReason, string> = {
6666
signing_provisioning_profile_missing: 'AGENT_DEVICE_IOS_PROVISIONING_PROFILE',
6767
signing_unspecified: 'Automatic Signing',
6868
devtools_security_developer_mode_disabled: 'DevToolsSecurity -enable',
69+
simulator_service_hub_unavailable: 'xcode-select -s',
6970
simulator_set_destination_not_found: '-DVTSimulatorSetLocation',
7071
// Both device remedies are owned by `core/devicectl.ts` and travel on the device report, so this
7172
// table quotes them instead of restating them; `runner-device-readiness.test.ts` is where the

‎packages/platform-apple/src/runner/runner-error-classification.ts‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,13 @@ const hasRunnerConnectFailureReason =
108108
(reason: RunnerConnectFailureReason): RunnerErrorDetailsMatch =>
109109
(details) =>
110110
details.runnerConnectFailureReason === reason;
111+
/**
112+
* The terminal verdict of a runner launch whose simulator service this macOS refused to load. It keys
113+
* on the reason the launch paths publish, never on the bless sentence they quote, so foreign text
114+
* carrying Xcode's own wording cannot, on its own, mark a failure terminal.
115+
*/
116+
const hasSimulatorServiceHubFailure: RunnerErrorDetailsMatch = (details) =>
117+
details.reason === SIMULATOR_SERVICE_HUB_UNAVAILABLE_REASON;
111118
/**
112119
* The preflight marks whatever it was waiting on when it stopped, and one of the things it waits on
113120
* is a caller that stopped waiting. A canceled request is not a wedged runner: the restart this
@@ -179,6 +186,7 @@ export const RUNNER_STARTUP_FAILURE_REASONS = [
179186
'signing_provisioning_profile_missing',
180187
'signing_unspecified',
181188
'devtools_security_developer_mode_disabled',
189+
'simulator_service_hub_unavailable',
182190
'simulator_set_destination_not_found',
183191
...RUNNER_DEVICE_READINESS_FAILURE_REASONS,
184192
'build_failed_unclassified',
@@ -190,6 +198,25 @@ export type RunnerStartupFailureReason = (typeof RUNNER_STARTUP_FAILURE_REASONS)
190198
export type RunnerDeviceReadinessFailureReason =
191199
(typeof RUNNER_DEVICE_READINESS_FAILURE_REASONS)[number];
192200

201+
/**
202+
* The reason for a simulator runner launch whose service hub the host macOS refused to load. macOS
203+
* gates `DTServiceHub` behind a version-restriction entitlement, so an Xcode older than the macOS it
204+
* runs on cannot bless the simulator service at all: every launch fails the same way until the host
205+
* selects an Xcode that supports its macOS. AMFI's refusal is visible only as the bless line in the
206+
* launch output, which is why the row keys on that phrase and why no host-side preflight exists.
207+
*/
208+
export const SIMULATOR_SERVICE_HUB_UNAVAILABLE_REASON: RunnerStartupFailureReason =
209+
'simulator_service_hub_unavailable';
210+
211+
/**
212+
* Names the cause in the message beside the connect-shaped failure the bless refusal produced. The
213+
* reason and hint come from {@link classifyRunnerStartupFailure}; this only folds the cause into the
214+
* sentence the caller reads first.
215+
*/
216+
export function simulatorServiceHubFailureMessage(message: string): string {
217+
return `${message}: this Xcode cannot start its simulator service on this macOS`;
218+
}
219+
193220
/**
194221
* The reason a startup failure carries when no rule proves a cause. Its hint is deliberately the
195222
* cache-recovery advice rather than anything about signing: an unclassified build is not evidence of
@@ -284,6 +311,19 @@ export const RUNNER_ERROR_RULES: readonly RunnerErrorRule[] = [
284311
},
285312
verdicts: { retryable: false, connectRetry: false, artifactSuspect: false },
286313
},
314+
{
315+
// A boot whose simulator service the host refused is not cured by wiping derived data, and no
316+
// restart or resend can make AMFI load a binary this macOS rejects.
317+
reason: 'simulator_service_hub_unavailable',
318+
match: { code: 'COMMAND_FAILED', details: hasSimulatorServiceHubFailure },
319+
verdicts: {
320+
retryable: false,
321+
connectRetry: false,
322+
restartBeforeSend: false,
323+
restartAfterReadinessPreflight: false,
324+
artifactSuspect: false,
325+
},
326+
},
287327
{
288328
// A device still mid-attachment is not a runner we can talk to yet, and waiting on
289329
// it inside this request is what the caller's own retry is for.
@@ -393,6 +433,22 @@ export const RUNNER_ERROR_RULES: readonly RunnerErrorRule[] = [
393433
hint: 'Check that the simulator still exists in the --ios-simulator-device-set this error names (`xcrun simctl --set <set> list devices`). If it does, the Xcode this error names no longer honors -DVTSimulatorSetLocation, so use a simulator in the default set.',
394434
},
395435
},
436+
{
437+
// macOS validates Xcode's `DTServiceHub` against a version-restriction entitlement and refuses a
438+
// build older than itself (AMFI: "disallowed without correct
439+
// com.apple.private.amfi.version-restriction entitlement version"), so no launch of this Xcode's
440+
// simulator service can succeed on this host. Both phrases are Xcode's own sentence and meet on
441+
// the same line of the launch output; the domain names the client, the complaint names the bless.
442+
reason: 'simulator_service_hub_bless_refused',
443+
match: {
444+
toolTextLineIncludesAll: ['dtservicehubclient', 'failed to bless service hub'],
445+
},
446+
verdicts: {},
447+
buildFailure: {
448+
reason: SIMULATOR_SERVICE_HUB_UNAVAILABLE_REASON,
449+
hint: 'This Xcode cannot start its simulator service on this macOS: macOS refuses to load its DTServiceHub. Select an Xcode that supports this macOS on the daemon host (`xcode-select -s /Applications/Xcode.app`, or DEVELOPER_DIR=/Applications/Xcode.app for the daemon process), then retry. Updating Xcode is the fix; restarting the runner or clearing derived data cannot change it.',
450+
},
451+
},
396452
{
397453
reason: 'bundle_identifier_registration_failed',
398454
match: { toolTextIncludesAll: ['failed registering bundle identifier'] },

0 commit comments

Comments
 (0)