Skip to content

Commit 9b76f60

Browse files
thymikeeclaude
andauthored
refactor(ios-runner): key TS runner retry rules on typed connect-failure reasons (#2820)
* refactor(ios-runner): key runner retry rules on typed connect-failure reasons The runner connect path now publishes `details.runnerConnectFailureReason` (`xcodebuild_exited_early`, `runner_connect_refused`, `runner_endpoint_probe_exhausted`) where it builds each failure, and the retry table keys those three rules on that field instead of our own message text. The remaining text rules read only foreign text (Node's fetch/net/http errors, Xcode's device-busy text) under a renamed `foreignMessageIncludesAll` field, each naming its tool. Test doubles that stood in for these failures now carry the typed reason, and each converted rule has a test that the reason, not the message, decides the verdicts. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * refactor(ios-runner): drop the restart composition typed reasons made redundant With the connect-failure reasons exclusive, no error that grants a restart before send can also carry a terminal connect verdict, so the restart predicate reads its own axis. Doc comments now say which failures carry a BootFailureReason and that the rows above the startup block include foreign-runtime text matchers. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * test(apple-runner): move connect-retry classification tests to their owning file runner-client.test.ts asserted shouldRetryRunnerConnectError and isRetryableRunnerError, both exported by runner-error-classification.ts and already covered there, growing it past the test-file size ratchet. Move the one uncovered case (runner_endpoint_probe_exhausted keeps the connect loop retrying) into runner-error-classification.test.ts and drop the rest as duplicate message-sniff coverage. Give runnerConnectFailure a reason-keyed default message so the retry suite's repeated connect-refused/probe-exhausted fixtures fit on one line again, bringing runner-command-retry.test.ts back to its merge-base length. --------- Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
1 parent 7e9031a commit 9b76f60

8 files changed

Lines changed: 222 additions & 95 deletions

‎packages/platform-apple/src/runner/__tests__/runner-client.test.ts‎

Lines changed: 0 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -30,10 +30,6 @@ vi.mock('../runner-macos-products.ts', async () => {
3030

3131
import type { DeviceInfo } from '@agent-device/kernel/device';
3232
import { RUNNER_COMMAND_TRAITS, isReadOnlyRunnerCommand } from '../runner-command-traits.ts';
33-
import {
34-
isRetryableRunnerError,
35-
shouldRetryRunnerConnectError,
36-
} from '../runner-error-classification.ts';
3733
import { withRunnerCommandId, type RunnerCommand } from '../runner-contract.ts';
3834
import {
3935
resolveRunnerBuildDestination,
@@ -412,32 +408,6 @@ test('assertSafeDerivedCleanup allows cleaning override path under project .tmp'
412408
});
413409
});
414410

415-
test('shouldRetryRunnerConnectError does not retry xcodebuild early-exit errors', () => {
416-
const err = new AppError(
417-
'COMMAND_FAILED',
418-
'Runner did not accept connection (xcodebuild exited early)',
419-
);
420-
assert.equal(shouldRetryRunnerConnectError(err), false);
421-
});
422-
423-
test('shouldRetryRunnerConnectError retries transient connect errors', () => {
424-
const err = new AppError('COMMAND_FAILED', 'Runner endpoint probe failed');
425-
assert.equal(shouldRetryRunnerConnectError(err), true);
426-
});
427-
428-
test('isRetryableRunnerError does not retry xcodebuild early-exit errors', () => {
429-
const err = new AppError(
430-
'COMMAND_FAILED',
431-
'Runner did not accept connection (xcodebuild exited early)',
432-
);
433-
assert.equal(isRetryableRunnerError(err), false);
434-
});
435-
436-
test('isRetryableRunnerError does not retry busy-connecting errors', () => {
437-
const err = new AppError('COMMAND_FAILED', 'Device is busy (Connecting to iPhone)');
438-
assert.equal(isRetryableRunnerError(err), false);
439-
});
440-
441411
test('xctestrunReferencesProjectRoot rejects stale worktree artifacts', async () => {
442412
const tmpDir = await makeTmpDir();
443413
const xctestrunPath = path.join(tmpDir, 'AgentDeviceRunner.xctestrun');

‎packages/platform-apple/src/runner/__tests__/runner-command-retry.test.ts‎

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
import { beforeEach, test, vi } from 'vitest';
22
import assert from 'node:assert/strict';
33
import { IOS_SIMULATOR } from './device-fixtures.ts';
4-
import { createTestRequestCancellation } from './runner-session-fixtures.ts';
4+
import { createTestRequestCancellation, runnerConnectFailure } from './runner-session-fixtures.ts';
55
import { AppError } from '@agent-device/kernel/errors';
66
import { Deadline } from '../host.ts';
77
import { appleRunnerTestHost } from '../test-host.ts';
@@ -71,7 +71,7 @@ test('prepareIosRunner marks a bad restored artifact and rebuilds once after hea
7171
.mockResolvedValueOnce(fixtures.restoredSession)
7272
.mockResolvedValueOnce(fixtures.rebuiltSession);
7373
mockExecuteRunnerCommandWithSession
74-
.mockRejectedValueOnce(new AppError('COMMAND_FAILED', 'Runner did not accept connection'))
74+
.mockRejectedValueOnce(runnerConnectFailure('runner_connect_refused'))
7575
.mockResolvedValueOnce({ uptimeMs: 42 });
7676

7777
const result = await prepareIosRunner(IOS_SIMULATOR, {
@@ -110,7 +110,7 @@ test('prepareIosRunner invalidates rebuilt sessions when bad-cache recovery heal
110110
.mockResolvedValueOnce(restoredSession)
111111
.mockResolvedValueOnce(rebuiltSession);
112112
mockExecuteRunnerCommandWithSession
113-
.mockRejectedValueOnce(new AppError('COMMAND_FAILED', 'Runner endpoint probe failed'))
113+
.mockRejectedValueOnce(runnerConnectFailure('runner_endpoint_probe_exhausted'))
114114
.mockRejectedValueOnce(new AppError('COMMAND_FAILED', 'Runner health timed out'));
115115

116116
await assert.rejects(
@@ -144,7 +144,7 @@ test('prepareIosRunner retries a fresh launch session when the health check cann
144144
.mockResolvedValueOnce(stuckSession)
145145
.mockResolvedValueOnce(relaunchedSession);
146146
mockExecuteRunnerCommandWithSession
147-
.mockRejectedValueOnce(new AppError('COMMAND_FAILED', 'Runner did not accept connection'))
147+
.mockRejectedValueOnce(runnerConnectFailure('runner_connect_refused'))
148148
.mockResolvedValueOnce({ uptimeMs: 42 });
149149

150150
const result = await prepareIosRunner(IOS_SIMULATOR, {
@@ -232,8 +232,8 @@ test('prepareIosRunner does not force a rebuild when the relaunched fresh sessio
232232
.mockResolvedValueOnce(stuckSession)
233233
.mockResolvedValueOnce(relaunchedSession);
234234
mockExecuteRunnerCommandWithSession
235-
.mockRejectedValueOnce(new AppError('COMMAND_FAILED', 'Runner did not accept connection'))
236-
.mockRejectedValueOnce(new AppError('COMMAND_FAILED', 'Runner did not accept connection'));
235+
.mockRejectedValueOnce(runnerConnectFailure('runner_connect_refused'))
236+
.mockRejectedValueOnce(runnerConnectFailure('runner_connect_refused'));
237237

238238
await assert.rejects(
239239
() =>
@@ -261,7 +261,7 @@ test('prepareIosRunner does not relaunch after non-retryable runner startup fail
261261

262262
mockEnsureRunnerSession.mockResolvedValueOnce(failedSession);
263263
mockExecuteRunnerCommandWithSession.mockRejectedValueOnce(
264-
new AppError('COMMAND_FAILED', 'xcodebuild exited early'),
264+
runnerConnectFailure('xcodebuild_exited_early'),
265265
);
266266

267267
await assert.rejects(
@@ -281,7 +281,7 @@ test('prepareIosRunner does not relaunch after request cancellation', async () =
281281
mockEnsureRunnerSession.mockResolvedValueOnce(stuckSession);
282282
mockExecuteRunnerCommandWithSession.mockImplementationOnce(() => {
283283
markRequestCanceled(requestId);
284-
throw new AppError('COMMAND_FAILED', 'Runner did not accept connection');
284+
throw runnerConnectFailure('runner_connect_refused');
285285
});
286286

287287
try {
@@ -303,7 +303,7 @@ test('mutating commands restart stale ready sessions when the preflight probe ne
303303

304304
mockEnsureRunnerSession.mockResolvedValueOnce(staleSession).mockResolvedValueOnce(freshSession);
305305
mockExecuteRunnerCommandWithSession
306-
.mockRejectedValueOnce(new AppError('COMMAND_FAILED', 'Runner did not accept connection'))
306+
.mockRejectedValueOnce(runnerConnectFailure('runner_connect_refused'))
307307
.mockResolvedValueOnce({ message: 'tapped' });
308308

309309
const result = await runAppleRunnerCommand(IOS_SIMULATOR, { command: 'tap', x: 120, y: 240 });
@@ -326,7 +326,7 @@ test('mutating commands retry startup sessions with stale bundle cleanup', async
326326

327327
mockEnsureRunnerSession.mockResolvedValueOnce(startupSession).mockResolvedValueOnce(freshSession);
328328
mockExecuteRunnerCommandWithSession
329-
.mockRejectedValueOnce(new AppError('COMMAND_FAILED', 'Runner did not accept connection'))
329+
.mockRejectedValueOnce(runnerConnectFailure('runner_connect_refused'))
330330
.mockResolvedValueOnce({ message: 'tapped' });
331331

332332
const result = await runAppleRunnerCommand(IOS_SIMULATOR, { command: 'tap', x: 120, y: 240 });
@@ -811,7 +811,7 @@ test('mutating commands invalidate the retry session without replaying again', a
811811

812812
mockEnsureRunnerSession.mockResolvedValueOnce(staleSession).mockResolvedValueOnce(freshSession);
813813
mockExecuteRunnerCommandWithSession
814-
.mockRejectedValueOnce(new AppError('COMMAND_FAILED', 'Runner did not accept connection'))
814+
.mockRejectedValueOnce(runnerConnectFailure('runner_connect_refused'))
815815
.mockRejectedValueOnce(new AppError('COMMAND_FAILED', 'fetch failed'))
816816
.mockResolvedValueOnce({ lifecycleState: 'notAccepted' });
817817

@@ -1153,7 +1153,7 @@ test('a failed replacement boot does not consume the request recycle budget', as
11531153
const requestId = 'req-recycle-transient-boot-failure';
11541154
mockEnsureRunnerSession
11551155
.mockResolvedValueOnce(makeRunnerSession({ port: 8100, state: 'ready' }))
1156-
.mockRejectedValueOnce(new AppError('COMMAND_FAILED', 'Runner did not accept connection'))
1156+
.mockRejectedValueOnce(runnerConnectFailure('runner_connect_refused'))
11571157
.mockResolvedValueOnce(makeRunnerSession({ port: 8101, state: 'ready' }));
11581158
mockExecuteRunnerCommandWithSession
11591159
.mockRejectedValueOnce(new AppError('COMMAND_FAILED', 'fetch failed'))
@@ -1204,7 +1204,7 @@ test('a later command in the same request cannot pay for a second recycle boot',
12041204

12051205
mockEnsureRunnerSession.mockResolvedValueOnce(staleSession).mockResolvedValueOnce(freshSession);
12061206
mockExecuteRunnerCommandWithSession
1207-
.mockRejectedValueOnce(new AppError('COMMAND_FAILED', 'Runner did not accept connection'))
1207+
.mockRejectedValueOnce(runnerConnectFailure('runner_connect_refused'))
12081208
.mockResolvedValueOnce({ message: 'tapped' });
12091209

12101210
// First command consumes the request's only recycle via restart-and-replay.

‎packages/platform-apple/src/runner/__tests__/runner-error-classification.test.ts‎

Lines changed: 99 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ import {
1414
shouldRestartRunnerBeforeCommandSend,
1515
shouldRetryRunnerConnectError,
1616
} from '../runner-error-classification.ts';
17+
import { runnerConnectFailure } from './runner-session-fixtures.ts';
1718

1819
function commandFailed(message: string, details?: Record<string, unknown>): AppError {
1920
return new AppError('COMMAND_FAILED', message, details);
@@ -27,20 +28,27 @@ test('every rule carries a unique reason', () => {
2728
// --- retryable axis (isRetryableRunnerError) ---
2829

2930
test('transport-shaped failures are retryable', () => {
30-
for (const message of [
31-
'Runner did not accept connection on port 8100',
32-
'fetch failed',
33-
'connect ECONNREFUSED 127.0.0.1:8100',
34-
'socket hang up',
35-
]) {
31+
assert.equal(
32+
isRetryableRunnerError(
33+
runnerConnectFailure(
34+
'runner_connect_refused',
35+
'Runner did not accept connection on port 8100',
36+
),
37+
),
38+
true,
39+
);
40+
for (const message of ['fetch failed', 'connect ECONNREFUSED 127.0.0.1:8100', 'socket hang up']) {
3641
assert.equal(isRetryableRunnerError(commandFailed(message)), true, message);
3742
}
3843
});
3944

4045
test('boot-shaped failures are not retryable', () => {
4146
assert.equal(
4247
isRetryableRunnerError(
43-
commandFailed('Runner did not accept connection (xcodebuild exited early)'),
48+
runnerConnectFailure(
49+
'xcodebuild_exited_early',
50+
'Runner did not accept connection (xcodebuild exited early)',
51+
),
4452
),
4553
false,
4654
);
@@ -51,7 +59,9 @@ test('boot-shaped failures are not retryable', () => {
5159
});
5260

5361
test('an explicitly retriable flag wins over any message denial', () => {
54-
const flagged = commandFailed('xcodebuild exited early', { retriable: true });
62+
const flagged = runnerConnectFailure('xcodebuild_exited_early', 'xcodebuild exited early', {
63+
retriable: true,
64+
});
5565
assert.equal(isRetryableRunnerError(flagged), true);
5666
});
5767

@@ -64,15 +74,22 @@ test('retryable requires an AppError with COMMAND_FAILED', () => {
6474

6575
test('connect loop keeps waiting by default, including for unknown errors', () => {
6676
assert.equal(
67-
shouldRetryRunnerConnectError(commandFailed('Runner did not accept connection')),
77+
shouldRetryRunnerConnectError(
78+
runnerConnectFailure('runner_connect_refused', 'Runner did not accept connection'),
79+
),
6880
true,
6981
);
7082
assert.equal(shouldRetryRunnerConnectError(new Error('anything')), true);
7183
assert.equal(shouldRetryRunnerConnectError(new AppError('INVALID_ARGS', 'nope')), true);
7284
});
7385

7486
test('connect loop stops for terminal verdicts', () => {
75-
assert.equal(shouldRetryRunnerConnectError(commandFailed('xcodebuild exited early')), false);
87+
assert.equal(
88+
shouldRetryRunnerConnectError(
89+
runnerConnectFailure('xcodebuild_exited_early', 'xcodebuild exited early'),
90+
),
91+
false,
92+
);
7693
const unattached = new AppError('DEVICE_NOT_FOUND', 'device not attached', {
7794
usbmuxDeviceAttached: false,
7895
});
@@ -140,27 +157,37 @@ test('a deadline on its own earns no recovery verdict', () => {
140157

141158
test('only a runner that never accepted a connection indicts the cached artifact', () => {
142159
assert.equal(
143-
shouldRebuildCachedRunnerArtifact(commandFailed('Runner endpoint probe failed')),
160+
shouldRebuildCachedRunnerArtifact(
161+
runnerConnectFailure('runner_endpoint_probe_exhausted', 'Runner endpoint probe failed'),
162+
),
144163
true,
145164
);
146165
assert.equal(
147-
shouldRebuildCachedRunnerArtifact(commandFailed('Runner did not accept connection')),
166+
shouldRebuildCachedRunnerArtifact(
167+
runnerConnectFailure('runner_connect_refused', 'Runner did not accept connection'),
168+
),
148169
true,
149170
);
150171
assert.equal(
151172
shouldRebuildCachedRunnerArtifact(
152-
commandFailed('Runner did not accept connection (simctl spawn)', { port: 8100 }),
173+
runnerConnectFailure(
174+
'runner_connect_refused',
175+
'Runner did not accept connection (simctl spawn)',
176+
{
177+
port: 8100,
178+
},
179+
),
153180
),
154181
true,
155182
);
156-
// Wiping derived data cannot fix a boot that refuses to compile, and its message
157-
// otherwise reads as a refused connection.
183+
// Wiping derived data cannot fix a boot that refuses to compile.
158184
assert.equal(
159185
shouldRebuildCachedRunnerArtifact(
160-
commandFailed('Runner did not accept connection (xcodebuild exited early)', {
161-
port: 8100,
162-
logPath: '/tmp/runner.log',
163-
}),
186+
runnerConnectFailure(
187+
'xcodebuild_exited_early',
188+
'Runner did not accept connection (xcodebuild exited early)',
189+
{ port: 8100, logPath: '/tmp/runner.log' },
190+
),
164191
),
165192
false,
166193
);
@@ -198,23 +225,67 @@ test('ordinary errors are never session-fatal', () => {
198225

199226
// --- restart-before-send axis (shouldRestartRunnerBeforeCommandSend) ---
200227

201-
test('a refused connection before send restarts the session, case-insensitively', () => {
228+
test('a refused connection before send restarts the session', () => {
202229
assert.equal(
203-
shouldRestartRunnerBeforeCommandSend(commandFailed('Runner did not accept connection')),
204-
true,
205-
);
206-
assert.equal(
207-
shouldRestartRunnerBeforeCommandSend(commandFailed('runner did not accept connection')),
230+
shouldRestartRunnerBeforeCommandSend(
231+
runnerConnectFailure('runner_connect_refused', 'Runner did not accept connection'),
232+
),
208233
true,
209234
);
210235
});
211236

212-
test('a terminal connect verdict refuses the restart even when the message matches', () => {
213-
const both = commandFailed('xcodebuild exited early: runner did not accept connection');
214-
assert.equal(shouldRestartRunnerBeforeCommandSend(both), false);
237+
test('an early exit or a foreign transport failure earns no restart before send', () => {
238+
const earlyExit = runnerConnectFailure(
239+
'xcodebuild_exited_early',
240+
'xcodebuild exited early: runner did not accept connection',
241+
);
242+
assert.equal(shouldRestartRunnerBeforeCommandSend(earlyExit), false);
215243
assert.equal(shouldRestartRunnerBeforeCommandSend(commandFailed('socket hang up')), false);
216244
});
217245

246+
// --- typed connect-failure reasons (agent-device's own connect path) ---
247+
248+
test('xcodebuild_exited_early is decided by the typed reason, not the message', () => {
249+
for (const message of ['Runner did not accept connection (xcodebuild exited early)', 'boom']) {
250+
const error = runnerConnectFailure('xcodebuild_exited_early', message);
251+
assert.equal(isRetryableRunnerError(error), false, message);
252+
assert.equal(shouldRetryRunnerConnectError(error), false, message);
253+
assert.equal(shouldRebuildCachedRunnerArtifact(error), false, message);
254+
assert.equal(shouldRestartRunnerBeforeCommandSend(error), false, message);
255+
}
256+
// The same words without the reason earn no terminal verdict.
257+
const untyped = commandFailed('Runner did not accept connection (xcodebuild exited early)');
258+
assert.equal(shouldRetryRunnerConnectError(untyped), true);
259+
});
260+
261+
test('runner_connect_refused is decided by the typed reason, not the message', () => {
262+
for (const message of ['Runner did not accept connection', 'boom']) {
263+
const error = runnerConnectFailure('runner_connect_refused', message);
264+
assert.equal(isRetryableRunnerError(error), true, message);
265+
assert.equal(shouldRetryRunnerConnectError(error), true, message);
266+
assert.equal(shouldRebuildCachedRunnerArtifact(error), true, message);
267+
assert.equal(shouldRestartRunnerBeforeCommandSend(error), true, message);
268+
}
269+
const untyped = commandFailed('Runner did not accept connection');
270+
assert.equal(isRetryableRunnerError(untyped), false);
271+
assert.equal(shouldRebuildCachedRunnerArtifact(untyped), false);
272+
assert.equal(shouldRestartRunnerBeforeCommandSend(untyped), false);
273+
});
274+
275+
test('runner_endpoint_probe_exhausted is decided by the typed reason, not the message', () => {
276+
for (const message of ['Runner endpoint probe failed', 'boom']) {
277+
const error = runnerConnectFailure('runner_endpoint_probe_exhausted', message);
278+
assert.equal(shouldRebuildCachedRunnerArtifact(error), true, message);
279+
assert.equal(isRetryableRunnerError(error), false, message);
280+
assert.equal(shouldRestartRunnerBeforeCommandSend(error), false, message);
281+
assert.equal(shouldRetryRunnerConnectError(error), true, message);
282+
}
283+
assert.equal(
284+
shouldRebuildCachedRunnerArtifact(commandFailed('Runner endpoint probe failed')),
285+
false,
286+
);
287+
});
288+
218289
// The literals are what the Swift runner encodes, so they are the contract and not the constant
219290
// names: a rename on one side has to fail here rather than silently split the pair (#2728).
220291
test('a refused screen capture keeps the runner reason and stays off the wire code', () => {

‎packages/platform-apple/src/runner/__tests__/runner-lifecycle-readiness-preflight.test.ts‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,11 @@ import {
88
import { appleRunnerTestHost } from '../test-host.ts';
99
import type { RunnerXctestrunArtifact } from '../runner-xctestrun.ts';
1010
import { IOS_SIMULATOR } from './device-fixtures.ts';
11-
import { createTestRequestCancellation, makeRunnerSession } from './runner-session-fixtures.ts';
11+
import {
12+
createTestRequestCancellation,
13+
makeRunnerSession,
14+
runnerConnectFailure,
15+
} from './runner-session-fixtures.ts';
1216

1317
const {
1418
mockEnsureRunnerSession,
@@ -216,7 +220,10 @@ test('a boot that exited early does not wipe a restored runner artifact', async
216220

217221
mockEnsureRunnerSession.mockResolvedValueOnce(restoredSession);
218222
mockExecuteRunnerCommandWithSession.mockRejectedValueOnce(
219-
new AppError('COMMAND_FAILED', 'Runner did not accept connection (xcodebuild exited early)'),
223+
runnerConnectFailure(
224+
'xcodebuild_exited_early',
225+
'Runner did not accept connection (xcodebuild exited early)',
226+
),
220227
);
221228

222229
await assert.rejects(

0 commit comments

Comments
 (0)