Skip to content

Commit 120ae34

Browse files
committed
fix(ios-runner): carry the device's answer onto the failure it explains
Developer Mode off was the only device state no later step could change, and it is the only one that keeps refusing a run before the build. The developer disk image is mounted on demand by CoreDevice during build and launch since iOS 17, so a phone that has just been rebooted reports it down while the very next build clears it: refusing there turned a self-clearing state into a failed run. The preflight now hands the states back, the build carries them, and an unavailable image is published onto a build that named no cause of its own — beside xcodebuild's own sentence, never in place of it. A log flush gained the deadline and abort it was missing, and a refused append is recorded instead of dropped, because an offset measured over bytes that never landed is not a boundary.
1 parent 73c5f5b commit 120ae34

13 files changed

Lines changed: 385 additions & 144 deletions

‎.device-evidence/CHECKLIST.md‎

Lines changed: 23 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -159,23 +159,36 @@ Expected: exit non-zero with
159159
`hint` is top-level and names `Settings > Privacy & Security > Developer Mode`. Record what
160160
`developerDiskImage` says; either value is acceptable as long as the toggle stays the reason.
161161

162-
### 4. Developer disk image down with the toggle on -> `device_developer_disk_image_unavailable`
162+
### 4. Developer disk image down with the toggle on -> carried onto the failure, not a refusal
163163

164164
This is the pairing the old hint got wrong, so it is the evidence that matters. Reach it with a
165165
device whose iOS build is newer than the installed Xcode supports, or before Xcode finishes
166166
installing device support for a freshly paired phone, with Developer Mode on.
167167

168-
Expected: `details.reason` is `device_developer_disk_image_unavailable`, `details.developerMode` is
169-
`enabled`, and the hint names device support WITHOUT mentioning `Settings > Privacy & Security`. If
170-
the toggle reason appears here instead, that is the bug this issue exists to fix: paste the whole
168+
This state does NOT stop the run before the build (#2683 review): since iOS 17 CoreDevice mounts the
169+
personalized disk image on demand during build and launch, a phone that has just been rebooted reports
170+
the image down while the very next build clears it. Run the `prepare ios-runner` command above and
171+
record which of the two outcomes you get — both are evidence, and the second one is the reason the
172+
pre-build refusal was removed:
173+
174+
- **The build clears it**: the command succeeds. Paste the `ios_runner_session_startup` timings showing
175+
`verify_device_readiness` ran and `ensure_xctestrun` followed it. Nothing may name device support.
176+
- **The build fails and names no cause of its own**: `details.reason` is
177+
`device_developer_disk_image_unavailable` and `details.developerDiskImage` is `unavailable`, and the
178+
hint names device support WITHOUT mentioning `Settings > Privacy & Security`.
179+
- **The build fails and names its own cause** (e.g. `signing_no_development_team`): that reason wins and
180+
`details.developerDiskImage` is still `unavailable`. The device state never overwrites xcodebuild's
181+
sentence.
182+
183+
If the toggle reason appears here instead, that is the bug this issue exists to fix: paste the whole
171184
error and the `/tmp/device-details.json` payload rather than adjusting a rule.
172185

173-
Capture the details payload at the same moment as the refusal, and check it says
174-
`tunnelState: "connected"` and `bootState: "booted"`. A `ddiServicesAvailable: false` read any other
175-
way is not this case: an asleep or unreachable device has the same field and no obstacle, and the run
176-
must not name device support for it. The hint you get here is the one `core/devicectl.ts` owns and the
177-
device report carries, so it is worded identically to the hint `devicectl` output produces for the
178-
same complaint — paste both strings and confirm they match character for character.
186+
Capture the details payload at the same moment as the run, and check it says `tunnelState:
187+
"connected"` and `bootState: "booted"`. A `ddiServicesAvailable: false` read any other way is not this
188+
case: an asleep or unreachable device has the same field and no obstacle, and the run must not name
189+
device support for it. The hint you get here is the one `core/devicectl.ts` owns and the device report
190+
carries, so it is worded identically to the hint `devicectl` output produces for the same complaint —
191+
paste both strings and confirm they match character for character.
179192

180193
### 5. A device that cannot be read claims nothing
181194

‎packages/platform-apple/src/runner/__tests__/runner-device-readiness.test.ts‎

Lines changed: 66 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -4,12 +4,13 @@ import { AppError, normalizeError } from '@agent-device/kernel/errors';
44
import type { DeviceInfo } from '@agent-device/kernel/device';
55
import { appleRunnerTestHost } from '../test-host.ts';
66
import type { IosPhysicalDeviceRunnerControl } from '../../core/physical-device-routing.ts';
7-
import type { IosDeviceRunnerReadiness } from '../host.ts';
8-
import { assertDeviceReadinessForIosRunner } from '../runner-device-readiness.ts';
7+
import type { IosDeviceReadiness } from '../host.ts';
8+
import { preflightIosRunnerDeviceReadiness } from '../runner-device-readiness.ts';
99
import { RUNNER_DEVICE_READINESS_FAILURE_REASONS } from '../runner-contract.ts';
1010
import { IOS_DEVICE, IOS_SIMULATOR, MACOS_DEVICE } from './device-fixtures.ts';
1111
import {
1212
deviceReadinessFixtures,
13+
type IosDeviceReadinessFixture,
1314
type IosDeviceReadinessReport,
1415
} from './runner-startup-failure-fixtures.ts';
1516

@@ -19,12 +20,21 @@ import {
1920
* disabled" and "developer disk image" complaints and always named the toggle, which sent people to
2021
* a Settings pane that was already correct whenever the image was the actual obstacle.
2122
*
22-
* These cases drive the recorded device reports through the preflight and assert on what reaches the
23-
* caller, including the two reports that must never be confused for one another.
23+
* Only one of those two states may stop a run before the build. The toggle is owner-only and no later
24+
* step turns it on; the developer disk image is mounted on demand by CoreDevice during build and
25+
* launch since iOS 17, so a phone that has just been rebooted reports it down while the very next build
26+
* clears it (#2683 review). These cases drive the recorded device reports through the preflight and
27+
* assert which one refuses, which one is carried forward, and that the two are never confused.
2428
*/
2529

2630
const REPORTS = deviceReadinessFixtures();
2731

32+
/** The one report the preflight is allowed to refuse a build for: the owner's toggle. */
33+
const REFUSALS = REPORTS.filter(
34+
(fixture): fixture is IosDeviceReadinessFixture =>
35+
fixture.reason === 'device_developer_mode_disabled',
36+
);
37+
2838
/**
2939
* Remedies no other module could produce, so a hint matching one of them can only have come from the
3040
* report this test handed over. That is the claim #2683 has to keep: the preflight reads the wording
@@ -44,11 +54,7 @@ const HINT_FOR_REASON = {
4454
const BUDGET = { budgetMs: 10_000 } as const;
4555

4656
const readDeviceReadiness = vi.fn(
47-
(
48-
_device: DeviceInfo,
49-
_budgetMs?: number,
50-
_signal?: AbortSignal,
51-
): Promise<IosDeviceRunnerReadiness> =>
57+
(_device: DeviceInfo, _budgetMs?: number, _signal?: AbortSignal): Promise<IosDeviceReadiness> =>
5258
Promise.reject(new Error('this case records no device report')),
5359
);
5460

@@ -59,8 +65,8 @@ beforeEach(() => {
5965
});
6066
});
6167

62-
for (const fixture of REPORTS) {
63-
test(`a device reporting ${fixture.deviceReport.developerMode} mode and ${fixture.deviceReport.developerDiskImage} disk image publishes ${fixture.reason}`, async () => {
68+
for (const fixture of REFUSALS) {
69+
test(`a device reporting ${fixture.deviceReport.developerMode} mode refuses the run with ${fixture.reason}`, async () => {
6470
readDeviceReadiness.mockResolvedValue(readableReport(fixture.deviceReport));
6571

6672
const error = await expectRefusal(IOS_DEVICE);
@@ -93,22 +99,57 @@ for (const fixture of REPORTS) {
9399
});
94100
}
95101

102+
for (const fixture of REPORTS.filter((f) => f.reason !== 'device_developer_mode_disabled')) {
103+
test(`a device reporting ${fixture.deviceReport.developerDiskImage} disk image with ${fixture.deviceReport.developerMode} mode builds anyway`, async () => {
104+
// The refusal #2683 shipped with was wrong here (#2683 review): iOS 17+ mounts the image on demand
105+
// during build and launch, so this state has to reach the build rather than stop it.
106+
readDeviceReadiness.mockResolvedValue(readableReport(fixture.deviceReport));
107+
108+
await assert.doesNotReject(() => preflightIosRunnerDeviceReadiness(IOS_DEVICE, BUDGET));
109+
});
110+
111+
test(`the ${fixture.reason} report is carried forward for the failure it explains`, async () => {
112+
readDeviceReadiness.mockResolvedValue(readableReport(fixture.deviceReport));
113+
114+
const states = await preflightIosRunnerDeviceReadiness(IOS_DEVICE, BUDGET);
115+
116+
assert.deepEqual(states, {
117+
developerMode: fixture.deviceReport.developerMode,
118+
developerDiskImage: fixture.deviceReport.developerDiskImage,
119+
developerDiskImageHint: REMEDIES.developerDiskImageUnavailable,
120+
});
121+
});
122+
}
123+
96124
test('a device whose report cannot be read is not given a reason', async () => {
97125
readDeviceReadiness.mockResolvedValue({
98126
available: false,
99127
reason: 'device_readiness_unreadable',
100128
hint: 'Read the device state directly with `xcrun devicectl device info details`.',
101-
} satisfies IosDeviceRunnerReadiness);
129+
} satisfies IosDeviceReadiness);
130+
131+
await assert.doesNotReject(() => preflightIosRunnerDeviceReadiness(IOS_DEVICE, BUDGET));
132+
});
133+
134+
test('a device whose report cannot be read carries no state forward', async () => {
135+
// Nothing was read, so nothing may be claimed later (#2683).
136+
readDeviceReadiness.mockResolvedValue({
137+
available: false,
138+
reason: 'device_readiness_unreadable',
139+
hint: 'Read the device state directly with `xcrun devicectl device info details`.',
140+
} satisfies IosDeviceReadiness);
102141

103-
await assert.doesNotReject(() => assertDeviceReadinessForIosRunner(IOS_DEVICE, BUDGET));
142+
assert.equal(await preflightIosRunnerDeviceReadiness(IOS_DEVICE, BUDGET), undefined);
104143
});
105144

106-
test('a device reporting both states healthy is not a failure', async () => {
145+
test('a device reporting both states healthy is not a failure and carries an available image', async () => {
107146
readDeviceReadiness.mockResolvedValue(
108147
readableReport({ developerMode: 'enabled', developerDiskImage: 'available' }),
109148
);
110149

111-
await assert.doesNotReject(() => assertDeviceReadinessForIosRunner(IOS_DEVICE, BUDGET));
150+
const states = await preflightIosRunnerDeviceReadiness(IOS_DEVICE, BUDGET);
151+
152+
assert.equal(states?.developerDiskImage, 'available');
112153
});
113154

114155
test('a device that reports neither state is not read as accusing its owner', async () => {
@@ -118,7 +159,7 @@ test('a device that reports neither state is not read as accusing its owner', as
118159
readableReport({ developerMode: 'unknown', developerDiskImage: 'unknown' }),
119160
);
120161

121-
await assert.doesNotReject(() => assertDeviceReadinessForIosRunner(IOS_DEVICE, BUDGET));
162+
await assert.doesNotReject(() => preflightIosRunnerDeviceReadiness(IOS_DEVICE, BUDGET));
122163
});
123164

124165
test('an unavailable disk image on a device with Developer Mode on is never named as the toggle', async () => {
@@ -129,11 +170,11 @@ test('an unavailable disk image on a device with Developer Mode on is never name
129170
readableReport({ developerMode, developerDiskImage: 'unavailable' }),
130171
);
131172

132-
const error = await expectRefusal(IOS_DEVICE);
173+
const states = await preflightIosRunnerDeviceReadiness(IOS_DEVICE, BUDGET);
133174

134-
assert.equal(error.details?.reason, 'device_developer_disk_image_unavailable');
135-
// Only the image remedy may be published: the toggle remedy mentions the Settings pane.
136-
assert.doesNotMatch(String(error.details?.hint), /Privacy & Security/);
175+
assert.equal(states?.developerDiskImage, 'unavailable');
176+
// Only the image remedy may be carried: the toggle remedy mentions the Settings pane.
177+
assert.doesNotMatch(String(states?.developerDiskImageHint), /Privacy & Security/);
137178
}
138179
});
139180

@@ -151,7 +192,7 @@ test('a device with Developer Mode off names the toggle even when the image is d
151192

152193
test('a simulator or the desktop target never asks the device', async () => {
153194
for (const device of [IOS_SIMULATOR, MACOS_DEVICE]) {
154-
await assert.doesNotReject(() => assertDeviceReadinessForIosRunner(device, BUDGET));
195+
await assert.doesNotReject(() => preflightIosRunnerDeviceReadiness(device, BUDGET));
155196
}
156197

157198
assert.equal(readDeviceReadiness.mock.calls.length, 0);
@@ -174,7 +215,7 @@ test('the probe is bounded by the startup budget it runs inside', async () => {
174215
readableReport({ developerMode: 'enabled', developerDiskImage: 'available' }),
175216
);
176217

177-
await assertDeviceReadinessForIosRunner(IOS_DEVICE, {
218+
await preflightIosRunnerDeviceReadiness(IOS_DEVICE, {
178219
budgetMs: 2_500,
179220
signal: controller.signal,
180221
});
@@ -196,7 +237,7 @@ test('a budget that ran out during the probe stops the startup even on a healthy
196237

197238
await assert.rejects(
198239
() =>
199-
assertDeviceReadinessForIosRunner(IOS_DEVICE, {
240+
preflightIosRunnerDeviceReadiness(IOS_DEVICE, {
200241
budgetMs: 10_000,
201242
signal: controller.signal,
202243
}),
@@ -206,14 +247,14 @@ test('a budget that ran out during the probe stops the startup even on a healthy
206247

207248
function readableReport(
208249
report: IosDeviceReadinessReport,
209-
): Extract<IosDeviceRunnerReadiness, { available: true }> {
250+
): Extract<IosDeviceReadiness, { available: true }> {
210251
return { available: true, ...report, remedies: REMEDIES };
211252
}
212253

213254
async function expectRefusal(device: DeviceInfo): Promise<AppError> {
214255
let caught: unknown;
215256
await assert.rejects(
216-
() => assertDeviceReadinessForIosRunner(device, BUDGET),
257+
() => preflightIosRunnerDeviceReadiness(device, BUDGET),
217258
(error: unknown) => {
218259
caught = error;
219260
return true;

‎packages/platform-apple/src/runner/__tests__/runner-failure-diagnostics.test.ts‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,29 @@ test("an earlier command's bytes still in the writer are not this command's eith
107107
assert.equal(error.details?.runnerFailureReason, undefined);
108108
});
109109

110+
test('a log the disk refuses leaves no marker', async () => {
111+
// The writer no longer swallows a failed append (#2683 review), and the boundary has to say so: an
112+
// offset measured over bytes that never landed would credit this command with output it did not
113+
// produce, so the marker is withheld and the tail goes unread. `blocker` is a regular file standing
114+
// where a directory has to be, which is the cheapest way to make every append fail with ENOTDIR.
115+
const dir = mkdtempForTestSync('agent-device-runner-log-refused-');
116+
onTestFinished(() => fs.rmSync(dir, { recursive: true, force: true }));
117+
fs.writeFileSync(path.join(dir, 'blocker'), '');
118+
const logPath = path.join(dir, 'blocker', 'runner.log');
119+
logChunk(AX_RUNTIME_CRASH, logPath);
120+
121+
assert.equal(await captureRunnerLogAttempt(logPath), undefined);
122+
});
123+
124+
test('a command that already stopped waiting draws no marker', async () => {
125+
// The boundary is a prelude to a command, so a caller that gave up gets no further delay and no
126+
// claim about a log it is no longer reading (#2683 review).
127+
const logPath = writeRunnerLog(PRELUDE);
128+
const canceled = AbortSignal.abort();
129+
130+
assert.equal(await captureRunnerLogAttempt(logPath, { signal: canceled }), undefined);
131+
});
132+
110133
async function expectFailure(
111134
logAttempt: Awaited<ReturnType<typeof captureRunnerLogAttempt>>,
112135
): Promise<AppError> {

‎packages/platform-apple/src/runner/__tests__/runner-startup-failure-fixtures.ts‎

Lines changed: 28 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,11 @@ export type RunnerStartupFailureFixture = Readonly<{
7676
output: string;
7777
/** The argv the exec reported, which is never evidence of a cause (#2680). */
7878
args?: readonly string[];
79-
/** The device's own states, which is the evidence the `device-readiness` site reads. */
79+
/**
80+
* The device's own states. On the `device-readiness` site this is the evidence the preflight reads;
81+
* on a `build-for-testing` entry it is what the startup carried onto that build, which is the pairing
82+
* the corroborated disk-image reason depends on (#2683).
83+
*/
8084
deviceReport?: IosDeviceReadinessReport;
8185
/** What the pending capture still has to show, and how to reach it. */
8286
note?: string;
@@ -226,6 +230,28 @@ export const RUNNER_STARTUP_FAILURE_FIXTURES: readonly RunnerStartupFailureFixtu
226230
"note: Using provisioning profile \"match-development\" to sign the app bundle (in target 'AgentDeviceRunner' from project 'AgentDeviceRunner')\nerror: cannot find 'AgentDeviceRunnerCommand' in scope (in target 'AgentDeviceRunnerUITests' from project 'AgentDeviceRunner')\n** TEST BUILD FAILED **\n",
227231
note: 'The hazard the bare `provisioning profile` trigger carried (#2688 review): a failing build can print the profile it used while the failure is a compile error. A benign mention must keep cache-recovery advice; it also says nothing Xcode calls code signing, which is its own honest row.',
228232
},
233+
{
234+
id: 'unclassified-build-on-device-with-image-down',
235+
reason: 'device_developer_disk_image_unavailable',
236+
site: 'build-for-testing',
237+
xcodeVersion: UNOBSERVED,
238+
provenance: 'invented-shape',
239+
output:
240+
"error: cannot find 'AgentDeviceRunnerCommand' in scope (in target 'AgentDeviceRunnerUITests' from project 'AgentDeviceRunner')\n** TEST BUILD FAILED **\n",
241+
deviceReport: { developerMode: 'enabled', developerDiskImage: 'unavailable' },
242+
note: 'The corroborated pairing (#2683 review): a build that names no cause, on a phone core read directly as reporting its image down. Naming the image beats cache-recovery advice; the state also travels as details.developerDiskImage.',
243+
},
244+
{
245+
id: 'team-id-failure-on-device-with-image-down',
246+
reason: 'signing_no_development_team',
247+
site: 'build-for-testing',
248+
xcodeVersion: UNOBSERVED,
249+
provenance: 'shipped-sniff-trigger',
250+
output:
251+
"error: Signing for \"AgentDeviceRunner\" requires a development team (in target 'AgentDeviceRunner' from project 'AgentDeviceRunner')\n** TEST BUILD FAILED **\n",
252+
deviceReport: { developerMode: 'enabled', developerDiskImage: 'unavailable' },
253+
note: "A build that named its own cause keeps it: a corroborated device state never overwrites xcodebuild's own sentence (#2683).",
254+
},
229255
{
230256
id: 'device-mode-off',
231257
reason: 'device_developer_mode_disabled',
@@ -246,7 +272,7 @@ export const RUNNER_STARTUP_FAILURE_FIXTURES: readonly RunnerStartupFailureFixtu
246272
provenance: 'invented-shape',
247273
output: '"developerModeStatus" : "enabled",\n"ddiServicesAvailable" : false,\n',
248274
deviceReport: { developerMode: 'enabled', developerDiskImage: 'unavailable' },
249-
note: 'The decisive pairing, and the one #2682 used to answer with Developer Mode advice: the toggle is on and only the image is down. The enabled half is the captured state; a device waiting on device support has not been captured.',
275+
note: 'The decisive pairing, and the one #2682 used to answer with Developer Mode advice: the toggle is on and only the image is down. It is NOT a pre-build refusal (#2683 review): iOS 17+ mounts the image on demand during build and launch, so this report has to survive to a failure — which is what `unclassified-build-on-device-with-image-down` records. The enabled half is captured on some devices; a device waiting on device support has not been captured.',
250276
},
251277
{
252278
id: 'devtools-security-disabled',

0 commit comments

Comments
 (0)