Skip to content

Commit 15808ae

Browse files
committed
fix(apple): require a profile and its complaint in one line
A whole-log AND proves two phrases exist, not that one qualifies the other: a note about the profile the build used, three lines above an unrelated expired-certificate warning, published `signing_provisioning_profile_missing` for a profile that was fine. The profile rows now read one line at a time and the expiry row asks for Xcode's own `has expired` phrase. The rules table grows a `toolTextLineIncludesAll` axis beside `toolTextIncludesAll`, and the bundle-identifier row that also paired two phrases moves onto it so no startup row is left reading a whole log; its cross-line negative rides along. `signing_no_profiles_for_bundle_id` shares the `PROFILE_UNUSABLE` advice instead of spelling it out a third time. `.device-evidence/CHECKLIST.md` belongs to the merged #2682 lane, so this stack's capture sheet lives beside it as `CHECKLIST-runner-failures.md`.
1 parent d9c9c5c commit 15808ae

3 files changed

Lines changed: 73 additions & 25 deletions

File tree

‎.device-evidence/CHECKLIST-runner-failures.md‎

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Device evidence checklist
1+
# Runner-failure evidence checklist (#2680, #2683)
22

33
Live evidence the coordinator runs serially on the connected iPhone. Each item names the exact
44
command, the environment it needs, and the rendered error that proves the change. Do not paraphrase
@@ -71,13 +71,15 @@ env AGENT_DEVICE_IOS_TEAM_ID="<your team id>" \
7171
prepare ios-runner --platform ios --device "<physical iPhone name>"
7272
```
7373

74-
Expected: `details.reason` is `signing_provisioning_profile_missing`, and the profile row only fires
75-
when xcodebuild also says what is wrong with the profile — one of its `IDEProvisioningErrorDomain`
76-
diagnostics, "doesn't include ...", or "has expired" (#2688 review). Paste the whole error: which of
77-
those lines printed is what promotes the `profile-xcode-signing-error`,
78-
`profile-does-not-cover-app-id` and `profile-expired` fixtures from `invented-shape` to `captured`. A
79-
different reason is worth recording just as much: say which one and treat the fixtures as unconfirmed
80-
rather than editing the rules to fit.
74+
Expected: `details.reason` is `signing_provisioning_profile_missing`, and the profile rows only fire
75+
when xcodebuild says what is wrong with the profile **on the same line as the profile**: its
76+
`IDEProvisioningErrorDomain` diagnostic naming the profile, `doesn't include ...`, or `has expired`
77+
(#2688 review). Paste the whole error and keep the line breaks — which line carried which phrase is
78+
what promotes the `profile-xcode-signing-error`, `profile-does-not-cover-app-id` and `profile-expired`
79+
fixtures from `invented-shape` to `captured`, and a capture that splits the two phrases across lines
80+
belongs to `profile-note-above-an-expired-certificate` instead. A different reason is worth recording
81+
just as much: say which one and treat the fixtures as unconfirmed rather than editing the rules to
82+
fit.
8183

8284
### 4. The line that claims no reason yet -> `build_failed_unclassified`
8385

‎packages/platform-apple/src/runner/__tests__/runner-startup-failure-fixtures.ts‎

Lines changed: 23 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ import type { RunnerStartupFailureReason } from '../runner-contract.ts';
77
* Provenance is the point of this file, so it is stated per entry and never as a blanket claim:
88
*
99
* - `captured` — `output` was pasted from a run, and `command` plus `xcodeVersion` (from
10-
* `xcodebuild -version`) were recorded with it by `.device-evidence/CHECKLIST.md`.
10+
* `xcodebuild -version`) were recorded with it by `.device-evidence/CHECKLIST-runner-failures.md`.
1111
* - `shipped-sniff-trigger` — the substrings a rule matches are the ones shipped in
1212
* `resolveSigningFailureHint` before #2680, which is evidence xcodebuild can emit them. The
1313
* sentence around them is ours, so `command` and `xcodeVersion` stay unrecorded.
@@ -93,6 +93,16 @@ export const RUNNER_STARTUP_FAILURE_FIXTURES: readonly RunnerStartupFailureFixtu
9393
"error: Signing for \"AgentDeviceRunner\" requires a development team (in target 'AgentDeviceRunner' from project 'AgentDeviceRunner')",
9494
note: 'Same text arriving in the thrown message instead of the exec details: the catch wraps a non-AppError with String(err), and the rule still has to see it.',
9595
},
96+
{
97+
id: 'app-identifier-and-availability-in-different-lines',
98+
reason: 'build_failed_unclassified',
99+
site: 'build-for-testing',
100+
xcodeVersion: UNOBSERVED,
101+
provenance: 'invented-shape',
102+
output:
103+
"error: App Identifier 'com.yourname.agentdevice.runner' is invalid (in target 'AgentDeviceRunner' from project 'AgentDeviceRunner')\nnote: The simulator device is not available for this destination\n** TEST BUILD FAILED **\n",
104+
note: 'The same cross-line hazard the profile rows gave up (#2688 review): one line faults the identifier and another says something is not available, and neither line pairs them. The reason needs both in one sentence, which is what `app-id-not-available` records.',
105+
},
96106
{
97107
id: 'no-profiles-for-bundle-id',
98108
reason: 'signing_provisioning_profile_missing',
@@ -158,8 +168,8 @@ export const RUNNER_STARTUP_FAILURE_FIXTURES: readonly RunnerStartupFailureFixtu
158168
xcodeVersion: UNOBSERVED,
159169
provenance: 'invented-shape',
160170
output:
161-
"error: Provisioning profile \"match-development\" is not a valid provisioning profile (in target 'AgentDeviceRunner' from project 'AgentDeviceRunner')\nError Domain=IDEProvisioningErrorDomain Code=17\n** TEST BUILD FAILED **\n",
162-
note: "Xcode names the profile beside its own IDEProvisioningErrorDomain diagnostics. Sentence and domain code are our reconstruction; Phase B capture has to record the real wording and this entry's xcodeVersion.",
171+
"error: Provisioning profile \"match-development\" is not a valid provisioning profile (in target 'AgentDeviceRunner' from project 'AgentDeviceRunner')\nError Domain=IDEProvisioningErrorDomain Code=17 \"Provisioning profile 'match-development' is not a valid provisioning profile.\"\n** TEST BUILD FAILED **\n",
172+
note: "Xcode repeats the profile inside the same line as its IDEProvisioningErrorDomain diagnostics, which is what the row reads: domain on one line and profile on another is two facts, not one complaint. Sentence and domain code are our reconstruction; Phase B capture has to record the real wording and this entry's xcodeVersion.",
163173
},
164174
{
165175
id: 'profile-does-not-cover-app-id',
@@ -191,6 +201,16 @@ export const RUNNER_STARTUP_FAILURE_FIXTURES: readonly RunnerStartupFailureFixtu
191201
"note: Using provisioning profile \"match-development\" to sign the app bundle (in target 'AgentDeviceRunner' from project 'AgentDeviceRunner')\nerror: cannot find 'AgentDeviceRunnerCommand' in scope (in target 'AgentDeviceRunnerUITests' from project 'AgentDeviceRunner')\n** TEST BUILD FAILED **\n",
192202
note: 'The hazard the bare `provisioning profile` trigger carried (#2688 review): a failing build can print the profile it used while the failure is a compile error. A benign mention must keep cache-recovery advice; it also says nothing Xcode calls code signing, which is its own honest row.',
193203
},
204+
{
205+
id: 'profile-note-above-an-expired-certificate',
206+
reason: 'build_failed_unclassified',
207+
site: 'build-for-testing',
208+
xcodeVersion: UNOBSERVED,
209+
provenance: 'invented-shape',
210+
output:
211+
"note: Using provisioning profile \"match-development\" to sign the app bundle (in target 'AgentDeviceRunner' from project 'AgentDeviceRunner')\nwarning: The certificate \"Apple Development: Example Dev (ABCD1234)\" has expired.\nerror: cannot find 'AgentDeviceRunnerCommand' in scope (in target 'AgentDeviceRunnerUITests' from project 'AgentDeviceRunner')\n** TEST BUILD FAILED **\n",
212+
note: 'The cross-line hazard a whole-log AND cannot see (#2688 review): a benign profile note three lines above an unrelated expired-certificate warning. Both phrases are in the captured log and neither qualifies the other, so the profile stays unclassified and the reader keeps cache-recovery advice rather than being sent to replace a profile that is fine.',
213+
},
194214
{
195215
id: 'devtools-security-disabled',
196216
reason: 'devtools_security_developer_mode_disabled',

‎packages/platform-apple/src/runner/runner-contract.ts‎

Lines changed: 40 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -178,6 +178,14 @@ type RunnerErrorMatch = {
178178
* the verdict this classifier already published can never carry a rule (#2680).
179179
*/
180180
toolTextIncludesAll?: readonly string[];
181+
/**
182+
* Every entry must appear in the SAME line of the lowercased {@link runnerToolText} (#2688 review).
183+
* {@link RunnerErrorMatch.toolTextIncludesAll} proves only that two phrases exist somewhere in a
184+
* captured log, which is a weaker claim than one phrase qualifying the other: a note about the
185+
* profile the build used, three lines above an unrelated expired-certificate warning, says nothing
186+
* about the profile. A row whose evidence is a noun and its complaint asks for both on one line.
187+
*/
188+
toolTextLineIncludesAll?: readonly string[];
181189
/** Required details evidence beyond code/message. */
182190
details?: RunnerErrorDetailsMatch;
183191
};
@@ -403,8 +411,11 @@ export const RUNNER_ERROR_RULES: readonly RunnerErrorRule[] = [
403411
},
404412
},
405413
{
414+
// The identifier and its availability have to meet in one line: `App Identifier` and `not
415+
// available` are two phrases a captured log can carry for reasons that have nothing to do with
416+
// each other, which is the same hazard the profile rows just gave up (#2688 review).
406417
reason: 'bundle_identifier_unavailable',
407-
match: { toolTextIncludesAll: ['app identifier', 'not available'] },
418+
match: { toolTextLineIncludesAll: ['app identifier', 'not available'] },
408419
verdicts: {},
409420
buildFailure: {
410421
reason: 'bundle_identifier_already_registered',
@@ -435,41 +446,43 @@ export const RUNNER_ERROR_RULES: readonly RunnerErrorRule[] = [
435446
},
436447
},
437448
{
449+
// "No profiles for 'com.example' were found" names the bundle id and the absence in one sentence,
450+
// so the phrase alone is the complaint and needs no second phrase to qualify it.
438451
reason: 'signing_no_profiles_for_bundle_id',
439452
match: { toolTextIncludesAll: ['no profiles for'] },
440453
verdicts: {},
441-
buildFailure: {
442-
reason: 'signing_provisioning_profile_missing',
443-
hint: 'Install/select a valid iOS provisioning profile, or set AGENT_DEVICE_IOS_PROVISIONING_PROFILE.',
444-
},
454+
buildFailure: PROFILE_UNUSABLE,
445455
},
446456
// A profile named in the tool's output is only evidence when the output also says what is wrong with
447-
// it (#2688 review). One bare `provisioning profile` substring was the shipped sniffer's trigger, and
448-
// it is a phrase a failing build can print while talking about something else: the codesign command
449-
// line, a build-settings dump, a note about the profile that was used. Each row below therefore
450-
// requires the profile plus the complaint Xcode attaches to it, and a failure that merely mentions a
451-
// profile stays unclassified rather than being sent to install a profile it already has.
457+
// that profile, in the same line (#2688 review). One bare `provisioning profile` substring was the
458+
// shipped sniffer's trigger, and it is a phrase a failing build can print while talking about
459+
// something else: the codesign command line, a build-settings dump, a note about the profile that was
460+
// used. Requiring a second phrase somewhere in the same log is no better — a note about the profile
461+
// used above an unrelated `has expired` certificate warning would then name the profile. Each row
462+
// below therefore asks for the profile and Xcode's complaint about it on one line, and a failure that
463+
// merely mentions a profile stays unclassified rather than being sent to install one it already has.
452464
{
453465
// Xcode's own signing-error domain beside the profile it rejected: the machine-readable half of its
454466
// `IDEProvisioningErrorDomain` diagnostics, which accompanies the prose rather than replacing it.
455467
reason: 'signing_provisioning_profile_xcode_error',
456-
match: { toolTextIncludesAll: ['provisioning profile', 'ideprovisioningerrordomain'] },
468+
match: { toolTextLineIncludesAll: ['provisioning profile', 'ideprovisioningerrordomain'] },
457469
verdicts: {},
458470
buildFailure: PROFILE_UNUSABLE,
459471
},
460472
{
461473
// "Provisioning profile \"X\" doesn't include application identifier ..." — the profile that is
462474
// installed but does not cover this app or capability.
463475
reason: 'signing_provisioning_profile_does_not_cover',
464-
match: { toolTextIncludesAll: ['provisioning profile', "doesn't include"] },
476+
match: { toolTextLineIncludesAll: ['provisioning profile', "doesn't include"] },
465477
verdicts: {},
466478
buildFailure: PROFILE_UNUSABLE,
467479
},
468480
{
469481
// "Provisioning profile \"X\" has expired" — installing it again is not the fix; replacing it is,
470-
// which is what the hint's "valid" is for.
482+
// which is what the hint's "valid" is for. The full phrase, on the profile's own line: `expired`
483+
// alone is what an expired certificate, a stale session, or a revoked key writes (#2688 review).
471484
reason: 'signing_provisioning_profile_expired',
472-
match: { toolTextIncludesAll: ['provisioning profile', 'expired'] },
485+
match: { toolTextLineIncludesAll: ['provisioning profile', 'has expired'] },
473486
verdicts: {},
474487
buildFailure: PROFILE_UNUSABLE,
475488
},
@@ -499,6 +512,7 @@ function matchesRunnerErrorRule(error: AppError, match: RunnerErrorMatch): boole
499512
if (match.code !== undefined && error.code !== match.code) return false;
500513
if (!matchesRunnerErrorDetails(error, match.details)) return false;
501514
if (!matchesRunnerToolText(error, match.toolTextIncludesAll)) return false;
515+
if (!matchesRunnerToolTextLine(error, match.toolTextLineIncludesAll)) return false;
502516
return matchesRunnerErrorMessage(error, match.messageIncludesAll);
503517
}
504518

@@ -533,6 +547,18 @@ function matchesRunnerToolText(error: AppError, parts: readonly string[] | undef
533547
return parts.every((part) => text.includes(part));
534548
}
535549

550+
/**
551+
* The same haystack read one line at a time, so a row can require its phrases to be in one sentence
552+
* rather than merely in one file (#2688 review). A captured build log is thousands of lines long, and
553+
* two unrelated lines can hold any pair of words.
554+
*/
555+
function matchesRunnerToolTextLine(error: AppError, parts: readonly string[] | undefined): boolean {
556+
if (!parts) return true;
557+
return runnerToolText(error)
558+
.split('\n')
559+
.some((line) => parts.every((part) => line.includes(part)));
560+
}
561+
536562
function runnerErrorVerdict<Axis extends keyof RunnerErrorVerdicts>(
537563
error: unknown,
538564
axis: Axis,

0 commit comments

Comments
 (0)