Skip to content

Commit f133c58

Browse files
committed
Merge upstream/main into fix/mutation-gate-stacked-base
The only conflict was the gate step env block in .github/workflows/mutation-testing.yml. Both sides were kept: PR_BASE_REF (this branch, it tells the gate which branch the unit targets) and PR_HEAD_SHA (upstream #2293, the pull request head used as provenance for the mutation PR comment). The run block now takes the event value when it is present and derives the head from the merge commit only as a fallback for a run with no pull_request payload, so one variable no longer silently overrides the other. scripts/stryker-diff.mjs and scripts/stryker-diff.test.mjs merged on their own; the pin for that derivation line was updated to the merged spelling, and the continue-on-error count pin (3) already matches the merged workflow.
2 parents e8de314 + a15ba8c commit f133c58

28 files changed

Lines changed: 4545 additions & 254 deletions

‎.github/workflows/mutation-pr-comment.yml‎

Lines changed: 428 additions & 0 deletions
Large diffs are not rendered by default.

‎.github/workflows/mutation-testing.yml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -119,12 +119,15 @@ jobs:
119119
env:
120120
HEAD_SHA: ${{ github.sha }}
121121
PR_BASE_REF: ${{ github.event.pull_request.base.ref }}
122+
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
122123
run: |
123124
BASE_SHA="$(git rev-parse "$HEAD_SHA^1")"
124125
# The merge commit's second parent is the pull request head. When that head's own
125126
# parent is another open pull request's head, this PR is a stacked unit, so the gate
126127
# measures only this unit's delta instead of every unmerged ancestor.
127-
PR_HEAD_SHA="$(git rev-parse "$HEAD_SHA^2" 2>/dev/null || git rev-parse "$HEAD_SHA")"
128+
# The event already carries that head for provenance, so it wins here; deriving it from
129+
# the merge commit stays as the fallback for a manual run with no pull_request payload.
130+
PR_HEAD_SHA="${PR_HEAD_SHA:-$(git rev-parse "$HEAD_SHA^2" 2>/dev/null || git rev-parse "$HEAD_SHA")}"
128131
node scripts/stryker-diff.mjs ci --base "$BASE_SHA" --head "$HEAD_SHA" --pr-head "$PR_HEAD_SHA" --pr-base-ref "$PR_BASE_REF" --stacked-map-file .mutation/stacked-map.json
129132
130133
- name: Upload mutation reports

‎docs/mutation-testing.md‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# Mutation report PR comments
2+
3+
Changed-code mutation testing analyzes the pull request merge commit. Its report
4+
keeps that merge SHA in the reproduction command, but uses the PR head SHA in the
5+
comment marker. The trusted comment workflow checks that marker against the
6+
completed run's head SHA and the PR's current head before posting.
7+
8+
The workflow maintains one advisory comment per PR. Reports from older heads and
9+
closed PRs are ignored. Counts remain visible; a section collapsed by default
10+
contains the full report, mutant details, guidance, and reproduction steps.
11+
The renderer supports headings, paragraphs, lists, emphasis, inline code, fenced
12+
code blocks, and tables using trusted HTML structure with escaped text. Table
13+
cells, URLs, mentions, issue references, commit SHAs, and emoji shortcodes remain
14+
literal code. Commands and code examples use
15+
code blocks; guidance renders as prose. Artifact text cannot introduce active
16+
links, images, HTML, or mentions.
17+
18+
Focused test names come from Stryker's report or the selected-test fallback. An
19+
empty list means the names were not recorded; it does not establish that the run
20+
failed. The summary identifies incomplete runs from their result and reports
21+
zero-mutant packages separately.
22+
23+
A missing, unreadable, oversized, or invalid report deliberately leaves the last
24+
valid comment unchanged. This includes failed runs that exit before producing a
25+
report. The visible head SHA identifies which commit the comment describes; it
26+
does not certify that later runs completed. A valid clean report updates an
27+
existing comment to an all-clear result, but does not create a new comment.
28+
29+
Run `pnpm test:mutation-ci` to check report generation and execute the trusted
30+
inline comment script with mocked GitHub and filesystem APIs. The tests cover
31+
SHA matching, stale heads, invalid artifacts, table rendering, hostile cell escaping, truncation, and
32+
comment creation and updates.
33+
34+
## Validate the GitHub workflow before merge
35+
36+
Use private disposable repositories with the consumer workflow on the base
37+
repository's default branch. Copy the consumer and formatter unchanged. A minimal
38+
producer can call the real formatter with deterministic survivor, clean, invalid,
39+
and delayed reports while retaining the production SHA environment variables,
40+
merge checkout, workflow name, and artifact upload action.
41+
42+
Exercise a same-repository PR and a genuine cross-owner fork PR. Check comment
43+
creation and updates, clean results with and without an existing comment, invalid
44+
or missing report retention, rendered HTML, and an old run finishing after a new
45+
head. Download the artifact to verify that its PR head marker differs from the
46+
analyzed merge SHA. Keep fork jobs read-only with no forwarded secrets.
47+
48+
This checks real triggers, token permissions, artifact downloads, and comment
49+
writes before merge. It does not rerun mutation analysis or establish the target
50+
repository's own Actions policy; verify that policy after deployment.

‎packages/config-eslint/api-provider-hygiene.js‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,3 +81,20 @@ export const apiSpecTsIgnoreConfig = {
8181
"@typescript-eslint/ban-ts-comment": ["error", { "ts-ignore": true }],
8282
},
8383
}
84+
85+
// A fixed sleep in a provider spec either flakes or hides a missing await. Poll with vi.waitFor on the
86+
// observable condition. This uses no-restricted-globals, not no-restricted-syntax, because later config
87+
// blocks replace no-restricted-syntax for the same files.
88+
export const apiSpecSleepConfig = {
89+
files: ["api/**/__tests__/**/*.spec.ts"],
90+
rules: {
91+
"no-restricted-globals": [
92+
"error",
93+
{
94+
name: "setTimeout",
95+
message:
96+
"Poll with vi.waitFor on the observable condition. A fixed sleep either flakes or hides a missing await.",
97+
},
98+
],
99+
},
100+
}

‎scripts/stryker-diff.mjs‎

Lines changed: 37 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,9 @@ import ts from "typescript"
1111
export const MAX_CHANGED_LINES = 500
1212
export const MAX_MUTANTS = 400
1313

14+
export const PR_COMMENT_MARKER = "<!-- zoo-code-mutation-report -->"
15+
export const PR_COMMENT_FILENAME = "pr-comment.md"
16+
1417
export const PACKAGE_CONFIGS = [
1518
{
1619
id: "core",
@@ -714,9 +717,13 @@ export function formatSummary(rows, advisories, manifest = {}) {
714717
if (row.testFiles?.length > 0) {
715718
lines.push(`- **${row.id}** (cwd \`${cwd}\`): ${row.testFiles.map((file) => `\`${file}\``).join(", ")}`)
716719
} else {
717-
lines.push(
718-
`- **${row.id}** (cwd \`${cwd}\`): the mutation run did not complete far enough to report its selected tests; use the exact reproduction command below.`,
719-
)
720+
const detail =
721+
row.result === "Advisory incomplete"
722+
? "The mutation run was incomplete; selected test file names were not recorded. See Advisory findings for the failure."
723+
: row.result === "No mutants generated"
724+
? "No mutants were generated; selected test file names were not recorded."
725+
: "Selected test file names were not recorded in the mutation report."
726+
lines.push(`- **${row.id}** (cwd \`${cwd}\`): ${detail}`)
720727
}
721728
}
722729
}
@@ -792,8 +799,32 @@ export function formatSummary(rows, advisories, manifest = {}) {
792799
return `${lines.join("\n")}\n`
793800
}
794801

795-
export function appendSummary(rows, advisories, manifest) {
802+
export function formatPrComment(rows, advisories, manifest) {
803+
// Analysis uses the merge commit; comment provenance uses the PR head.
804+
const headSha = process.env.PR_HEAD_SHA || manifest?.headSha
805+
if (!headSha || !/^[0-9a-f]{40}$/i.test(headSha)) return undefined
806+
const blocking = rows.reduce((total, row) => total + (row.blocking?.length ?? 0), 0)
807+
return [
808+
PR_COMMENT_MARKER,
809+
`<!-- zoo-code-mutation-report head:${headSha.toLowerCase()} blocking:${blocking} advisories:${advisories.length} -->`,
810+
"",
811+
formatSummary(rows, advisories, manifest),
812+
].join("\n")
813+
}
814+
815+
export function appendSummary(rows, advisories, manifest, commentFile) {
796816
for (const advisory of advisories) console.warn(formatAdvisoryCommand(advisory))
817+
const comment = commentFile ? formatPrComment(rows, advisories, manifest) : undefined
818+
if (comment) {
819+
try {
820+
fs.mkdirSync(path.dirname(commentFile), { recursive: true })
821+
fs.writeFileSync(commentFile, comment)
822+
} catch (error) {
823+
console.warn(
824+
`::warning title=Mutation test advisory::Could not write the mutation PR comment file: ${escapeWorkflowData(error.message)}`,
825+
)
826+
}
827+
}
797828
if (!process.env.GITHUB_STEP_SUMMARY) return
798829
try {
799830
fs.appendFileSync(process.env.GITHUB_STEP_SUMMARY, formatSummary(rows, advisories, manifest))
@@ -936,7 +967,7 @@ export function runManifest(
936967
}
937968
}
938969

939-
appendSummary(rows, advisories, manifest)
970+
appendSummary(rows, advisories, manifest, path.join(reportRoot, PR_COMMENT_FILENAME))
940971
return rows
941972
}
942973

@@ -1077,7 +1108,7 @@ function main() {
10771108
stackedOn: invocation.stackedOn,
10781109
}
10791110
if (manifest.packages.length === 0) {
1080-
appendSummary([], manifest.advisories, manifest)
1111+
appendSummary([], manifest.advisories, manifest, path.join(reportRoot, PR_COMMENT_FILENAME))
10811112
console.log("No changed executable lines in mutation-tested packages; mutation testing is not applicable.")
10821113
return
10831114
}

0 commit comments

Comments
 (0)