|
| 1 | +# Mutation report PR comments |
| 2 | + |
| 3 | +Changed-code mutation testing analyzes the pull request merge commit. Its report |
| 4 | +keeps that merge SHA in the reproduction command, but uses the PR head SHA in the |
| 5 | +comment marker. The trusted comment workflow checks that marker against the |
| 6 | +completed run's head SHA and the PR's current head before posting. |
| 7 | + |
| 8 | +The workflow maintains one advisory comment per PR. Reports from older heads and |
| 9 | +closed PRs are ignored. Counts remain visible; a section collapsed by default |
| 10 | +contains the full report, mutant details, guidance, and reproduction steps. |
| 11 | +The renderer supports headings, paragraphs, lists, emphasis, inline code, fenced |
| 12 | +code blocks, and tables using trusted HTML structure with escaped text. Table |
| 13 | +cells, URLs, mentions, issue references, commit SHAs, and emoji shortcodes remain |
| 14 | +literal code. Commands and code examples use |
| 15 | +code blocks; guidance renders as prose. Artifact text cannot introduce active |
| 16 | +links, images, HTML, or mentions. |
| 17 | + |
| 18 | +Focused test names come from Stryker's report or the selected-test fallback. An |
| 19 | +empty list means the names were not recorded; it does not establish that the run |
| 20 | +failed. The summary identifies incomplete runs from their result and reports |
| 21 | +zero-mutant packages separately. |
| 22 | + |
| 23 | +A missing, unreadable, oversized, or invalid report deliberately leaves the last |
| 24 | +valid comment unchanged. This includes failed runs that exit before producing a |
| 25 | +report. The visible head SHA identifies which commit the comment describes; it |
| 26 | +does not certify that later runs completed. A valid clean report updates an |
| 27 | +existing comment to an all-clear result, but does not create a new comment. |
| 28 | + |
| 29 | +Run `pnpm test:mutation-ci` to check report generation and execute the trusted |
| 30 | +inline comment script with mocked GitHub and filesystem APIs. The tests cover |
| 31 | +SHA matching, stale heads, invalid artifacts, table rendering, hostile cell escaping, truncation, and |
| 32 | +comment creation and updates. |
| 33 | + |
| 34 | +## Validate the GitHub workflow before merge |
| 35 | + |
| 36 | +Use private disposable repositories with the consumer workflow on the base |
| 37 | +repository's default branch. Copy the consumer and formatter unchanged. A minimal |
| 38 | +producer can call the real formatter with deterministic survivor, clean, invalid, |
| 39 | +and delayed reports while retaining the production SHA environment variables, |
| 40 | +merge checkout, workflow name, and artifact upload action. |
| 41 | + |
| 42 | +Exercise a same-repository PR and a genuine cross-owner fork PR. Check comment |
| 43 | +creation and updates, clean results with and without an existing comment, invalid |
| 44 | +or missing report retention, rendered HTML, and an old run finishing after a new |
| 45 | +head. Download the artifact to verify that its PR head marker differs from the |
| 46 | +analyzed merge SHA. Keep fork jobs read-only with no forwarded secrets. |
| 47 | + |
| 48 | +This checks real triggers, token permissions, artifact downloads, and comment |
| 49 | +writes before merge. It does not rerun mutation analysis or establish the target |
| 50 | +repository's own Actions policy; verify that policy after deployment. |
0 commit comments