Skip to content

RQ2 — Link-integrity bot (warning-only) #1284

Description

@njhensley

Parent: #1265 · Labels: enhancement, area/ci (auto, .github/**) · Size: S–M · Depends on: RQ1 (links must exist to verify), a reachable TestGrid-epic public UI/API (TG4a), and a concrete TestGrid-epic "coordinate presence" query surface (TG4a deliverable) — see below.

Goal. A CI job that verifies Evidence deep-links resolve once a recipe has a TestGrid presence — advisory/warning-only, like make bom-check and the warning-only recipe-evidence.yaml gate. Never blocks merges; never edits the doc.

Scope.

  • Add .github/workflows/testgrid-link-check.yaml, structurally cloning the shipped .github/workflows/bom-refresh.yaml: weekly schedule cron + workflow_dispatch; permissions: contents: read on the top job; built-in GITHUB_TOKEN; runs-on: ubuntu-latest; timeout-minutes. Difference from bom-refresh: instead of opening a PR on drift, it emits a Markdown report to $GITHUB_STEP_SUMMARY (and optionally an artifact) listing any Evidence link that 404s / lands on an empty coordinate for a recipe the TestGrid epic reports as present. No create-pull-request, no write token, no doc edit.
  • Presence signal is a hard dependency on the TestGrid epic, not a mock. RQ2's only useful classification (missing-but-present warn vs not-yet-present ok) requires asking the TestGrid epic "which coordinates have at least one build?" the TestGrid epic's TG4a must expose a coordinate-presence / list-populated-coordinates endpoint as a named deliverable; RQ2 depends on it concretely. Without it, RQ2 cannot distinguish a legitimately-empty coordinate from a broken link and collapses to all-noise or all-silent.
  • Resolution check uses a bounded HTTP client (&http.Client{Timeout: defaults.HTTPClientTimeout} — never http.DefaultClient) and a pkg/defaults-bounded io.LimitReader on any response body, per repo HTTP-client rules. Harden against SSRF/redirect: pin the resolution base to the known TestGrid-epic UI origin (host allowlist), disable redirect-following to off-origin hosts, and treat only same-origin 2xx as resolved — so that if a link target ever becomes influenced by recipe/overlay content, the weekly bot cannot be steered off-origin. Treat "present in TestGrid but link dead" as a warning row; "not yet present" is expected and not a warning (mirrors ADR-009 distinguishing unattested from failure, 009-recipe-health-tracking.md:245-251, and the recipe-evidence.yaml warning-only posture). The transient "evidence pointer landed but the TestGrid column has not materialized yet" state is expected, not a warning (the publish/ingest lag is normal).
  • Note on lychee scope (verified): fern-docs-ci.yaml runs lychee with --offline on docs/**/*.md (fern-docs-ci.yaml:56) and .lychee.toml excludes (^|/)design(/|$) (:23) — so lychee resolves no external/dynamic URLs at all in this repo, by configuration, not merely "would resolve them poorly." Coordinate-resolution against a live UI is therefore necessarily a separate job regardless of URL shape — this bot is the right home, not a lychee config tweak. Do not attempt to fold this into lychee.
  • Inherit the repo's /ok reviewer-comment policy only if the job ever opens a PR (it should not in V1); since it is report-only, no PR-refire apparatus is needed.

Out of scope. Editing the matrix (RQ1 owns the doc). Blocking merges. Alerting/paging (deferred — TestGrid is the failure-signal surface). Verifying TestGrid content correctness (the TestGrid epic's summarizer owns that). Re-implementing the mapping function (imports the TestGrid epic's, as RQ1 does).

Key files. New .github/workflows/testgrid-link-check.yaml; pattern source .github/workflows/bom-refresh.yaml (weekly cron + peter-evans/create-pull-request shape) and .github/workflows/recipe-evidence.yaml (warning-only, read-only contents: read token at :50; header self-describes as "Workflow 1 of 2 for the warning-only recipe-evidence gate", :15); .lychee.toml / .github/workflows/fern-docs-ci.yaml (to document why this is separate from lychee); the TestGrid epic's shared mapping function (to recompute expected coordinates) and TG4a's presence endpoint.

Tests. A small Go helper (under tools/ or reused from RQ1) that, given a recipe list and a presence signal, computes expected links and classifies each as resolved / missing-but-present (warn) / not-yet-present (ok); table-driven unit tests over those three cases plus the off-origin-redirect-rejected case; a dry-run mode asserting the workflow produces a deterministic report and exits 0 even when warnings exist (warning-only). Workflow lint via the repo's existing actions-lint path.

Acceptance. Weekly job reports dead Evidence links for present recipes, never blocks merges, never edits the doc, uses a bounded HTTP client pinned to the TestGrid epic origin with off-origin redirects refused, keys off the TestGrid epic's concrete presence endpoint, and treats "not yet present" / "pointer-landed-but-no-column-yet" as expected. Clone-of-bom-refresh structure confirmed in review.


Child of #1265 · staged from docs/design/012-recipe-quality-surfaces.md (RQ2).

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions