Skip to content

Commit 06d734c

Browse files
docs(agents): document the fork-PR e2e split and check-run gate (#3257)
docs/agents/e2e-testing.md is the mandated reference for E2E CI changes but described a single pull-requests.yaml pipeline. Document the split (same-repo e2e in pull-requests.yaml; fork e2e in the privileged e2e-fork.yaml via workflow_run), that the required "E2E Tests" context is a synthetic check-run rather than a job (so renaming the e2e job must preserve it), that fork builds export OCI archives instead of pushing, and the repo prerequisite Actions → "Require approval for all outside collaborators". Refs: #3257 Assisted-By: Claude <[email protected]> Co-Authored-By: Claude Opus 4.8 <[email protected]> Signed-off-by: Myasnikov Daniil <[email protected]>
1 parent 056ff92 commit 06d734c

1 file changed

Lines changed: 14 additions & 0 deletions

File tree

‎docs/agents/e2e-testing.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -100,6 +100,20 @@ Be precise about TIA's scope — it is narrower than "skip E2E for unrelated PRs
100100
- **Exclude loop devices** from host LVM scanning in the Talos machine config so the host does not activate volume groups inside loop-mounted e2e disk images.
101101
- **Fail fast on node readiness** (≈5m, then bail) rather than marching into LB/NFS tests that will also fail — it saves several minutes per attempt and keeps the real failure at the top of the log.
102102

103+
### 10. The E2E workflow split: same-repo vs fork, and the "E2E Tests" check-run
104+
105+
A fork's `pull_request` run carries no registry credentials, so it cannot push the images the e2e sandbox pulls. E2E therefore runs across **two** workflows:
106+
107+
- **`.github/workflows/pull-requests.yaml`** runs e2e in-tree for **same-repo** PRs (the `e2e` job, named `E2E (in-tree)`). Fork PRs skip this job.
108+
- **`.github/workflows/e2e-fork.yaml`** runs e2e for **fork** PRs via a privileged `workflow_run` that fires after the fork's Pull Request run completes. It pushes the fork-built images to the registry and runs the same suite in an ephemeral sandbox that itself has no push credentials.
109+
110+
Two consequences to keep in mind when touching either workflow:
111+
112+
- **Fork builds export, they do not push (`OCI_EXPORT_DIR`).** The fork's unprivileged build sets `OCI_EXPORT_DIR` (see `hack/common-envs.mk`) so every image is written to a per-image OCI archive artifact instead of being pushed; the privileged `e2e-fork.yaml` later pushes those archives to the registry **by digest**. Any build recipe that shells out to push directly — e.g. a `skopeo copy … docker://…` that bypasses the `image-tags`/`oci-output` macros — must honour `OCI_EXPORT_DIR` too, or a fork build dies with an anonymous-push `denied` (#3257). This is covered by `hack/common-envs_test.bats`.
113+
- **"E2E Tests" is a synthetic check-run, not a job.** The required merge-gate context named `E2E Tests` is an explicit GitHub check-run — concluded by the `e2e-report` job for same-repo PRs and by `e2e-fork.yaml` for forks — **not** the `e2e` job itself. This is deliberate: a required *job* that is skipped (as the `e2e` job is for forks) counts as passed by branch protection, which is exactly how fork PRs used to merge with e2e never run (#3257). If you rename the `e2e` job, do **not** name it `E2E Tests` and do not wire branch protection to a job name — the check-run must stay the gate.
114+
- **The check-run is published by the COZYSTACK_CI GitHub App, not the default `GITHUB_TOKEN`.** A token-created check-run is owned by the app that created it, and GitHub files it under that app's check-suite. Created with the default token (the "GitHub Actions" app) a *floating* check-run — one with no owning job — attaches to an arbitrary GitHub Actions check-suite for the head SHA, in practice the earliest-registered one, which is a tiny labeler workflow; that is why `E2E Tests` used to render under "PR size label". Publishing it as the app (both `e2e-report` in `pull-requests.yaml` and the `open`/`report` steps in `e2e-fork.yaml` pass `github-token: ${{ steps.app-token.outputs.token }}`) gives the check its own dedicated suite shown under the app's name, and is stricter — only the CI app can satisfy the gate, not any workflow holding the default token. The app token is minted with `permission-checks: write` only; in `e2e-fork.yaml`'s `resolve` the PR/artifact lookups stay on the default `workflow_run` token, so the decision step and the check-publishing step use different tokens by design. **Consequence for branch protection:** the required `E2E Tests` context must be required *from the COZYSTACK_CI app's id*, not from GitHub Actions (`app_id 15368`). If it is pinned to GitHub Actions, the app-published check will not satisfy the gate and every PR stays "Expected"; conversely a `checks.create`/`checks.update` must always run as the same app that opened the pending run (the fork `report` step re-mints the token for exactly this reason).
115+
116+
**Security prerequisite (repo setting).** The split is only safe if fork runs require maintainer approval before they execute. The repository must keep **Settings → Actions → "Require approval for all outside collaborators"** enabled, so a fork's `pull_request` run — which produces the artifacts `e2e-fork.yaml` consumes — cannot run untrusted code (or feed the privileged run) without a maintainer's go-ahead.
103117
## Chainsaw v0.2.15 gotchas
104118

105119
The suite is pinned to Chainsaw **v0.2.15** (the latest release as of May 2026); none of the traps below are fixed upstream yet, so the workarounds stay until a bump is possible. Each one has already cost a debugging session — this is the "no one gets it right the first time" list, worth a read before writing a new assert.

0 commit comments

Comments
 (0)