You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 06d734c
Browse filesBrowse the repository at this point in the historyBrowse files
docs(agents): document the fork-PR e2e split and check-run gate (#3257)
docs/agents/e2e-testing.md is the mandated reference for E2E CI changes but
described a single pull-requests.yaml pipeline. Document the split (same-repo
e2e in pull-requests.yaml; fork e2e in the privileged e2e-fork.yaml via
workflow_run), that the required "E2E Tests" context is a synthetic check-run
rather than a job (so renaming the e2e job must preserve it), that fork builds
export OCI archives instead of pushing, and the repo prerequisite Actions →
"Require approval for all outside collaborators".
Refs: #3257
Assisted-By: Claude <[email protected]>
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Signed-off-by: Myasnikov Daniil <[email protected]>
Copy file name to clipboardExpand all lines: docs/agents/e2e-testing.md
+14Lines changed: 14 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -100,6 +100,20 @@ Be precise about TIA's scope — it is narrower than "skip E2E for unrelated PRs
100
100
- **Exclude loop devices** from host LVM scanning in the Talos machine config so the host does not activate volume groups inside loop-mounted e2e disk images.
101
101
- **Fail fast on node readiness** (≈5m, then bail) rather than marching into LB/NFS tests that will also fail — it saves several minutes per attempt and keeps the real failure at the top of the log.
102
102
103
+
### 10. The E2E workflow split: same-repo vs fork, and the "E2E Tests" check-run
104
+
105
+
A fork's `pull_request` run carries no registry credentials, so it cannot push the images the e2e sandbox pulls. E2E therefore runs across **two** workflows:
106
+
107
+
- **`.github/workflows/pull-requests.yaml`** runs e2e in-tree for **same-repo** PRs (the `e2e` job, named `E2E (in-tree)`). Fork PRs skip this job.
108
+
- **`.github/workflows/e2e-fork.yaml`** runs e2e for **fork** PRs via a privileged `workflow_run` that fires after the fork's Pull Request run completes. It pushes the fork-built images to the registry and runs the same suite in an ephemeral sandbox that itself has no push credentials.
109
+
110
+
Two consequences to keep in mind when touching either workflow:
111
+
112
+
- **Fork builds export, they do not push (`OCI_EXPORT_DIR`).** The fork's unprivileged build sets `OCI_EXPORT_DIR` (see `hack/common-envs.mk`) so every image is written to a per-image OCI archive artifact instead of being pushed; the privileged `e2e-fork.yaml` later pushes those archives to the registry **by digest**. Any build recipe that shells out to push directly — e.g. a `skopeo copy … docker://…` that bypasses the `image-tags`/`oci-output` macros — must honour `OCI_EXPORT_DIR` too, or a fork build dies with an anonymous-push `denied` (#3257). This is covered by `hack/common-envs_test.bats`.
113
+
- **"E2E Tests" is a synthetic check-run, not a job.** The required merge-gate context named `E2E Tests` is an explicit GitHub check-run — concluded by the `e2e-report` job for same-repo PRs and by `e2e-fork.yaml` for forks — **not** the `e2e` job itself. This is deliberate: a required *job* that is skipped (as the `e2e` job is for forks) counts as passed by branch protection, which is exactly how fork PRs used to merge with e2e never run (#3257). If you rename the `e2e` job, do **not** name it `E2E Tests` and do not wire branch protection to a job name — the check-run must stay the gate.
114
+
- **The check-run is published by the COZYSTACK_CI GitHub App, not the default `GITHUB_TOKEN`.** A token-created check-run is owned by the app that created it, and GitHub files it under that app's check-suite. Created with the default token (the "GitHub Actions" app) a *floating* check-run — one with no owning job — attaches to an arbitrary GitHub Actions check-suite for the head SHA, in practice the earliest-registered one, which is a tiny labeler workflow; that is why `E2E Tests` used to render under "PR size label". Publishing it as the app (both `e2e-report` in `pull-requests.yaml` and the `open`/`report` steps in `e2e-fork.yaml` pass `github-token: ${{ steps.app-token.outputs.token }}`) gives the check its own dedicated suite shown under the app's name, and is stricter — only the CI app can satisfy the gate, not any workflow holding the default token. The app token is minted with `permission-checks: write` only; in `e2e-fork.yaml`'s `resolve` the PR/artifact lookups stay on the default `workflow_run` token, so the decision step and the check-publishing step use different tokens by design. **Consequence for branch protection:** the required `E2E Tests` context must be required *from the COZYSTACK_CI app's id*, not from GitHub Actions (`app_id 15368`). If it is pinned to GitHub Actions, the app-published check will not satisfy the gate and every PR stays "Expected"; conversely a `checks.create`/`checks.update` must always run as the same app that opened the pending run (the fork `report` step re-mints the token for exactly this reason).
115
+
116
+
**Security prerequisite (repo setting).** The split is only safe if fork runs require maintainer approval before they execute. The repository must keep **Settings → Actions → "Require approval for all outside collaborators"** enabled, so a fork's `pull_request` run — which produces the artifacts `e2e-fork.yaml` consumes — cannot run untrusted code (or feed the privileged run) without a maintainer's go-ahead.
103
117
## Chainsaw v0.2.15 gotchas
104
118
105
119
The suite is pinned to Chainsaw **v0.2.15** (the latest release as of May 2026); none of the traps below are fixed upstream yet, so the workarounds stay until a bump is possible. Each one has already cost a debugging session — this is the "no one gets it right the first time" list, worth a read before writing a new assert.
0 commit comments