You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit ae36143
Browse filesBrowse the repository at this point in the historyBrowse files
fix(e2e): validate the leak allowlist against every suite layout
lg_allowlist_validate accepted only <suite>/chainsaw-test.y*ml, while the
selectors already see step-file suites and a Test nested below the suite
directory. A suite in one of those layouts could not be allowlisted at
all: the run stopped before the first suite. Validate the allowlist with
the rule hack/select-e2e.sh enumerates suites with, spelled the same way
on both sides and cross-referenced so they move together.
Describe the three layouts once in the e2e-chainsaw README, point the
agent doc at it, and list kubernetes-proxmox as the parked suite.
Closes#4659
Signed-off-by: Marian Koreniuk <[email protected]>
Assisted-by: LLM
(cherry picked from commit 782e0ab)
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com>
Copy file name to clipboardExpand all lines: docs/agents/e2e-testing.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,7 +2,7 @@
2
2
3
3
Guidance for writing, changing, and reviewing Cozystack's end-to-end (E2E) tests and the CI that runs them. Read this **before** touching anything under `hack/e2e-chainsaw/`, the bootstrap/OpenAPI BATS (`hack/e2e-*.bats`), `hack/*.sh` test helpers, either E2E CI workflow (`.github/workflows/pull-requests.yaml` for same-repo pull requests, `.github/workflows/e2e-fork.yaml` for forks), or `packages/core/testing/`. §10 covers how the two split the work and which invariants hold them together.
4
4
5
-
The app suite is **Kyverno Chainsaw** (`hack/e2e-chainsaw/`, one directory per app, each with a `chainsaw-test.yaml`; §7 lists the other layouts Chainsaw accepts). Cluster bootstrap (`hack/e2e-install-cozystack.bats`, `hack/e2e-prepare-cluster.bats`) and the OpenAPI checks (`hack/e2e-test-openapi.bats`) remain BATS.
5
+
The app suite is **Kyverno Chainsaw** (`hack/e2e-chainsaw/`, one directory per app, usually with a `chainsaw-test.yaml`; the README there describes the layouts a suite can take, and §7 how they are selected). Cluster bootstrap (`hack/e2e-install-cozystack.bats`, `hack/e2e-prepare-cluster.bats`) and the OpenAPI checks (`hack/e2e-test-openapi.bats`) remain BATS.
A suite is a directory directly under `hack/e2e-chainsaw/` in or below which Chainsaw finds a Test. Three layouts do that, and the selector (`hack/select-e2e.sh`), the runner (`hack/e2e-chainsaw-run.sh`) and the leak-guard allowlist all recognise the same three:
22
+
23
+
-`chainsaw-test.yaml`, the usual one, or `chainsaw-test.yml` when the `.yaml` is absent;
24
+
- numbered step files such as `01-install.yaml`, which Chainsaw turns into a Test on its own when neither of the above exists;
25
+
- either of those in a subdirectory of the suite, since Chainsaw walks every subdirectory of the directory it is handed.
26
+
27
+
A parked `chainsaw-test.yaml.disabled` is none of these: the directory is then not a suite, is selected by nothing and cannot be allowlisted.
28
+
29
+
Suites: `postgres`, `bucket`, `mariadb`, `mongodb`, `redis`, `valkey`, `qdrant`, `clickhouse`, `kafka`, `kafka-metadata`, `rabbitmq`, `etcd`, `openbao`, `harbor`, `foundationdb`, `external-dns`, `kuberture`, `vminstance`, `gateway`, `opensearch`, `monitoring`, `kubernetes-latest`, `kubernetes-previous`, `securitygroup`, `seaweedfs`, `computeplane`. `kubernetes-proxmox` is parked as `chainsaw-test.yaml.disabled` and runs only in `.github/workflows/e2e-proxmox.yaml`, which needs a Proxmox hypervisor.
Each suite is a directory with a `chainsaw-test.yaml` (one or more `Test` docs) plus the manifests it applies. A step has `try` (operations: `apply`, `assert`, `error`, `script`, `delete`, …), optional `catch` (diagnostics on failure), and optional `finally` (teardown regardless of outcome).
41
+
Each suite is a directory with one or more `Test` docs (in one of the layouts above, usually a `chainsaw-test.yaml`) plus the manifests it applies. A step has `try` (operations: `apply`, `assert`, `error`, `script`, `delete`, …), optional `catch` (diagnostics on failure), and optional `finally` (teardown regardless of outcome).
34
42
35
43
Declarative suites assert on `status.conditions` and concrete fields. Inherently imperative suites keep their logic in `script` steps:
echo"leak-guard: allowlist names '$suite', which is not a suite directory under $root">&2
213
+
if! lg_suite_has_test"$root""$suite";then
214
+
echo"leak-guard: allowlist names '$suite', which is not a suite directory under $root (no chainsaw-test.yaml, chainsaw-test.yml or numbered step file in or below it)">&2
0 commit comments