Skip to content

Run k8s e2e tests against the kind infrastructure in CI#339

Merged
nikolasmatt merged 21 commits intoagentregistry-dev:mainfrom
nikolasmatt:run-e2e-against-kind
Mar 16, 2026
Merged

Run k8s e2e tests against the kind infrastructure in CI#339
nikolasmatt merged 21 commits intoagentregistry-dev:mainfrom
nikolasmatt:run-e2e-against-kind

Conversation

@nikolasmatt
Copy link
Copy Markdown
Collaborator

@nikolasmatt nikolasmatt commented Mar 12, 2026

Description

Refactors the e2e test infrastructure to move all cluster/daemon setup out of Go test code and into Makefile targets, and adds a docker-only e2e mode that skips Kind entirely.

Motivation: The previous TestMain mixed infrastructure provisioning (Kind cluster creation, kagent install, daemon startup) with test orchestration. Separating concerns makes the test suite faster to iterate on locally and more debuggable in CI.

What changed:

  • e2e_test.go now contains only test setup — all infrastructure is provisioned by make test-e2e-k8s / make test-e2e-docker before the Go process starts
  • make setup-kind-cluster is fully self-contained and idempotent: creates the Kind cluster, installs MetalLB, PostgreSQL, agentregistry, and kagent (skips steps that already exist)
  • Linux/CI networking handled in setup-kind.sh: API server bound on 0.0.0.0, kubeconfig rewritten to the Docker bridge gateway (172.17.0.1), and 127.0.0.1 + 172.17.0.1 added to the API server cert SANs via kubeadmConfigPatches strategic merge
  • kind sourced via go tool kind (go.mod tool directive) — no separate install step
  • install-kagent.sh and install-helm-unittest.sh extracted as dedicated scripts; kagent CLI downloaded to bin/ and invoked directly rather than via $PATH
  • make local-registry manages a shared kind-registry container on port 5001 (used by both docker and k8s backends to avoid CI port conflicts)
  • make test-e2e dispatches to test-e2e-docker or test-e2e-k8s based on E2E_BACKEND; defaults to k8s
  • e2e CI workflow runs test-e2e-k8s and test-e2e-docker in parallel via matrix with fail-fast: false; cluster debug dump moved to make dump-kind-state
  • MCP publish tests clean up stale registry entries before and after each run to ensure idempotency
  • GOOGLE_API_KEY and OPENAI_API_KEY default to fake-key-for-setup in the Makefile — no test skips for missing keys
  • Pinned kagent version installed in cluster to v0.8.0-beta6 (latest) since the install script some times failed in CI due to Github API rate limits.

Change Type

/kind cleanup

Changelog

NONE

@nikolasmatt nikolasmatt marked this pull request as ready for review March 13, 2026 21:31
Copilot AI review requested due to automatic review settings March 13, 2026 21:31
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors the e2e test harness so that Kubernetes/Kind (and related infra like MetalLB, registry, kagent) is provisioned via Make targets/scripts before the Go test process starts, and adds a docker-only e2e mode that skips Kind/k8s-only tests—enabling faster local iteration and more reliable CI execution.

Changes:

  • Move e2e infra setup/teardown out of TestMain and into make test-e2e-k8s / make test-e2e-docker, with backend-based test skipping.
  • Rework Kind setup to support CI networking (API server bind/kubeconfig patch) and optional local registry wiring.
  • Update CI workflow to run docker and k8s e2e in parallel via a matrix, and add a Make target for dumping Kind state on failure.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
scripts/kind/setup-kind.sh Uses go tool kind, adds Linux networking/kubeconfig patching, and makes registry config optional via env vars.
scripts/kind/kind-config.yaml Switches to strategic merge patch to add API server cert SANs needed for CI networking.
scripts/kind/install-kagent.sh New script to download/install kagent CLI into bin/ and install kagent into the cluster.
scripts/install-helm-unittest.sh New standalone installer script for the helm-unittest plugin with retry logic.
e2e/mcp_publish_test.go Adds stale MCP entry cleanup to improve idempotency across interrupted runs.
e2e/helpers.go Adds IsK8sBackend() and getEnv() helpers for backend selection and env defaults.
e2e/e2e_test.go Simplifies TestMain to require pre-provisioned infra and gates k8s-only prereqs/logging by backend.
e2e/deploy_test.go Skips k8s deploy targets/tests when running with docker backend; improves test logging and idempotent cleanup.
e2e/agent_registry_test.go Adds stale MCP entry cleanup for idempotent test runs.
Makefile Introduces test-e2e-{k8s,docker}, run-{k8s,docker}, local-registry, dump-kind-state, and install-kagent; moves defaults for API keys.
.github/workflows/e2e.yaml Runs e2e against k8s and docker backends in parallel and calls dump-kind-state on k8s failures.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

@nikolasmatt nikolasmatt added this pull request to the merge queue Mar 16, 2026
Merged via the queue into agentregistry-dev:main with commit 5afd4b9 Mar 16, 2026
9 checks passed
@nikolasmatt nikolasmatt deleted the run-e2e-against-kind branch March 16, 2026 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants