Prerequisites

• I searched existing issues

Feature Summary

Extend CI to validate deployer-specific bundle output for every supported --deployer value, not just the default (helm). Today, argocd and argocd-helm bundles are generated but never schema-validated or applied, so semantically broken Argo CD Application manifests can land on main undetected.

Problem / Use Case

aicr bundle supports three deployers (pkg/bundler/config/config.go:26-39):

• helm (default) — per-component Helm bundles + deploy.sh
• argocd — app-of-apps with one Application per component (pkg/bundler/deployer/argocd/templates/application.yaml.tmpl)
• argocd-helm — Helm chart whose templates are Argo CD Application manifests

Current CI coverage:

Gaps:

• No schema validation of Application CRDs (apiVersion, required spec.source, destination.server, project, etc.).
• No dry-run apply (kubectl apply --dry-run=server or kubeconform with the Argo CD CRD bundle).
• No end-to-end "install Argo CD, apply app-of-apps, observe Sync" path.
• A template regression in application.yaml.tmpl would only be caught after a user tries to deploy a release.

Concrete risks that would slip past today's CI:

• Invalid apiVersion / kind after an Argo CD CRD bump.
• Missing required fields (e.g., multi-source spec.sources[].repoURL).
• Broken Helm value references when argocd-helm is rendered through Argo CD (vs. raw helm template).
• Wave/finalizer annotations that pass yamllint but cause sync ordering bugs.

Proposed Solution

Add a CI gate that exercises every deployer. Three options, ordered cheapest → most thorough; recommend landing Option A first and treating B/C as follow-ups.

Option A — Static schema validation (fast, no cluster)

• Add a step that, for each deployer, runs aicr bundle --deployer <d> over the representative recipes already used by KWOK.
• Validate every emitted YAML with kubeconform against:
  • upstream Kubernetes schemas, and
  • Argo CD CRDs (argoproj.io/v1alpha1 Application, AppProject) loaded from a pinned argo-cd release.
• Extend tests/chainsaw/cli/bundle-variants/chainsaw-test.yaml to parse Application YAML and assert required spec.source{,s}, destination, project, sync-policy fields — not just string presence.

Pros: <30 s, no cluster, catches the bulk of regressions, easy to add to make qualify.
Cons: does not exercise actual Argo CD reconciliation (sources, repo creds, kustomize/helm render).

Option B — Render-through-Argo CD (medium)

• In KWOK or a Kind cluster, install Argo CD (pin a chart version in .settings.yaml), apply the generated app-of-apps with --dry-run=server, and assert each child Application reaches Synced=Unknown/Healthy=Unknown without ComparisonError / ConditionError.
• Skip actual workload sync (no GPU/operator install) — purpose is template correctness, not deploy success.

Pros: catches reference errors, Helm value plumbing, multi-source paths.
Cons: adds Argo CD install time (~30-60 s) and a new CI dependency.

Option C — Full deploy parity with KWOK (most thorough)

• Matrix the existing KWOK workflow over deployer ∈ {helm, argocd, argocd-helm}.
• For Argo CD variants, install Argo CD into the KWOK cluster, apply the bundle, and reuse the existing pod-Running assertions.

Pros: end-to-end parity, would have caught every historical bundle bug.
Cons: ~3× KWOK runtime; Argo CD reconciling against repos that don't exist in CI needs a local repo-server or targetRevision: HEAD against the test workspace.

Success Criteria

• make qualify (or a dedicated make validate-bundles target wired into CI) fails when any deployer emits a manifest that fails schema validation.
• Per-deployer coverage matrix is documented in docs/contributor/ and DEVELOPMENT.md.
• A deliberate regression in pkg/bundler/deployer/argocd/templates/application.yaml.tmpl (e.g., dropping spec.destination.server) causes a red CI run.
• At least Option A is implemented; Option B/C tracked as follow-up issues with explicit accept/defer decision.

Alternatives Considered

• Status quo + pre-release manual smoke test — relies on humans, doesn't scale, already failed to catch [hypothetical] regressions.
• Generate then argocd app validate against a live Argo CD — requires a hosted Argo CD; equivalent to Option B but with stronger dependencies.
• Schema-validate only argocd and skip argocd-helm — leaves the Helm-chart-of-Applications path uncovered; rejected.

Component

Bundlers (gpu-operator, network-operator, etc.)

Priority

Important (would improve my workflow)

Compatibility / Breaking Changes

None for end users. CI runtime increases (Option A: negligible; B: ~1 min; C: ~3× current KWOK time). Adds kubeconform and the Argo CD CRD bundle as pinned CI tools — should be wired through .settings.yaml to honor the "local development equals CI" rule.

Operational Considerations

• Pin Argo CD CRD source by release tag in .settings.yaml to keep "local = CI".
• New failure surface: schema validator false positives when Argo CD bumps CRDs — mitigated by version-pinning and a Renovate rule.
• No production blast radius; CI-only change.

Willing to contribute

Maybe, with guidance