Summary

helm-diff renders every release against the live cluster's REST mapper before any release is installed. On a fresh cluster this fails for charts whose templates reference a CRD installed by a different chart in the same bundle (e.g., nodewright-customizations uses kind: Skyhook, registered by nodewright-operator). helmfile's needs: directive sequences installs but not the diff pre-pass, so the document aborts before any mutation with no matches for kind "Skyhook" in version "skyhook.nvidia.com/v1alpha1".

This PR implements option C1 from the issue: when any referenced component is registry-marked installsCRDs, the helmfile deployer emits a multi-helmfiles: document so CRDs land before consumers render.

Fixes: #914
Related: N/A

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• Documentation update (demo workaround removed; generated README explains the layout)

Component(s) Affected

• Bundlers (pkg/bundler, pkg/component/*)
• Recipe engine / data (pkg/recipe)
• Docs/examples (docs/, examples/)

Implementation Notes

Layout when split is active:

bundle/
  helmfile.yaml          # helmfiles: [crds.yaml, releases.yaml]
  crds.yaml              # CRD-owner releases, with their own repositories + needs chain
  releases.yaml          # every other release, separate repositories + needs chain
  001-cert-manager/      # per-component folders unchanged
  002-…/

helmfile processes the helmfiles: list sequentially, so by the time releases.yaml renders, every CRD installed by crds.yaml is registered with the cluster's REST mapper. The cross-sub-helmfile needs: edge (e.g., gpu-operator → cert-manager) dissolves because sub-helmfile sequencing already enforces ordering.

Decision tree in Generator.writeHelmfileLayout:

• No CRD-owner components → single helmfile.yaml (legacy path, byte-identical for non-mixed bundles).
• Every component is a CRD-owner → single helmfile.yaml (split would add no value).
• Mixed → emit the three-file split.

Registry marks (installsCRDs: true) added to:

• cert-manager — Certificate / Issuer / ClusterIssuer / Order / Challenge / CertificateRequest.
• nodewright-operator — Skyhook (the originating failure).
• kube-prometheus-stack — ServiceMonitor / PodMonitor / Alertmanager / Prometheus / etc.
• nvsentinel — HealthCheck / NodeHealth.
• agentgateway-crds — CRDs-only chart.
• slinky-slurm-operator-crds — CRDs-only chart.

Other components (gpu-operator, network-operator, nvidia-dra-driver-gpu, kueue, kubeflow-trainer, etc.) also install CRDs but are not consumed by other charts in the bundle — leaving them unmarked keeps the main releases.yaml chain longer and the split surface smaller. Easy to opt in if a future consumer surfaces the same bug.

Tests added:

• pkg/bundler/deployer/helmfile/crd_split_test.go — TestSplitFoldersByCRD (partition shape, pre/post inheritance), TestSplitFoldersByCRD_NilCRDSet, TestGenerate_SplitLayout (end-to-end three-file layout + needs dissolution), TestGenerate_AllCRD_CollapsesToSingleFile, TestGenerate_NoCRD_KeepsSingleFile, TestRegistry_CRDOwnersStillMarked (sentinel guard against rename/unmark).
• pkg/bundler/deployer/helmfile/helmfile_test.go — assertHelmfileShape refactored into split-aware top-level walker + leaf checker; upstream_helm_only scenario flipped to the split layout with new crds.yaml + releases.yaml goldens.
• tests/chainsaw/cli/bundle-helmfile/chainsaw-test.yaml — every step that inspected helmfile.yaml's leaf shape now resolves a LEAVES list (one entry in the single-file case, two in the split case) and iterates.

Generated README gains: helm-diff plugin prerequisite (helmfile invokes it under the hood for apply / diff) and a paragraph explaining the helmfiles: layout when HasCRDLayer is true.

Testing

make qualify

All 21 chainsaw tests + go test -race + golangci-lint + yamllint pass. Coverage on pkg/bundler/deployer/helmfile unchanged.

Manual end-to-end via chainsaw test ./tests/chainsaw/cli/bundle-helmfile --no-cluster against an eks-h100-ubuntu-training recipe (which triggers the split because cert-manager + kube-prometheus-stack + nodewright-operator + nvsentinel are all marked): top-level helmfile.yaml carries helmfiles: [crds.yaml, releases.yaml], both sub-files parse, helmfile build resolves the cross-file release graph cleanly.

Risk Assessment

• Low — Additive on bundles without CRD-marked components (single-file layout preserved byte-for-byte). For bundles that DO have marked components, the previous output was broken on fresh clusters; the new output is the documented helmfile pattern. Easy to revert via git revert.

Rollout notes: No CLI flag, no API surface change. Operators of existing bundles will see the split layout the next time they regenerate; helmfile apply continues to work without flags. The README in the generated bundle explains the new layout.

Checklist

• Tests pass locally (make test with -race)
• Linter passes (make lint)
• I did not skip/disable tests to make CI green
• I added/updated tests for new functionality
• I updated docs if user-facing behavior changed
• Changes follow existing patterns in the codebase
• Commits are cryptographically signed (git commit -S)