Summary

Based on CNCF AI Conformance WG feedback, the conformance submission should certify a commercial product on a specific K8s distribution, not standalone tooling. We will convert the existing AICR conformance PR to certify NIM on EKS.

Background

Discussion in k8s-ai-conformance Slack:

• CNCF conformance requires submitting a product per K8s offering (e.g., NIM on EKS, NIM on AKS)
• AICR serves as the validation tooling (analogous to sonobuoy for K8s conformance), not the certified product
• The certified product is NVIDIA NIM deployed on EKS

Proposal

Add a new AICR recipe that deploys NIM on EKS using the k8s-nim-operator:

1. New components in recipes/registry.yaml:

   • k8s-nim-operator — Helm chart from helm.ngc.nvidia.com/nvidia/helm-charts/k8s-nim-operator
   • Deploys the NIM Operator which manages NIM lifecycle (autoscaling, health monitoring, model caching)

2. New recipe overlay: h100-eks-ubuntu-inference-nim.yaml

   • Inherits from h100-eks-ubuntu-inference
   • Adds k8s-nim-operator component
   • Includes a NIMService CR to deploy a small model (e.g., Llama 3.2 1B) for conformance validation
   • Full conformance validation suite (all 11 checks)

3. Conformance PR conversion:

   • Update the existing CNCF conformance PR to use the NIM recipe
   • Evidence collected against NIM on EKS deployment
   • Follow-up submissions for GKE, AKS, OKE, etc.

Why NIM Operator (not raw NIM Helm chart)

• Production deployment method with full lifecycle management
• Natively supports autoscaling and health checks required by conformance
• NIMService CR demonstrates the product experience, not just a container
• Stronger certification narrative: "NIM Operator on EKS"

Tasks

• Add k8s-nim-operator component to recipes/registry.yaml
• Create component values at recipes/components/k8s-nim-operator/values.yaml
• Create NIMService CR manifest for conformance test workload
• Create h100-eks-ubuntu-inference-nim.yaml recipe overlay
• Validate conformance checks pass against NIM deployment on EKS
• Convert CNCF conformance PR to use NIM on EKS