cc @terrytangyuan @janetkuo @puja108 @ritazh

cc @dims @mchmarny

@jeefy , can you take a look when you get a chance? Thanks.

xpost from a slack thread for future posterity.
https://kubernetes.slack.com/archives/C09813W8DC2/p1774920621774989?thread_ts=1774363606.075179&cid=C09813W8DC2

Thank you for submitting this PR @yuanchen8911
IMO AICR is valuable as an accelerator for implementing and validating the k8s AI conformance KARs, and I’d like to see it being a part of the k8s AI conformance program.

That said, I’m concerned about merging this PR as-is / right now because it risks effectively introducing “use AICR” as an implied requirement (even if unintentionally). Before we do anything that looks like a requirement shift, I think we need the program/vendor alignment and some bake time like we have done with all existing requirements and processes:

1. New requirement concern: If the submission implies that “AICR is the way to become AI-conformant,” that’s a new dependency/tooling requirement. We should discuss this with all vendors to confirm it’s sufficiently common/neutral and doesn’t privilege one implementation path over others.
2. Back-compat / impact on existing conformant products: Once there’s agreement in principle, we need to verify that all existing (and in-flight) conforming platforms can still meet the requirements without being forced to adopt AICR, or that we intentionally treat AICR as optional/reference.
3. Process / bake time: Similar to how we graduate requirements from SHOULD → MUST, anything that functions like a new “MUST run tool X” needs advance notice, feedback collection, and time before it’s treated as baseline. More details: https://github.com/kubernetes-sigs/ai-conformance/blob/main/README.md#process-details

On your proposed next steps:

(1) Platform certification (AICR-enabled EKS/GKE/AKS/etc.): Assuming we get enough support for this new requirement as part of the program, then this is a good first step that these vendors can take.
(2) Tooling for the program: This feels like the clearest win. Let’s take this to the subproject community meetings and determine if AICR should be a new requirement or a “reference implementation/recipes/validation tooling".

xpost from a slack thread for future posterity. https://kubernetes.slack.com/archives/C09813W8DC2/p1774920621774989?thread_ts=1774363606.075179&cid=C09813W8DC2

Thank you for submitting this PR @yuanchen8911 IMO AICR is valuable as an accelerator for implementing and validating the k8s AI conformance KARs, and I’d like to see it being a part of the k8s AI conformance program.

That said, I’m concerned about merging this PR as-is / right now because it risks effectively introducing “use AICR” as an implied requirement (even if unintentionally). Before we do anything that looks like a requirement shift, I think we need the program/vendor alignment and some bake time like we have done with all existing requirements and processes:

1. New requirement concern: If the submission implies that “AICR is the way to become AI-conformant,” that’s a new dependency/tooling requirement. We should discuss this with all vendors to confirm it’s sufficiently common/neutral and doesn’t privilege one implementation path over others.
2. Back-compat / impact on existing conformant products: Once there’s agreement in principle, we need to verify that all existing (and in-flight) conforming platforms can still meet the requirements without being forced to adopt AICR, or that we intentionally treat AICR as optional/reference.
3. Process / bake time: Similar to how we graduate requirements from SHOULD → MUST, anything that functions like a new “MUST run tool X” needs advance notice, feedback collection, and time before it’s treated as baseline. More details: https://github.com/kubernetes-sigs/ai-conformance/blob/main/README.md#process-details

On your proposed next steps:

(1) Platform certification (AICR-enabled EKS/GKE/AKS/etc.): Assuming we get enough support for this new requirement as part of the program, then this is a good first step that these vendors can take. (2) Tooling for the program: This feels like the clearest win. Let’s take this to the subproject community meetings and determine if AICR should be a new requirement or a “reference implementation/recipes/validation tooling".

Thanks, @ritazh!

Re 1. What we're proposing to certify is a conformant K8s platform + AICR runtime (recipe). No K8s platform today is AI-conformant on its own, not EKS, GKE, AKS, OKE, or OpenShift. It's always the platform plus additional runtime components (GPU operators, device plugins, NCCL, gang scheduling, etc.) that make it AI-conformant. So what the program actually certifies is a K8s platform plus runtime components, a configured product built on top of a conformant base.

Every existing submission follows this pattern. OpenShift bundles the GPU operator stack, Gardener deploys additional components. They just ship it as one branded product. AICR makes this explicit rather than hiding it behind a single product name.

From this perspective, "conformant K8s platform + AICR runtime" is a valid platform to certify. 

To clarify, the platform certification isn't proposing AICR as a requirement for others. It's NVIDIA certifying its own configured platform, the same way any other vendor submits their product. No existing conformant platform would need to adopt AICR, and it doesn't change the requirements for anyone else.

We're happy to take the tooling contribution to the subproject community meetings in parallel. But that will requires some substantial discussion and work while the platform submission itself should be straightforward. It's just another vendor certification, not a new requirement or process change.

Would it help if we revised the PR t make it a specific base K8s distro (e.g., starting with EKS) + AICR runtime) as a named platform certification to make this distinction more explicit?

Re 2. We'll present and discuss the AICR validating tools and plan with the WG in a meeting.

The slack discussion thread: https://kubernetes.slack.com/archives/C09813W8DC2/p1774907536181959?thread_ts=1774363606.075179&cid=C09813W8DC2

@ritazh, I think we're aligning on the plan. Regarding priority and ordering of certification vs. validation tooling, we're happy to take the tooling contribution to the community meetings in parallel. But that will require substantial discussion and work TBH, while the platform submission is straightforward. It's just another vendor certification, not a new requirement or process change. Please let us know if we can proceed with reworking the PR to certify a specific K8s distro + AICR runtime components (deployment recipes)? Thanks.

No K8s platform today is AI-conformant on its own, not EKS, GKE, AKS, OKE, or OpenShift. It's always the platform plus additional runtime components (GPU operators, device plugins, NCCL, gang scheduling, etc.) that make it AI-conformant.

The AI Conformance program does not certify "on its own" outcomes. The pattern is (abstractly):

1. Demonstrate that the standard platform features are used to create a cluster, e.g., gcloud container clusters create
2. Demonstrate that the standard platform configuration is used to apply the desired platform configuration, e.g., gcloud container clusters update
3. Demonstrate that the standard Kubernetes API is used to apply the desired Kubernetes configuration, e.g., kubectl apply -f

To use a concrete example, here's the gang scheduling requirement, which describes requirements that accommodate the flexible AI/ML k8s landscape (ref: kubernetes-sigs/ai-conformance#54):

The platform must allow for the installation and successful operation of at least one gang scheduling solution that ensures all-or-nothing scheduling for distributed AI workloads (e.g. Kueue, Volcano, etc.) To be conformant, the vendor must demonstrate that their platform can successfully run at least one such solution.

This pattern of platform + apply specific, standard, and repeatable configuration is exactly why a tool like AICR can be a helpful accelerant for the AI Conformant platform ecosystem. :)

To clarify, the platform certification isn't proposing AICR as a requirement for others. It's NVIDIA certifying its own configured platform, the same way any other vendor submits their product.

Because AICR is not a Kubernetes cluster distro or platform (ref: https://github.com/NVIDIA/aicr/?tab=readme-ov-file#what-ai-cluster-runtime-is-not), it is not something to certify as AI Conformant per se. Similarly, we would not want to certify argo/flux/Cluster API as AI Conformant per se (even though all of these tools can be used for provisioning AI Conformant clusters).

I'm very excited that the launch of AICR coincides w/ early days of AI Conformance, as there's great opportunity to collaborate and standardize together!

Per @ritazh's feedback, the CNCF AI Conformance WG requires certifying a product on a K8s platform, not standalone tooling/software. It sounds like an NVIDIA product/service such as NIM on EKS is the right framing. AICR serves as the deployment and validation tooling while NIM is the certified product.

Converting this PR to draft while we rework the submission accordingly.

Hi @ritazh @jackfrancis @jeefy @terrytangyuan @janetkuo @puja108 @dims @mchmarny,

Based on our discussion in the Slack thread, we've reworked this PR to certify NVIDIA NIM on EKS as the product, rather than AICR (which is now positioned as deployment/validation tooling only).

Key changes:

• Product: NVIDIA NIM on EKS — an AI inference platform managed by the NIM Operator on Amazon EKS
• AICR's role: Deployment and validation tooling (analogous to sonobuoy for K8s conformance), not the certified product
• Evidence: All 9 MUST requirements pass, collected on EKS v1.35

This PR is ready for review. One open question in the description: k8sConformanceUrl references the underlying EKS conformance entry since NIM is a layered product, not a K8s distribution. Happy to adjust if a different approach is preferred.

Follow-up submissions for NIM on GKE/AKS/OKE are planned.

Thanks @jeefy for the clarification — that makes sense. Since NIM sits on top of EKS (which has its own K8s conformance), referencing EKS's conformance results via k8sConformanceUrl is the right pattern here.

Also fixed the broken virtualized_accelerator evidence URL that was causing the validate check to fail (NVIDIA restructured their docs). CI should be green now.

The documentation request from @dims and @puja108 has been addressed by the now-merged PR #89 (FAQ on certifying products on existing distributions).

This submission already follows that guidance — k8sConformanceUrl in v1.35/nvidia-nim-eks/PRODUCT.yaml references the underlying EKS conformance entry, consistent with the layered-product pattern described in the FAQ.

Rebased this branch onto main so the merged FAQ is included.

Thank you for your updates @yuanchen8911 !

I would like to request a hold on this PR until we can provide better guidance on the requirements for layered products. I've opened #91 to clarify what a product built on an existing conformant distribution must demonstrate independently.

Since this may affect how the community interprets eligibility and certification scope, it would benefit from broader discussion. If you have input, please comment on #91 or join the next AI Conformance project meetings

/hold

Thank you for your updates @yuanchen8911 !

I would like to request a hold on this PR until we can provide better guidance on the requirements for layered products. I've opened #91 to clarify what a product built on an existing conformant distribution must demonstrate independently.

Since this may affect how the community interprets eligibility and certification scope, it would benefit from broader discussion. If you have input, please comment on #91 or join the next AI Conformance project meetings

/hold

Thanks @ritazh! It will be helpful to align on guidance for layered products. Happy to adjust this submission as needed once the requirements are clarified. Please keep us updated. cc @dims

@jeefy perhaps you missed the /hold comment in #79 (comment) we wanted to hold off on merging this PR until we can provide better guidance on the requirements for layered products. #91 was intended to be a WIP PR and waiting on more community input.

Note: this PR along with #91 are unlike typical submissions in 1.35/$distro to be merged by CNCF staff. These are changes to the requirements, guidance, and process to the k8s AI conformance program and therefore, should be discussed in the community and should be approved by the leads.

I did miss it because there wasn't a hold label on it. 🙃 I'll revert.

Since this was accidentally merged, add comments and feedback to #95 to clarify what a layered product built on an existing conformant distribution must demonstrate independently.

Since this may affect how the community interprets eligibility and certification scope, it would benefit from broader discussion. If you have input, please comment on #95 or join the next AI Conformance project meetings