Skip to content

[Epic]: Topology discovery and network operator tuning using k8s-launch-kit profiles #827

Description

@mchmarny

Summary

Integrate NVIDIA/k8s-launch-kit (l8k) into AICR so the snapshot/recipe pipeline can automatically discover GPU/NIC fabric topology and emit optimized network-operator configuration without hand-tuning.

Motivation

Today AICR ships network-operator as a static Helm component (recipes/registry.yaml:108-124, recipes/components/network-operator/values.yaml). Users must hand-author NicClusterPolicy / SriovNetworkNodePolicy / IPPool for their fabric, deployment type, and rail configuration. l8k already encodes that knowledge as profiles (sriov, rdma_shared, host_device × infiniband/ethernet, plus spectrum-x) and ships per-SKU presets (PowerEdge XE9680, ThinkSystem SR680a-V3, UCSC-885A-M8-H22).

By integrating l8k as a library, AICR can:

  • Discover NIC/fabric/rail topology and persist it in the snapshot.
  • Pick the right l8k profile from user intent + cluster capabilities.
  • Generate network-operator deployment artifacts as part of an AICR bundle.
  • Build a curated topology preset library shared by both projects.

Approach: two phases, decoupled

Phase 2 can ship before Phase 1 — recipe-side integration does not require live discovery and benefits users immediately via presets.

Phase 1 — Snapshot integration (l8k as Collector) — #828

Implement AICR's Collector interface (pkg/collector/types.go:27) on top of l8k's discovery API; expose l8k as a Go library; persist topology in the snapshot.

Phase 2 — Recipe integration (l8k as Component generator) — #829

Inject l8k profile templates into AICR's bundler so aicr bundle produces a network-operator chart bundle plus the right CR manifests for the discovered/declared deployment type.

Upstream dependency

Phase 1 (Option C) depends on l8k exposing a read-only discovery API: NVIDIA/k8s-launch-kit#62.

Constraints discovered during scoping

  1. l8k discovery is mutating, not read-only. pkg/networkoperatorplugin/discovery.go:99-260 creates/patches NicClusterPolicy, waits for nic-configuration-daemon, and execs into pods. Requires network-operator already installed. AICR collectors today are read-only and don't take pod-exec RBAC. Phase 1 will need either upstream read-only mode (preferred) or an AICR policy decision to allow mutating collectors.
  2. AICR has no Go Component interface. Components are declarative (registry.yaml + values.yaml). Bundler is template-driven (pkg/bundler/bundler.go:470-518); no hook for snapshot-driven dynamic value computation. Phase 2 must design this hook.
  3. Output shape mismatch. l8k's GenerateProfileDeploymentFiles returns rendered CR manifests, not Helm chart values.yaml. Phase 2 must reconcile (hybrid bundle is the leading option).
  4. Snapshot is not passed to the bundler today — only the resolved RecipeResult. Phase 2 must thread it through.

Acceptance criteria

  • Integrate k8s-launch-kit as a snapshot Collector #828 closed: snapshot pipeline can persist NIC/fabric/rail topology when l8k is available; degrades cleanly when not.
  • Integrate k8s-launch-kit profiles as a recipe Component generator #829 closed: aicr bundle produces a network-operator bundle whose deployment artifacts are derived from recipe inputs (fabric, deployment type) and, when present, snapshot topology.
  • l8k consumed strictly as a Go library — no shelling out to l8k or kubectl, no in-cluster l8k deployment.
  • Documentation updated: component catalog, recipe-development guide, snapshot/collector reference.
  • KWOK e2e covers the (fabric × deployment-type) matrix.

Out of scope

  • Replacing the existing network-operator Helm component entry. Phase 2 augments it; it does not replace it.
  • Embedding l8k's CLI inside AICR.
  • Discovery of non-NVIDIA NICs.

References

  • l8k plugin contract: pkg/plugin/plugin.go
  • l8k discovery flow: pkg/networkoperatorplugin/discovery.go:99-260
  • l8k profiles: profiles/<name>/profile.yaml
  • l8k presets: pkg/presets/, presets/
  • AICR collector contract: pkg/collector/types.go:27, pkg/collector/factory.go:30
  • AICR bundler value flow: pkg/bundler/bundler.go:470-518
  • AICR registry: recipes/registry.yaml

Metadata

Metadata

Assignees

Type

Fields

No fields configured for Epic.

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions