Repository navigation
[Feature]: Add HAMi integration as an AICR recipe/component #481
Description
Prerequisites
- I searched existing issues
Feature Summary
HAMi is a CNCF sandbox project. It is a cloud-native GPU virtualization middleware that brings sharing, isolation and scheduling of heterogeneous accelerators to AI workloads on Kubernetes.
Support integrating HAMi as a “recipe” capability in this project, so users can enable GPU resource-isolation and topology-aware scheduling through AICR’s existing recipe/overlay/bundle workflow.
Problem/Use Case
HAMi provides a unified scheduler extender and device plugins for GPU sharing, and has been adopted by over 200+ companies, i think it would be great to offer an out-of-the-box recipe/component that installs and configures HAMi in a way consistent with AICR’s deployment model.
Otherwise, users must manage HAMi via manual Helm installation (and custom cluster configuration), which:
- increases setup effort and configuration drift risk,
- complicates upgrades and reproducibility across environments,
- makes it harder to combine HAMi with AICR-managed components (e.g., GPU operator, gang scheduling, inference/training overlays).
Proposed Solution
- Add a new AICR component and recipe for HAMi, installed via Helm including:
- HAMi scheduler extender components
- HAMi device plugin components
- any required admission webhook/mutating webhook components
- Register HAMi in
recipes/registry.yamlso it can be selected consistently by AICR recipes. - Add one or more overlays so HAMi can be enabled for relevant inference recipes, while keeping the default behavior unchanged.
- Add health checks/assertions (Chainsaw-based, consistent with other components) so
aicr validatecan confirm HAMi is installed and healthy.
Note HAMi will not be enabled by default in any recipe, it can only be installed by manually set
overrides.enabled=trueduring bundle.
Success Criteria
- A user can generate a recipe/bundle that includes HAMi and successfully installs it with correct namespace/RBAC/config when manually enable in during bundle.
aicr validate(or component health checks) can detect HAMi readiness/health.- In a cluster, workloads that rely on HAMi’s sharing/scheduling can be scheduled successfully, and AICR bundle output remains reproducible.
- Clear documentation describes how users enable HAMi and what cluster prerequisites are required.
Alternatives Considered
Manual Helm installation of HAMi outside AICR (current approach).
Component
Bundlers (gpu-operator, network-operator, etc.)
Priority
Important (would improve my workflow)
Compatibility / Breaking Changes
Conflict with 'KAI-scheduler'
Conflict with 'NVIDIA/k8s-device-plugin'
Conflict with 'NVIDIA/Mig-manager'
because of that, we won't enable hami by default in any existing overlays, so, either we set 'overlays.enabled to false' or we add our 'hami' overlays.
Operational Considerations
No response
Are you willing to contribute?
Yes, I can open a PR
Activity
github-actions commentedon Apr 2, 2026
Welcome to AICR, @archlitchi! Thanks for opening your first issue.
A maintainer will triage this shortly. In the meantime:
archlitchi commentedon Apr 2, 2026
@yuanchen8911 could you help me review that
yuanchen8911 commentedon Apr 7, 2026
Thanks for the detailed proposal, @archlitchi, and for offering to contribute. HAMi clearly addresses a real use case around GPU sharing and isolation.
After reviewing the request, the main issue is that HAMi is not an additive component in AICR's current model. AICR's base recipes already include gpu-operator, nvidia-dra-driver-gpu, and kai-scheduler, and HAMi conflicts with the device plugin / MIG ownership and scheduling semantics in that stack. This is not the same kind of integration as adding something like kgateway or kubeflow-trainer, which layer on top of the existing base.
Because of that, "disabled by default, enabled at bundle time" is not sufficient. Bundle-time
--setonly affects components already present in the resolved recipe; it does not add HAMi or remove conflicting base components after resolution. Supporting HAMi would require a distinct opt-in overlay branch or recipe family that explicitly replaces the conflicting defaults, plus HAMi-specific validation and health checks.That is a substantially larger scope than "register a new component," so I don't think the current issue is scoped correctly for the existing recipe tree. A better path would be one of:
If you want to pursue this further, the next useful step would be a concrete design for a HAMi-specific recipe branch, including which base components would be removed or replaced and how validation would work.
We can have a discussion offline if needed.
@mchmarny — flagging for your awareness given the architectural scope.
archlitchi commentedon Apr 8, 2026
Thanks for the review, i'll do some experiment on my AWS cluster, and later come up with a design so we could discuss on it.
11 remaining items
mchmarny commentedon Apr 22, 2026
Yes, that sounds less invasive. @yuanchen8911 thoughts?
yuanchen8911 commentedon Apr 30, 2026
Thanks @archlitchi. Before discussing where this fits in the recipe model, a few clarifications would help:
Scope of the swap. Can we assume the
*-hamioverlay keeps everything else from the parent recipe (gpu-operator,kai-scheduler,nfd,cert-manager, observability, etc.) and only replacesnvidia-dra-driver-gpuwith HAMi's DRA driver + webhook?DRA API compatibility. What DeviceClass/driver names does HAMi DRA expose for NVIDIA GPUs, and does the webhook preserve the existing API for resource request? Do we need HAMi-specific validators?
Hardware support. Which NVIDIA accelerators does HAMi's DRA path actually support?
Once those are clear, we can decide whether the right model is GPU allocation mixins, a criteria field, or a smaller experimental overlay set.
archlitchi commentedon Apr 30, 2026
@yuanchen8911 thanks for the comment, i'll try to answer that:
2.1 Yes, we uses 'hami-core-gpu.project-hami.io', we could add the HAMi-specific validators and workload templates, or we can add a configuration in HAMi-DRA chart to allow reuse gpu.nvidia.com
2.2 About resourceName 'nvidia.com/gpu'. yes, we can use that resourceName during pod allocation after install HAMi-DRA, we will translate them into a resourceClaim during HAMi-DRA-webhook, and erase the resourceName in container resources. so it won't be conflict with 'nvidia-device-plugin' in gpu-operator.
yuanchen8911 commentedon Apr 30, 2026
Thanks @archlitchi. A few follow-ups:
ResourceClaim/ResourceClaimTemplatewithdeviceClassName: gpu.nvidia.com, served bynvidia-dra-driver-gpu.The traditional
resources.limits: nvidia.com/gpu: 1path is a separate request style. In Kubernetes v1.36, KEP-5004 /DRAExtendedResourcemakes it possible for a DRA driver to back extended-resource requests natively.Can HAMi-DRA serve as a drop-in replacement for
nvidia-dra-driver-gpuon AICR's current path. Should we expect HAMi-specific DeviceClass names and workload templates?Scheduler requirements. Does HAMi-DRA require a HAMi-specific scheduler, extender, or scheduler config override? AICR uses the default kube-scheduler plus KAI Scheduler for gang scheduling; we want to confirm HAMi-DRA works with that unchanged.
MIG. Not a blocker for the initial integration. We would document the limitation though.
Proposed next step: start narrow with H100, and list the concrete AICR-side changes needed:
archlitchi commentedon Apr 30, 2026
Thanks @yuanchen8911
shivamerla commentedon May 6, 2026
@archlitchi I’m curious about the motivation for needing a drop-in replacement for the DRA Driver for NVIDIA GPUs in this case. Since
dra-driver-nvidia-gpuis being developed underkubernetes-sigs, my preference would be to continue standardizing around that implementation and close any feature gaps upstream rather than introducing a parallel/custom replacement.As Yuan mentioned, there are already standard DRA-based approaches for most of the functionality being discussed:
DRAExtendedResourcesfeatureDRAPartitionableDevicesDRADeviceTaintsDRAPartitionableDevicesAPI)DRAPartitionableDevicesAPI)It would help to better understand the specific user workflows or operational requirements that make users choose one implementation over the other, and why that would justify maintaining a separate standardized recipe in AICR instead of extending the upstream driver.
archlitchi commentedon May 15, 2026
@shivamerla
yes, that's a good question, many people ask me the same thing during kubeCon Europe, there are three reasons behind this.
you can find more details on this repo: https://github.com/Project-HAMi/HAMi-DRA
mchmarny commentedon Jun 13, 2026
Thanks for the detailed back-and-forth here, and for narrowing this to the HAMi-DRA approach.
After weighing it, we've decided this is out of scope for the supported AICR recipe tree, so I'm going to close it out.
HAMi-DRA is a parallel implementation of the GPU DRA driver, and our direction is to standardize on the upstream kubernetes-sigs
nvidia-dra-driver-gpuand close any sharing/isolation gaps there (DRAExtendedResources, DRAPartitionableDevices for MIG/MPS) rather than ship and validate a second, differently-designed driver. The HAMi-core design — a CUDA/NVML interception layer injected into each container — is also not an approach we want to build a first-class, supported recipe around. We reached the same conclusion on the OpenShell side for the same reasons (NVIDIA/OpenShell#1065).This doesn't block HAMi users: the manual Helm path still works, and a community-maintained recipe set remains the most realistic home for it, free of the core tree's validation and release constraints. If upstream DRA leaves a concrete gap that only HAMi's approach fills, that's best raised on the
nvidia-dra-driver-gpuproject so it can be addressed in the standard path.Appreciate the engagement and the cross-project discussion.