Summary
Close the subset of performance-phase floor gaps that don't require testbed access:
recipes/overlays/gb200-eks-ubuntu-inference-dynamo.yaml — add an inference-perf block using the same placeholder smoke thresholds as h100-eks-ubuntu-inference-dynamo.yaml:65-79. GB200 outperforms H100, so the H100 floors (inference-throughput >= 5000, inference-ttft-p99 <= 200) are trivially satisfied — they assert "the system runs," not "GB200 is fast enough." Comment that the values are placeholders pending GB200-tuned numbers.
recipes/overlays/h100-eks-ubuntu-inference-nim.yaml — add the same placeholder inference-perf block (NIM uses the same check name and threshold set).
pkg/recipe/validation_phase_floor_test.go::requiresPerformance — return false when accelerator is unset. This exempts accelerator-unbound intermediates (aks-training, eks-training, gke-cos-training, oke-training) from the strict-mode perf floor because the threshold is accelerator-specific and only meaningful on concrete leaves.
Motivation / Context
AICR_VALIDATION_FLOOR_STRICT=1 go test ./pkg/recipe/... -run TestOverlayValidationPhaseFloor flags 13 overlays today. PR #1001 closes the deployment-phase gap (separate axis). This issue closes 6 of the 13 remaining perf warnings without needing testbeds:
| Closed by |
Overlay |
| Perf block added |
gb200-eks-ubuntu-inference-dynamo, h100-eks-ubuntu-inference-nim |
| Floor-test refinement |
aks-training, eks-training, gke-cos-training, oke-training |
The remaining 7 are testbed-blocked AKS / OCI / LKE leaves, tracked in dedicated issues.
Design notes
- The two Dynamo / NIM placeholder blocks follow the precedent set by
h100-eks-ubuntu-inference-dynamo.yaml — loose smoke-test floors with a comment explaining they're not tight perf gates. Tightening to empirical thresholds is a separate per-accelerator workload-tuning effort.
- The
requiresPerformance refinement is a 3-line rule change: if c.Accelerator == "" || c.Accelerator == "any" { return false }. The rationale: an intent-level intermediate without accelerator binding has no meaningful perf threshold; concrete leaves (h100-eks-training ≥ 300 GB/s, gb200-eks-training ≥ 40 NET / 500 NVLS GB/s) carry their own threshold via per-phase replace. The floor test was over-broad for these intermediates.
Done when
go test ./pkg/recipe/... -run TestOverlayValidationPhaseFloor -v continues to pass in default mode.
AICR_VALIDATION_FLOOR_STRICT=1 go test ./pkg/recipe/... -run TestOverlayValidationPhaseFloor -v failures drop from 13 to 7 (testbed-blocked only).
- The 6 newly-closed cases never reappear in either mode.
Related
Summary
Close the subset of performance-phase floor gaps that don't require testbed access:
recipes/overlays/gb200-eks-ubuntu-inference-dynamo.yaml— add aninference-perfblock using the same placeholder smoke thresholds ash100-eks-ubuntu-inference-dynamo.yaml:65-79. GB200 outperforms H100, so the H100 floors (inference-throughput >= 5000,inference-ttft-p99 <= 200) are trivially satisfied — they assert "the system runs," not "GB200 is fast enough." Comment that the values are placeholders pending GB200-tuned numbers.recipes/overlays/h100-eks-ubuntu-inference-nim.yaml— add the same placeholderinference-perfblock (NIM uses the same check name and threshold set).pkg/recipe/validation_phase_floor_test.go::requiresPerformance— return false whenacceleratoris unset. This exempts accelerator-unbound intermediates (aks-training,eks-training,gke-cos-training,oke-training) from the strict-mode perf floor because the threshold is accelerator-specific and only meaningful on concrete leaves.Motivation / Context
AICR_VALIDATION_FLOOR_STRICT=1 go test ./pkg/recipe/... -run TestOverlayValidationPhaseFloorflags 13 overlays today. PR #1001 closes the deployment-phase gap (separate axis). This issue closes 6 of the 13 remaining perf warnings without needing testbeds:gb200-eks-ubuntu-inference-dynamo,h100-eks-ubuntu-inference-nimaks-training,eks-training,gke-cos-training,oke-trainingThe remaining 7 are testbed-blocked AKS / OCI / LKE leaves, tracked in dedicated issues.
Design notes
h100-eks-ubuntu-inference-dynamo.yaml— loose smoke-test floors with a comment explaining they're not tight perf gates. Tightening to empirical thresholds is a separate per-accelerator workload-tuning effort.requiresPerformancerefinement is a 3-line rule change:if c.Accelerator == "" || c.Accelerator == "any" { return false }. The rationale: an intent-level intermediate without accelerator binding has no meaningful perf threshold; concrete leaves (h100-eks-training≥ 300 GB/s,gb200-eks-training≥ 40 NET / 500 NVLS GB/s) carry their own threshold via per-phase replace. The floor test was over-broad for these intermediates.Done when
go test ./pkg/recipe/... -run TestOverlayValidationPhaseFloor -vcontinues to pass in default mode.AICR_VALIDATION_FLOOR_STRICT=1 go test ./pkg/recipe/... -run TestOverlayValidationPhaseFloor -vfailures drop from 13 to 7 (testbed-blocked only).Related