Skip to content

Improve inference performance validation: tunable, per-accelerator params wired into recipes #1116

Description

@yuanchen8911

Summary

The inference-perf check defaulted to a tiny smoke-test model (Qwen/Qwen3-0.6B) with only global tuning knobs, so it couldn't characterize modern accelerators or adapt per GPU type. It is now configurable per recipe, backed by a model-weights cache, with per-accelerator thresholds re-measured on real hardware — so aicr validate --phase performance reflects GPU compute, not serving overhead, and works out of the box.

Implemented on branch feat/inference-perf-configurable-model (stacked on merged PR #1096); validated end to end on RTX PRO 6000, H100 (EKS + GKE), and GB200.

Main changes

1. Per-recipe configuration

inference-model and inference-concurrency-per-gpu are now read from the recipe's validation.performance.constraints — symmetric with the inference-throughput / inference-ttft-p99 thresholds that already live there. Each <accel>-<service>-…-inference-dynamo overlay pins its own values. Resolution precedence is recipe constraint > catalog env (AICR_INFERENCE_PERF_*) > compiled default. New defaults: Qwen/Qwen3-8B at 256 concurrency/GPU (the empirical throughput sweet spot across all tested accelerators).

2. Model-weights cache

A PVC-backed cache is on by default: a one-time populate Job downloads the model once, and all workers mount it read-only (HF_HUB_OFFLINE=1). This removes the per-worker Hugging Face download that tripped per-IP HTTP 429 throttling on large models, and needs no token for ungated models. Disable with AICR_INFERENCE_PERF_MODEL_CACHE_SIZE=off. The cache fails fast with actionable guidance when no StorageClass is resolvable (instead of leaving the PVC Pending until timeout); on EKS the aws-ebs-csi-driver component now provisions a default gp3 StorageClass so it works zero-config (GKE already has standard-rwo).

3. Updated / improved thresholds

The per-accelerator gates (previously placeholders tuned to 0.6B at conc 16: >= 5000 / <= 200) were re-measured at 8B / 256 and pinned alongside the model + concurrency in each overlay:

Accelerator (overlay) throughput gate TTFT p99 gate measured (8B/256)
RTX PRO 6000 (eks) >= 50000 <= 1000 59,260 tok/s · 445 ms · 100% util
H100 (eks + gke) >= 50000 <= 1000 ~108k tok/s · ~700 ms · 100% util
GB200 (eks) >= 50000 <= 2000 65,952 tok/s · 1,240 ms · 80% util
B200 (gke) >= 50000 <= 2000 untested — mirrors GB200 (flagged in-file)

The evaluator applies a 10% tolerance on top of these.

Remaining work

  1. Precision (FP8 / NVFP4, typically via the checkpoint ID) — not yet a knob.
  2. Tensor-parallel size (TP > 1) — the Dynamo deploy template still hardcodes TP=1 / one worker per GPU. Needed for models that exceed one GPU (e.g. 32B BF16 on an 80 GB H100) and to fully load a Blackwell GPU on GB200.

Motivation (from 8B characterization)

At the original 0.6B size the benchmark was serving-overhead-bound and never loaded the GPU. Defaulting to 8B at 256 concurrency/GPU fixes that on Hopper/Ada — H100 and RTX PRO 6000 reach ~100% GPU util at their throughput peak. GB200 reaches only ~80% on 8B (it is over-provisioned for a model this small), which is what the remaining precision + TP / larger-model work is for. TTFT SLO context: interactive ~200 ms, voice ~500 ms, MLPerf server 2 s (70B) – 6 s (405B).

Related

Metadata

Metadata

Assignees

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions