Skip to content

docs: replace vLLM metric names that current vLLM no longer exports - #2835

Merged
Gregory-Pereira merged 1 commit into
llm-d:mainfrom
fjvbn2003:docs-current-vllm-metric-names
Oct 16, 2026
Merged

Gregory-Pereira merged 1 commit into
llm-d:mainfrom
fjvbn2003:docs-current-vllm-metric-names

Conversation

@fjvbn2003

Copy link
Copy Markdown
Contributor

Three docs point readers at vLLM metrics that a current vLLM server does not expose, so the suggested queries return no data.

File Old name Replacement Why
guides/turn-priority-fairness/README.md (Observability table) vllm:gpu_prefix_cache_hit_rate rate(vllm:prefix_cache_hits_total[5m]) / rate(vllm:prefix_cache_queries_total[5m]) That gauge only existed in the old V0 engine. The V1 engine, including the vLLM 0.26 this guide was validated on, exports only the prefix_cache_hits / prefix_cache_queries counters.
docs/architecture/autoscaling/wva.md (SLO analyzer metrics, avg_itl row) vllm:time_per_output_token_seconds vllm:inter_token_latency_seconds The WVA collector queries vllm:inter_token_latency_seconds_{sum,count} (internal/collector/registration/queueing_model.go at v0.9.0). The doc did not match the code.
docs/infrastructure/providers/digitalocean/monitoring/inference-gateway-dashboard.json ("Time Per Output Token Latency" panel, 3 queries) vllm:time_per_output_token_seconds_bucket vllm:inter_token_latency_seconds_bucket vLLM renamed this histogram in 0.11. It hid the old name in 0.12 and removed it in 0.15 (vllm/v1/metrics/loggers.py).

How I checked: I ran vLLM v0.31.0 (vllm/vllm-openai, the current release) on a single 8 GB consumer GPU with a tiny dummy-weight model, sent 5 completions that share a 160-token prefix, and scraped /metrics:

$ curl -s localhost:8000/metrics | grep -cE '^vllm:(gpu_prefix_cache_hit_rate|time_per_output_token_seconds)'
0
$ curl -s localhost:8000/metrics | grep -E '^vllm:(prefix_cache_(hits|queries)_total|inter_token_latency_seconds_count)'
vllm:prefix_cache_queries_total{engine="0",model_name="tiny"} 805.0
vllm:prefix_cache_hits_total{engine="0",model_name="tiny"} 640.0
vllm:inter_token_latency_seconds_count{engine="0",model_name="tiny"} 75.0

The dashboard JSON still parses. I did not load it into Grafana; only the metric name inside the three histogram_quantile expressions changed. The panel title is unchanged.

This change was drafted with AI assistance and reviewed by me.

🤖 Generated with Claude Code

Three places point readers at vLLM metrics that a current vLLM server
does not expose, so the queries return no data:

- guides/turn-priority-fairness: vllm:gpu_prefix_cache_hit_rate was a
  V0-engine gauge. The V1 engine (and the vLLM 0.26 this guide was
  validated on) only exports the vllm:prefix_cache_hits_total and
  vllm:prefix_cache_queries_total counters, so show the hit ratio as a
  rate of the two.
- docs/architecture/autoscaling/wva.md: the avg_itl rows name
  vllm:time_per_output_token_seconds, but the WVA collector (v0.9.0,
  internal/collector/registration/queueing_model.go) queries
  vllm:inter_token_latency_seconds. vLLM renamed the histogram in 0.11
  and removed the old name in 0.15.
- the DigitalOcean monitoring dashboard: the "Time Per Output Token
  Latency" panel queries vllm:time_per_output_token_seconds_bucket;
  switch it to vllm:inter_token_latency_seconds_bucket.

Signed-off-by: YoungjuKim <[email protected]>
@github-actions

Copy link
Copy Markdown
Contributor

llm-d.ai preview

The site was built for 3d1587e. Download the site-preview artifact from the CI run, unzip it into build/ and run npx serve build, then open http://localhost:3000/docs/dev.

@Gregory-Pereira Gregory-Pereira left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@Gregory-Pereira
Gregory-Pereira merged commit 2b0b8de into llm-d:main Oct 16, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants