Repository navigation
docs: replace vLLM metric names that current vLLM no longer exports - #2835
Merged
Gregory-Pereira merged 1 commit intoOct 16, 2026
Merged
Conversation
Three places point readers at vLLM metrics that a current vLLM server does not expose, so the queries return no data: - guides/turn-priority-fairness: vllm:gpu_prefix_cache_hit_rate was a V0-engine gauge. The V1 engine (and the vLLM 0.26 this guide was validated on) only exports the vllm:prefix_cache_hits_total and vllm:prefix_cache_queries_total counters, so show the hit ratio as a rate of the two. - docs/architecture/autoscaling/wva.md: the avg_itl rows name vllm:time_per_output_token_seconds, but the WVA collector (v0.9.0, internal/collector/registration/queueing_model.go) queries vllm:inter_token_latency_seconds. vLLM renamed the histogram in 0.11 and removed the old name in 0.15. - the DigitalOcean monitoring dashboard: the "Time Per Output Token Latency" panel queries vllm:time_per_output_token_seconds_bucket; switch it to vllm:inter_token_latency_seconds_bucket. Signed-off-by: YoungjuKim <[email protected]>
ahg-g
approved these changes
Oct 16, 2026
Contributor
llm-d.ai previewThe site was built for 3d1587e. Download the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three docs point readers at vLLM metrics that a current vLLM server does not expose, so the suggested queries return no data.
guides/turn-priority-fairness/README.md(Observability table)vllm:gpu_prefix_cache_hit_raterate(vllm:prefix_cache_hits_total[5m]) / rate(vllm:prefix_cache_queries_total[5m])prefix_cache_hits/prefix_cache_queriescounters.docs/architecture/autoscaling/wva.md(SLO analyzer metrics,avg_itlrow)vllm:time_per_output_token_secondsvllm:inter_token_latency_secondsvllm:inter_token_latency_seconds_{sum,count}(internal/collector/registration/queueing_model.goat v0.9.0). The doc did not match the code.docs/infrastructure/providers/digitalocean/monitoring/inference-gateway-dashboard.json("Time Per Output Token Latency" panel, 3 queries)vllm:time_per_output_token_seconds_bucketvllm:inter_token_latency_seconds_bucketvllm/v1/metrics/loggers.py).How I checked: I ran vLLM v0.31.0 (
vllm/vllm-openai, the current release) on a single 8 GB consumer GPU with a tiny dummy-weight model, sent 5 completions that share a 160-token prefix, and scraped/metrics:The dashboard JSON still parses. I did not load it into Grafana; only the metric name inside the three
histogram_quantileexpressions changed. The panel title is unchanged.This change was drafted with AI assistance and reviewed by me.
🤖 Generated with Claude Code