Repository navigation
Conversation
…anel The "GPU KV Cache Hit Rate" panel in the bundled vLLM dashboard still queries vllm:gpu_prefix_cache_hits_total and vllm:gpu_prefix_cache_queries_total. vLLM hid these deprecated counters in v0.11.0 and removed them in v0.12.0, so the panel shows "No data" with any current engine image, including the chart's default lmcache/vllm-openai:latest (vLLM 0.29.0). Query the replacement counters vllm:prefix_cache_hits_total and vllm:prefix_cache_queries_total instead. vLLM has exported them since v0.9.2. vllm-project#989 updated the other renamed metrics in this dashboard; this is the one it missed. Signed-off-by: YoungjuKim <[email protected]>
fjvbn2003
requested review from
ApostaC,
YuhanLiu11 and
ruizhang0101
as code owners
October 16, 2026 15:59
Contributor
There was a problem hiding this comment.
Code Review
This pull request updates the vLLM dashboard configuration by renaming the prefix cache metrics, changing vllm:gpu_prefix_cache_hits_total and vllm:gpu_prefix_cache_queries_total to vllm:prefix_cache_hits_total and vllm:prefix_cache_queries_total respectively. There are no review comments, and I have no feedback to provide.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The "GPU KV Cache Hit Rate" panel in
helm/dashboards/vllm-dashboard.jsonstill queries the old prefix-cache counters:vLLM deprecated these counters in favor of
vllm:prefix_cache_hits/vllm:prefix_cache_queries, hid them by default in v0.11.0 (only shown with--show-hidden-metrics-for-version), and removed them in v0.12.0 (vllm/v1/metrics/loggers.py). So with any current engine the panel shows No data. That includes the chart's default engine image,lmcache/vllm-openai:latest, which ships vLLM 0.29.0.This PR switches the expression to the current counters:
vLLM has exported
vllm:prefix_cache_{hits,queries}since v0.9.2. #989 already moved this dashboard to the other renamed metrics (kv_cache_usage_perc,request_time_per_output_token_seconds) but missed this panel. The router-side parser has the same problem; that is tracked separately in #1140 / #1146. This PR only touches the Grafana dashboard.How I checked
vLLM v0.31.0 (
vllm/vllm-openai, the current release) on a single 8 GB consumer GPU. I used a tiny randomly initialised Qwen2-style model (--load-format dummy --skip-tokenizer-init), sent 5 completions that share a 160-token prefix, then scraped/metrics:The old expression matches no series. The new one gives 640 / 805 = 0.795, which is what the panel should show.
lmcache/vllm-openai:latest(vLLM 0.29.0, LMCache 0.5.5):vllm/v1/metrics/loggers.pyin the image definesvllm:prefix_cache_hits/vllm:prefix_cache_queriesand has novllm:gpu_prefix_cache*name.I checked that the JSON still parses. I did not import the dashboard into a live Grafana; the change is a metric-name swap inside one PromQL expression.
I have not run
pre-commitlocally for this one-line change; I will fix anything CI flags.-swhen doinggit commit[Bugfix],[Feat], and[CI].This change was drafted with AI assistance and reviewed by me.
🤖 Generated with Claude Code