Skip to content

[Bugfix][Helm] Use vllm:prefix_cache_* in the GPU KV Cache Hit Rate panel - #1147

Open
fjvbn2003 wants to merge 1 commit into
vllm-project:mainfrom
fjvbn2003:fix-prefix-cache-panel-metrics
Open

fjvbn2003 wants to merge 1 commit into
vllm-project:mainfrom
fjvbn2003:fix-prefix-cache-panel-metrics

Conversation

@fjvbn2003

Copy link
Copy Markdown

The "GPU KV Cache Hit Rate" panel in helm/dashboards/vllm-dashboard.json still queries the old prefix-cache counters:

vllm:gpu_prefix_cache_hits_total{endpoint="service-port"} / vllm:gpu_prefix_cache_queries_total

vLLM deprecated these counters in favor of vllm:prefix_cache_hits / vllm:prefix_cache_queries, hid them by default in v0.11.0 (only shown with --show-hidden-metrics-for-version), and removed them in v0.12.0 (vllm/v1/metrics/loggers.py). So with any current engine the panel shows No data. That includes the chart's default engine image, lmcache/vllm-openai:latest, which ships vLLM 0.29.0.

This PR switches the expression to the current counters:

vllm:prefix_cache_hits_total{endpoint="service-port"} / vllm:prefix_cache_queries_total

vLLM has exported vllm:prefix_cache_{hits,queries} since v0.9.2. #989 already moved this dashboard to the other renamed metrics (kv_cache_usage_perc, request_time_per_output_token_seconds) but missed this panel. The router-side parser has the same problem; that is tracked separately in #1140 / #1146. This PR only touches the Grafana dashboard.

How I checked

  • vLLM v0.31.0 (vllm/vllm-openai, the current release) on a single 8 GB consumer GPU. I used a tiny randomly initialised Qwen2-style model (--load-format dummy --skip-tokenizer-init), sent 5 completions that share a 160-token prefix, then scraped /metrics:

    $ curl -s localhost:8000/metrics | grep -c gpu_prefix_cache
    0
    $ curl -s localhost:8000/metrics | grep '^vllm:prefix_cache_.*_total'
    vllm:prefix_cache_queries_total{engine="0",model_name="tiny"} 805.0
    vllm:prefix_cache_hits_total{engine="0",model_name="tiny"} 640.0
    

    The old expression matches no series. The new one gives 640 / 805 = 0.795, which is what the panel should show.

  • lmcache/vllm-openai:latest (vLLM 0.29.0, LMCache 0.5.5): vllm/v1/metrics/loggers.py in the image defines vllm:prefix_cache_hits / vllm:prefix_cache_queries and has no vllm:gpu_prefix_cache* name.

  • I checked that the JSON still parses. I did not import the dashboard into a live Grafana; the change is a metric-name swap inside one PromQL expression.

I have not run pre-commit locally for this one-line change; I will fix anything CI flags.


  • Make sure the code changes pass the pre-commit checks.
  • Sign-off your commit by using -s when doing git commit
  • Try to classify PRs for easy understanding of the type of changes, such as [Bugfix], [Feat], and [CI].

This change was drafted with AI assistance and reviewed by me.

🤖 Generated with Claude Code

…anel

The "GPU KV Cache Hit Rate" panel in the bundled vLLM dashboard still
queries vllm:gpu_prefix_cache_hits_total and
vllm:gpu_prefix_cache_queries_total. vLLM hid these deprecated counters
in v0.11.0 and removed them in v0.12.0, so the panel shows "No data" with
any current engine image, including the chart's default
lmcache/vllm-openai:latest (vLLM 0.29.0).

Query the replacement counters vllm:prefix_cache_hits_total and
vllm:prefix_cache_queries_total instead. vLLM has exported them since
v0.9.2.

vllm-project#989 updated the other renamed metrics in this dashboard; this is the
one it missed.

Signed-off-by: YoungjuKim <[email protected]>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the vLLM dashboard configuration by renaming the prefix cache metrics, changing vllm:gpu_prefix_cache_hits_total and vllm:gpu_prefix_cache_queries_total to vllm:prefix_cache_hits_total and vllm:prefix_cache_queries_total respectively. There are no review comments, and I have no feedback to provide.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant