Skip to content

[Model Runner V2][Perf] Warm up GLM-5.2 DSA indexer prefill metadata kernel#47285

Merged
LucasWilkinson merged 1 commit into
vllm-project:mainfrom
chaunceyjiang:warm_prefill_chunk_metadata_kernel
Jul 2, 2026
Merged

[Model Runner V2][Perf] Warm up GLM-5.2 DSA indexer prefill metadata kernel#47285
LucasWilkinson merged 1 commit into
vllm-project:mainfrom
chaunceyjiang:warm_prefill_chunk_metadata_kernel

Conversation

@chaunceyjiang

@chaunceyjiang chaunceyjiang commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator

Purpose

[Perf] Warm up GLM-5.2 DSA indexer prefill metadata kernel

Test Plan

VLLM_USE_V2_MODEL_RUNNER=1 vllm serve /mnt/data1/models/zai-org/GLM-5.2-FP8 -tp 8 --tool-call-parser glm47 --reasoning-parser glm45  --enable-auto-tool-choice  --chat-template-content-format=string --served-model-name zai-org/GLM-5.2-FP8 --max-model-len 272640 --port 8001 --jit-monitor-verbose

Test Result

before

(APIServer pid=3465377) INFO:     127.0.0.1:55664 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(Worker_TP2 pid=3477064) WARNING 07-01 18:02:58 [jit_monitor.py:125] Triton kernel JIT compilation during inference: _build_prefill_chunk_metadata_kernel (constexprs={BLOCK_SIZE=1024, COMPRESS_RATIO=1, DCP_INTERLEAVE=1, DCP_WORLD=1}; signature={BLOCK_SIZE='constexpr', COMPRESS_RATIO='constexpr', DCP_INTERLEAVE='constexpr', DCP_RANK='i32', DCP_WORLD='constexpr', cu_compressed_seq_len_ke_ptr='*i32', cu_compressed_seq_len_ks_ptr='*i32', cu_compressed_seq_lens_ptr='*i32', query_slice_start='i32', query_slice_stop='i32', query_start_loc_ptr='*i32', row_start_cu_compressed_seq_lens_ptr='*i32', token_to_seq_ptr='*i32', uncompressed_seq_lens_ptr='*i32'}; extra_compile_info={configs=[{(0,): [['tt.divisibility', 16]], (1,): [['tt.divisibility', 16]], (2,): [['tt.divisibility', 16]], (3,): [['tt.divi..., device=2, enable_fp_fusion=True, extern_libs=(('libdevice', '/mnt/data4/jxy/venv/lib/python3.12/site-packages/triton/backends/nvidia/lib/libdevice.10.bc'),), is_warmup=False, launch_cooperative_grid=False, num_ctas=1, num_stages=3, num_warps=4, specialization_data='{"name": "vllm.v1.attention.backends.mla.indexer._build_prefill_chunk_metadata_kernel", "signature": {"query_start_l...}; key="[('*i32', 'D'), ('*i32', 'D'), ('*i32', 'D'), ('*i32', 'D'), ('*i32', 'D'), ('*i32', 'D'), ('*i32', 'D'), ('i32', 'D...). This causes a latency spike; consider extending warmup to cover this shape/config.

after

(APIServer pid=3834534) INFO:     127.0.0.1:34634 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=3834534) INFO:     127.0.0.1:34634 - "POST /v1/chat/completions HTTP/1.1" 200 OK

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@chaunceyjiang chaunceyjiang changed the title [Perf] Warm up GLM-5.2 DSA indexer prefill metadata kernel [Model Runner V2][Perf] Warm up GLM-5.2 DSA indexer prefill metadata kernel Jul 2, 2026

@LopezCastroRoberto LopezCastroRoberto left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you for extending the warmup coverage!

@LucasWilkinson LucasWilkinson added the ready ONLY add when PR is ready to merge/full CI is needed label Jul 2, 2026
@LucasWilkinson
LucasWilkinson enabled auto-merge (squash) July 2, 2026 14:32
@LucasWilkinson
LucasWilkinson merged commit 320ee28 into vllm-project:main Jul 2, 2026
77 checks passed
allenh1 pushed a commit to allenh1/vllm that referenced this pull request Jul 3, 2026
Sync 2026-07-03: picks up Model Runner V2 improvements (motivated by V2
viability reopening via PR#26). Notable: MRV2 scheduler req-slot bugfix
(vllm-project#46974), MRV2 spec-decode int32 block-verify overflow fix (vllm-project#47383),
MRV2 DSA-indexer prefill warmup for GLM-5.2/DSA (vllm-project#47285), V2-default for
dense models (vllm-project#44443 — DSv4 is MoE so stays V1), DeepGEMM tag -> nv-dev
a6b593d2 for SM120 (vllm-project#47304), TLI heterogeneous-vocab spec decode (vllm-project#38174),
Delete PagedAttention (vllm-project#47361 — no impact, our sparse-MLA runner is a
flashinfer symbol), Xqa decode kernels (vllm-project#43232).

Conflict: vllm/v1/spec_decode/llm_base_proposer.py (our DSv4 MTP
spec_step_idx routing vs upstream TLI) — resolved keeping BOTH: TLI's
heterogeneous-vocab branch routed through our _compute_logits(spec_step_idx).

Our SM12x stack preserved (persistent_topk, topk.cu, PR#26 padded-Q kernel,
dspark, attention, R1 hardening all UNCHANGED); config/vllm.py DSv4->V1
default + DSpark env-control intact.
jakki-amd pushed a commit to jakki-amd/vllm that referenced this pull request Jul 6, 2026
lkk12014402 pushed a commit to lkk12014402/vllm that referenced this pull request Jul 8, 2026
mayuyuace pushed a commit to mayuyuace/vllm that referenced this pull request Jul 9, 2026
moulalis pushed a commit to red-hat-data-services/vllm-cpu that referenced this pull request Jul 16, 2026
…up cherry-picks (sync-v0.24.0) (#592)

## Summary

Stacks two RHAIIS 3.5.0 GA cherry-pick stories onto `sync-v0.24.0`:

- **INFERENG-7752** — openai/privacy-filter support + `transformers >=
5.6.0`
- **INFERENG-8175** — JIT/Triton kernel warmup coverage (2 of 3 PRs; see
note)

## Changes

### INFERENG-7752 (privacy-filter)
- Cherry-pick of upstream
[vllm#41026](vllm-project/vllm#41026) (merged
06-28), 2 content commits. Clean, no conflicts.
- `requirements/common.txt`: `transformers >= 5.5.3` → `>= 5.6.0` (model
config lands in transformers 5.6.0).

### INFERENG-8175 (kernel warmup)
- [vllm#46634](vllm-project/vllm#46634) — Triton
kernel warmup coverage (DSv4).
- [vllm#47285](vllm-project/vllm#47285) —
GLM-5.2 DSA indexer warmup.
- **Deferred:
[vllm#46182](vllm-project/vllm#46182
(CuTeDSL/FA4 MLA) — moves the FlashAttention submodule pin to a commit
incompatible with our v0.24.0 base and rewrites the MLA prefill backend.
Held pending Runtime-team review (high risk on a v0.24.0 base).

## Cherry-pick conflicts & resolutions

| PR | Conflict | Resolution |
|----|----------|-----------|
| #41026 | none | clean auto-merge |
| #46634 | `vllm/model_executor/warmup/kernel_warmup.py` (2 additive
hunks) | Kept incoming warmup wiring on top of our carry-forward
DSv4/minimax warmup calls. **Backfill:** the wiring imports
`qwen_triton_warmup`, a module added to upstream *after* the v0.24.0 tag
(absent from our base). Backfilled the self-contained module (`torch` +
`logger` only) from #46634's tree so imports resolve and the branch
stays buildable. |
| #47285 | none | clean (only touches `sparse_mla_triton_warmup.py` from
#46634) |

## Testing

- `py_compile` on all changed/added Python modules — pass.
- Full validation via **accept-sync** (BUILD →
IMAGES/TEST/LM-EVAL/GUIDELLM).

## AI assistance

AI-assisted (Claude Code); changes reviewed by @wjhrdy.
philippesic pushed a commit to philippesic/vllm-semantic-cache that referenced this pull request Jul 19, 2026
plasticchris pushed a commit to plasticchris/vllm that referenced this pull request Jul 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants