[Bugfix] Restore overridden HF config fields and support index_skip_topk_offset for DSA topk sharing#27114
Conversation
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
|
/tag-and-rerun-ci |
|
/tag-and-rerun-ci |
1 similar comment
|
/tag-and-rerun-ci |
|
/rerun-failed-ci |
1 similar comment
|
/rerun-failed-ci |
|
/rerun-failed-ci |
6 similar comments
|
/rerun-failed-ci |
|
/rerun-failed-ci |
|
/rerun-failed-ci |
|
/rerun-failed-ci |
|
/rerun-failed-ci |
|
/rerun-failed-ci |
|
/rerun-failed-ci |
12 similar comments
|
/rerun-failed-ci |
|
/rerun-failed-ci |
|
/rerun-failed-ci |
|
/rerun-failed-ci |
|
/rerun-failed-ci |
|
/rerun-failed-ci |
|
/rerun-failed-ci |
|
/rerun-failed-ci |
|
/rerun-failed-ci |
|
/rerun-failed-ci |
|
/rerun-failed-ci |
|
/rerun-failed-ci |
JustinTong0323
left a comment
JustinTong0323
left a comment
There was a problem hiding this comment.
reuse_mtp_topk_indices defaults to False (not None), which breaks two-batch-overlap on every DeepSeek-family run — the deepep CI jobs (TestMTPWithTBO / TestDeepseekMTP) die at scheduler init, not just the MTP path.
The Spec V1 deprecation (sgl-project#25464) deleted eagle_worker.py, which carried the only setter for forward_batch.reuse_mtp_topk_indices. The receiver plumbing (deepseek_nextn, ForwardBatch fields, TBO filter) from sgl-project#27114 survived but went dead. Re-add the setup/reset block to the V2 EagleDraftWorker.draft_forward so DSA/NSA topk indices are shared across MTP draft steps again (topk==1 only).
…opk_offset for DSA topk sharing (sgl-project#27114)
3 participants
Restore HF config fields clobbered by GlmMoeDsaConfig and add index_skip_topk_offset handling for DSA topk sharing
And in the PR description, reference the upstream transformers fix:
The GlmMoeDsaConfig drops/clobbers raw checkpoint fields the DSA path needs (qk_rope_head_dim, index_topk_freq), so we re-read them from config.json and restore. Fixed upstream by this pr — this workaround can be removed once SGLang requires transformers >= 5.10.
Also adds handling for the index_skip_topk_offset config. when set, skip_topk / next_skip_topk are computed relative to the offset instead of layer 1.
CI States
Latest PR Test (Base): ✅ Run #27008071717
Latest PR Test (Extra): ❌ Run #27008071538