)

* [NPU] Support GLM-4.7-Flash on NPU (sgl-project#153)

* [NPU] recover accuracy for gemma3-4b-it from 54% to 72% (reduced by transformer5.3) (sgl-project#155)

* [NPU] recover accuracy for gemma3-4b-it for transformer5.3

* Br fix qwen2 5 ascend (sgl-project#159)

* fix qwen2_5_math_rm_72b

* add documentation for GLM-4.7-Flash on Ascend (sgl-project#162)

* add documentation for GLM-4.7-Flash

* update

* Revert "Use LazyValue for routed_experts_weights_of_layer initialization" (sgl-project#161)

Co-authored-by: j30065060 <[email protected]>

* fix(grok): fallback to standard weight loading when no presharded files found (sgl-project#156)

Co-authored-by: Jianzhao Xu <[email protected]>

* revert: revert qwen3_5.py, use separate layers (sgl-project#172)

* [bugfix]GLM-4V model (sgl-project#176)

Co-authored-by: Jianzhao Xu <[email protected]>

* NPU can use piece cuda graph when the piece cuda graph is explicitly declared (sgl-project#169)

Co-authored-by: chx96642264 <[email protected]>

* Bug fix for llama eagle3 (sgl-project#177)

* fix eagle3 accept rate (sgl-project#179)

* Support MTP for Qwen3.5 (sgl-project#154)

* feat: support Ascend NPU MTP adaptation for GDN attention backend

* fix bug (sgl-project#181)

Co-authored-by: j30065060 <[email protected]>

* revert pr 19321 for accuracy temporarily (sgl-project#178)

* revert pr 19321 for accuracy temporarily

* Bug fix for not import is npu (sgl-project#182)

* Revert "Revert "Use LazyValue for routed_experts_weights_of_layer initializat…" (sgl-project#190)

* fix: qwen3.5 precision & quant model load error (sgl-project#191)

* [NPU] change fused_qkvzba_split_reshape_cat_npu to fused_qkvzba_split_reshape_cat_contiguous (sgl-project#198)

* [NPU] Use causal_conv1d and fix qwen-next modelslim (sgl-project#207)

* fix: use causal_conv1d from sgl-kernel-npu

* merge sgl-project main (sgl-project#221)

* [AMD] Fix AMD CI monitor GitHub API rate limit exhaustion (sgl-project#21527)

* [CI] Register missing jit_kernel test files (sgl-project#21547)

* [diffusion] fix: return None instead of raising RuntimeError when no model info found (sgl-project#21319)

Co-authored-by: Mick <[email protected]>

* [rl][sgl] fix tensor mismatch after pause (sgl-project#21514)

* [Hicache & JIT_kernel] Support page first layout  & mla jit kernel (sgl-project#18311)

* test: point DSV3 int8 MLA CI models to lmsys Hugging Face org (sgl-project#21561)

* [CI] Relax several thresholds in flaky CIs (sgl-project#21562)

* feat: add gc_threshold arg (sgl-project#21481)

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Fix flaky test_pp_single_node (sgl-project#21564)

* Split workflow for releasing runtime docker (sgl-project#21563)

* fix tp capture in vit cuda graph (sgl-project#17255)

* [1/n] lora support - Auto detect lora target modules (sgl-project#21439)

Co-authored-by: Baizhou Zhang <[email protected]>

* [fix] qwen3.5 fuse_moe_triton_tune bug (sgl-project#20232)

* Remove sync when enabling return_logprob (sgl-project#20972)

* Scope streaming backlog coalescing to incremental_streaming_output mode (sgl-project#21037)

Signed-off-by: Vladislav Nosivskoy <[email protected]>
Co-authored-by: Lianmin Zheng <[email protected]>

* docs: flesh out MAINTAINER.md oncall lists and link GitHub profiles (sgl-project#21575)

* [NVIDIA] Enable automatic NUMA configuration (sgl-project#19452)

* [diffusion] UX: aggregate expected dtype-cast logs during weight loading (sgl-project#21552)

* [diffusion] refactor: Unify `TeaCacheParams` and `WanTeaCacheParams` (sgl-project#20706)

Co-authored-by: Mick <[email protected]>

* [diffusion] chore: remove redundant identity preprocess_text functions(sgl-project#20633)

Co-authored-by: Fengyuan Yu <[email protected]>

* Update CODEOWNERS for transformers.py and docs (sgl-project#21555)

Co-authored-by: Lianmin Zheng <[email protected]>

* reduce CPU peak memory in multimodal tensor hashing (sgl-project#21123)

* Fix HFRunner hang when subprocess dies during init (sgl-project#21582)

* Fix Piecewise CUDA Graph crash with `-enable-mixed-chunk` (sgl-project#20441)

Co-authored-by: jianyingzhu <[email protected]>

* [CI] Replace upload/download-artifact with job outputs in release-docker workflow (sgl-project#21579)

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

* Patch transformers is_base_mistral in CI to avoid HF 429 rate limiting (sgl-project#21586)

* [CI] Move v32 cp test to deepep running suite (sgl-project#21585)

* [AMD] Add GLM-4.7-FP8 accuracy CI test for MI35x (sgl-project#21534)

Co-authored-by: Claude Opus 4.6 <[email protected]>

* [Clean] Remove deprecated environs (sgl-project#21536)

* [diffusion] fix: fix Flux2-Klein prompt tokenization length to 512 and add regression coverage (sgl-project#21407)

* [CI] hot-fix ci lint (sgl-project#21608)

* [diffusion] feat: support overlay model materialization (sgl-project#21600)

* [VLM] Optimize ShmPointerMMData for multi-pickle safety and deferred unwrap (sgl-project#21465)

* feat: enable CUDA graph and timestamp for the whisper model(sgl-project#21190)

* [NPU] Update quantization&CI documentation (sgl-project#21100)

Co-authored-by: Tamir Baydasov <[email protected]>

* Skip ci for .md files (sgl-project#21482)

* Support skip-softmax attention (sgl-project#19089)

* fix: piecewise_cuda_graph get correct qo_indptr (sgl-project#21452)

Co-authored-by: Avery Huang <[email protected]>

* fix bench_serving sglang backend to support image dataset  (sgl-project#21294)

* [AMD] Add peft>=0.18.0 to diffusion_hip deps for transformers 5.x compat for AMD diffusion model (sgl-project#21442)

Co-authored-by: HaiShaw <[email protected]>

* [GDN] Fuse GDN kkt + solve_tril into one kernel (sgl-project#21411)

Co-authored-by: luoyuan.luo <[email protected]>

* [Diffusion] Align diffusion benchmark skill presets with nightly comparison cases (sgl-project#21616)

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Clean up detokenizer and remove dead multimodal_gen code (sgl-project#21588)

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

* [CI] Skip flaky elastic EP test (sgl-project#21619)

* feat(ci): add GB300 nightly benchmark test suites (sgl-project#21487)

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

* [CI] Lossen test_return_routed_experts threshold (sgl-project#21270)

* Add subprocess liveness monitor to detect scheduler crashes (sgl-project#18582)

Co-authored-by: 继优 <[email protected]>
Co-authored-by: shuwenn <[email protected]>

* fix: scheduler launch hang when non-current rank dies (sgl-project#20287)

* Wrap IPv6 addresses in gRPC, bench_serving, and log messages (sgl-project#21236)

Co-authored-by: hnyls2002 <[email protected]>
Co-authored-by: Liangsheng Yin <[email protected]>

* [HiCache] fix: graceful shutdown of pending async tasks in bench_mix.py (sgl-project#20276)

* Clean up _wait_for_scheduler_ready implementation (sgl-project#21626)

* fix cuda graph capturing error in sm120 mxfp8 triton path (sgl-project#19835)

* [sgl] disable piecewise cuda graph when a model doesn't have layers (sgl-project#21565)

* [Feature] Optimizations for JPEG input on NVIDIA GPU (sgl-project#19749)

* [VLM] perf: optimize CUDA IPC for multimodal transfer by caching IPC pool handles (sgl-project#21418)

* [Fix] SGLANG_USE_CUDA_IPC_TRANSPORT=1 and SGLANG_ENABLE_MM_SPLITTING=1 do not work at the same time. (sgl-project#19915)

* [Fix] Remove redundant allreduce fusion block and skip TP=1 (sgl-project#20621)

* Simplify routed experts test and move base64 encoding to tokenizer manager (sgl-project#21634)

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

* [Cleanup] Remove unused BatchMultimodalOutput and BatchMultimodalDecodeReq (sgl-project#21640)

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

* Clean up TokenizerManager: remove dead code and improve rid validation (sgl-project#21639)

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

* README: coding agent sponsorship for long-term contributors (sgl-project#21642)

* Fix circular reference in CustomTestCase.__init_subclass__ (sgl-project#21650)

Co-authored-by: wan4ch <[email protected]>

* [Fix] Fix Qwen3.5 MoE model loading and Mamba cache sharding in PP mode (sgl-project#21448)

Co-authored-by: zhangxiaolei123456 <[email protected]>

* [diffusion] CI: fix dashboard chart (nightly) display issues (sgl-project#21653)

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update sponsorship details in README.md (sgl-project#21658)

* [Fix] Handle pre-release tags in nightly wheel version parsing (sgl-project#21656)

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

* [Intel GPU] Enable DeepSeek R1 inference on XPU (sgl-project#18461)

Signed-off-by: P V R K Jyothendra Varma <[email protected]>

* [Doc] Update tips for developer new-comers (sgl-project#21659)

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* [CI] [FlashInfer v0.6.7] Use offline quantized checkpoint for MXFP8 Gemm tests (sgl-project#21625)

* MFU metrics in Prometheus  (sgl-project#19395)

* fix topk softmax performance issue (sgl-project#14702)

* [CPU] add kernel apply_rotary_pos_emb_cpu for Qwen3-VL and Qwen3-Omni (sgl-project#13121)

Co-authored-by: Ma Mingfei <[email protected]>

* [CPU] Implement MXFP4 Gemm kernels for intel AMX to support GPT OSS series. (sgl-project#14385)

* [AMD] Fused rope kv store (sgl-project#21315)

Co-authored-by: wunhuang <[email protected]>

* [NPU] Update DeepSeek-V3.2 model deployment instructions in documentation (sgl-project#21468)

Co-authored-by: wuxue (C) <[email protected]>

* [AMD] Support AMD MXFP4 Qwen3.5-397B-A17B model (sgl-project#21234)

* [Fix] Fix weight_loader property assignment for qwen3-next FP8 models (sgl-project#21662)

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

* fix mamba cache leak when adder fails to add a matched req. (sgl-project#21404)

* fix: Mistral Small 4 fails to start due to config/weight format mismatch (sgl-project#21620)

Co-authored-by: mengxiancheng03 <[email protected]>
Co-authored-by: Baizhou Zhang <[email protected]>
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

* [diffusion] feat: enhance overlay mechanism (sgl-project#21648)

* [diffusion] CI: relax pr-test threshold (sgl-project#21682)

* [NPU][Diffusion] fix sp modulate for qwen-image-edit (sgl-project#20974)

Co-authored-by: 高鑫 <[email protected]>

* [NPU] fix eagle3 accept rate (sgl-project#21255)

* DeepSeek-R1-0528-w4a8: DeepEP Low Latency Dispatch Adopts FP8 Communication (sgl-project#14162)

Co-authored-by: undefined <[email protected]>

* [NPU] GLM-5 optimize with fused kernels (sgl-project#18617)

* [NPU][diffusion]: support parallel decoding of qwen-image (sgl-project#20757)

Co-authored-by: 高鑫 <[email protected]>

* [diffusion] [NPU] support ring attention on NPU with FA (sgl-project#21383)

* [diffusion][doc]: add ring sp performance benchmark page (sgl-project#20998)

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* [GLM-V and GLM-4.7] Cast to FP32 before gate projection for GLM model. (sgl-project#21660)

* fix nemotron capture for non attention layers (sgl-project#21436)

* [Bugfix][NPU] Skip FRACTAL_NZ format for MoE weights with unaligned dimensions (sgl-project#21209)

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: ronnie_zheng <[email protected]>

* [AMD] Add SGLANG_DISAGGREGATION_NUM_PRE_ALLOCATE_REQS env var for configurable KV transfer overlap (sgl-project#20410)

Co-authored-by: HaiShaw <[email protected]>

* [AMD][MoRI] bump MoRI to v0.1.0 (sgl-project#21673)

* [AMD] fix performance regression issue when run gpt-oss with "--context-length 13824" (sgl-project#21691)

* Remove flashinfer wheel cache cleanup that deletes other versions (sgl-project#21711)

Co-authored-by: Alison Shao <[email protected]>

* [misc] multiprocess compilation to speed up test (sgl-project#21483)

* Fix human-eval CI install on 5090 runners (sgl-project#21714)

Co-authored-by: Alison Shao <[email protected]>

* Revert "DeepSeek-R1-0528-w4a8: DeepEP Low Latency Dispatch Adopts FP8 Communication" (sgl-project#21719)

* [Fix] Update supported custom_mem_pool types for mooncake (sgl-project#21728)

Co-authored-by: 百麒 <[email protected]>

* [Perf]Remove H2D  for Qwen3.5 SpecV2 (sgl-project#20864)

* [AMD] Fix CI multimodal-gen-test-1-gpu-amd for gen model  (sgl-project#21621)

* [diffusion] fix: fix Flux.2 with tp(sgl-project#21664)

* Add explicit disable flag for FlashInfer allreduce fusion (sgl-project#21446)

* [NPU] fix conflict between empty_cache and use_mem_pool (sgl-project#21507)

* [AMD] Use tgemm.mm for MoEGate router gemm in deepseek_v2.py (sgl-project#21657)

* [CI]Remove msgm-en and mmlu tests which cause timeout (sgl-project#21733)

* Fix disaggregation hybrid attention ci (sgl-project#21745)

* Rename rerun-ut to rerun-test (sgl-project#21747)

* bugfix(model):fix deepstack index out of range error (sgl-project#21727)

Co-authored-by: xiaoqi.31 <[email protected]>

* [diffusion] fix: fix typo (sgl-project#21746)

Signed-off-by: Xiaodong Ye <[email protected]>

* [CI] Fix rerun-test suite detection to skip commented registrations (sgl-project#21753)

* [PD] Refactor Disagg Conn and Fix Hang with total_request/total_tokens Balancing (sgl-project#21299)

Co-authored-by: Weiliangl User <[email protected]>

* [CI] Fix ring test timeout (sgl-project#21751)

* Enable evict swa with piecewise cuda graph (sgl-project#21754)

* Fix kimi-linear launch server error (sgl-project#21752)

Co-authored-by: luoyuan.luo <[email protected]>

* [PD] Tiny cleanup after KVReceiver refactor (sgl-project#21760)

Signed-off-by: Shangming Cai <[email protected]>

* Fix remote weight info nnode>1 and dp>1 (sgl-project#17389)

* [diffusion] UX: replace deprecated ORJSONResponse with orjson_response (sgl-project#21755)

Co-authored-by: Claude Opus 4.6 <[email protected]>

* [diffusion] fix: fix Wan2.2-I2V-A14B video max size issue(sgl-project#21390)

Signed-off-by: Xiaodong Ye <[email protected]>
Co-authored-by: Mick <[email protected]>

* [HiMambaTree]: Optimize mamba host lock mechanism (sgl-project#21750)

* [AMD] Fix Handle missing rope_theta in get_rope_config for Grok-1 (sgl-project#21518)

* [bugfix] Fix rope theta config for MiniMax after transformers v5 update (sgl-project#21241)

* Fix ineffective is_base_mistral CI patch for HF API rate limiting (sgl-project#21729)

* [2/n] lora - Shared outer experts and support qwen3_30b_a3b_instruct (sgl-project#21466)

Co-authored-by: Baizhou Zhang <[email protected]>

* Fix cuda graph max bs capture upper bound (sgl-project#21005)

* [Fix] Fall back to triton MOE for GPT-OSS on Blackwell with driver >= 595 (sgl-project#21780)

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

* Cache nvidia wheels locally to skip repeated 830 MB downloads in CI (sgl-project#21778)

* Add Trivy vulnerability scanning to nightly dev Docker builds (sgl-project#21772)

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

* [CI] Remove more redundant PCG tests (sgl-project#21554)

* [moe] add customized option to moe-a2a-backend (sgl-project#21786)

* Add CompletionSampler for non-chat eval in run_eval (sgl-project#21785)

* Remove redundant test_moe_eval_accuracy_large (sgl-project#21787)

* Increase hicache eval to 200 examples (sgl-project#21791)

* Switch MooncakeSpec to EAGLE3 + Llama-3.1 (sgl-project#21794)

* Reduce redundant speculative decoding CI tests (sgl-project#21779)

* Fix killall.py crash when sglang is not yet installed (sgl-project#21797)

* Remove obsolete sgl-kernel legacy paths (sgl-project#21528)

* [jit_kernel] Optimize fused_qknorm_rope: deduplicate sincosf for interleave RoPE  (sgl-project#21654)

* CUTLASS NVFP4 GEMM improvement of SM120 (sgl-project#21314)

* [gRPC] Preserve original ImportError in grpc_server.py (sgl-project#21801)

Signed-off-by: Chang Su <[email protected]>

* [Misc] Tiny: Add test network timeouts and dynamic max-parallel for 5090/2-gpu runners (sgl-project#21800)

* Fix draft extend cuda graph when spec_step=1 (sgl-project#21709)

* [Diffusion] Add `--uvicorn-access-log-exclude-prefixes` to suppress noisy access logs (sgl-project#20379)

* Add latency and throughput metrics to run_eval (sgl-project#21793)

* [diffusion] CI: improve ci reliability (sgl-project#21763)

* [bugfix]GLM-4V model (sgl-project#17122)

* Fix CVEs in Docker image: pillow, linux-libc-dev, and broken sgl-model-gateway build (sgl-project#21789)

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

* fix: only showing recent runners from ci failure analysis (sgl-project#21015)

* [MPS] Fix Triton stub sub-module imports on Python 3.12+ (sgl-project#21551)

Co-authored-by: karanb192 <[email protected]>
Co-authored-by: R0CKSTAR <[email protected]>
Co-authored-by: R0CKSTAR <[email protected]>

* [KDA] Fuse scaled_dot_kkt + solve_tril + recompute_w_u for KDA (sgl-project#21604)

Co-authored-by: luoyuan.luo <[email protected]>

* chore: bump flashinfer version to 0.6.7 (sgl-project#21422)

Co-authored-by: sglang-bot <[email protected]>
Co-authored-by: Baizhou Zhang <[email protected]>

* [3/n] lora moe - Support Qwen3-VL-30B-A3B-Instruct  (sgl-project#21469)

Co-authored-by: Baizhou Zhang <[email protected]>

* [Feature Restoration] repetition_penalty is essential for GLM-V models (sgl-project#21258)

Co-authored-by: Xinyuan Tong <[email protected]>
Co-authored-by: Xinyuan Tong <[email protected]>
Co-authored-by: hnyls2002 <[email protected]>
Co-authored-by: Liangsheng Yin <[email protected]>

* VLM: change default mm-attention backend from triton_attn to fa4 (on blackwell) (sgl-project#21595)

* Fix added tokens config with sensible filter (sgl-project#17905)

* [AMD] Optimize Qwen3-VL decode - fuse QK-norm + 3D mRoPE + KV cache write (sgl-project#21458)

Co-authored-by: Bingxu Chen <[email protected]>
Co-authored-by: HaiShaw <[email protected]>

* [Bugfix] Fix PP tied embeddings weight loading for qwen3.5 4B dense model (sgl-project#21347)

* [CI] Fix lint that was not applied in sgl-project#21458 (sgl-project#21818)

* Bug fix for llama eagle3 (sgl-project#21397)

* glm_interleave for GLM-V (sgl-project#21671)

* style refinement for hisparse (sgl-project#21198)

* [Bug][VLM] Fix shared memory race condition in ShmPointerMMData broadcast for multi-GPU VLM serving (sgl-project#21655)

* [Bugfix] Fix effective_mamba_size over-allocation (sgl-project#20858)

Co-authored-by: Shangming Cai <[email protected]>

* Fix in-place mode in pause generation (sgl-project#21705)

* [diffusion] fix: respect --prompt-path (sgl-project#21756)

* [NPU] update ascend docs (sgl-project#21807)

* [VLM] remove AsyncMMDataProcessor wrapper (sgl-project#21651)

* Use CustomTestCase for TestSessionControl to enable CI retry (sgl-project#21830)

* [NPU]Add a full test pipeline on NPU, resolve issues in the NPU test architecture (sgl-project#20751)

* [diffusion][CI]: Add individual component accuracy CI for diffusion models (sgl-project#18709)

Co-authored-by: Xiaoyu Zhang <[email protected]>

* [Feature] JIT rmsnorm update (with claude) (sgl-project#21834)

* [Diffusion][NPU] add ring sp performance benchmark page in npu (sgl-project#21811)

* fix(MiMo-V2-Flash): add mimo reasoning parser (sgl-project#21414)

* [diffusion] hardware: support FA3 attention backend on MUSA (attn backend, 14/N) (sgl-project#18648)

Signed-off-by: Xiaodong Ye <[email protected]>
Co-authored-by: Mick <[email protected]>

* fix: pre-init tokenizer_manager to avoid AttributeError in shutdown (sgl-project#21824)

* [FlashInver v0.6.7] Integrate flashinfer_trtllm mxfp8 gemm (sgl-project#21576)

* [Misc] Add network timeout to eval dataset downloads (sgl-project#21873)

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

* [refactor] Clean up duplicate flashinfer trtllm moe code (sgl-project#21233)

* [DSA] Support trtllm sparse mla kernel for prefill batches  (sgl-project#21783)

* [Disagg] GPU staging buffer with dynamic ring allocator for heterogeneous TP KV transfer (sgl-project#19890)

* Add merge prohibition policy during CI maintenance mode (sgl-project#21882)

* [Misc] Fix comparator e2e tests: add polars dep + fix dp-attention test (sgl-project#21804)

Co-authored-by: Alison Shao <[email protected]>

* revert: remove TTL-based hard pin from HiRadixCache (sgl-project#21884)

* Unify GSM8K eval path to Chat API for regression CI readiness (sgl-project#21667)

* [HiCache] fix: Clone host indices to avoid memory leak (sgl-project#21624)

Co-authored-by: Zhiqiang Xie <[email protected]>

* [HiCache & PD]Fixed detailed cache hit breakdown in PD scenarios. (sgl-project#21764)

* [CI] Add Llama 3.1 8B Instruct FP4 CI test on SM120 (sgl-project#20648)

* [CI] Add Per-Tensor, Blockwise FP8 Tests on SM120 (sgl-project#20717)

Co-authored-by: Brayden Zhong <[email protected]>

* Allow /rerun-test to checkout fork PR branch for trusted users (sgl-project#21890)

* Direct model loading from object storage with Runai Model Streamer (sgl-project#17948)

Signed-off-by: Noa Neria <[email protected]>

* fix pcg torch dynamo recompile in mxfp8 Triton path (sgl-project#21888)

Co-authored-by: Hanlin Bi <[email protected]>

* chore: bump mooncake version to 0.3.10.post1 (sgl-project#21844)

* [VLM] Add VLM TP=4 per-commit CI test and improve MMMU eval prompt/parser (sgl-project#21841)

* fix(ci): update est_time for 57 tests based on runtime analysis (sgl-project#21896)

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

* [CI] Increase multimodal server test timeout from 60 to 90 minutes (sgl-project#21897)

* [CI] Remove crashing Kimi K2.5 EAGLE3/MTP variants, keep TP8 and TP8+DP8 (sgl-project#21898)

* [diffusion] CI: add initial nvfp4 ci test for b200 (sgl-project#21767)

Co-authored-by: Mick <[email protected]>

* Migrate all callers from /get_server_info to /server_info (sgl-project#21463)

* Support PP key for file backend (sgl-project#21901)

* Enable multi-thread weight loading by default (sgl-project#20289)

* Skip Go stdlib and NVIDIA tool CVEs in Trivy scan (sgl-project#21905)

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

* [Kernel] Fuse temperature + softmax in sampling for decode speedup (sgl-project#20501)

* Multi tool streaming fix (sgl-project#20004)

* Return HTTP 400 for streaming validation errors (sgl-project#21900)

* [Spec][Ngram] 4/N: Remove `max_match_window_size` and `min_match_window_size`, matching all suffixes of the Trie (sgl-project#21225)

* Fix ngram doc for speculative_num_draft_tokens default (sgl-project#21910)

* [NVIDIA] Enable fp8 flashinfer_trtllm_routed MoE for MiniMax-M2.5 (sgl-project#20394)

* scheduler: add prefill-only update in merge batch (sgl-project#21840)

* [DSA] Set trtllm kernels as nsa default for Blackwell (sgl-project#21914)

* Revert "Rollback flashmla to older version [1/2]" (sgl-project#21922)

* test: add manual init test for mooncake transfer engine (sgl-project#21842)

Co-authored-by: yunzhi <[email protected]>

* Fix spec v2 + logprob when max_num_token is set (sgl-project#20799)

* Migrate ngram corpus from torch cpp_extension to TVM FFI jit_kernel (sgl-project#21920)

Co-authored-by: DarkSharpness <[email protected]>

* [NPU] Support  GLM-4.7-Flash on NPU (sgl-project#21408)

* [CI] Fix gpu deps import in cpu test (sgl-project#21950)

* [Parallel State Refactor 1/n] Remove stream of PyNCCL (sgl-project#20866)

* [diffusion] chore: fix stage profiler for multi-stage denoising (sgl-project#21955)

* [CI] [Tracing] Add ci for tracing and fix bugs (sgl-project#21740)

* Remove logging for subprocess watchdog start (sgl-project#21968)

* [4/n] Support gpt oss 20b lora (sgl-project#21570)

* [MUSA][9/N] Add FA3 attention backend support through MATE (MUSA AI Tensor Engine) (sgl-project#17985)

Co-authored-by: R0CKSTAR <[email protected]>

* [Feature] Stronger transformers modeling backend with TP, PP, MoE, VLMs, and torch compile (sgl-project#19163)

* [CI] Remove stale Ascend suite entries from test/srt/run_suite.py (sgl-project#21978)

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

* Skip broken AutoModel mapping entries when resolving Llava submodules (sgl-project#21892)

* [CI] Add timeouts to Slack upload urlopen and WebClient (sgl-project#21903)

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

* [Diffusion][NPU] Add support for MOVA (sgl-project#21633)

Co-authored-by: zhangshuai (S) <[email protected]>

* Remove maxItems=1 restriction when tool_choice is specified (sgl-project#20208)

* [Feature] NVFP4 Marlin fallback for non-Blackwell GPUs (SM75+) (sgl-project#19652)

* [PP] qwen3 vl skip layer id for pp (sgl-project#19135)

* [VLM] Enable per-image MM splitting by default and remove MULTI_IMAGES modality (sgl-project#21899)

* [Bugfix] Fix incorrect dp-attention parallel info in bench_one_batch (sgl-project#21519)

* Revert "[MUSA][9/N] Add FA3 attention backend support through MATE (MUSA AI Tensor Engine)" (sgl-project#22002)

* [NPU] Optimized the wording in the npu docs (sgl-project#21998)

* [Parallel State Refactor 2/n] Unify code path of AMD deterministic all reduce (sgl-project#20871)

* [AMD] Resolve the performance degression when launch server with "--enable-aiter-allreduce-fusion" (sgl-project#21947)

Co-authored-by: wunhuang <[email protected]>

* chore: bump sgl-kernel version to 0.4.1 (sgl-project#21447)

Co-authored-by: sglang-bot <[email protected]>

* [Workflow] Avoid triggering nightly tests in kernel bump workflow (sgl-project#22010)

* [Workflow] Fix kernel release jobs skipped on push events (sgl-project#22011)

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

* [PD]: Add support for HiSparse to directly transfer the cache from Prefill to Decode DRAM. (sgl-project#21591)

Co-authored-by: Tingwei Huang <[email protected]>
Co-authored-by: Shangming Cai <[email protected]>
Co-authored-by: Zhiqiang Xie <[email protected]>

* [Misc] Update CI permission (sgl-project#22014)

* [ROCM][RL] Shuffle Weight In-Place to Preserve Parameter Attributes (sgl-project#21825)

* [CI] Fix duplicate job names that bypass branch protection (sgl-project#22001)

* fix: remove duplicate words in comments (sgl-project#22007)

* [PD] Tiny register info field cleanup for mooncake backend (sgl-project#22016)

* [NPU] optimize glm4.7 (sgl-project#19246)

* [AMD] Enable FP8 KV cache and FP8 attention kernel for NSA on MI300/MI355 with TileLang backend (sgl-project#21511)

* [AMD] Add MiniMax-M2.5 nightly perf benchmarks for MI30x and MI35x (sgl-project#21524)

---------

Signed-off-by: Vladislav Nosivskoy <[email protected]>
Signed-off-by: P V R K Jyothendra Varma <[email protected]>
Signed-off-by: Xiaodong Ye <[email protected]>
Signed-off-by: Shangming Cai <[email protected]>
Signed-off-by: Chang Su <[email protected]>
Signed-off-by: Noa Neria <[email protected]>
Co-authored-by: Bingxu Chen <[email protected]>
Co-authored-by: Xiaoyu Zhang <[email protected]>
Co-authored-by: yang1002378395-cmyk <[email protected]>
Co-authored-by: Mick <[email protected]>
Co-authored-by: Bi Xue <[email protected]>
Co-authored-by: huangtingwei <[email protected]>
Co-authored-by: Lianmin Zheng <[email protected]>
Co-authored-by: Baizhou Zhang <[email protected]>
Co-authored-by: Muqi Li <[email protected]>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Qiaolin Yu <[email protected]>
Co-authored-by: narutolhy <[email protected]>
Co-authored-by: Ethan (Yusheng) Su <[email protected]>
Co-authored-by: zhangxiaolei <[email protected]>
Co-authored-by: Vladislav Nosivskoy <[email protected]>
Co-authored-by: Trevor Morris <[email protected]>
Co-authored-by: Eitan Turok <[email protected]>
Co-authored-by: Fengyuan Yu <[email protected]>
Co-authored-by: Fengyuan Yu <[email protected]>
Co-authored-by: Adarsh Shirawalmath <[email protected]>
Co-authored-by: Yuhao Yang <[email protected]>
Co-authored-by: Liangsheng Yin <[email protected]>
Co-authored-by: Jianying <[email protected]>
Co-authored-by: jianyingzhu <[email protected]>
Co-authored-by: Kangyan-Zhou <[email protected]>
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
Co-authored-by: Jacob0226 <[email protected]>
Co-authored-by: Aditya Sharma <[email protected]>
Co-authored-by: Yuan Luo <[email protected]>
Co-authored-by: Xinyuan Tong <[email protected]>
Co-authored-by: Артем Савкин <[email protected]>
Co-authored-by: Tamir Baydasov <[email protected]>
Co-authored-by: Shu Wang <[email protected]>
Co-authored-by: eigen <[email protected]>
Co-authored-by: Avery Huang <[email protected]>
Co-authored-by: jacky.cheng <[email protected]>
Co-authored-by: HaiShaw <[email protected]>
Co-authored-by: luoyuan.luo <[email protected]>
Co-authored-by: Shangming Cai <[email protected]>
Co-authored-by: Junrong Lin <[email protected]>
Co-authored-by: Simon (Jiyou) Li <[email protected]>
Co-authored-by: 继优 <[email protected]>
Co-authored-by: shuwenn <[email protected]>
Co-authored-by: psaab <[email protected]>
Co-authored-by: hnyls2002 <[email protected]>
Co-authored-by: Hanlin Bi <[email protected]>
Co-authored-by: wili <[email protected]>
Co-authored-by: saatwiknagpal <[email protected]>
Co-authored-by: Mohammad Miadh Angkad <[email protected]>
Co-authored-by: wan4ch <[email protected]>
Co-authored-by: Feng Su <[email protected]>
Co-authored-by: Ying Sheng <[email protected]>
Co-authored-by: Polisetty V R K Jyothendra Varma <[email protected]>
Co-authored-by: Ziang Li <[email protected]>
Co-authored-by: Aishwarya Ramasethu <[email protected]>
Co-authored-by: Ma Mingfei <[email protected]>
Co-authored-by: blzheng <[email protected]>
Co-authored-by: kk <[email protected]>
Co-authored-by: wunhuang <[email protected]>
Co-authored-by: Michelle Wu <[email protected]>
Co-authored-by: wuxue (C) <[email protected]>
Co-authored-by: Hubert Lu <[email protected]>
Co-authored-by: strgrb <[email protected]>
Co-authored-by: LiYomi <[email protected]>
Co-authored-by: mengxiancheng03 <[email protected]>
Co-authored-by: GXIN <[email protected]>
Co-authored-by: 高鑫 <[email protected]>
Co-authored-by: heziiop <[email protected]>
Co-authored-by: xieminghe1 <[email protected]>
Co-authored-by: undefined <[email protected]>
Co-authored-by: Makcum888e <[email protected]>
Co-authored-by: yuefeng Wu <[email protected]>
Co-authored-by: Yuxuan Zhang <[email protected]>
Co-authored-by: Vedant V Jhaveri <[email protected]>
Co-authored-by: ronnie_zheng <[email protected]>
Co-authored-by: Zhai Feiyue <[email protected]>
Co-authored-by: jhchouuu <[email protected]>
Co-authored-by: Alison Shao <[email protected]>
Co-authored-by: Alison Shao <[email protected]>
Co-authored-by: DarkSharpness <[email protected]>
Co-authored-by: Alison Shao <[email protected]>
Co-authored-by: Lewis <[email protected]>
Co-authored-by: 百麒 <[email protected]>
Co-authored-by: Jincong Chen <[email protected]>
Co-authored-by: xiazhahe <[email protected]>
Co-authored-by: Thomas Wang <[email protected]>
Co-authored-by: Ke Bao <[email protected]>
Co-authored-by: xiaoqi <[email protected]>
Co-authored-by: xiaoqi.31 <[email protected]>
Co-authored-by: R0CKSTAR <[email protected]>
Co-authored-by: weireweire <[email protected]>
Co-authored-by: Weiliangl User <[email protected]>
Co-authored-by: JD <[email protected]>
Co-authored-by: Zhangheng <[email protected]>
Co-authored-by: Michael <[email protected]>
Co-authored-by: Yilong Zhao <[email protected]>
Co-authored-by: Johnsonms <[email protected]>
Co-authored-by: Brayden Zhong <[email protected]>
Co-authored-by: Chang Su <[email protected]>
Co-authored-by: KnightLTC <[email protected]>
Co-authored-by: Douglas Yang <[email protected]>
Co-authored-by: Karan Bansal <[email protected]>
Co-authored-by: karanb192 <[email protected]>
Co-authored-by: R0CKSTAR <[email protected]>
Co-authored-by: sglang-bot <[email protected]>
Co-authored-by: sglang-bot <[email protected]>
Co-authored-by: Xinyuan Tong <[email protected]>
Co-authored-by: sbeurnier <[email protected]>
Co-authored-by: YC Yen-Ching Tseng <[email protected]>
Co-authored-by: Wenyao Gao <[email protected]>
Co-authored-by: Alex Nails <[email protected]>
Co-authored-by: khalilzhk <[email protected]>
Co-authored-by: Zhiqiang Xie <[email protected]>
Co-authored-by: yudian0504 <[email protected]>
Co-authored-by: yunkchen <[email protected]>
Co-authored-by: wduan-hai <[email protected]>
Co-authored-by: amote-i <[email protected]>
Co-authored-by: Cherry_ming <[email protected]>
Co-authored-by: Ratish P <[email protected]>
Co-authored-by: YAMY <[email protected]>
Co-authored-by: Alison Shao <[email protected]>
Co-authored-by: ishandhanani <[email protected]>
Co-authored-by: Derek Yu <[email protected]>
Co-authored-by: Noa Neria <[email protected]>
Co-authored-by: Hanlin Bi <[email protected]>
Co-authored-by: Prozac614 <[email protected]>
Co-authored-by: David Cheung <[email protected]>
Co-authored-by: Mook <[email protected]>
Co-authored-by: Khoa Pham <[email protected]>
Co-authored-by: foraxe <[email protected]>
Co-authored-by: yunzhi <[email protected]>
Co-authored-by: DarkSharpness <[email protected]>
Co-authored-by: Todobe <[email protected]>
Co-authored-by: ori <[email protected]>
Co-authored-by: Thomas <[email protected]>
Co-authored-by: zhangshuai (S) <[email protected]>
Co-authored-by: lviy <[email protected]>
Co-authored-by: Tingwei Huang <[email protected]>
Co-authored-by: Yuzhen Zhou <[email protected]>
Co-authored-by: Ricardo-M-L <[email protected]>
Co-authored-by: Kelon <[email protected]>
Co-authored-by: cen121212 <[email protected]>

* BugFix for MLAPO for Deepseek eagle3 on Ascend (sgl-project#222)

* adapt mtp + prefix for ascend gdn backend (sgl-project#202)

* Minimax 2.5 optimization (sgl-project#237)

* Move ring test to nightly (sgl-project#22267)

* [diffusion] CI: fix consistency check (sgl-project#22251)

* [Disagg][NIXL] Support Mamba state slice transfer for heterogeneous TP (Step 2/2 for Qwen3.5) (sgl-project#22240)

* [Mamba] eliminate D2H if tracking mamba states (sgl-project#20522)

Co-authored-by: hzh0425 <[email protected]>

* fix: Auto-correct page_size for Mamba no_buffer radix cache mode (sgl-project#20538)

* Support HTTP2 server (sgl-project#21700)

* [NPU] enable mla prepare fused kernel only when being mla attn (sgl-project#22024)

* [HiSparse] Optimize the scheduling of decode backup. (sgl-project#21932)

Co-authored-by: hzh0425 <[email protected]>
Co-authored-by: Zhiqiang Xie <[email protected]>

* Kernels community fa3 (sgl-project#20796)

* [tiny] migrate /get_server_info; print accept length in accuracy tests (sgl-project#22282)

* [AMD] Fix test_kimi_k25_mxfp4.py : stage-c-test-large-8-gpu-amd-mi35x (linux-mi35x-gpu-8, 1) (sgl-project#22188)

* [Feature] Add DFLASH speculative decoding support (sgl-project#22077)

Co-authored-by: Jian Chen <[email protected]>
Co-authored-by: Zhijian Liu <[email protected]>
Co-authored-by: Richard Gong <[email protected]>
Co-authored-by: David Wang <[email protected]>
Co-authored-by: yilian49 <[email protected]>
Co-authored-by: xm:D <[email protected]>

* Reduce unnecessary kernels and copies in the NSA indexer (sgl-project#22232)

* [CI] Update nightly test models for H200/B200 (sgl-project#22288)

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

* Add fast-fail to multimodal-gen CI (sgl-project#22284)

* [NVIDIA] Enable FP4 flashinfer trtllm routed moe (sgl-project#21240)

* [CI] Migrate mgsm_en eval to gsm8k to remove openaipublic dependency (sgl-project#21931)

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: Kangyan-Zhou <[email protected]>

* Revert "[CI] Update nightly test models for H200/B200 (sgl-project#22288)" (sgl-project#22297)

* Only upload CUDA coredumps on test failure (sgl-project#22301)

---------

Signed-off-by: Vladislav Nosivskoy <[email protected]>
Signed-off-by: P V R K Jyothendra Varma <[email protected]>
Signed-off-by: Xiaodong Ye <[email protected]>
Signed-off-by: Shangming Cai <[email protected]>
Signed-off-by: Chang Su <[email protected]>
Signed-off-by: Noa Neria <[email protected]>
Co-authored-by: Estrella-xx <[email protected]>
Co-authored-by: McZyWu <[email protected]>
Co-authored-by: amote-i <[email protected]>
Co-authored-by: longxin9715 <[email protected]>
Co-authored-by: j30065060 <[email protected]>
Co-authored-by: jianzhao-xu <[email protected]>
Co-authored-by: Jianzhao Xu <[email protected]>
Co-authored-by: iridiumine <[email protected]>
Co-authored-by: chx96642264 <[email protected]>
Co-authored-by: chx96642264 <[email protected]>
Co-authored-by: heziiop <[email protected]>
Co-authored-by: cen121212 <[email protected]>
Co-authored-by: Bingxu Chen <[email protected]>
Co-authored-by: Xiaoyu Zhang <[email protected]>
Co-authored-by: yang1002378395-cmyk <[email protected]>
Co-authored-by: Mick <[email protected]>
Co-authored-by: Bi Xue <[email protected]>
Co-authored-by: huangtingwei <[email protected]>
Co-authored-by: Lianmin Zheng <[email protected]>
Co-authored-by: Baizhou Zhang <[email protected]>
Co-authored-by: Muqi Li <[email protected]>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Qiaolin Yu <[email protected]>
Co-authored-by: narutolhy <[email protected]>
Co-authored-by: Ethan (Yusheng) Su <[email protected]>
Co-authored-by: zhangxiaolei <[email protected]>
Co-authored-by: Vladislav Nosivskoy <[email protected]>
Co-authored-by: Trevor Morris <[email protected]>
Co-authored-by: Eitan Turok <[email protected]>
Co-authored-by: Fengyuan Yu <[email protected]>
Co-authored-by: Fengyuan Yu <[email protected]>
Co-authored-by: Adarsh Shirawalmath <[email protected]>
Co-authored-by: Yuhao Yang <[email protected]>
Co-authored-by: Liangsheng Yin <[email protected]>
Co-authored-by: Jianying <[email protected]>
Co-authored-by: jianyingzhu <[email protected]>
Co-authored-by: Kangyan-Zhou <[email protected]>
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
Co-authored-by: Jacob0226 <[email protected]>
Co-authored-by: Aditya Sharma <[email protected]>
Co-authored-by: Yuan Luo <[email protected]>
Co-authored-by: Xinyuan Tong <[email protected]>
Co-authored-by: Артем Савкин <[email protected]>
Co-authored-by: Tamir Baydasov <[email protected]>
Co-authored-by: Shu Wang <[email protected]>
Co-authored-by: eigen <[email protected]>
Co-authored-by: Avery Huang <[email protected]>
Co-authored-by: jacky.cheng <[email protected]>
Co-authored-by: HaiShaw <[email protected]>
Co-authored-by: luoyuan.luo <[email protected]>
Co-authored-by: Shangming Cai <[email protected]>
Co-authored-by: Junrong Lin <[email protected]>
Co-authored-by: Simon (Jiyou) Li <[email protected]>
Co-authored-by: 继优 <[email protected]>
Co-authored-by: shuwenn <[email protected]>
Co-authored-by: psaab <[email protected]>
Co-authored-by: hnyls2002 <[email protected]>
Co-authored-by: Hanlin Bi <[email protected]>
Co-authored-by: wili <[email protected]>
Co-authored-by: saatwiknagpal <[email protected]>
Co-authored-by: Mohammad Miadh Angkad <[email protected]>
Co-authored-by: wan4ch <[email protected]>
Co-authored-by: Feng Su <[email protected]>
Co-authored-by: Ying Sheng <[email protected]>
Co-authored-by: Polisetty V R K Jyothendra Varma <[email protected]>
Co-authored-by: Ziang Li <[email protected]>
Co-authored-by: Aishwarya Ramasethu <[email protected]>
Co-authored-by: Ma Mingfei <[email protected]>
Co-authored-by: blzheng <[email protected]>
Co-authored-by: kk <[email protected]>
Co-authored-by: wunhuang <[email protected]>
Co-authored-by: Michelle Wu <[email protected]>
Co-authored-by: wuxue (C) <[email protected]>
Co-authored-by: Hubert Lu <[email protected]>
Co-authored-by: strgrb <[email protected]>
Co-authored-by: LiYomi <[email protected]>
Co-authored-by: mengxiancheng03 <[email protected]>
Co-authored-by: GXIN <[email protected]>
Co-authored-by: 高鑫 <[email protected]>
Co-authored-by: heziiop <[email protected]>
Co-authored-by: xieminghe1 <[email protected]>
Co-authored-by: undefined <[email protected]>
Co-authored-by: Makcum888e <[email protected]>
Co-authored-by: yuefeng Wu <[email protected]>
Co-authored-by: Yuxuan Zhang <[email protected]>
Co-authored-by: Vedant V Jhaveri <[email protected]>
Co-authored-by: ronnie_zheng <[email protected]>
Co-authored-by: Zhai Feiyue <[email protected]>
Co-authored-by: jhchouuu <[email protected]>
Co-authored-by: Alison Shao <[email protected]>
Co-authored-by: Alison Shao <[email protected]>
Co-authored-by: DarkSharpness <[email protected]>
Co-authored-by: Alison Shao <[email protected]>
Co-authored-by: Lewis <[email protected]>
Co-authored-by: 百麒 <[email protected]>
Co-authored-by: Jincong Chen <[email protected]>
Co-authored-by: xiazhahe <[email protected]>
Co-authored-by: Thomas Wang <[email protected]>
Co-authored-by: Ke Bao <[email protected]>
Co-authored-by: xiaoqi <[email protected]>
Co-authored-by: xiaoqi.31 <[email protected]>
Co-authored-by: R0CKSTAR <[email protected]>
Co-authored-by: weireweire <[email protected]>
Co-authored-by: Weiliangl User <[email protected]>
Co-authored-by: JD <[email protected]>
Co-authored-by: Zhangheng <[email protected]>
Co-authored-by: Michael <[email protected]>
Co-authored-by: Yilong Zhao <[email protected]>
Co-authored-by: Johnsonms <[email protected]>
Co-authored-by: Brayden Zhong <[email protected]>
Co-authored-by: Chang Su <[email protected]>
Co-authored-by: KnightLTC <[email protected]>
Co-authored-by: Douglas Yang <[email protected]>
Co-authored-by: Karan Bansal <[email protected]>
Co-authored-by: karanb192 <[email protected]>
Co-authored-by: R0CKSTAR <[email protected]>
Co-authored-by: sglang-bot <[email protected]>
Co-authored-by: sglang-bot <[email protected]>
Co-authored-by: Xinyuan Tong <[email protected]>
Co-authored-by: sbeurnier <[email protected]>
Co-authored-by: YC Yen-Ching Tseng <[email protected]>
Co-authored-by: Wenyao Gao <[email protected]>
Co-authored-by: Alex Nails <[email protected]>
Co-authored-by: Zhiqiang Xie <[email protected]>
Co-authored-by: yudian0504 <[email protected]>
Co-authored-by: yunkchen <[email protected]>
Co-authored-by: wduan-hai <[email protected]>
Co-authored-by: Cherry_ming <[email protected]>
Co-authored-by: Ratish P <[email protected]>
Co-authored-by: YAMY <[email protected]>
Co-authored-by: Alison Shao <[email protected]>
Co-authored-by: ishandhanani <[email protected]>
Co-authored-by: Derek Yu <[email protected]>
Co-authored-by: Noa Neria <[email protected]>
Co-authored-by: Hanlin Bi <[email protected]>
Co-authored-by: Prozac614 <[email protected]>
Co-authored-by: David Cheung <[email protected]>
Co-authored-by: Mook <[email protected]>
Co-authored-by: Khoa Pham <[email protected]>
Co-authored-by: foraxe <[email protected]>
Co-authored-by: yunzhi <[email protected]>
Co-authored-by: DarkSharpness <[email protected]>
Co-authored-by: Todobe <[email protected]>
Co-authored-by: ori <[email protected]>
Co-authored-by: Thomas <[email protected]>
Co-authored-by: zhangshuai (S) <[email protected]>
Co-authored-by: lviy <[email protected]>
Co-authored-by: Tingwei Huang <[email protected]>
Co-authored-by: Yuzhen Zhou <[email protected]>
Co-authored-by: Ricardo-M-L <[email protected]>
Co-authored-by: Kelon <[email protected]>
Co-authored-by: cen121212 <[email protected]>
Co-authored-by: silencejade <[email protected]>
Co-authored-by: shadowxz109 <[email protected]>
Co-authored-by: Henson-Zh-Ali <[email protected]>
Co-authored-by: Rain Jiang <[email protected]>
Co-authored-by: David Wang <[email protected]>
Co-authored-by: Jian Chen <[email protected]>
Co-authored-by: Zhijian Liu <[email protected]>
Co-authored-by: Richard Gong <[email protected]>
Co-authored-by: yilian49 <[email protected]>
Co-authored-by: xm:D <[email protected]>
Co-authored-by: Thomas Wang <[email protected]>

…ch logic

9 stages, all green.

A. sglang `_aginfer_link_stats` emission contract
   A0 4 directions (HBM↔DRAM, DRAM↔DISK) × 3 keys each
   A1 cold-start recent_throughput_bps == 0 (pre-T26)
   A2 cold-start time_since_last_sample_s > LINK_IDLE_SECONDS
   A3 peak_bw_bps > 0 for every direction

B. Daemon bw_free branch in build_paper_state
   B0 idle link (time_since > 1.0) → bw_free = peak; stale `recent` ignored
   B1 busy link → bw_free = peak − recent
   B2 saturated link → bw_free clamps to 0 (no negative migration cost)
   B3 peak_bw_bps <= 0 → fatal(peak_bw_bps_non_positive); subprocess exit 1
   B4 idle threshold boundary: t_idle = 1.0 is NOT idle (predicate is `> 1.0`)

DEFERRED to T26 (PLAN §1 T13 + §3 T26 updated, task sgl-project#172 tracks):
   - compare recent_throughput_bps vs ground-truth wall-clock per
     migrate (idle + contended)
   - time_since_last_sample_s monotonicity on a quiet link

These need T26's HiCache/Mooncake measurement signal — there's no
ground truth pre-T26.  When T26 lands, the follow-on stages should
wire into verify/t13/ (or verify/t26/).

…tasks + T35 verify

Audit verdict per T (the 16 pending PLAN §3/§4 items):

DONE (covered by existing impl + tests):
  * T29 SGLANG_KV_POLICY_MODULE → _load_eviction_scorer exists;
    verify/t38/ stage C1 pins resolver
  * T32 all-rank atomic actions → tokenizer-server fanout exists;
    verify/t15/run_dp2_real.py exercises multi-rank wire
  * T35 authoritative_tier → baselines/base.py + new verify/t35/
    (8 stages: all residence combinations + empty-raises)

STRUCTURAL (compile-time invariant, no dedicated verify needed):
  * T19 atomic unit visibility → page_aligned() called in insert
    path at unified_radix_cache.py:574/599/731; run-time pin
    deferred to integration_stress flavor B

PARTIAL (impl half done):
  * T24 HASH_COLLISION → detection live with pair-dedupe; webhook
    firing pending ("T24 webhook pending" log) → sgl-project#182
  * T25 idempotent endpoints → migrate ✓, threshold ✓, pause/hint
    gated on T21/T40

OPEN — new tracking tasks created:
  * sgl-project#180  T18 state-dump single-snapshot read-lock
  * sgl-project#181  T21 PUT /aginfer/program_paused endpoint
  * sgl-project#182  T24 wire HASH_COLLISION webhook firing
  * sgl-project#183  T30+T39 proxy disconnect (paired)
  * sgl-project#184  T40 F2 hint emitter (daemon → sglang)
  * sgl-project#185  T41 F5 SESSION_END-for-PAUSED handler

OPEN — already tracked elsewhere:
  * T26 (HiCache/Mooncake EMA) → sgl-project#172 (T13 deferred)
  * T27 hint clear ordering → blocks on T40 (sgl-project#184)
  * T28 should_write_through → sgl-project#178

OUT OF SCOPE:
  * T31 harbor /aginfer/session_end — lives in harbor/client,
    not in sglang/daemon codebase

PLAN.md updated with per-T status notes referencing the audit
date + task IDs.  verify/t35/ added (8 stages, all green).

…first conflict was add/add

Merge 2eced7a brings tip 1357538: block-scaled fp8 dequant, the
CustomAllreduce construction fix under HTCCL, draft-solo placement and
graph-plan exemptions, group-wide spec-kernel capability dispatch, and the
Nordstern L0 scripts. 21 files, +1831/-25. Rollback tag pre-merge2-r3.

First textual conflict on this branch, and it was an add/add in
test_uneven_tp_args.py -- my two standalone tests against their
TestTreeSpecDcpGuardHardenedForNewFlagPaths class. Independent, both kept, their
import already present; resolved file passes 80 tests.

The two files touched by BOTH lines were inspected BEFORE merging, since every
earlier merge here hid a semantic conflict under zero textual ones. fp8.py:
their needs_device_kernel() drops the floor for block-scaled fp8 (now has a
plain-torch route) and my sgl-project#171 supports_current_device() sits below it --
compatible, and on CUDA the hook returns None so the numeric path is untouched.
model_runner.py: their draft-solo exemption at ~1085 vs my bf16 helper at ~297
and its call site at ~1700 -- disjoint.

Regression gate was arm E, because a9ced80 and the parallel_state.py delta
touch the HTCCL path it exercises:

  GREEN, capture end 3.78/3.80/3.81 s (band 3.76-3.81 over the last four boots)
  HTCCL device transport up, capturable; ownership weights [2, 1, 2]
  "plan differs": 0; health 200
  accepts code 2.909-3.459, prose 3.200/3.048/3.325, mixed 3.325/3.325/3.459

So the CustomAllreduce change did not disturb the NVIDIA path. The 3 of 9
char-loop flags are the known validator-threshold artefact, not a regression.

No JIT source touched, so the sgl-project#172 cold-build hazard did not apply; 0 incomplete
cache dirs before and after.

…URES_VS_UPSTREAM

…idated on the main rig, three merges green

Phase 1 (sgl-project#173/sgl-project#169, feat/triton-weighted-dcp). G1 index math on device 6
passed/33 subtests. G2: the 27B uneven-DCP arm now boots on --attention-backend
triton (previously refused outright), while the spec and weightless refusals
still fire naming themselves; MLA and SWA are not GPU-reachable here and stay
CPU-pinned. G4, the parity anchor, is green in BOTH eager and CUDA-graph decode:
eager and graphs are byte-identical within each backend, which is the D3
capture-stable buffer contract, and short_code is byte-identical across all four
cells. G5 (kv >= tp, the only path that issues the write-side kv-head gather)
and G6 (1-2 token prefix under [13,30,21], the D5 empty-shard provocation, 4/4
completed on both backends) are green. sgl-project#169 boots 2 and 5 are red at startup
with their exact messages, boot 6 boots and serves coherently, boot 4 keeps
max_total_num_tokens=98328 and accepts in regime.

Two measurement corrections are recorded rather than the conclusions they would
have produced. The first chunking prompt was a 40x-repeated paragraph, which put
the model on a knife edge and made both arms "diverge at token 4"; replacing it
with non-repeating text and adding a self-determinism control showed that
flashinfer diverges from ITSELF at the same character 43 across two identical
requests, as does the DCP-off baseline, while triton is the only self-
deterministic one there. The between-backend gap is inside the within-backend
spread, so that prompt cannot discriminate the backends. No first-token
divergence and zero non-ASCII output anywhere.

sgl-project#169 boot 3 is red, but via boot 2's guard. Root-caused rather than reinterpreted:
the token vector is only installed by scheduler.py:4949, which is gated on
base_plan is not None, so without a --rank-tp-ratio the vector never installs and
resolve_cp_token_ratios -- which holds the "vector without a plan" honesty guard
-- is never called. The gate is cc4ff87 and is byte-identical on
integration/r3-probe, so this is pre-existing. Consequences registered as open:
that honesty guard is unreachable from the boot path, and branch (2) of the
triton geometry guard is dead code at boot.

Phase 2 (sgl-project#172, fix/jit-coldbuild-robustness), falsifier first. The pre-fix tree
reproduces on a cold cache: capture begins, cudaErrorLaunchFailure exactly 30 s
later, surfacing in fused_add_rmsnorm_cute -- the symptom, not the site. The
fixed tree on the same cold-cache setup boots GREEN, logs the cold-build window
on all three ranks, and spends the nvcc time inside it (capture 174.9 s); the
warm second boot captures in 3.83/3.85/3.86 s. A boot killed with -9 while cicc
compiled gptq_marlin_repack for compute_86 left the documented residue, and the
next boot discarded it by name and rebuilt with no manual intervention. The
escape hatch is exact: MULT=1 restores the old deadline constant identically.

Honest limit: the pre-fix tree booted green (eager) on a copy of that same
residue, because ninja finished the interrupted build. That residue shape is
therefore recoverable and the control does not prove the pre-fix tree is broken
by it; the fatal shape stays covered by the unit tests.

Phase 3. Merged fix/dcp-geometry-leftovers (c0395f9), feat/triton-weighted-dcp
(bc32749) and fix/jit-coldbuild-robustness (7cb555a). One conflict, the
predicted doc tail, both sides kept. The branch had moved under this work by two
sgl-project#171/Vega commits touching model_runner.py and test_bf16_fallback_vendor.py,
which none of the merged branches touches. On the merge tip the merged suites are
80 passed/167 subtests and the other line's tests 32 passed. Integrated arm E,
warm: capture 3.88/3.89/3.90 s, ownership [2,1,2], max_total_num_tokens=98328,
plan differs 0, health 200, accepts 3.571/2.857/2.740.

Device-order note, since --rank-gpu-id indexes TORCH order and the two disagree
here: torch 0=5090/1=3080/2=3080 against nvml 0=3080/1=5090/2=3080. Resolved at
runtime for every boot, never assumed.