Add nvidia/SPEED-Bench as a new dataset option in bench_serving.

Motivation

SPEED-Bench (SPEculative Evaluation Dataset) is a unified benchmark specifically designed to evaluate Speculative Decoding (SD) algorithms across diverse semantic domains and realistic serving regimes (paper). It combines two purpose-built dataset splits:

• Qualitative split — 880 semantically diverse prompts across 11 task domains (Coding, Math, Reasoning, RAG, etc.) to measure drafter acceptance rates and acceptance lengths.
• Throughput split — fixed-length input sequences (1K–32K tokens) categorized by output entropy (low_entropy / mixed / high_entropy) to assess system-level throughput under high concurrency across the compute-bound → memory-bound transition that governs speculative decoding cost-benefit trade-offs.

Adding it as a native dataset option enables reproducible, category-specific speculative decoding throughput benchmarking directly via bench_serving, without requiring custom preprocessing scripts.

Modifications

• New SpeedBenchDataset plugin (python/sglang/benchmark/datasets/speed_bench.py) that reads a pre-downloaded throughput_1k JSONL file, optionally filters by category (low_entropy / mixed / high_entropy), applies the model's chat template, and fixes the output length.
• Registers "speed-bench" in DATASET_MAPPING (python/sglang/benchmark/datasets/__init__.py).
• Adds --speed-bench-category and --speed-bench-output-len CLI args to bench_serving.py.

Example usage:

python -m sglang.bench_serving \
  --dataset-name speed-bench \
  --dataset-path /path/to/throughput_1k.jsonl \
  --speed-bench-category mixed \
  --speed-bench-output-len 512 \
  --num-prompts 80

Accuracy Tests

N/A — this PR only adds a dataset loader; it does not modify any model forward or kernel code.

Speed Tests and Profiling

N/A — this PR does not affect inference speed. The dataset plugin is used to drive existing bench_serving infrastructure.

Checklist

• Format your code according to the Format code with pre-commit. (all pre-commit hooks passed)
• Add unit tests according to the Run and add unit tests. Added to test/registered/bench_fn/test_benchmark_datasets_api.py: test_speed_bench_sampler, test_speed_bench_category_filter, test_speed_bench_output_len_override, test_speed_bench_empty_category_raises, test_speed_bench_no_path_raises, and coverage in test_dataset_mapping_and_dispatch.
• Update documentation according to Write documentations. Updated docs_new/docs/developer_guide/bench_serving.mdx: added speed-bench to the dataset list with a description, added its CLI flags (--speed-bench-category, --speed-bench-output-len), and added a numbered example for speculative decoding throughput benchmarking.
• Accuracy and speed benchmark results: N/A (dataset loader only).
• Follow the SGLang code style guidance.

CI States

Latest PR Test (Base): ✅ Run #26535922407
Latest PR Test (Extra): ❌ Run #26535921652