Motivation

GenerateReqInput.extra_key is typed Optional[Union[List[str], str]], but batch normalization did not normalize/expand it and __getitem__ passed extra_key=self.extra_key to every sub-request instead of indexing it (issue #26755).

A batched request with a per-request list crashes downstream prefix-cache matching:

{"text": ["P", "P"], "extra_key": ["tenant-A", "tenant-B"], "sampling_params": [{"max_new_tokens": 1}, {"max_new_tokens": 1}]}

both sub-requests inherit extra_key=["tenant-A", "tenant-B"], and RadixKey.child_key() produces (['tenant-A', 'tenant-B'], 50) which fails with TypeError: unhashable type: 'list'.

Modifications

python/sglang/srt/managers/io_struct.py:

1. New _normalize_extra_key(num) wired into _normalize_batch_inputs, mirroring the existing _normalize_custom_logit_processor style:
   • None → unchanged
   • str → [str] * num (broadcast to every sub-request)
   • list of length batch_size → expanded by parallel_sample_num
   • list of mismatched length → ValueError with a clear message
   • Anything else → ValueError
2. __getitem__ now reads self.extra_key[i] when non-None and None otherwise — matches the custom_logit_processor / lora_path pattern already used a few lines above in the same method.

After normalization the invariant becomes: extra_key is None or len(extra_key) == batch_size * parallel_sample_num, identical to the convention used by lora_path / custom_logit_processor / logprob_params.

Tests

test/registered/unit/managers/test_io_struct.py::TestGenerateReqInputNormalization::test_extra_key_normalization covers:

• Per-request list → normalized list intact, __getitem__ returns scalar per index (the bug fix).
• Scalar string → broadcast to a list, __getitem__ returns the same string for every index.
• None → stays None, __getitem__ broadcasts None.
• parallel_sample_num=2 → list doubled by parallel_sample_num.
• Wrong-length list → ValueError with batch size in the message.
• Single (non-batched) request with scalar extra_key → unchanged.

Registered to base-b-test-cpu / stage-b-test-1-gpu-small-amd / cuda base-b 1-gpu-large via the file's existing register_*_ci(...) block.

Accuracy

Not applicable. No change to sampling, model forward, or scheduling. Only the routing of extra_key from the batched input to its sub-requests changes; downstream consumers (RadixKey, prefix cache) already expect a scalar per request.

Speed Tests and Profiling

Not applicable. Normalization adds one Python list operation per batched request at admission time; negligible vs. tokenization and prefill.

Checklist

• Format your code according to pre-commit.
• Add unit tests.
• Update documentation — N/A, no API change.
• Provide accuracy and speed benchmark results — N/A, see sections above.

Closes #26755.

CI States

Latest PR Test (Base): ⏳ Run #26981186663
Latest PR Test (Extra): ❌ Run #26981186627