Summary
A single request carrying a negative token id in the token-id form of the input field of /v1/embeddings or /pooling terminates the vLLM engine. The request is answered HTTP 500 and the server then serves nothing further: /health refuses the connection and every subsequent request from every client fails until the process is restarted.
No authentication, no prior request, and no unusual value are required. -1 is enough.
Details
/v1/embeddings and /pooling accept token ids directly from the caller (input: list[int] | list[list[int]] | str | list[str] on CompletionRequestMixin, in vllm/entrypoints/pooling/base/protocol.py). The engine validated only the upper bound of those ids against the vocabulary, in InputProcessor._validate_model_input:
if max_input_id > max(tokenizer.max_token_id, model_vocab_size - 1):
raise ... # "Token id ... is out of vocabulary" -> HTTP 400
There was no corresponding lower-bound check, so a negative id passed validation and was used directly as an index into the embedding table on the GPU. That trips a device-side assertion inside PyTorch's index kernel:
aten/src/ATen/native/cuda/Indexing.cu:1553: indexSelectSmallIndex:
block: [4,0,0], thread: [32,0,0]
Assertion `srcIndex < srcSelectDimSize` failed.
This is not a catchable Python IndexError. A CUDA device-side assertion poisons the CUDA context: after it fires, every subsequent CUDA operation in the process fails, including operations belonging to unrelated requests. That is why the process cannot recover and why /health refuses the connection rather than answering 503. A try/except around the forward pass cannot turn this into a 4xx — the ids have to be bounded before they reach the device.
This is not a PyTorch defect. The assertion is PyTorch's protection working as designed; without it an out-of-range id would read unmapped memory. PyTorch checks index values host-side only on CPU, because doing so on CUDA would require a synchronization on every indexing op (pytorch/pytorch#71572). The resulting contract is that callers must validate indices before handing them over, and vLLM was not doing so on this path.
Proof of concept
Against a freshly started server (the first request is a control showing that the token-id input form is legal for this endpoint):
M=Qwen/Qwen3-Embedding-0.6B
vllm serve $M --port 8000 --max-model-len 2048 --enforce-eager &
# 1. control -- valid token ids
curl -sS -X POST http://localhost:8000/v1/embeddings \
-H 'Content-Type: application/json' \
-d "{\"model\":\"$M\",\"input\":[[9707,1879]]}"
# -> 200, /health -> 200
# 2. the same request with one id made negative
curl -sS -X POST http://localhost:8000/v1/embeddings \
-H 'Content-Type: application/json' \
-d "{\"model\":\"$M\",\"input\":[[-1,1879]]}"
# -> 500 {"error":{"message":"EngineCore encountered an issue. See stack trace
# (above) for the root cause.","type":"InternalServerError","code":500}}
# then /health refuses the connection and the server is gone
/pooling behaves identically. Either list position triggers it, and -1 behaves the same as INT64_MIN, so this is any negative id rather than a boundary value.
Impact
Any deployment that serves a pooling or embedding model through the OpenAI-compatible API server. One unauthenticated HTTP request is a complete denial of service for that server process and every client using it. Restarting under a supervisor does not mitigate it, because the same request can be sent again immediately.
Confidentiality and integrity are unaffected; the impact is availability only.
Verification
Measured on stock upstream release images (vllm/vllm-openai, aarch64), one request per freshly started server, with a valid control request immediately before the negative one:
| build |
endpoint |
control |
negative id -1 |
/health after |
valid request after |
device-side asserts in log |
| v0.27.1 (unpatched) |
/v1/embeddings |
200 |
500 |
connection refused |
connection refused |
3 |
| v0.27.1 (unpatched) |
/pooling |
200 |
500 |
connection refused |
connection refused |
3 |
| v0.28.0 (patched) |
/v1/embeddings |
200 |
400 Token id -1 is out of vocabulary |
200 |
200 |
0 |
| v0.28.0 (patched) |
/pooling |
200 |
400 Token id -1 is out of vocabulary |
200 |
200 |
0 |
INT64_MIN (-9223372036854775808) behaves identically to -1 on both builds, and either position in the id list triggers it, so this is any negative id rather than a boundary value.
On v0.27.1 the engine process reports:
torch.AcceleratorError: CUDA error: device-side assert triggered
with the underlying kernel assertion being
aten/src/ATen/native/cuda/Indexing.cu:1553: indexSelectSmallIndex:
Assertion `srcIndex < srcSelectDimSize` failed.
Patches
Fixed in 0.28.0 by #51795 (commit b216db3ed0), which adds the missing lower-bound check to InputProcessor._validate_model_input so a negative id is rejected as HTTP 400 before the ids reach the device:
if min_input_id < 0:
raise VLLMValidationError(f"Token id {min_input_id} is out of vocabulary")
The validation path is shared by generate, embedding and pooling requests, so the check covers all three.
Workarounds
For deployments that cannot upgrade to 0.28.0:
- Reject request bodies containing negative integers in
input at a reverse proxy or API gateway in front of vLLM.
- Do not expose the token-id form of the input; accept only string input from untrusted callers.
- Restrict network access to
/v1/embeddings and /pooling.
Process supervision is not a workaround: the request can simply be repeated.
Not affected
/v1/completions — its prompt field constrains token ids with Field(ge=0), so pydantic rejects a negative id before the engine sees it. That constraint has been present since v0.17.0, and was confirmed on an unpatched v0.27.1 build: CompletionRequest(model=..., prompt=[[-1, 5]]) raises ValidationError.
/score and /rerank — these do not accept a token-id input form at all.
References
Summary
A single request carrying a negative token id in the token-id form of the
inputfield of/v1/embeddingsor/poolingterminates the vLLM engine. The request is answered HTTP 500 and the server then serves nothing further:/healthrefuses the connection and every subsequent request from every client fails until the process is restarted.No authentication, no prior request, and no unusual value are required.
-1is enough.Details
/v1/embeddingsand/poolingaccept token ids directly from the caller (input: list[int] | list[list[int]] | str | list[str]onCompletionRequestMixin, invllm/entrypoints/pooling/base/protocol.py). The engine validated only the upper bound of those ids against the vocabulary, inInputProcessor._validate_model_input:There was no corresponding lower-bound check, so a negative id passed validation and was used directly as an index into the embedding table on the GPU. That trips a device-side assertion inside PyTorch's index kernel:
This is not a catchable Python
IndexError. A CUDA device-side assertion poisons the CUDA context: after it fires, every subsequent CUDA operation in the process fails, including operations belonging to unrelated requests. That is why the process cannot recover and why/healthrefuses the connection rather than answering 503. Atry/exceptaround the forward pass cannot turn this into a 4xx — the ids have to be bounded before they reach the device.This is not a PyTorch defect. The assertion is PyTorch's protection working as designed; without it an out-of-range id would read unmapped memory. PyTorch checks index values host-side only on CPU, because doing so on CUDA would require a synchronization on every indexing op (pytorch/pytorch#71572). The resulting contract is that callers must validate indices before handing them over, and vLLM was not doing so on this path.
Proof of concept
Against a freshly started server (the first request is a control showing that the token-id input form is legal for this endpoint):
/poolingbehaves identically. Either list position triggers it, and-1behaves the same asINT64_MIN, so this is any negative id rather than a boundary value.Impact
Any deployment that serves a pooling or embedding model through the OpenAI-compatible API server. One unauthenticated HTTP request is a complete denial of service for that server process and every client using it. Restarting under a supervisor does not mitigate it, because the same request can be sent again immediately.
Confidentiality and integrity are unaffected; the impact is availability only.
Verification
Measured on stock upstream release images (
vllm/vllm-openai, aarch64), one request per freshly started server, with a valid control request immediately before the negative one:-1/healthafter/v1/embeddings/pooling/v1/embeddingsToken id -1 is out of vocabulary/poolingToken id -1 is out of vocabularyINT64_MIN(-9223372036854775808) behaves identically to-1on both builds, and either position in the id list triggers it, so this is any negative id rather than a boundary value.On v0.27.1 the engine process reports:
with the underlying kernel assertion being
Patches
Fixed in 0.28.0 by #51795 (commit
b216db3ed0), which adds the missing lower-bound check toInputProcessor._validate_model_inputso a negative id is rejected as HTTP 400 before the ids reach the device:The validation path is shared by generate, embedding and pooling requests, so the check covers all three.
Workarounds
For deployments that cannot upgrade to 0.28.0:
inputat a reverse proxy or API gateway in front of vLLM./v1/embeddingsand/pooling.Process supervision is not a workaround: the request can simply be repeated.
Not affected
/v1/completions— itspromptfield constrains token ids withField(ge=0), so pydantic rejects a negative id before the engine sees it. That constraint has been present since v0.17.0, and was confirmed on an unpatched v0.27.1 build:CompletionRequest(model=..., prompt=[[-1, 5]])raisesValidationError./scoreand/rerank— these do not accept a token-id input form at all.References