[Compatibility] Qwen3.8-27B via LiteLLM hosted_vllm: 400 "System message must be at the beginning." (openai-chat wire sends system+developer) #6674

Client or integration

Codex CLI

Provider or upstream service

LiteLLM -> hosted_vllm -> vLLM serving Qwen/Qwen3.8-27B (OpenAI-compatible endpoint)

OpenCodex version

2.78.0

Endpoint or capability

/v1/chat/completions (openai-chat adapter; Codex responses -> chat translation)

Current behaviour

When routing Codex to an OpenAI-compatible endpoint that serves Qwen3.8-27B through LiteLLM + vLLM, every request fails with HTTP 400.

litellm.BadRequestError: Hosted_vllmException - {"error":{"message":"System message must be at the beginning.","type":"BadRequestError","param":null,"code":400}}

The proxy (openai-chat adapter, messagesToChatFormat in src/adapters/openai-chat/messages.ts) emits a leading system message for the system prompt, then a separate developer message for Codex's developer/instructions turn:

[
  {
    "role": "system",
    "content": "<system prompt>"
  },
  {
    "role": "developer",
    "content": "<developer instructions>"
  },
  {
    "role": "user",
    "content": "..."
  }
]

Qwen3.8-27B's pinned vLLM chat template treats both system and developer as leading instruction blocks, so system followed by developer is rejected with "System message must be at the beginning."

Reproduced directly against the endpoint:

• [system, user] -> 200 OK
• [system, developer, user] -> 400 "System message must be at the beginning."
• [developer, user] (no leading system) -> 200 OK
• [system, user, user] (developer demoted to user) -> 200 OK

Expected behaviour

Any served name for Qwen3.8-27B (e.g. openai/Qwen3.8-27B, a common LiteLLM provider prefix) should work through the proxy. The developer instruction should either be folded into the single leading system message or emitted with a role the model accepts (user), instead of producing a leading system + developer pair.

Codex itself expects the agent/developer instructions to be preserved; the point is the wire encoding, not dropping the content.

Minimal redacted request or reproduction

curl https://llm.xxx.com/v1/chat/completions \
  -H "Authorization: Bearer <key>" -H "Content-Type: application/json" \
  -d '{"model":"openai/Qwen3.8-27B","messages":[
      {"role":"system","content":"You are a helpful assistant."},
      {"role":"developer","content":"developer instruction"},
      {"role":"user","content":"hi"}],
      "max_tokens":10}'


The same endpoint accepts `[system, user]` and `[developer, user]`.

opencodex provider config `Varmmego-Qwen3.8-27B`:
- adapter: openai-chat
- defaultModel: openai/Qwen3.8-27B
- foldDeveloperRoleToSystem: false

Actual response or error

HTTP 400
{"error":{"message":"litellm.BadRequestError: Hosted_vllmException - {\"error\":{\"message\":\"System message must be at the beginning.\",\"type\":\"BadRequestError\",\"param\":null,\"code\":400}}. Received Model Group=openai/Qwen3.8-27B\nAvailable Model Group Fallbacks=None","type":null,"param":null,"code":"400"}}

Upstream documentation

Qwen3.8-27B model: https://huggingface.co/Qwen/Qwen3.8-27B

The chat template is model-owned (pinned in the served vLLM tokenizer); the error originates from vLLM request pre-validation, not from LiteLLM. LiteLLM is only a thin OpenAI-compatible proxy in front of vLLM.

Suggested mapping or implementation notes

Incomplete workaround is the root cause

opencodex 2.78 ALREADY ships a model-specific workaround for this exact model. In src/adapters/openai-chat/messages.ts, qwen38LeadingSystemTemplate demotes a developer message to user when it is enabled, precisely because Qwen3.8-27B's pinned vLLM chat template rejects a non-leading system and every developer role.

However, the trigger regex only matches Qwen/Qwen3.8-27B or the bare Qwen3.8-27B:

const qwen38LeadingSystemTemplate = !nativeOpenAI && /^(?:Qwen\/)?(?:Qwen3\.8-27B|qwen3-8-27b(?:-(?:fp8|lora))?)$/i.test(parsed.modelId);

It does NOT match a namespaced id like openai/Qwen3.8-27B, which is the form LiteLLM exposes (the openai/ prefix comes from the LiteLLM provider config, not from Qwen). So for users routing Qwen3.8-27B through LiteLLM (a very common setup), the workaround never fires and the request still goes out as [system, developer, user] -> 400.

Net effect: the workaround exists for this model family but is ineffective for the most common deployment shape.

Suggested fix

Accept an optional provider prefix before the model name, e.g. /(?:^|\/)Qwen3\.8-27B$/i, or key the behaviour off the underlying served model rather than the full routed id.

Additional context and attachments

Reported from a Codex -> opencodex 2.78.0 -> LiteLLM -> vLLM setup on Windows.

Checks

• I searched existing provider and compatibility issues.
• The request and response were redacted.
• The expected behaviour is based on an upstream specification or a concrete client requirement.