json_schema responses: the grammar allows 64 whitespace characters before the JSON value, and a client cannot ask for compact output #730

Summary

With a json_schema response format, the grammar lets the model write up to 64 whitespace characters before the JSON value and in every gap between tokens. Qwen3.8-27B with its DFlash2 draft often uses that allowance, padding the answer with \r\n\n\n runs before the opening brace. Those tokens cost decode time, and with a small max_tokens the answer ends with finish_reason: "length" before any JSON is written. A client cannot ask for compact output, because _grammar_compatible_schema replaces any x-guidance object in the request's schema.

Reproduction

Splash 1.6.0 (also present on main at 9124ff2), mlx-community/Qwen3.8-27B-4bit with incoai/Qwen3.8-27B-DFlash2, --offline --language-only, macOS on Apple silicon.

{
  "model": "mlx-community/Qwen3.8-27B-4bit",
  "messages": [{"role": "user", "content": "Say ok."}],
  "max_tokens": 16,
  "temperature": 0,
  "chat_template_kwargs": {"enable_thinking": false},
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "answer",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": {"ok": {"type": "boolean"}},
        "required": ["ok"],
        "additionalProperties": false
      }
    }
  }
}

A different prompt on the same request ("Is the sky blue? Answer ok=true for yes.") gives {\n "ok": true\n} in 10 tokens, so the padding depends on the prompt rather than appearing every time.

Where it comes from

• server/tool_schema.py L215-217: WHITESPACE = [ \t\n\r]{0,64}. The comment above it already notes that Qwen models prefer whitespace, most of all under speculative decoding.
• L256: output["x-guidance"] = {"lenient": True, "whitespace_pattern": WHITESPACE} overwrites any x-guidance the client sent.
• L1021: start: {think}WS %json ... allows the same run again before the value.

Suggestion

Any one of these would let a client that wants compact JSON get it, without changing the default:

1. Keep a client-supplied x-guidance whitespace setting (whitespace_pattern, or whitespace_flexible: false) instead of replacing it, or merge it.
2. Add a server option (or a request field) for compact constrained JSON: no leading whitespace and single-space or empty gaps.
3. Drop the leading WS when no reasoning precedes the JSON. The value can then start at the first token, and the 64-character cap still applies inside.

For context, we run greedy, schema-constrained review requests through Splash. The answers are correct and the 64-character cap keeps the waste bounded, so this is an efficiency and predictability request, not a correctness bug.