[Bugfix][Parser] Stream DeepSeek V4/V3.2 string tool arguments incrementally - #58780

Purpose

Fixes #52846.

DeepSeek V4, V4.1 and V3.2 streamed a long string tool argument as one fragment when </｜DSML｜parameter> arrived. The configs set arg_structural_chars=frozenset(">"), so the engine skipped every plain body delta.

The fix now uses the per-parser arg_value_extender hook, the same way GLM (#61253) and Gemma4 (#61494) do. The earlier engine-level gate change is dropped.

• vllm/parser/deepseek_v4.py: _dsml_arg_value_extender returns an extender while the last parameter is an open string="true" value. Each delta streams as escaped JSON text without rerunning the converter. It stops on a parameter tag, keeping a tail so a tag split across deltas is caught.
• Wired into deepseek_v4_config, deepseek_v32_config, and deepseek_v41_config (spaced DSML tags via functools.partial).
• _param_patterns: the trailing-partial regex holds back an unfinished next <｜DSML｜parameter tag, so a streamed value never includes it (implicit close case).

Test Plan

New tests:

• tests/tool_parsers/test_deepseekv4_tool_parser.py: test_streaming_long_string_argument_is_incremental (4096-char body in 32-char chunks, needs >1 non-empty body delta and exact round-trip), test_streaming_non_string_argument_is_withheld_until_close, test_streaming_string_argument_with_markup_is_incremental.
• tests/tool_parsers/test_deepseekv32_tool_parser.py::TestExtractToolCallsStreaming::test_long_string_argument_streams_before_close
• tests/parser/engine/test_deepseek_v41.py::test_long_string_argument_streams_before_close (spaced DSML)
• tests/parser/engine/test_deepseek_v4.py::TestArgValueExtender (split close tag, closed value, non-string value, repeated key)

Commands (CPU only):

pytest tests/tool_parsers/test_deepseekv4_tool_parser.py tests/tool_parsers/test_deepseekv32_tool_parser.py \
  tests/tool_parsers/test_deepseekv31_tool_parser.py tests/tool_parsers/test_deepseekv3_tool_parser.py

pytest tests/parser/engine/test_deepseek_v4.py tests/parser/engine/test_deepseek_v41.py \
  tests/parser/engine/test_glm47_moe.py tests/parser/engine/test_gemma4_streaming_reasoning.py \
  tests/parser/engine/test_parser_engine.py tests/tool_parsers/test_glm47_moe_tool_parser.py \
  tests/tool_parsers/test_gemma4_tool_parser.py tests/tokenizers_/test_deepseek_v41.py

pytest tests/parser tests/tool_parsers --ignore=tests/parser/cohere

ruff check <changed files>; ruff format --check <changed files>

Test Result

On this branch:

deepseek v3/v31/v32/v4 tool parsers:  82 passed, 1 xfailed
engine + glm47 + gemma4 + v41 set:     589 passed
tests/parser + tests/tool_parsers:     7672 passed, 1 skipped, 30 xfailed, 41 errors
ruff: All checks passed! / 6 files already formatted

The 41 errors are setup errors in test_harmony.py and test_llama3_json_tool_parser.py (gated or unreachable HF downloads on my machine). tests/parser/cohere was skipped: cohere_melody is not installed.

Same new tests against upstream/main parser code:

FAILED test_deepseekv4_tool_parser.py::test_streaming_long_string_argument_is_incremental - assert 0 > 1
FAILED test_deepseekv32_tool_parser.py::...::test_long_string_argument_streams_before_close - assert 0 > 1
FAILED parser/engine/test_deepseek_v41.py::test_long_string_argument_streams_before_close - assert 2 > (4096 // 64)

V4, 4096-char body in 128 chunks: 0/128 body chunks emit arguments on main, 128/128 on this branch.

AI disclosure: I used Claude Code to port the fix and write tests; I reviewed the diff and test results.