Skip to content

dev/mock-llm.mjs: 长任务 N with K omitted emits no text block per step — .map(Number) turns the missing K capture into NaN, defeating the size = 4 default #55

Closed

Description

@BlueX888

What happened?

A 长任务 N turn in the dev mock model (ds-bot/dev/mock-llm.mjs) with the optional K omitted — documented default 4 — does not grow the context after the opener step. Every step after the opener returns no text block at all: 0 characters of 步骤N filler instead of the documented ~4000. There is no error to copy; the per-step filler is silently missing, so the multi-step context-growth scenario this command exists for does nothing.

  • What I did: started the mock model with node dev/mock-llm.mjs and drove its /v1/messages endpoint through a 长任务 3 turn (K omitted), answering each list_bots tool call with an empty result.
  • What I expected: steps 1 and 2 each add about 4000 characters of filler. dev/mock-llm.mjs:19 documents 长任务 N [K] (N steps of K×1000, default 4), and the branch comment at dev/mock-llm.mjs:159 repeats "(default 4)".
  • What happened instead: the opener (步骤0) correctly emits 4000 characters, but every later step emits no text block; only the closing 长任务做完了,一共 3 步。 line arrives. With K given explicitly (长任务 3 4) the same steps work and each emits 4000 characters.

Actual output, verbatim from two independent runs of the reproduction script below:

opener step0 text length: 4000 (expected ~4000, 步骤0 filler)
step1 text: null
step1 text length: 0 (documented: ~4000 chars of 步骤1 filler; default K = 4)
control (K given as 4): step1 text length: 4000
=== K omitted ("长任务 3", documented default K=4) ===
[长任务 3] step 1: textBlocks=0 chars=0 toolCalls=list_bots
[长任务 3] step 2: textBlocks=0 chars=0 toolCalls=list_bots
[长任务 3] step 3: textBlocks=1 chars=14 firstText="长任务做完了,一共 3 步。" toolCalls=none
---
K omitted total text chars: 4014 (documented: 3 steps x ~4000 + finisher)
K=4 explicit total text chars: 12014
VERDICT: K-omitted turn is missing its per-step filler

(The 4014 total is just the 4000-character opener plus the 14-character finisher; the two missing per-step fillers would have added about 8000 more.)

Steps to reproduce

From a checkout of this repository at commit bdc5fda64e2ea1e83a4d146440072a2413025df4, save and run this script with Node (node repo-longtask-e2e.mjs). It starts the mock model itself on port 8791 and talks to its /v1/messages endpoint; no DSH install and no API key are involved.

// repo-longtask-e2e.mjs — "长任务 3" with K omitted must grow the context ~4k
// characters per step; instead every step after the opener returns no text block.
import { spawn } from 'node:child_process'

const PORT = 8791
const child = spawn('node', ['dev/mock-llm.mjs'], {
  cwd: 'ds-bot', // the plugin folder inside this repository
  env: { ...process.env, MOCK_PORT: String(PORT), MOCK_DELAY_MS: '10' },
  stdio: ['ignore', 'pipe', 'pipe'],
})
await new Promise(resolve => child.stdout.on('data', resolve))

async function call(messages) {
  const response = await fetch(`http://127.0.0.1:${PORT}/v1/messages`, {
    method: 'POST',
    headers: { 'content-type': 'application/json' },
    body: JSON.stringify({ model: 'mock-1', stream: true, max_tokens: 8192, messages }),
  })
  const text = await response.text()
  const events = [...text.matchAll(/data: (.+)\n/g)].map(m => JSON.parse(m[1]))
  const blocks = []
  for (const event of events) {
    if (event.type === 'content_block_start') blocks.push({ ...event.content_block, index: event.index })
    if (event.type === 'content_block_delta' && event.delta.type === 'text_delta') blocks[event.index].text = (blocks[event.index].text ?? '') + event.delta.text
    if (event.type === 'content_block_delta' && event.delta.type === 'input_json_delta') blocks[event.index].input = JSON.parse(event.delta.partial_json)
    if (event.type === 'message_delta') blocks.stop_reason = event.delta.stop_reason
  }
  return blocks
}

// 1. opener turn
const first = await call([{ role: 'user', content: '长任务 3' }])
const step0 = first.find(b => b.type === 'text')
const toolUse = first.find(b => b.type === 'tool_use')
console.log('opener step0 text length:', step0.text.length, '(expected ~4000, 步骤0 filler)')

// 2. first tool result -> should be 步骤1 filler (~4000 chars), actually is empty
const second = await call([
  { role: 'user', content: '长任务 3' },
  { role: 'assistant', content: [step0, { type: 'tool_use', id: toolUse.id ?? 'call-1-1', name: 'list_bots', input: toolUse.input ?? {} }] },
  { role: 'user', content: [{ type: 'tool_result', tool_use_id: toolUse.id ?? 'call-1-1', content: '[]' }] },
])
const step1 = second.find(b => b.type === 'text')
console.log('step1 text:', JSON.stringify(step1?.text ?? null))
console.log('step1 text length:', step1?.text?.length ?? 0, '(documented: ~4000 chars of 步骤1 filler; default K = 4)')

// 3. control: with K given explicitly, the same step works
const third = await call([
  { role: 'user', content: '长任务 3 4' },
  { role: 'assistant', content: [{ type: 'text', text: 'x' }, { type: 'tool_use', id: 'call-2-1', name: 'list_bots', input: {} }] },
  { role: 'user', content: [{ type: 'tool_result', tool_use_id: 'call-2-1', content: '[]' }] },
])
const step1k = third.find(b => b.type === 'text')
console.log('control (K given as 4): step1 text length:', step1k.text.length)

child.kill()
process.exit(0)

It prints the first output block above: opener 4000, step 1 missing (null, 0 chars), control 4000.

Diagnostics

  • Repository commit: bdc5fda64e2ea1e83a4d146440072a2413025df4 ("Let every Bot update its own settings with update_bot"), main branch.
  • Node.js v22.23.2, macOS 26.6.2 (arm64).
  • Settings → Feedback diagnostics do not apply here: the defect is in the dev mock model itself (ds-bot/dev/mock-llm.mjs), reached by running it directly; no DSH install, no real model, no API key, and no error text — the failure is silent.

How did you install DS Bot?

Git address (a development checkout of this repository itself, for working on the plugin).

Privacy

I checked that this report has no API keys, tokens or personal data.

Root cause

ds-bot/dev/mock-llm.mjs:162, in the tool-result branch of decide():

const [, steps, size = 4] = /^长任务\s*(\d+)(?:\s+(\d+))?/.exec(textOf(messages[openerAt].content)).map(Number)

For 长任务 3, .exec() returns ['长任务 3', '3', undefined], and .map(Number) turns the omitted K capture into NaN. A destructuring default only replaces undefined, not NaN, so size = 4 never applies. Then filler(NaN * 1000, ...) (dev/mock-llm.mjs:78) builds Array.from({ length: Math.ceil(NaN / 20) }) — an empty array — and returns '', and the SSE writer skips empty text blocks entirely (if (block.text === '') continue, dev/mock-llm.mjs:541), so the step emits only the tool call and no text.

Expected behaviour, per the file's own documentation (dev/mock-llm.mjs:19 and the comment at dev/mock-llm.mjs:159): the omitted K falls back to 4, and the sibling opener branch at dev/mock-llm.mjs:341 already parses the same regex correctly with Number(order[2] ?? 4) — which is why the opener step works with K omitted.

I have a small fix ready that mirrors that same ?? 4 pattern (keep the match object, convert each capture separately), and am happy to be assigned to it.

Related issues

I searched the open and closed issues and pull requests for 长任务, mock-llm and NaN, and found no existing report of this.

Activity

  1. BlueX888 commented on Oct 12, 2026

    @BlueX888
    Author

    Reproduced this. Root cause: ds-bot/dev/mock-llm.mjs:162 — .map(Number) on the regex match turns the missing K capture into NaN, so the size = 4 default never applies and filler(NaN * 1000) returns an empty text block that the SSE writer skips.

    Minimal reproduction:

    node repo-longtask-e2e.mjs   # the reproduction script from the report, run from the repo root; it starts dev/mock-llm.mjs itself on port 8791
    

    Observed output:

    opener step0 text length: 4000 (expected ~4000, 步骤0 filler)
    step1 text: null
    step1 text length: 0 (documented: ~4000 chars of 步骤1 filler; default K = 4)
    control (K given as 4): step1 text length: 4000
    

    Happy to open a PR with this fix, or to be assigned if you prefer to take it.

  2. BlueX888 commented on Oct 12, 2026

    @BlueX888
    Author

    Since pull requests are currently limited to collaborators: the fix from my last comment also sits on my fork — BlueX888/DeepSeek-Bot, branch fix/prep-mock-llm-longtask-nan-size (commit 9f7f6c1). Verified red→green there: the reproduction above goes from step1 text length: 0 to 4000, and a regression test (a 长任务 turn grows every step even with K left at its default, test/mock-llm.test.js) passes with the rest of the suite at the main baseline. Feel free to pull from that branch.

  3. Fei2411 commented on Oct 12, 2026

    @Fei2411
    Owner

    已完成修复

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions