fix(agents): add cross-turn separator in deltaBuffer for blockStreaming#35344
fix(agents): add cross-turn separator in deltaBuffer for blockStreaming#35344lisitan wants to merge 3 commits intoopenclaw:mainfrom
Conversation
When a skill's SKILL.md frontmatter has name: 12306 (without quotes), YAML parses it as a number instead of a string. This causes TypeError when calling name.startsWith() or name.trim() on a number. This fix adds String() coercion in three places: - bundled-context.ts: skill.name.trim() check - config.ts: allowlist.includes() check - workspace.ts: normalized.includes() check Also adds comprehensive test coverage for numeric skill names. Note: The underlying @mariozechner/pi-coding-agent package has the same issue in validateName() function. Users should quote numeric skill names in YAML (name: "12306") until that package is fixed. Fixes openclaw#35252 Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <[email protected]> Co-Authored-By: Happy <[email protected]>
When blockStreaming is enabled, the deltaBuffer was being reset at message_start without preserving content from the previous turn. This caused text from consecutive API turns (e.g., before and after a tool call) to be concatenated without any separator, producing garbled output like "...with durableNO Fresh session...". This fix adds a "\n\n" separator before resetting the deltaBuffer if it contains content from a previous turn. The separator is added to both deltaBuffer and blockBuffer (or blockChunker) to ensure proper text separation in the streaming UI. After a page refresh, the history renders correctly, confirming this is purely a streaming display issue. Fixes openclaw#35308 Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <[email protected]> Co-Authored-By: Happy <[email protected]>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: adeb7d564d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| } | ||
| } | ||
|
|
||
| ctx.resetAssistantMessageState(ctx.state.assistantTexts.length); |
There was a problem hiding this comment.
Preserve turn separator across state reset
Appending "\n\n" just before resetAssistantMessageState(...) does not actually separate turns in streaming output, because the reset immediately clears deltaBuffer, blockBuffer, and the blockChunker buffer (src/agents/pi-embedded-subscribe.ts), so the separator is discarded before any later text_delta/drain can emit it. In blockStreaming sessions that span multiple assistant messages (for example around tool calls), this leaves the original concatenation bug unresolved.
Useful? React with 👍 / 👎.
Greptile SummaryThis PR bundles two distinct fixes: (1) a
Confidence Score: 1/5
Last reviewed commit: adeb7d5 |
| if (ctx.state.deltaBuffer.trim()) { | ||
| ctx.state.deltaBuffer += "\n\n"; | ||
| if (ctx.blockChunker) { | ||
| ctx.blockChunker.append("\n\n"); | ||
| } else { | ||
| ctx.state.blockBuffer += "\n\n"; | ||
| } | ||
| } | ||
|
|
||
| ctx.resetAssistantMessageState(ctx.state.assistantTexts.length); |
There was a problem hiding this comment.
Separator is immediately erased by resetAssistantMessageState
The \n\n is appended to all three buffers (deltaBuffer, blockBuffer/blockChunker) and then immediately cleared by the very next call to ctx.resetAssistantMessageState(...).
Looking at the implementation of resetAssistantMessageState in pi-embedded-subscribe.ts:
const resetAssistantMessageState = (nextAssistantTextBaseline: number) => {
state.deltaBuffer = ""; // clears the \n\n we just added
state.blockBuffer = ""; // clears the \n\n we just added
blockChunker?.reset(); // clears the \n\n we just appended
// ...
};And EmbeddedBlockChunker.reset() simply does this.#buffer = "", and append() only buffers without emitting — so blockChunker.append("\n\n") followed immediately by blockChunker.reset() discards the separator entirely.
As a result, the fix is a no-op: the \n\n separator is added and then erased before any streaming output can observe it. The bug (#35308) remains unfixed.
The separator needs to be added after resetAssistantMessageState, preserving the separator in the freshly-cleared buffers so the next turn's text is appended after it:
const hadPreviousTurnContent = ctx.state.deltaBuffer.trim().length > 0;
ctx.resetAssistantMessageState(ctx.state.assistantTexts.length);
if (hadPreviousTurnContent) {
ctx.state.deltaBuffer = "\n\n";
if (ctx.blockChunker) {
ctx.blockChunker.append("\n\n");
} else {
ctx.state.blockBuffer = "\n\n";
}
}
// Use assistant message_start as the earliest "writing" signal for typing.
void ctx.params.onAssistantMessageStart?.();Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-subscribe.handlers.messages.ts
Line: 76-85
Comment:
**Separator is immediately erased by `resetAssistantMessageState`**
The `\n\n` is appended to all three buffers (`deltaBuffer`, `blockBuffer`/`blockChunker`) and then **immediately cleared** by the very next call to `ctx.resetAssistantMessageState(...)`.
Looking at the implementation of `resetAssistantMessageState` in `pi-embedded-subscribe.ts`:
```typescript
const resetAssistantMessageState = (nextAssistantTextBaseline: number) => {
state.deltaBuffer = ""; // clears the \n\n we just added
state.blockBuffer = ""; // clears the \n\n we just added
blockChunker?.reset(); // clears the \n\n we just appended
// ...
};
```
And `EmbeddedBlockChunker.reset()` simply does `this.#buffer = ""`, and `append()` only buffers without emitting — so `blockChunker.append("\n\n")` followed immediately by `blockChunker.reset()` discards the separator entirely.
As a result, the fix is a no-op: the `\n\n` separator is added and then erased before any streaming output can observe it. The bug (#35308) remains unfixed.
The separator needs to be added **after** `resetAssistantMessageState`, preserving the separator in the freshly-cleared buffers so the next turn's text is appended after it:
```typescript
const hadPreviousTurnContent = ctx.state.deltaBuffer.trim().length > 0;
ctx.resetAssistantMessageState(ctx.state.assistantTexts.length);
if (hadPreviousTurnContent) {
ctx.state.deltaBuffer = "\n\n";
if (ctx.blockChunker) {
ctx.blockChunker.append("\n\n");
} else {
ctx.state.blockBuffer = "\n\n";
}
}
// Use assistant message_start as the earliest "writing" signal for typing.
void ctx.params.onAssistantMessageStart?.();
```
How can I resolve this? If you propose a fix, please make it concise.| it("handles numeric skill names without throwing TypeError", async () => { | ||
| // Note: The underlying @mariozechner/pi-coding-agent package also has the same bug | ||
| // in validateName() function. This test verifies that our fix in bundled-context.ts | ||
| // handles the case where skill.name might be a number type. | ||
|
|
||
| // For now, we use quoted YAML to ensure the skill loads successfully | ||
| // The real-world scenario where YAML parses unquoted numbers is handled by | ||
| // the String() coercion in bundled-context.ts | ||
| const skillDir = path.join(tempDir, "12306"); | ||
| await fs.mkdir(skillDir, { recursive: true }); | ||
|
|
||
| // Use quoted name to ensure it's parsed as string by YAML | ||
| const skillContent = `--- | ||
| name: "12306" | ||
| description: Test skill with numeric name | ||
| --- | ||
|
|
||
| # Test Skill | ||
|
|
||
| This skill has a numeric name. | ||
| `; | ||
| await fs.writeFile(path.join(skillDir, "SKILL.md"), skillContent, "utf-8"); | ||
|
|
||
| const context = resolveBundledSkillsContext(); | ||
|
|
||
| // The fix ensures that even if skill.name is a number, it's converted to string | ||
| expect(context.names.has("12306")).toBe(true); | ||
| expect(context.dir).toBe(tempDir); | ||
| }); | ||
|
|
||
| it("handles string skill names normally", async () => { |
There was a problem hiding this comment.
Test doesn't exercise the actual bug it claims to cover
The test uses a quoted YAML name (name: "12306"), which YAML parses as a string — meaning skill.name is already a string when it reaches resolveBundledSkillsContext. The String() coercion on line 35 of bundled-context.ts is therefore a no-op here, and this test would pass identically both before and after the fix.
The actual bug is triggered when YAML parses an unquoted integer (e.g., name: 12306) and the underlying loadSkillsFromDir returns a skill whose .name is a number. The test comments acknowledge this by noting the upstream validateName() function would also throw, but this means the regression test for the actual code path doesn't exist.
Consider mocking loadSkillsFromDir to return a skill with a numeric name directly, so the String() coercion is actually exercised:
vi.mock("@mariozechner/pi-coding-agent", () => ({
loadSkillsFromDir: () => ({ skills: [{ name: 12306, source: "openclaw-bundled" }] }),
}));Without this, the fix in bundled-context.ts has no automated regression coverage against the actual numeric-name scenario.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/skills/bundled-context.test.ts
Line: 22-52
Comment:
**Test doesn't exercise the actual bug it claims to cover**
The test uses a **quoted** YAML name (`name: "12306"`), which YAML parses as a `string` — meaning `skill.name` is already a `string` when it reaches `resolveBundledSkillsContext`. The `String()` coercion on line 35 of `bundled-context.ts` is therefore a no-op here, and this test would pass identically both before and after the fix.
The actual bug is triggered when YAML parses an **unquoted** integer (e.g., `name: 12306`) and the underlying `loadSkillsFromDir` returns a skill whose `.name` is a `number`. The test comments acknowledge this by noting the upstream `validateName()` function would also throw, but this means the regression test for the actual code path doesn't exist.
Consider mocking `loadSkillsFromDir` to return a skill with a numeric name directly, so the `String()` coercion is actually exercised:
```typescript
vi.mock("@mariozechner/pi-coding-agent", () => ({
loadSkillsFromDir: () => ({ skills: [{ name: 12306, source: "openclaw-bundled" }] }),
}));
```
Without this, the fix in `bundled-context.ts` has no automated regression coverage against the actual numeric-name scenario.
How can I resolve this? If you propose a fix, please make it concise.The previous fix added \n\n before resetAssistantMessageState(), which immediately cleared the separator. This fix: 1. Captures if separator is needed before reset 2. Calls resetAssistantMessageState() to clear buffers 3. Adds separator AFTER reset Fixes PR openclaw#35344 review feedback
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e697dc1fa2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ctx.state.deltaBuffer = “\n\n”; | ||
| if (ctx.blockChunker) { | ||
| ctx.blockChunker.append(“\n\n”); |
There was a problem hiding this comment.
Use ASCII quotes for separator string literals
These separator assignments use typographic quotes (“/”) instead of normal string delimiters, which makes the file invalid TypeScript and prevents this handler from compiling/running. Replace them with ASCII quotes ("\n\n") so the stream message path remains executable.
Useful? React with 👍 / 👎.
| if (needsSeparator) { | ||
| ctx.state.deltaBuffer = “\n\n”; | ||
| if (ctx.blockChunker) { | ||
| ctx.blockChunker.append(“\n\n”); |
There was a problem hiding this comment.
Preserve separator when paragraph chunk flushing is enabled
Appending "\n\n" as a standalone chunk to blockChunker still loses the turn boundary in chunkMode="newline" flows: resolveBlockStreamingChunking enables flushOnParagraph for that mode (src/auto-reply/reply/block-streaming.ts:186), and EmbeddedBlockChunker.#drainParagraphs drops leading paragraph breaks as empty chunks (src/agents/pi-embedded-block-chunker.ts:192-199). In those environments, assistant text before and after a tool call is still concatenated without the intended separator.
Useful? React with 👍 / 👎.
When buffer starts with paragraph break (index === 0), keep it instead of stripping it. This ensures cross-turn separator is prepended to next paragraph in flushOnParagraph mode. Added test case for cross-turn separator preservation. Fixes PR openclaw#35344 review feedback
Summary
Fixes #35308 - Text concatenation bug when blockStreaming is enabled.
When blockStreaming is enabled, the
deltaBufferwas being reset atmessage_startwithout preserving content from the previous turn. This caused text from consecutive API turns (e.g., before and after a tool call) to be concatenated without any separator.Problem Example
Before fix:
The string
durableis the end of pre-tool text andNOis the start of post-tool text — joined with no separator.After fix:
Changes
Adds a
\n\nseparator inhandleMessageStartbefore resetting thedeltaBufferif it contains content from a previous turn. The separator is added to:deltaBufferblockBuffer(orblockChunkerif available)This ensures proper text separation in the streaming UI while maintaining the correct behavior for the first assistant message.
Testing
blockStreaming(agents.defaults.blockStreamingDefault = "on")\n\nImpact
src/agents/pi-embedded-subscribe.handlers.messages.tshandleMessageStartNote
After a page refresh, the history renders correctly via the history endpoint, confirming this is purely a streaming display bug and the underlying data is intact.
Checklist
resetAssistantMessageState