Skip to content

fix(feishu): prevent duplicate text in streaming card updates#34746

Closed
Daily-AC wants to merge 4 commits intoopenclaw:mainfrom
Daily-AC:fix/feishu-streaming-duplicate-text
Closed

fix(feishu): prevent duplicate text in streaming card updates#34746
Daily-AC wants to merge 4 commits intoopenclaw:mainfrom
Daily-AC:fix/feishu-streaming-duplicate-text

Conversation

@Daily-AC
Copy link
Copy Markdown

@Daily-AC Daily-AC commented Mar 4, 2026

Summary

Fixes #34108 and #33883 — streaming card updates rendering the same content 3+ times.

Root Cause

mergeStreamingText in reply-dispatcher.ts uses nextText.startsWith(streamText) to detect cumulative payloads. When this check fails (e.g., leading whitespace or directive stripping changes the prefix), the function falls through to streamText += nextText — appending the entire cumulative text again instead of replacing.

Fix

Add a nextText.includes(streamText) check before the append fallback. If the incoming text contains the current stream text as a substring, it is the full cumulative output and should replace rather than append.

Testing

All 16 existing reply-dispatcher tests pass.

@openclaw-barnacle openclaw-barnacle bot added channel: feishu Channel integration: feishu size: XS labels Mar 4, 2026
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 4, 2026

Greptile Summary

This PR adds a nextText.includes(streamText) heuristic to mergeStreamingText in reply-dispatcher.ts to fix duplicate-content rendering when cumulative streaming payloads fail the existing startsWith prefix check (e.g. due to leading whitespace or directive stripping). The intent is correct, but the heuristic is broader than necessary and introduces a new edge case.

  • Logic risk: String.includes matches the current streamText anywhere inside nextText, including as part of an unrelated word. When streamText is short (common early in a streaming session), a genuine delta can accidentally satisfy this check, causing the accumulated text to be silently replaced with only the delta — the opposite of the original bug, but still data loss.
  • Missing tests: No test is added to cover the new includes branch, nor a regression test for the original duplicate-content scenario. The 16 passing tests all pre-date this change and don't exercise the new code path.
  • A more targeted fix — such as trimming leading whitespace before the startsWith comparison (nextText.trimStart().startsWith(streamText.trimStart())) — would address the root cause described in the PR without the broader false-positive surface.

Confidence Score: 3/5

  • The fix reduces duplicate-content bugs but may cause silent data loss in edge cases where a short accumulated streamText matches as a substring of a true delta.
  • The heuristic is overly broad — String.includes with a short needle will match in many unintended places. This introduces a new failure mode (text replacement instead of append) that is the mirror image of the bug being fixed. No new tests guard against either the original bug or this regression. The change is limited to a single file and a small function, limiting blast radius, but the correctness concern warrants attention before merging.
  • extensions/feishu/src/reply-dispatcher.ts — specifically the mergeStreamingText function

Last reviewed commit: e69de74

Comment on lines +163 to +166
if (nextText.includes(streamText)) {
streamText = nextText;
return;
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

includes check can produce false-positive replacements for true deltas

nextText.includes(streamText) treats nextText as a full cumulative payload whenever streamText appears as a substring anywhere inside it — including as part of an unrelated word. This will misfire for true delta chunks if the short accumulated streamText appears as a substring inside the incoming delta.

For example:

  • streamText = "is" (early in the stream, only two characters accumulated)
  • nextText = "This result is important" (a genuine delta to be appended)
  • "This result is important".includes("is")true ("is" appears in "Th**is**" and standalone)
  • Result: streamText is silently replaced with the delta text, permanently dropping everything before "is"

The same false-positive risk applies to any common short word or phrase that may appear as a substring in subsequent delta chunks. The risk is highest early in a streaming session when streamText is still short.

A more conservative guard — e.g. only triggering when nextText is meaningfully longer than streamText, or requiring the match to be in the second half of nextText — would reduce false positives. For instance:

// Require nextText to end with streamText, or that streamText occupies
// at least half of nextText, to reduce false-positive substring matches.
if (
  nextText.includes(streamText) &&
  nextText.length >= streamText.length * 2
) {
  streamText = nextText;
  return;
}

Or alternatively, since the bug is triggered by leading whitespace/directive stripping, trimming both sides before comparing is a more targeted fix:

if (nextText.trimStart().startsWith(streamText.trimStart())) {
  streamText = nextText;
  return;
}
Prompt To Fix With AI
This is a comment left during a code review.
Path: extensions/feishu/src/reply-dispatcher.ts
Line: 163-166

Comment:
**`includes` check can produce false-positive replacements for true deltas**

`nextText.includes(streamText)` treats `nextText` as a full cumulative payload whenever `streamText` appears as a substring anywhere inside it — including as part of an unrelated word. This will misfire for true delta chunks if the short accumulated `streamText` appears as a substring inside the incoming delta.

For example:
- `streamText = "is"` (early in the stream, only two characters accumulated)
- `nextText = "This result is important"` (a genuine delta to be appended)
- `"This result is important".includes("is")``true` (`"is"` appears in `"Th**is**"` and standalone)
- Result: `streamText` is silently **replaced** with the delta text, permanently dropping everything before `"is"`

The same false-positive risk applies to any common short word or phrase that may appear as a substring in subsequent delta chunks. The risk is highest early in a streaming session when `streamText` is still short.

A more conservative guard — e.g. only triggering when `nextText` is meaningfully longer than `streamText`, or requiring the match to be in the second half of `nextText` — would reduce false positives. For instance:

```ts
// Require nextText to end with streamText, or that streamText occupies
// at least half of nextText, to reduce false-positive substring matches.
if (
  nextText.includes(streamText) &&
  nextText.length >= streamText.length * 2
) {
  streamText = nextText;
  return;
}
```

Or alternatively, since the bug is triggered by leading whitespace/directive stripping, trimming both sides before comparing is a more targeted fix:

```ts
if (nextText.trimStart().startsWith(streamText.trimStart())) {
  streamText = nextText;
  return;
}
```

How can I resolve this? If you propose a fix, please make it concise.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 4, 2026

Additional Comments (1)

extensions/feishu/src/reply-dispatcher.ts
No test coverage for the new branch

The PR description notes that all 16 existing tests pass, but none of them exercise the newly added nextText.includes(streamText) path. The bug scenario (cumulative payload that fails startsWith due to leading whitespace/directive stripping) and the potential false-positive scenario (delta whose text happens to contain the current streamText as a substring) are both untested.

Adding targeted unit tests for mergeStreamingText would both validate the fix and guard against future regressions. For example:

it("mergeStreamingText handles cumulative payload with leading whitespace prefix mismatch", async () => {
  // onPartialReply("Hello") → streamText = "Hello"
  // onPartialReply(" Hello world") → prefix mismatch but includes "Hello"
  // expected: streamText = " Hello world" (replace, not append)
  ...
});

it("mergeStreamingText does not treat a short delta as cumulative", async () => {
  // streamText = "is" (first chunk)
  // nextText = "This is the answer" (true delta starting differently)
  // expected: streamText = "isThis is the answer" (append)
  ...
});
Prompt To Fix With AI
This is a comment left during a code review.
Path: extensions/feishu/src/reply-dispatcher.ts
Line: 149-172

Comment:
**No test coverage for the new branch**

The PR description notes that all 16 existing tests pass, but none of them exercise the newly added `nextText.includes(streamText)` path. The bug scenario (cumulative payload that fails `startsWith` due to leading whitespace/directive stripping) and the potential false-positive scenario (delta whose text happens to contain the current `streamText` as a substring) are both untested.

Adding targeted unit tests for `mergeStreamingText` would both validate the fix and guard against future regressions. For example:

```ts
it("mergeStreamingText handles cumulative payload with leading whitespace prefix mismatch", async () => {
  // onPartialReply("Hello") → streamText = "Hello"
  // onPartialReply(" Hello world") → prefix mismatch but includes "Hello"
  // expected: streamText = " Hello world" (replace, not append)
  ...
});

it("mergeStreamingText does not treat a short delta as cumulative", async () => {
  // streamText = "is" (first chunk)
  // nextText = "This is the answer" (true delta starting differently)
  // expected: streamText = "isThis is the answer" (append)
  ...
});
```

How can I resolve this? If you propose a fix, please make it concise.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e69de74499

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +163 to +164
if (nextText.includes(streamText)) {
streamText = nextText;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid replacing stream text on arbitrary substring matches

Using nextText.includes(streamText) to classify cumulative partials can drop already-streamed content when the upstream sends true deltas that happen to repeat prior output. In that case a delta chunk like "abc def" after "abc" matches this condition, so streamText is replaced instead of appended and the earlier prefix is lost in the card update/final close path. This regression appears whenever partial payloads are delta-style (which this function still claims to support via the append fallback), so the replacement check should be constrained to a safer cumulative-only signal.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 31e1f3d121

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +170 to +171
if (streamText.trimStart().startsWith(nextText.trimStart())) {
return;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Tighten reverse-prefix dedupe before dropping partial chunks

The new streamText.trimStart().startsWith(nextText.trimStart()) fast-path can suppress valid delta chunks whenever a new token happens to match the beginning of the response (or is whitespace-only after trimming). For example, after streamText = "Hello there. ", an incoming delta "Hello" is dropped because "Hello there. ".trimStart().startsWith("Hello") is true, so the repeated word is never appended; the same happens for "\n" because trimming yields an empty string. This regresses content accuracy in streaming updates and can leak into the final card text when the flow closes from partials without a final payload.

Useful? React with 👍 / 👎.

YiLin added 4 commits March 5, 2026 01:44
mergeStreamingText appended the full cumulative text when
nextText.startsWith(streamText) failed due to leading whitespace or
directive-stripping differences.  This caused the card to render the
same content 3+ times, growing multiplicatively with each streaming
update cycle.

Add a nextText.includes(streamText) check to detect cumulative
payloads even when prefixes diverge, and replace rather than append.

Fixes openclaw#34108
Fixes openclaw#33883
Replace the broad String.includes heuristic with trimStart().startsWith()
to avoid false positives when streamText is short. This precisely handles
the leading-whitespace prefix mismatch without the risk of accidentally
replacing accumulated text with a delta that happens to contain a short
streamText as a substring.
@Daily-AC Daily-AC force-pushed the fix/feishu-streaming-duplicate-text branch from a5c9c18 to 5112a68 Compare March 5, 2026 01:44
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5112a68886

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +170 to +171
if (streamText.trimStart().startsWith(nextText.trimStart())) {
return;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Remove reverse-prefix suppression for streamed deltas

The new reverse-prefix guard drops valid delta chunks whenever a chunk starts with text that already appeared at the beginning of the response. In mergeStreamingText, streamText.trimStart().startsWith(nextText.trimStart()) returns true for inputs like existing "Hello there. " plus delta "Hello" (or whitespace-only chunks where trimStart() becomes empty), so the function returns early and never appends text; if the stream closes from partials, the final card content is missing those tokens.

Useful? React with 👍 / 👎.

@vincentkoc
Copy link
Copy Markdown
Member

This no longer patches a live gap on current main.

I re-checked the current Feishu streaming implementation, and the overlap / cumulative-text handling is already present in extensions/feishu/src/streaming-card.ts inside mergeStreamingText(...):

  • next.includes(previous)
  • previous.includes(next)
  • suffix/prefix overlap merge

So the underlying bug family is real and separate from #43683, but this specific PR is now stale against the current codebase and still carries the earlier review concern about broad includes(...) matching without targeted regression coverage on the old path in reply-dispatcher.ts.

Closing as already fixed on main rather than as the canonical landing path.

@vincentkoc vincentkoc closed this Mar 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

channel: feishu Channel integration: feishu size: XS

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Feishu streaming card renders duplicate text blocks

2 participants