Skip to content

[Bug]: Streaming text silently dropped when tool boundary arrives during fallback mode #8124

Description

@austinmw

Bug Description

When streaming to Telegram, if a message edit fails (flood control) and the agent then calls a tool before the stream finishes, text that was already generated is silently dropped. The user sees a frozen partial message with ▉ and the missing words are never delivered.

This is distinct from #7183, which covers only the final cursor-removal edit failing after full text was already delivered. This bug loses actual content.

Steps to Reproduce

Save the script below as repro.py in the repo root and run:

source venv/bin/activate && python repro.py
Reproduction script (verified on v0.8.0)
import asyncio, sys
sys.path.insert(0, ".")
from gateway.platforms.base import SendResult
from gateway.stream_consumer import GatewayStreamConsumer, StreamConsumerConfig

class FakeTelegram:
    """Simulates Telegram with persistent flood control once triggered."""
    MAX_MESSAGE_LENGTH = 4096
    def __init__(self):
        self.visible_messages = {}
        self.message_order = []
        self.edit_log = []
        self._next_id = 1
        self._flood_active = False
    async def send(self, *, chat_id, content, reply_to=None, metadata=None):
        mid = f"msg_{self._next_id}"; self._next_id += 1
        self.visible_messages[mid] = content
        self.message_order.append(mid)
        return SendResult(success=True, message_id=mid)
    async def edit_message(self, *, chat_id, message_id, content):
        self.edit_log.append(f"EDIT {message_id}: {content!r}")
        if self._flood_active:
            self.edit_log[-1] += "  <-- FAILED (flood still active)"
            return SendResult(success=False, error="flood_control:6.0")
        if "more" in content and content.endswith(" ▉"):
            self._flood_active = True
            self.edit_log[-1] += "  <-- FAILED (flood control triggered)"
            return SendResult(success=False, error="flood_control:6.0")
        self.visible_messages[message_id] = content
        return SendResult(success=True, message_id=message_id)
    def truncate_message(self, text, limit):
        return [text] if len(text) <= limit else [text[i:i+limit] for i in range(0, len(text), limit)]

async def main():
    tg = FakeTelegram()
    consumer = GatewayStreamConsumer(tg, "chat_123",
        StreamConsumerConfig(edit_interval=0.01, buffer_threshold=5, cursor=" ▉"))
    consumer.on_delta("Hello")
    task = asyncio.create_task(consumer.run())
    await asyncio.sleep(0.08)
    consumer.on_delta(" world"); await asyncio.sleep(0.08)
    consumer.on_delta(" more"); await asyncio.sleep(0.08)  # this edit fails
    consumer.on_delta(None)  # tool boundary
    consumer.on_delta("Here is the tool result.")
    consumer.finish()
    await task
    visible = " ".join(tg.visible_messages[m] for m in tg.message_order)
    for e in tg.edit_log: print(e)
    for mid in tg.message_order: print(f"{mid}: {tg.visible_messages[mid]!r}")
    print(f"Agent generated: \"Hello world more\" + [tool] + \"Here is the tool result.\"")
    print(f"User received:   {visible!r}")
    print("REPRODUCED" if "more" not in visible else "NOT REPRODUCED")

asyncio.run(main())

Expected Behavior

All generated text is delivered to the user, even when progressive edits fail and tool boundaries occur mid-stream.

Actual Behavior

Text buffered after the edit failure but before the tool boundary is silently dropped:

EDIT msg_1: 'Hello world ▉'
EDIT msg_1: 'Hello world more ▉'  <-- FAILED (flood control triggered)
EDIT msg_1: 'Hello world more'  <-- FAILED (flood still active)

msg_1: 'Hello world ▉'
msg_2: 'Here is the tool result.'

Agent generated: "Hello world more" + [tool] + "Here is the tool result."
User received:   'Hello world ▉ Here is the tool result.'

REPRODUCED

The word "more" was generated but never delivered. The user also sees a stuck ▉ cursor in msg_1.

Affected Component

Gateway (Telegram/Discord/Slack/WhatsApp)

Messaging Platform

Telegram

Operating System

Ubuntu 24.04 (aarch64 — NVIDIA DGX Spark)

Python Version

3.11.15

Hermes Version

v0.8.0 (2026.4.8)

Root Cause Analysis

Verified sequence with line numbers from gateway/stream_consumer.py:

  1. Edit fails → _send_or_edit() enters fallback mode (lines 545-547): sets _fallback_prefix, _fallback_final_send=True, _edit_supported=False
  2. Text continues accumulating in _accumulated via queue drain (line 159) regardless of _edit_supported
  3. _NEW_SEGMENT arrives → run() calls _reset_segment_state(preserve_no_edit=True) at line 272
  4. _reset_segment_state() (lines 113-117) clears _accumulated, _fallback_final_send, and _fallback_prefix. The preserve_no_edit guard only protects the __no_edit__ sentinel — not normal fallback state
  5. _DONE arrives → the check at line 241 (if self._fallback_final_send) is now False, and _accumulated only contains the post-boundary text. Pre-boundary text is gone.

Related: #7183 (covers only the narrower final-cursor-removal case)

Proposed Fix

Before resetting segment state on _NEW_SEGMENT (line 272), flush the pending fallback continuation. Or add a guard in _reset_segment_state() to preserve fallback state when _fallback_final_send is True.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions