Repository navigation
Async tool reply raises "cannot schedule new speech" when its agent is handing off #7632
Description
Bug Description
An async tool (one that calls ctx.update() and returns later) delivers its final result through _ToolExecutor._deliver_reply, which waits for the owning activity to be idle and then calls session.generate_reply().
If the result lands while the owning activity is draining for a handoff (for example, the LLM called the async tool and a handoff tool in the same step), wait_for_idle() still returns, because it only raises once the activity is closed. generate_reply() then routes to session._next_activity, which hasn't started yet, and two errors follow:
- the reply task fails with
RuntimeError: cannot schedule new speech, the speech scheduling is draining/pausing, the speech will be cancelled(an unretrieved task exception) - the LLM task created for that reply logs
Error in _llm_inference_task:RuntimeError: no activity context found, the agent is not running
When the owning activity is instead paused under an AgentTask (a parallel tool awaits a task), the same reply is routed to the task's activity. The task agent is then asked to summarize results that aren't in its chat context.
We hit this in production on 1.6.6. It reproduces on main (a7c62e3).
Expected Behavior
No errors, and the reply is never spoken by another agent. Dropping it is fine, the same way _deliver_reply already drops it once the owning activity has closed: the results are already in the owning agent's chat context.
Reproduction Steps
import asyncio
from livekit.agents import Agent, AgentSession, RunContext, function_tool
from livekit.agents.llm import LLM, ChatChunk, ChoiceDelta, FunctionToolCall, LLMStream
from livekit.agents.types import DEFAULT_API_CONNECT_OPTIONS
class ScriptedStream(LLMStream):
async def _run(self) -> None:
if self._llm.calls == 1: # the user turn: call both tools in one step
calls = [
FunctionToolCall(name="send_link", arguments="{}", call_id="c1"),
FunctionToolCall(name="handoff", arguments="{}", call_id="c2"),
]
self._event_ch.send_nowait(
ChatChunk(id="1", delta=ChoiceDelta(role="assistant", tool_calls=calls))
)
return
await asyncio.sleep(1.0) # the draining agent's tool reply outlasts send_link's work
self._event_ch.send_nowait(
ChatChunk(id="2", delta=ChoiceDelta(role="assistant", content="ok"))
)
class ScriptedLLM(LLM):
calls = 0
def chat(self, *, chat_ctx, tools=None, conn_options=DEFAULT_API_CONNECT_OPTIONS, **kwargs):
self.calls += 1
return ScriptedStream(self, chat_ctx=chat_ctx, tools=tools or [], conn_options=conn_options)
class Second(Agent):
def __init__(self) -> None:
super().__init__(instructions="second")
class First(Agent):
def __init__(self) -> None:
super().__init__(instructions="first")
@function_tool
async def send_link(self, ctx: RunContext) -> str:
"""Send a link."""
await ctx.update("sending the link")
await asyncio.sleep(0.3)
return "link sent"
@function_tool
async def handoff(self) -> Agent:
"""Hand off."""
await asyncio.sleep(0.05)
return Second()
async def main() -> None:
async with AgentSession(llm=ScriptedLLM()) as session:
await session.start(First())
await session.run(user_input="send me the link")
await asyncio.sleep(2)
asyncio.run(main())Running it logs Error in _llm_inference_task ... no activity context found. The reply task's own cannot schedule new speech exception is unretrieved, so it surfaces through error reporting rather than logs (Sentry captures it as unhandled).
Operating System
Linux
Models Used
Any. The repro uses a scripted LLM.
Package Versions
livekit-agents==1.6.6 # production
livekit-agents main @ a7c62e301 # reproProposed Solution
# _ToolExecutor._deliver_reply, after the idle wait
if self._owning_activity is not None and self._owning_activity.scheduling_paused:
logger.debug("dropping tool reply — owning activity is paused")
self._pending_updates.clear()
returnAdditional Context
Session-scoped executors (AsyncToolset passed to AgentSession) go through the same path during a handoff. Those should retarget the reply to the new agent rather than drop it, so they're not covered by the fix above.
Activity