Skip to content

HttpStream sends streaming activities faster than the 1 request/second limit #452

Description

@Athosone

What happens

When a bot pushes status updates through HttpStream.update(), the stream can
send several POSTs back to back, above the 1 request/second limit in the Teams
streaming docs. Teams throttles the stream and ends it: the bot gets a
403 ContentStreamNotAllowed, HttpStream._send turns it into a
StreamCancelledError, and the user sees a red error toast.

It is easiest to hit on the informative update() path, for example forwarding
workflow or agent progress events while a longer answer is generated.

  • Package: microsoft-teams-apps 2.0.11 (same code on main / 2.0.13-dev)
  • File: packages/apps/src/microsoft_teams/apps/http_stream.py

The docs are explicit: "The throttling limit is 1 request per second", make
subsequent calls only after a successful response, and "You can set only one
informative message. Your bot reuses this message for each update."

How to reproduce

Drive HttpStream with a mock client that records the time of each POST and
simulates a ~20 ms round trip:

# A) eight informative updates in a tight loop (bursty progress)
for i in range(8):
    stream.update(f"progress {i}")
await asyncio.sleep(1.0)

# B) five informative updates, 200 ms apart (steady progress)
for i in range(5):
    stream.update(f"step {i}")
    await asyncio.sleep(0.2)
A) eight rapid update():  8 POSTs over 153 ms  -> ~52 req/s   (limit is 1)
B) five update() @ 200ms: 5 POSTs, min gap 201 ms             (should be >= 1000 ms)

Both go well past 1 req/s, enough to get the stream throttled against the real
service.

Why it happens

Two separate causes in http_stream.py.

First, _flush() drains the queue and sends each collected informative update in
a loop with nothing between the calls, then the text chunk immediately after:

for typing_update in informative_updates:
    await self._send_activity(typing_update)   # no delay between calls
if self._text:
    await self._send_activity(TypingActivityInput(text=self._text))  # also immediate

So eight queued update() calls become eight POSTs in one flush. The text path
is already coalesced into a single chunk, so only the informative path bursts.

Second, the only pacing is call_later(0.5, ...) (already 2 req/s), armed only
when a backlog survives a flush:

if self._queue and not self._timeout:
    self._timeout = asyncio.get_running_loop().call_later(0.5, ...)

Once a flush empties the queue, _timeout stays None, so the next emit()
schedules an immediate flush and a steady stream of update() calls gets no
pacing at all. That is case B.

Possible fix

The throttle is better handled inside HttpStream than in every caller, and #453
does that with a per-stream limiter. The SDK creates one HttpStream per
conversation turn, so a stream-scoped limiter matches the per-conversation scope
Teams throttles on. Two parts:

  1. Pace all sends to at most 1 req/s, including across flushes. The first send
    goes out right away so the progress bar appears quickly; the rest wait for the
    interval, which is configurable toward the docs' 1.5 to 2s advice.
  2. By default coalesce a burst of informative updates in one flush to the latest,
    matching the docs' one-reused-message behavior. A flag paces out every update
    instead, for callers who want each one delivered.

Environment

  • microsoft-teams-apps 2.0.11, Python 3.12
  • msteams channel, 1:1 chat, streaming

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions