Skip to content

[Fix] streaming: do not reset timeout for each emit, do not wait forever on close stream#197

Merged
MehakBindra merged 17 commits into
mainfrom
mehak/streaming
Dec 18, 2025
Merged

[Fix] streaming: do not reset timeout for each emit, do not wait forever on close stream#197
MehakBindra merged 17 commits into
mainfrom
mehak/streaming

Conversation

@MehakBindra

@MehakBindra MehakBindra commented Oct 31, 2025

Copy link
Copy Markdown
Member

Issue : microsoft/teams.ts#374

Main change::

  • Emit behavior updated

    • If no flush is scheduled and no timeout is pending, emit flushes immediately.
    • Otherwise, messages are queued and flushed later.
  • Flush improvements

    • Explicitly acquire and release the lock to prevent concurrent flushes.
    • If the lock is unavailable, flush returns immediately without blocking.
  • Test fixes

    • Added _wait_for_id_and_queue to prevent hanging tests: ensures stream.close() does not wait forever if flush fails.
    • Removed asyncio.sleep delays in tests; replaced with mocked call_later for deterministic scheduling.
    • Added a test to verify concurrent emits are serialized and do not flush at the same time.
  • remove timer.py (Previously used to differentiate between ongoing tasks and tasks not yet started, which is no longer needed as we now only cancel timeout from flush, not emit)

streaming_bug_py.mp4

Copilot AI review requested due to automatic review settings October 31, 2025 01:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors the HTTP streaming flush mechanism to trigger immediately on the first emit rather than after a 500ms delay. The changes improve responsiveness while still maintaining rate-limiting through subsequent 500ms delays between flushes.

Key changes:

  • Modified HttpStream.emit() to trigger immediate flush on first call instead of using a timer
  • Updated concurrency control in _flush() using manual lock acquire/release with early returns
  • Changed timer initialization from get_event_loop() to get_running_loop() for better async safety
  • Added optional server parameter to HttpPlugin constructor for dependency injection

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
packages/apps/src/microsoft/teams/apps/http_stream.py Refactored emit and flush logic to trigger immediate flush, replaced async context manager with manual lock handling
packages/apps/tests/test_http_stream.py Updated tests to reflect immediate flush behavior, added concurrency test, adjusted timing expectations
packages/apps/src/microsoft/teams/apps/utils/timer.py Changed from get_event_loop() to get_running_loop() for proper async context
packages/apps/src/microsoft/teams/apps/http_plugin.py Added optional server parameter for dependency injection, updated server initialization logic
tests/echo/src/main.py Added uvicorn server configuration and HttpPlugin with manual server setup (test code)
Comments suppressed due to low confidence (2)

packages/apps/src/microsoft/teams/apps/http_stream.py:212

  • Lock is acquired but not released when i == 0. The early return on line 212 bypasses the finally block at line 231, causing a deadlock on subsequent flush attempts. Move this check before acquiring the lock or use self._lock.release() before returning.
            if i == 0:
                self._logger.debug("No activities to flush")
                return

packages/apps/src/microsoft/teams/apps/http_stream.py:34

  • Documentation is outdated. The new implementation triggers an immediate flush on the first emit() call (not after 0.5 seconds), and subsequent emits do not cancel/reschedule timeouts. Update to reflect the immediate flush behavior and that the timeout is only used for subsequent flushes when queue has remaining items.
    1. emit() adds activities to a queue and cancels any pending flush timeout
    2. emit() schedules _flush() to run after 0.5 seconds via Timeout
    3. If another emit() happens before flush executes, the timeout is cancelled and rescheduled

Comment thread packages/apps/src/microsoft/teams/apps/http_stream.py Outdated
Comment thread tests/echo/src/main.py Outdated
Comment thread tests/echo/src/main.py Outdated
@MehakBindra MehakBindra changed the title Mehak/streaming [Fix] streaming: do not reset timeout for each chunk emit, do not wait forever on close stream Oct 31, 2025
@MehakBindra MehakBindra changed the title [Fix] streaming: do not reset timeout for each chunk emit, do not wait forever on close stream [Fix] streaming: do not reset timeout for each emit, do not wait forever on close stream Oct 31, 2025
Comment thread packages/apps/tests/test_http_stream.py Outdated
Comment thread packages/apps/tests/test_http_stream.py Outdated
Comment thread packages/apps/tests/test_http_stream.py Outdated

@aacebo aacebo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should try replacing the sleep statements in the tests with stubbing the http client and making it throw 429 at different times, ensuring that the streamer can recover from http rate limits, then also some other error codes to make sure it behaves as expected.

Right now the sleep statements will affect the overall test suite performance but also, this doesn't accurately simulate network rate limits

@heyitsaamir heyitsaamir left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unsure why we need the polling?

Comment thread packages/apps/src/microsoft_teams/apps/http_stream.py
Comment thread packages/apps/src/microsoft_teams/apps/http_stream.py
Comment thread packages/apps/tests/test_http_stream.py
Mehak Bindra and others added 2 commits December 17, 2025 08:14
Removed print statement for model response.
Comment thread packages/apps/src/microsoft_teams/apps/http_stream.py
Comment thread packages/apps/src/microsoft_teams/apps/http_stream.py
Comment thread packages/apps/src/microsoft_teams/apps/http_stream.py Outdated
@MehakBindra
MehakBindra merged commit 4577e0b into main Dec 18, 2025
6 checks passed
@MehakBindra
MehakBindra deleted the mehak/streaming branch December 18, 2025 15:45
@heyitsaamir heyitsaamir mentioned this pull request Jan 27, 2026
heyitsaamir added a commit that referenced this pull request Jan 28, 2026
# Release version 2.0.0a9

Commits since v2.0.0a8:
  - Add support for Python 3.14 (#259)
  - Bump qs from 6.14.0 to 6.14.1 in /examples/tab/Web (#240)
  - Bump urllib3 from 2.5.0 to 2.6.3 (#251)
  - Bump authlib from 1.6.5 to 1.6.6 (#252)
  - Bump starlette from 0.48.0 to 0.49.1 (#261)
  - Bump fastmcp from 2.12.4 to 2.14.0 (#260)
  - Bump python-multipart from 0.0.20 to 0.0.22 (#254)
  - Bump virtualenv from 20.35.3 to 20.36.1 (#256)
  - Bump pyasn1 from 0.6.1 to 0.6.2 (#255)
  - Bump azure-core from 1.36.0 to 1.38.0 (#257)
  - Bump aiohttp from 3.13.0 to 3.13.3 (#258)
- fix meetingStart and meetingEnd casing bug & add meetings sample
(#246)
  - Remove jitter during streaming (#248)
  - Add SERVICE_URL to override default service_url for Teams (#247)
- [Fix] streaming: do not reset timeout for each emit, do not wait
forever on close stream (#197)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants