[Fix] streaming: do not reset timeout for each emit, do not wait forever on close stream#197
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR refactors the HTTP streaming flush mechanism to trigger immediately on the first emit rather than after a 500ms delay. The changes improve responsiveness while still maintaining rate-limiting through subsequent 500ms delays between flushes.
Key changes:
- Modified
HttpStream.emit()to trigger immediate flush on first call instead of using a timer - Updated concurrency control in
_flush()using manual lock acquire/release with early returns - Changed timer initialization from
get_event_loop()toget_running_loop()for better async safety - Added optional server parameter to
HttpPluginconstructor for dependency injection
Reviewed Changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/apps/src/microsoft/teams/apps/http_stream.py | Refactored emit and flush logic to trigger immediate flush, replaced async context manager with manual lock handling |
| packages/apps/tests/test_http_stream.py | Updated tests to reflect immediate flush behavior, added concurrency test, adjusted timing expectations |
| packages/apps/src/microsoft/teams/apps/utils/timer.py | Changed from get_event_loop() to get_running_loop() for proper async context |
| packages/apps/src/microsoft/teams/apps/http_plugin.py | Added optional server parameter for dependency injection, updated server initialization logic |
| tests/echo/src/main.py | Added uvicorn server configuration and HttpPlugin with manual server setup (test code) |
Comments suppressed due to low confidence (2)
packages/apps/src/microsoft/teams/apps/http_stream.py:212
- Lock is acquired but not released when i == 0. The early return on line 212 bypasses the
finallyblock at line 231, causing a deadlock on subsequent flush attempts. Move this check before acquiring the lock or useself._lock.release()before returning.
if i == 0:
self._logger.debug("No activities to flush")
return
packages/apps/src/microsoft/teams/apps/http_stream.py:34
- Documentation is outdated. The new implementation triggers an immediate flush on the first emit() call (not after 0.5 seconds), and subsequent emits do not cancel/reschedule timeouts. Update to reflect the immediate flush behavior and that the timeout is only used for subsequent flushes when queue has remaining items.
1. emit() adds activities to a queue and cancels any pending flush timeout
2. emit() schedules _flush() to run after 0.5 seconds via Timeout
3. If another emit() happens before flush executes, the timeout is cancelled and rescheduled
70c86a3 to
8e2bccf
Compare
aacebo
left a comment
There was a problem hiding this comment.
I think you should try replacing the sleep statements in the tests with stubbing the http client and making it throw 429 at different times, ensuring that the streamer can recover from http rate limits, then also some other error codes to make sure it behaves as expected.
Right now the sleep statements will affect the overall test suite performance but also, this doesn't accurately simulate network rate limits
heyitsaamir
left a comment
There was a problem hiding this comment.
Unsure why we need the polling?
Removed print statement for model response.
# Release version 2.0.0a9 Commits since v2.0.0a8: - Add support for Python 3.14 (#259) - Bump qs from 6.14.0 to 6.14.1 in /examples/tab/Web (#240) - Bump urllib3 from 2.5.0 to 2.6.3 (#251) - Bump authlib from 1.6.5 to 1.6.6 (#252) - Bump starlette from 0.48.0 to 0.49.1 (#261) - Bump fastmcp from 2.12.4 to 2.14.0 (#260) - Bump python-multipart from 0.0.20 to 0.0.22 (#254) - Bump virtualenv from 20.35.3 to 20.36.1 (#256) - Bump pyasn1 from 0.6.1 to 0.6.2 (#255) - Bump azure-core from 1.36.0 to 1.38.0 (#257) - Bump aiohttp from 3.13.0 to 3.13.3 (#258) - fix meetingStart and meetingEnd casing bug & add meetings sample (#246) - Remove jitter during streaming (#248) - Add SERVICE_URL to override default service_url for Teams (#247) - [Fix] streaming: do not reset timeout for each emit, do not wait forever on close stream (#197)
Issue : microsoft/teams.ts#374
Main change::
Emit behavior updated
Flush improvements
Test fixes
remove
timer.py(Previously used to differentiate between ongoing tasks and tasks not yet started, which is no longer needed as we now only cancel timeout from flush, not emit)streaming_bug_py.mp4