fix(msteams): throttle typing indicator to prevent 429 rate limit spiral#53188
fix(msteams): throttle typing indicator to prevent 429 rate limit spiral#53188gumclaw wants to merge 2 commits intoopenclaw:mainfrom
Conversation
During long-running agent sessions, the typing indicator fires in a tight loop without throttling. This quickly hits Teams' API rate limit (HTTP 429), then falls back to proactive messaging which also loops, creating an escalating spiral that blocks message delivery. Changes: - Throttle typing to at most once every 3 seconds - Suppress typing for 30 seconds after a 429 error - Set a stop flag when markDispatchIdle() is called so typing does not continue after the response is sent Fixes openclaw#53184
Greptile SummaryThis PR fixes a real and well-described production issue: the MS Teams typing indicator firing in a tight loop during long agent runs, hitting the Teams 429 rate limit and creating a cascading failure spiral. The approach — a 3-second minimum interval, a 30-second post-429 suppression window, and a Key changes:
Issue found:
Confidence Score: 4/5
Prompt To Fix All With AIThis is a comment left during a code review.
Path: extensions/msteams/src/reply-dispatcher.ts
Line: 90-96
Comment:
**Non-429 errors are silently swallowed**
The `try/catch` wraps the entire `withRevokedProxyFallback` call but only acts on 429-class errors. Any other error (network failure, invalid activity, SDK error, etc.) is now silently discarded here, whereas before the PR it would propagate out of `sendTypingIndicator` and be caught by the pipeline's `onStartError` handler (which logs via `logTypingFailure`). This loses observability for non-429 failures.
Re-throw anything that isn't a rate-limit error so the existing error-logging path stays intact:
```suggestion
} catch (err: unknown) {
const errStr = String(err);
if (errStr.includes("429") || errStr.includes("quota exceeded")) {
typingSuppressedUntil = Date.now() + TYPING_BACKOFF_MS;
params.log.debug?.(`msteams typing suppressed for ${TYPING_BACKOFF_MS}ms after 429`);
} else {
throw err;
}
}
```
How can I resolve this? If you propose a fix, please make it concise.Reviews (1): Last reviewed commit: "fix(msteams): throttle typing indicator ..." | Re-trigger Greptile |
| } catch (err: unknown) { | ||
| const errStr = String(err); | ||
| if (errStr.includes("429") || errStr.includes("quota exceeded")) { | ||
| typingSuppressedUntil = Date.now() + TYPING_BACKOFF_MS; | ||
| params.log.debug?.(`msteams typing suppressed for ${TYPING_BACKOFF_MS}ms after 429`); | ||
| } | ||
| } |
There was a problem hiding this comment.
Non-429 errors are silently swallowed
The try/catch wraps the entire withRevokedProxyFallback call but only acts on 429-class errors. Any other error (network failure, invalid activity, SDK error, etc.) is now silently discarded here, whereas before the PR it would propagate out of sendTypingIndicator and be caught by the pipeline's onStartError handler (which logs via logTypingFailure). This loses observability for non-429 failures.
Re-throw anything that isn't a rate-limit error so the existing error-logging path stays intact:
| } catch (err: unknown) { | |
| const errStr = String(err); | |
| if (errStr.includes("429") || errStr.includes("quota exceeded")) { | |
| typingSuppressedUntil = Date.now() + TYPING_BACKOFF_MS; | |
| params.log.debug?.(`msteams typing suppressed for ${TYPING_BACKOFF_MS}ms after 429`); | |
| } | |
| } | |
| } catch (err: unknown) { | |
| const errStr = String(err); | |
| if (errStr.includes("429") || errStr.includes("quota exceeded")) { | |
| typingSuppressedUntil = Date.now() + TYPING_BACKOFF_MS; | |
| params.log.debug?.(`msteams typing suppressed for ${TYPING_BACKOFF_MS}ms after 429`); | |
| } else { | |
| throw err; | |
| } | |
| } |
Prompt To Fix With AI
This is a comment left during a code review.
Path: extensions/msteams/src/reply-dispatcher.ts
Line: 90-96
Comment:
**Non-429 errors are silently swallowed**
The `try/catch` wraps the entire `withRevokedProxyFallback` call but only acts on 429-class errors. Any other error (network failure, invalid activity, SDK error, etc.) is now silently discarded here, whereas before the PR it would propagate out of `sendTypingIndicator` and be caught by the pipeline's `onStartError` handler (which logs via `logTypingFailure`). This loses observability for non-429 failures.
Re-throw anything that isn't a rate-limit error so the existing error-logging path stays intact:
```suggestion
} catch (err: unknown) {
const errStr = String(err);
if (errStr.includes("429") || errStr.includes("quota exceeded")) {
typingSuppressedUntil = Date.now() + TYPING_BACKOFF_MS;
params.log.debug?.(`msteams typing suppressed for ${TYPING_BACKOFF_MS}ms after 429`);
} else {
throw err;
}
}
```
How can I resolve this? If you propose a fix, please make it concise.There was a problem hiding this comment.
Fixed in cf6192b. Non-429 errors now re-throw, so they still hit the existing onStartError/logTypingFailure path. Only 429/quota-exceeded cases are suppressed with backoff.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b71e51f36a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| } catch (err: unknown) { | ||
| const errStr = String(err); | ||
| if (errStr.includes("429") || errStr.includes("quota exceeded")) { | ||
| typingSuppressedUntil = Date.now() + TYPING_BACKOFF_MS; | ||
| params.log.debug?.(`msteams typing suppressed for ${TYPING_BACKOFF_MS}ms after 429`); |
There was a problem hiding this comment.
Re-throw non-429 typing failures
This catch now absorbs every typing error, but only applies special handling for 429-like strings, which means non-throttling failures never propagate to the typing start guard. In createTypingCallbacks (src/channels/typing.ts), that guard relies on thrown errors to count consecutive failures and stop the keepalive loop; with this change, persistent failures such as auth/conversation errors are treated as success and the loop keeps retrying until TTL while also bypassing onStartError logging. After setting 429 backoff, rethrow (or explicitly surface) non-429 failures so the existing failure-trip behavior still works.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed in cf6192b. Non-429 typing failures now re-throw, so the existing failure counting and onStartError logging still work. Only 429/quota-exceeded cases get suppressed with backoff.
Address review feedback: the catch block was silently swallowing all errors, not just 429s. Non-rate-limit errors should propagate to the pipeline's onStartError handler for logging via logTypingFailure.
|
Addressed the review feedback in cf6192b: non-429 errors now re-throw so they reach |
|
Closing this because the original symptom appears to have been superseded by the move to the official Teams SDK, so this PR no longer looks like the right path forward. If the typing/rate-limit spiral still reproduces on the current Teams implementation, we can open a fresh PR against the updated code path with current logs. |
During long-running agent sessions (browser automation, complex tool chains), the MS Teams typing indicator fires in a tight loop without any throttle. This quickly hits Teams' API rate limit (HTTP 429 "API calls quota exceeded"), then falls back to proactive messaging which also loops, creating an escalating spiral that blocks message delivery.
Changes
typingStoppedis set whenmarkDispatchIdle()is called, preventing typing from continuing after the response is sentTesting
Tested on a live Gumroad Teams workspace:
Fixes #53184