feat: add LLM idle timeout for streaming responses#55072
feat: add LLM idle timeout for streaming responses#55072obviyus merged 3 commits intoopenclaw:mainfrom
Conversation
Greptile SummaryThis PR adds a configurable idle timeout for LLM streaming responses, aborting with a descriptive error message if no token is received within the threshold (default 60 s). The feature is well-scoped and fits cleanly into the existing Key implementation decisions look correct:
One minor style nit: Confidence Score: 5/5Safe to merge — all previous P1 concerns (empty abort body, timer leak on non-done path, schema rejecting 0) are resolved in this version. The only remaining finding is a P2 style issue (value import used only for its type). All core logic — timer lifecycle, abort propagation, config validation — is correct and covered by 13 tests. No files require special attention.
|
| Filename | Overview |
|---|---|
| src/agents/pi-embedded-runner/run/llm-idle-timeout.ts | New file implementing idle timeout for LLM streaming via timer-per-next() + Promise.race. Logic is sound: timer is cleared on all three return paths (done, non-done, throw), controller.abort is called on timeout, and resolveLlmIdleTimeoutMs correctly converts seconds to ms with capping. One style nit: streamSimple is imported as a value but used only for its type. |
| src/agents/pi-embedded-runner/run/llm-idle-timeout.test.ts | 13 test cases covering config resolution edge cases and stream wrapper behavior including timeout firing and controller abort verification. Coverage is solid. |
| src/agents/pi-embedded-runner/run/attempt.ts | Integration wires streamWithIdleTimeout into the attempt loop when idleTimeoutMs > 0, and extends makeAbortError to preserve the original Error message when the abort reason is already an Error instance. |
| src/config/zod-schema.agent-defaults.ts | Uses .nonnegative() (allowing 0 to disable) — correctly aligned with the runtime behavior of resolveLlmIdleTimeoutMs. |
| src/config/schema.base.generated.ts | JSON schema uses minimum: 0 (inclusive) for idleTimeoutSeconds, consistent with the Zod schema and runtime behavior. |
| src/config/types.agent-defaults.ts | Adds AgentLlmConfig type with idleTimeoutSeconds?: number and wires it into AgentDefaultsConfig. Straightforward type scaffolding. |
Prompt To Fix All With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-runner/run/llm-idle-timeout.ts
Line: 2
Comment:
**Value import used only for its type**
`streamSimple` is imported as a value but used exclusively as a type in `ReturnType<typeof streamSimple>` on line 52. This should be `import type` to avoid a potential runtime side-effect load of `@mariozechner/pi-ai`. Projects using `verbatimModuleSyntax: true` would also require this.
Alternatively, you can drop the dependency on `streamSimple` entirely by deriving the type from `StreamFn`'s own return type (e.g. `Awaited<ReturnType<StreamFn>>`), which keeps the module self-contained.
```suggestion
import type { streamSimple } from "@mariozechner/pi-ai";
```
How can I resolve this? If you propose a fix, please make it concise.Reviews (3): Last reviewed commit: "fix: stop rewriting abort reason names i..." | Re-trigger Greptile
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8d7237f65d
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
Thanks for the thorough review! Both issues have been fixed in the latest commit:
@greptile-apps Could you re-review the latest commit |
8e4335c to
3fc715e
Compare
3fc715e to
5451a87
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5451a871a4
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
2fc137d to
81d5f75
Compare
|
Addressed all review comments:
|
dd5759e to
81d5f75
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dd5759e7a0
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
81d5f75 to
895bf06
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 077f9623c7
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
077f962 to
f7a72b7
Compare
|
@greptileai please rescore |
|
Patched the idle-timeout path.
Verified with:
|
Problem: When LLM stops responding, the agent hangs for ~5 minutes with no feedback. Users had to use /stop to recover. Solution: Add idle timeout detection for LLM streaming responses.
d52d7e5 to
2e7f201
Compare
obviyus
left a comment
There was a problem hiding this comment.
Reviewed latest changes; landing now.
Fixes #55065
Summary
Adds an idle timeout mechanism for LLM streaming responses. If the model doesn't return any token within the specified timeout, the request is aborted with a user-friendly error message.
Changes
New configuration option:
agents.defaults.llm.idleTimeoutSeconds0= disabled (never timeout)> 0= timeout in secondsCore implementation:
src/agents/pi-embedded-runner/run/llm-idle-timeout.tsresolveLlmIdleTimeoutMs(): resolves timeout from configstreamWithIdleTimeout(): wraps stream function with idle timeout usingPromise.raceIntegration: Modified
attempt.tsto wrapstreamFnwith idle timeoutTests:
llm-idle-timeout.test.tswith 13 test cases covering:Configuration Example
```json
{
"agents": {
"defaults": {
"llm": {
"idleTimeoutSeconds": 60
}
}
}
}
```
User Experience
Before: Agent hangs indefinitely when LLM is unresponsive, user must use `/stop`
After: After 60s (configurable) of no response, user sees:
```
⏱️ LLM idle timeout (60s): no response from model
```