Skip to content

feat(llm-drivers): add per-provider request_timeout_secs config#3004

Merged
houko merged 3 commits into
mainfrom
port/per-provider-timeout
Apr 23, 2026
Merged

feat(llm-drivers): add per-provider request_timeout_secs config#3004
houko merged 3 commits into
mainfrom
port/per-provider-timeout

Conversation

@houko

@houko houko commented Apr 23, 2026

Copy link
Copy Markdown
Contributor

Summary

Port from Hermes: per-provider (and per-model) LLM request timeout configuration.

Config:

[provider_request_timeout_secs]
ollama = 300
anthropic = 120

Changes:

  • KernelConfig: new provider_request_timeout_secs: HashMap<String, u64> field
  • DriverConfig: new request_timeout_secs: Option<u64> field
  • OpenAIDriver, AnthropicDriver, GeminiDriver: apply timeout via RequestBuilder::timeout() on both complete() and stream() paths
  • DriverCache::cache_key includes timeout value to prevent cross-timeout cache hits
  • All 5 kernel DriverConfig build sites inject the per-provider value

CLI drivers (claude-code, qwen-code) use their own message_timeout_secs and are unaffected.

Test plan

  • No config → existing default timeout behaviour unchanged
  • ollama = 300 → Ollama requests use 300s timeout
  • Slow model times out at configured value, not global default

🤖 Ported from Hermes with Claude Code

@github-actions github-actions Bot added size/M 50-249 lines changed has-conflicts PR has merge conflicts that need resolution area/kernel Core kernel (scheduling, RBAC, workflows) ready-for-review PR is ready for maintainer review labels Apr 23, 2026
Port Hermes per-provider request timeout to Rust. Each HTTP API provider
(OpenAI-compatible, Anthropic, Gemini) can now override the HTTP client's
default read timeout via provider_request_timeout_secs in config.toml:

  [provider_request_timeout_secs]
  ollama = 300
  anthropic = 120

DriverConfig gains request_timeout_secs: Option<u64>; KernelConfig gains
provider_request_timeout_secs: HashMap<String, u64>. The field is wired
through all DriverConfig construction sites in the kernel and applied via
reqwest RequestBuilder::timeout() in the complete/stream hot paths of the
three HTTP drivers. DriverCache key includes the timeout value so distinct
timeout configs produce distinct cached driver instances.
@houko
houko force-pushed the port/per-provider-timeout branch from bc1c3b7 to 28eff57 Compare April 23, 2026 03:23
@github-actions github-actions Bot removed the has-conflicts PR has merge conflicts that need resolution label Apr 23, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bc1c3b7e21

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@@ -92,146 +100,13 @@ impl OpenAIDriver {
extra_headers: Vec::new(),
use_api_key_header: true,
url_query: Some(format!("api-version={}", api_version)),
moonshot_file_cache: Default::default(),
request_timeout_secs: None,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Honor timeout in Azure OpenAI constructor

create_driver_from_entry now forwards config.request_timeout_secs for most HTTP providers, but OpenAIDriver::new_azure_with_proxy hardcodes request_timeout_secs: None, so provider_request_timeout_secs for azure-openai is silently ignored. In deployments that rely on Azure and set a larger timeout for long responses, requests will still use the default client timeout and fail unexpectedly despite the new config being present.

Useful? React with 👍 / 👎.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@houko
houko merged commit 5f56012 into main Apr 23, 2026
19 checks passed
@houko
houko deleted the port/per-provider-timeout branch April 23, 2026 06:11
@github-actions github-actions Bot removed the ready-for-review PR is ready for maintainer review label Apr 23, 2026
houko added a commit that referenced this pull request Apr 26, 2026
… (en + zh)

Three new `config.toml` sections that landed Apr 22-23:
- #3004 `[provider_request_timeout_secs]` map
- #2993 / #2991 `[browser].cdp_endpoint` + `cdp_auth_token_env`
- #2994 `[kernel].cron_session_max_tokens` / `cron_session_max_messages`
houko added a commit that referenced this pull request Apr 26, 2026
…pi / observability (#3189)

* docs(providers): add Novita / SearXHG / Bedrock entries

* docs(config): document [parallel_tools] section + max_history_messages

* docs(scheduler): cron multi-delivery + pre_script + silent_marker

* docs: live context.md, chat attachments, CLI slash commands

Salvages three of the six items from the (quota-killed) misc-features
agent run. Each is a focused append, no rewrite of existing content.

- agent/templates: live Live Context section explains the per-turn
  re-read of `context.md` (#3115), the .identity/ vs root path lookup,
  the 32 KB cap with stale-cache fallback, and the cache_context opt-out.
- integrations/api/agents: attachment-resolution table (image / PDF /
  text+code) covering the supported MIME / extension list and the
  200 KB truncation, plus a note about adjacent-text coalescing for
  small chat-tuned local models (#3094).
- integrations/cli/commands: 'Slash Commands in Chat' section listing
  the registry-backed CLI commands (/model /status /help /clear /kill
  /exit) and pointing at channel-side commands for the broader set
  (#3122 #3123).

* docs: tool invoke API, trajectory export, doctor audit, CLI default-pick, SDK codegen

Picks up the items the second misc-features agent didn't get to before
hitting the org quota.

- integrations/api/system: POST /api/tools/{name}/invoke (#3025) — fail-
  closed allowlist, agent_id requirement for approval-gated tools.
- integrations/api/agents: GET /sessions/{id}/trajectory (#3097) —
  json/jsonl formats, redactor pipeline, audit use case.
- operations/troubleshooting: 'Audit checks' subsection enumerating the
  three registered AuditChecks (#3082) — VaultKey, ApiListenAddr,
  ConfigTomlSchema — with what each asserts and how to extend.
- configuration/providers/tools: 'CLI logins as first-class default
  providers' section (#3061) — detection priority claude-code > codex-
  cli > gemini-cli > qwen-code, override via [default_model] in
  config.toml.
- integrations/sdk: 'Auto-generated from openapi.json' section (#3046) —
  scripts/codegen-sdks.py, pre-commit hook, openai-tag skip rule.

* docs(zh): mirror providers / config-core / cron / features-index sections

* docs(zh): mirror agent / api / cli / providers-tools / troubleshooting / sdk additions

Round 2 of the zh translation pass — picks up the seven mid-priority
files that the first commit didn't cover. All sections mirror the EN
content from #3189 but read naturally as Chinese (terms like agent /
session / fallback / NoEntry left in English where they're API
surface, prose translated to 中文).

- agent/templates: Live Context section, .identity/ vs root fallback,
  cache_context = true opt-out.
- api/agents: attachment-resolution table + trajectory export endpoint
  (#3025 / #3094 / #3097).
- api/system: POST /api/tools/{name}/invoke endpoint + fail-closed
  allowlist explanation.
- cli/commands: 'Slash Commands in Chat' section listing CLI registry
  commands and pointing at channel commands.
- providers/tools: 'CLI logins as first-class default providers'
  detection priority and override.
- troubleshooting: 'Audit checks' subsection + table.
- sdk: 'Auto-generated from openapi.json' section.

* docs: refresh /new slash-command semantics (#3071 review fix)

PR review on #3189 (houko) caught two stale references describing the
pre-#3071 behavior of /new ('wipe the session' / 'clear history'). The
new semantics fork off a fresh session while the previous one stays
resumable on the same channel — phrasing updated in 4 places (en + zh
× 2 files), plus a one-line clarifier in the slash-command roster
section so the cross-reference doesn't silently inherit the old
mental model:

- configuration/channels: 'wipe the session' → 'fork off into a new
  session (the prior conversation stays resumable on the channel)'
- integrations/api/realtime: '(clear history)' → 'forks into a fresh
  session — the prior session is preserved and remains resumable'
- integrations/cli/commands roster paragraph: explicit note that /new
  is a fork, not a wipe.

Same wording mirrored to docs/src/app/zh/.

* docs(channels): 9 channel-related Apr 22-25 PRs (en + zh)

Covers the channel + bridge feature wave:
- #3020 per-agent ChannelOverrides — fixes the contradicting 'without
  modifying the agent manifest' phrasing in both en and zh.
- #3077 prefix_agent_name + Signal plain-text default
- #3009 Slack reactions_enabled + #3005 Feishu @mention preservation
  consolidated into a 'Reactions and Processing State' section
- #3008 Signal media attachments
- #3012 WhatsApp send_voice + dm_policy / group_policy
- #3011 Webhook deliver_only mode
- #2972 channel file downloads (file_download_dir / max_bytes config)

* docs(config-core): provider timeout / browser CDP / cron session caps (en + zh)

Three new `config.toml` sections that landed Apr 22-23:
- #3004 `[provider_request_timeout_secs]` map
- #2993 / #2991 `[browser].cdp_endpoint` + `cdp_auth_token_env`
- #2994 `[kernel].cron_session_max_tokens` / `cron_session_max_messages`

* docs(api): GET /attach + Owner Notice envelope (en + zh)

- #3078 SSE attach endpoint for multi-client co-watching — describes
  the per-session SessionStreamHub fan-out, attacher behaviour, and
  the curl example.
- #2965 ReplyEnvelope.owner_notice + notify_owner tool — message body
  field, StreamEvent variant, WhatsApp OWNER_JIDS fan-out.

* docs(observability + ops): Tempo stack, cache_hit_ratio metric, daemon log tee (en + zh)

- #3064 + #3149 — bundled observability stack (otel-collector / Tempo
  / Grafana), bring-up commands, [telemetry] config, business spans,
  cache_hit_ratio metric formula and where it surfaces
- #3022 — `librefang start --foreground` now tees logs to
  ~/.librefang/logs/daemon-YYYY-MM-DD.log; same-day append; 7-day
  rotation pruned on daemon start

* docs: hooks transform / lazy tools / mcp taint / capability detection / providers extras

Final batch of catch-up docs covering the remaining Apr 22-25 user-facing
features:

- agent/hooks: transform_tool_result hook contract (#3003)
- agent/templates: tool_search / tool_load lazy loading + lazy_tools
  opt-out (#3047)
- integrations/mcp-a2a: TaintRuleId enum + per-tool skip_rules policy
  (#2999)
- configuration/features: compaction summaries in user's conversation
  language (#3007)
- configuration/providers/local: embedding auto-detection priority
  list (#3099) + Ollama capability detection (Modality / metadata
  pipeline, #3074 / #3133 / #3134 / #3140)
- configuration/providers/management: custom image-gen provider via
  generic OpenAI-compat driver (#2998) + Moonshot file-upload helper
  via /v1/files (#2966)

All sections mirrored to docs/src/app/zh/.

* docs: address houko fix-PR audit (8 items, en + zh)

PR review caught 1 stale-doc contradiction + 7 undocumented behaviour
changes from the fix-PR cohort.

- #3114 — configuration/core: 'api_key empty = unauthenticated' was
  stale; now documents the loopback-only policy (non-loopback bypass
  was closed)
- #3170 — features/observability: auto_start is opt-in; home_dir-
  scoped Docker labels; RAII cleanup
- #2989 — api/agents POST /message: per-request session_id override
- #3048 — Interrupting a Running Agent Turn: /stop cascades into
  agent_send subagents (recursive)
- #3056 + #3000 — desktop: dedicated Connection screen via custom
  URI scheme with retry / open-in-browser / uninstall
- #2952 — triggers: response routes to agent's home channel
- #2955 — triggers: task_posted assignee_match = 'self' | 'any'
- #3069 — providers/local: 60-second reprobe + manual 'Test
  connection' refresh

All sections mirrored in docs/src/app/zh/.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/kernel Core kernel (scheduling, RBAC, workflows) size/M 50-249 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant