Repository navigation
feat(server): API chat provider for Claude, Gemini, OpenAI and OpenRouter keys - #4
Conversation
…uter keys Normal chats could only run on agent CLIs, so a model with just an API key (Claude or Gemini on the API, OpenRouter, a local Ollama) had no way in. Add an "API chat" provider. Each instance picks a service and holds its key; the status check lists the service's models for the picker. Threads run a small ACP agent the server starts as a hidden command. It streams replies from the Anthropic or OpenAI-compatible API, keeps each conversation on the server, takes pasted images and files, and runs T3's MCP tools over HTTP, so shared memory works there too. A rate limit or spent quota answers with a usage-limit code that maps to the usage_limit failure class, so the fallback list applies. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_0128hvo8SEmbdhTeGeMM1fJe
|
ⓘ Qodo reviews are paused because your trial has ended. Ask your workspace admin to add credits to resume reviews. Manage billing |
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2ba8915711
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| makeRuntime, | ||
| // The agent's only tools are T3's, over HTTP. | ||
| mcpOverHttp: true, | ||
| promptFailure: apiChatPromptFailure, |
There was a problem hiding this comment.
When a user attaches a text file or document, the shared ACP adapter only adds its saved filesystem path through providerMessageTextWithAttachmentPaths; it embeds bytes only for images unless the flavor supplies buildAttachmentPrompt (packages/provider-acp/src/server/adapter.ts:8087–8160). This flavor supplies no attachment builder, and the API chat agent has no filesystem-reading tool, so the remote model receives a path instead of the document contents. Add attachment serialization here so the advertised attached-file support works for non-image files.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Confirmed: without buildAttachmentPrompt, non-image files reached the agent only as a saved path it can't open. Fixed in 641017c: the flavor now embeds text files and the extracted text of Office documents as resource blocks, with images as image blocks. Other binaries such as PDFs stay path-only, and the user guide now says so. This is a P2, so the commit goes up with the next push to this PR.
Generated by Claude Code
| } catch (error) { | ||
| // A failed prompt leaves no trace, so retrying it doesn't send it twice. | ||
| session.turns.length = firstTurn; | ||
| throw toRpcError(error); |
There was a problem hiding this comment.
If a tool executes successfully and the next model request returns a rate limit or another error, this catch removes the entire prompt, including the completed tool call and result. The side effect is not rolled back: for example, memory_save creates a fresh UUID and persists the entry in apps/server/src/mcp/toolkits/memory/handlers.ts. Subsequent messages therefore lose the record of work already performed, and retrying can duplicate it. Only discard a failed prompt before any externally visible work has occurred; retain and persist completed tool rounds on later failures.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Confirmed: a 429 after a tool round dropped the whole prompt, including tool results whose effects (such as a saved memory) had already happened. Fixed in 641017c: a failed prompt is dropped only when no tool has run. Otherwise history keeps everything up to the last completed tool round and is saved. The rate-limit test now covers both cases. This is a P2, so the commit goes up with the next push to this PR.
Generated by Claude Code
…fter a failure The API chat model cannot open files, so attachments now travel in the prompt: images as images, and text files or the extracted text of Office documents inline. A prompt that fails after a tool ran now keeps that tool round in the history, since the tool's effect (such as a saved memory) can't be undone. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_0128hvo8SEmbdhTeGeMM1fJe
Requested by Alek · project thread
Problem
Normal chats could only run on an installed agent CLI. A model you have only an API key for (Claude or Gemini on the API, OpenAI, OpenRouter, a local Ollama) had no way in, so there was no plain "pick a model and chat" mode.
Change
A new API chat provider (
packages/provider-api-chat):__api-chat-agent), so nothing is installed. It does the following:tool_useand OpenAItool_calls, including Gemini's thought signatures), so shared memory works in API chat too.rate_limit,RESOURCE_EXHAUSTEDorinsufficient_quotaanswer comes back as JSON-RPC code -32029, withretry-afteras the reset time. The adapter maps that code to theusage_limitfailure class, so the fallback list from feat(server): continue usage-limited threads on a fallback agent #2 moves the thread on.docs/user/providers-api-chat.md.Not included: reasoning-effort controls and context compaction. A conversation that outgrows the model's context fails with the service's error.
Scope
This is the "normal chat with a model picker" step of the AgentMayhem plan, after #1 (shared memory) and #2 (usage-limit fallback).
Verification
vp run check:changed: format, lint, knip, typecheck (42 packages) and tests all pass.services.test.tscovers SSE parsing, both stream formats with split tool-call arguments, max-token cut-offs, in-stream errors, failure classification, request bodies (role merging, tool turns as text when no tools are available), and model listing.agent.test.tsdrives the stdio agent with a fake API and a fake MCP server. It covers the tool loop, saving and then replaying onsession/load, the rate-limit error code leaving the failed prompt out of the history, and Stop keeping partial text.adapter.test.tscovers the -32029 →usage_limitmapping and its reset time.memory_savethrough T3's real MCP server, and "recall" got the memory back throughmemory_search.Rate limit → fallback agent:
Model and harness: Claude Code
Generated by Claude Code