Skip to content

Commit e6954e8

Browse files
committed
feat(rust): LLM HTTP transport (build_body -> POST -> SSE -> decode)
PR #27: the transport that ties the request + decode sides together, end-to-end and testable over plain HTTP (TLS-decoupled). - opencode-llm::transport::complete(client, url, headers, protocol, request): build_body -> reqwest POST (JSON) -> read response -> non-2xx becomes LlmError::Status { code, retryable, message } (429/5xx retryable), network failures become LlmError::Http -> else decode_sse into Vec<LlmEvent>. Collect-then-decode (incremental streaming is a refinement). - LlmError gains Http + Status; opencode-llm gains reqwest (already no-TLS in the workspace), with tokio + axum as dev-deps (both already vetted). - Tests run end-to-end against an in-test axum server (no wiremock, no TLS): happy-path event parity ("Hi" + Finish) and 429/401 status classification. Deliberately TLS-decoupled: real HTTPS needs rustls+ring (owner-approved), whose cargo-deny license entry for `ring` can't be verified locally — that re-add is a focused follow-up so it can't block this transport logic. Executor (retry/backoff/redaction) consumes Status::retryable in a later increment. Verified: build/clippy -D warnings/fmt/test (llm 12) and openapi-diff all green. No new external crates (reqwest/tokio/axum already in the tree). https://claude.ai/code/session_017Lo2jkGdTgRvsPG6tcwEqz
1 parent 013229f commit e6954e8

5 files changed

Lines changed: 189 additions & 2 deletions

File tree

‎Cargo.lock‎

Lines changed: 3 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎RUST_MIGRATION.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,8 @@ cargo-dist · cargo-deny. **Out of scope:** tree-sitter (TUI-only).
8787
### Phase 3 — LLM 🟡
8888
- ✅ **Protocol decode pipeline** (PR #25): `opencode-llm` defines the normalized `LlmEvent`/`Usage`/`FinishReason` model + the `Protocol` trait (decode side: `decode_frame` → `step(state) → Vec<LlmEvent>` → `terminal`) + `sse_frames`/`decode_sse` (the `sseFraming` analog). First impl: **anthropic-messages** — folds the Anthropic SSE stream (`message_start`/`content_block_*`/`message_delta`/`message_stop`/`ping`/`error`) into normalized events, threading open blocks + streaming tool-arg buffers + merged usage. **Parity proven network-free** by replaying real `http-recorder` cassettes (`streams-text`, `streams-tool-call`) and asserting the event stream (text, tool-call input `{"city":"Paris"}`, finish reason, token usage).
8989
- ✅ **Request lowering** (`body.from`, PR #26): the `Protocol` trait gains `Body` + `build_body`; `LlmRequest`/`Message`/`ContentPart`/`ToolDefinition`/`ToolChoice`/`Generation` model the normalized request (core subset). anthropic-messages lowers it to the Anthropic body (system text blocks, user/assistant/tool → content blocks incl. `tool_use`/`tool_result`, tools, `tool_choice` auto/any/tool, `max_tokens` default 4096, sampling params). **Parity proven** against the cassettes' recorded `request.body` (text + tool-call). Media/reasoning/cache/thinking/mid-conversation-system deferred.
90-
- ⬜ Next: the reqwest/SSE **transport** (re-adds TLS — **`rustls`+`ring`**, owner-approved; needs a `cargo-deny` license-allow entry for `ring`) + `route/executor.ts` (retry/backoff + redaction); then openai-chat → openai-responses → gemini → bedrock-converse. No public cutover (the runner consumes it; Phase 4).
90+
- ✅ **HTTP transport** (PR #27): `opencode-llm::transport::complete` — `build_body` → reqwest `POST` (JSON) → SSE response → `decode_sse` → `Vec<LlmEvent>`; non-2xx → `LlmError::Status { code, retryable, message }` (429/5xx retryable), network → `LlmError::Http`. Tested end-to-end against an in-test **axum** server (happy-path event parity + 429/401 classification) over plain HTTP — no new external crates, no TLS yet. Collect-then-decode (incremental streaming is a refinement).
91+
- ⬜ Next: re-add **TLS** to reqwest for real HTTPS provider calls (**`rustls`+`ring`**, owner-approved; isolated PR so the `cargo-deny` license-allow for `ring` can be iterated alone) + `route/executor.ts` (retry/backoff + redaction, consuming `Status::retryable`); then openai-chat → openai-responses → gemini → bedrock-converse. No public cutover (the runner consumes it; Phase 4).
9192

9293
### Phase 4 — Session runner 🟡
9394
- ✅ **Control-flow spike** (PR #24, pure logic — no IO/cutover): `opencode-core::runner` proves the `session/runner/llm.ts` control flow maps to panic-free Rust. `die(TurnTransitionError)`/`catchDefect` → `TurnTransition` (`RebuildPreparedTurn{promotion}` / `ContinueAfterOverflowCompaction`) returned as `Err` + the `run_turn` restart driver (flips `OverflowRecovery` `Enabled→Disabled`; a second overflow is `DoubleOverflow`); `needsContinuation` → `TurnOutcome::{Continue,Done}`; the outer continuation loop → `run_session` (step-limited); `FiberSet` + `raceFirst(join,awaitEmpty)` → `ToolExecutor` (over `tokio::task::JoinSet`) with `drain` (fail-fast vs all-settled) + `cancel_all`. 14 unit tests exercise every transition + tool race/timeout/cancel.

‎crates/opencode-llm/Cargo.toml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,3 +10,8 @@ description = "opencode LLM protocol router (ported from packages/llm) — Phase
1010
serde.workspace = true
1111
serde_json.workspace = true
1212
thiserror.workspace = true
13+
reqwest.workspace = true
14+
15+
[dev-dependencies]
16+
tokio.workspace = true
17+
axum.workspace = true

‎crates/opencode-llm/src/lib.rs‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212
//! openai-responses → gemini → bedrock-converse.
1313
1414
pub mod anthropic;
15+
pub mod transport;
1516

1617
use serde::{Deserialize, Serialize};
1718

@@ -113,12 +114,25 @@ pub enum LlmEvent {
113114
},
114115
}
115116

116-
/// Errors decoding a protocol stream.
117+
/// Errors building, sending, or decoding a protocol stream.
117118
#[derive(Debug, thiserror::Error)]
118119
pub enum LlmError {
119120
/// A frame could not be parsed into the protocol's event type.
120121
#[error("failed to decode frame: {0}")]
121122
Decode(String),
123+
/// A network/transport error reaching the provider.
124+
#[error("http transport error: {0}")]
125+
Http(String),
126+
/// The provider returned a non-success HTTP status.
127+
#[error("provider returned status {code}: {message}")]
128+
Status {
129+
/// HTTP status code.
130+
code: u16,
131+
/// Whether the request is worth retrying (the executor uses this; Phase 3 follow-up).
132+
retryable: bool,
133+
/// Response body / error message.
134+
message: String,
135+
},
122136
}
123137

124138
/// A provider-family protocol pipeline. This increment covers the **decode** side: parse one frame
Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
//! HTTP transport for protocol streams (`packages/llm/src/route/transport/http.ts` analog): lower the
2+
//! request, POST it, and decode the SSE response into normalized events.
3+
//!
4+
//! This increment collects the response then decodes it (the events are identical to incremental
5+
//! decoding; chunk-by-chunk streaming is a refinement). It runs over plain HTTP — the workspace
6+
//! `reqwest` has no TLS backend yet; re-adding `rustls`+`ring` for real HTTPS provider calls is a
7+
//! focused follow-up. Retry/backoff + secret redaction (`route/executor.ts`) are a later increment;
8+
//! [`LlmError::Status::retryable`] is the classification that executor will consume.
9+
10+
use crate::{decode_sse, LlmError, LlmEvent, LlmRequest, Protocol};
11+
12+
/// Whether a non-success status is worth retrying (429 + 5xx-ish), mirroring `route/executor.ts`'s
13+
/// status classification.
14+
fn is_retryable(status: u16) -> bool {
15+
matches!(status, 429 | 500 | 502 | 503 | 504 | 529)
16+
}
17+
18+
/// POST `protocol.build_body(request)` to `url` (with extra `headers`, e.g. `x-api-key`) and decode
19+
/// the SSE response into the normalized event stream. Non-2xx responses become
20+
/// [`LlmError::Status`]; network failures become [`LlmError::Http`].
21+
pub async fn complete<P: Protocol>(
22+
client: &reqwest::Client,
23+
url: &str,
24+
headers: &[(&str, &str)],
25+
protocol: &P,
26+
request: &LlmRequest,
27+
) -> Result<Vec<LlmEvent>, LlmError> {
28+
let body = protocol.build_body(request)?;
29+
let mut builder = client.post(url).json(&body);
30+
for (name, value) in headers {
31+
builder = builder.header(*name, *value);
32+
}
33+
let response = builder
34+
.send()
35+
.await
36+
.map_err(|e| LlmError::Http(e.to_string()))?;
37+
let status = response.status().as_u16();
38+
let text = response
39+
.text()
40+
.await
41+
.map_err(|e| LlmError::Http(e.to_string()))?;
42+
if status >= 400 {
43+
return Err(LlmError::Status {
44+
code: status,
45+
retryable: is_retryable(status),
46+
message: text,
47+
});
48+
}
49+
decode_sse(protocol, &text)
50+
}
51+
52+
#[cfg(test)]
53+
mod tests {
54+
use super::*;
55+
use crate::anthropic::AnthropicMessages;
56+
use crate::{LlmRequest, Message};
57+
use axum::{http::StatusCode, response::IntoResponse, routing::post, Router};
58+
59+
const SSE_BODY: &str = "event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"usage\":{\"input_tokens\":3,\"output_tokens\":1}}}\n\nevent: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"text\",\"text\":\"\"}}\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"Hi\"}}\n\nevent: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":0}\n\nevent: message_delta\ndata: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"end_turn\"},\"usage\":{\"input_tokens\":3,\"output_tokens\":4}}\n\nevent: message_stop\ndata: {\"type\":\"message_stop\"}\n\n";
60+
61+
/// Spawn an in-test HTTP server (axum, already a vetted workspace dep — no wiremock/TLS) and return
62+
/// its base URL.
63+
async fn spawn(router: Router) -> String {
64+
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
65+
let addr = listener.local_addr().unwrap();
66+
tokio::spawn(async move {
67+
axum::serve(listener, router).await.unwrap();
68+
});
69+
format!("http://{addr}")
70+
}
71+
72+
fn request() -> LlmRequest {
73+
LlmRequest {
74+
model: "claude-haiku-4-5-20251001".into(),
75+
messages: vec![Message::user_text("hi")],
76+
..Default::default()
77+
}
78+
}
79+
80+
#[tokio::test]
81+
async fn posts_and_decodes_the_sse_response() {
82+
let app = Router::new().route(
83+
"/v1/messages",
84+
post(|| async { ([("content-type", "text/event-stream")], SSE_BODY) }),
85+
);
86+
let base = spawn(app).await;
87+
let client = reqwest::Client::new();
88+
let events = complete(
89+
&client,
90+
&format!("{base}/v1/messages"),
91+
&[("x-api-key", "test"), ("anthropic-version", "2023-06-01")],
92+
&AnthropicMessages,
93+
&request(),
94+
)
95+
.await
96+
.unwrap();
97+
98+
let text: String = events
99+
.iter()
100+
.filter_map(|e| match e {
101+
LlmEvent::TextDelta { text, .. } => Some(text.as_str()),
102+
_ => None,
103+
})
104+
.collect();
105+
assert_eq!(text, "Hi");
106+
assert!(matches!(events.last(), Some(LlmEvent::Finish { .. })));
107+
}
108+
109+
#[tokio::test]
110+
async fn rate_limit_is_retryable_status() {
111+
let app = Router::new().route(
112+
"/v1/messages",
113+
post(|| async { (StatusCode::TOO_MANY_REQUESTS, "slow down").into_response() }),
114+
);
115+
let base = spawn(app).await;
116+
let client = reqwest::Client::new();
117+
let err = complete(
118+
&client,
119+
&format!("{base}/v1/messages"),
120+
&[],
121+
&AnthropicMessages,
122+
&request(),
123+
)
124+
.await
125+
.unwrap_err();
126+
match err {
127+
LlmError::Status {
128+
code, retryable, ..
129+
} => {
130+
assert_eq!(code, 429);
131+
assert!(retryable);
132+
}
133+
other => panic!("expected Status, got {other:?}"),
134+
}
135+
}
136+
137+
#[tokio::test]
138+
async fn unauthorized_is_non_retryable_status() {
139+
let app = Router::new().route(
140+
"/v1/messages",
141+
post(|| async { (StatusCode::UNAUTHORIZED, "bad key").into_response() }),
142+
);
143+
let base = spawn(app).await;
144+
let client = reqwest::Client::new();
145+
let err = complete(
146+
&client,
147+
&format!("{base}/v1/messages"),
148+
&[],
149+
&AnthropicMessages,
150+
&request(),
151+
)
152+
.await
153+
.unwrap_err();
154+
match err {
155+
LlmError::Status {
156+
code, retryable, ..
157+
} => {
158+
assert_eq!(code, 401);
159+
assert!(!retryable);
160+
}
161+
other => panic!("expected Status, got {other:?}"),
162+
}
163+
}
164+
}

0 commit comments

Comments
 (0)