Repository navigation
Commit 5bf7867
fix(agent): retry LLM completion on malformed 2xx JSON body (#5351)
## Problem
A provider can return HTTP 200 with a **truncated JSON body** — cleanly
closed connection, correct framing, content cut off mid-value. Both LLM
HTTP loops treated this as a terminal error on the first attempt:
`AgentError::Llm("json: EOF while parsing a value")`, surfaced as code
-32000 at the ACP boundary, killing the agent turn before it produced
anything.
Observed live in a tb2.1 bench trial (write-compressor, tb21-twins-1):
deepseek via OpenRouter returned a truncated body, the agent died
mid-prompt with 0 turns completed, and the trial scored 0 on a provider
hiccup.
Meanwhile the same loops already retry timeouts, 429s, 5xxs, 499s, and
mid-body stream stalls — a truncated-but-complete body was the one
transient upstream fault that fell through to terminal.
## Fix
In both `post()` and `openrouter_post()`
(`crates/buzz-agent/src/llm.rs`): when the fully-received success body
fails `serde_json::from_slice`, `continue` the **existing** retry loop
instead of returning terminal — same `MAX_RETRIES` (3) bound, same
`backoff_with_jitter`. On exhaustion, the error goes through
`terminal_llm_error` so it carries cumulative duration + attempt count
like every other retried failure (previously the `json:` error carried
neither).
`post_anthropic` routes through `post()`, so
Anthropic/OpenAI/Databricks/mesh and OpenRouter are all covered.
## Why this cannot re-run a tool call
Hard requirement: tool calls are not idempotent, and this change must
not introduce any possibility of replaying one.
1. **The retry lives inside the HTTP POST helper, below the parse
boundary.** Tool calls are only ever extracted from a *successfully
parsed* response value
(`parse_openai`/`parse_anthropic`/`parse_responses`, all downstream of
these helpers' `Ok` return). A malformed body never parses, therefore no
tool call was ever extracted from it, therefore nothing downstream of it
ever dispatched.
2. **What is re-sent is the completion request itself** — the identical
`body_bytes` captured once at function entry. Sending a completion
request executes no tools; it asks the model for the next message.
3. **Same safety class as existing behavior.** The loop already re-sends
this identical request on 429/5xx/timeout/stream-stall; this adds one
more transient-fault arm to the same loop with the same bytes.
## Tests
Three new tests mirroring the existing 499/dropped-connection fixtures
(raw `TcpListener` stubs):
- `post_retries_malformed_json_body_and_succeeds` — truncated 200 body
on attempt 1, valid JSON on attempt 2; asserts success and **exactly 2**
server-side requests
- `post_exhausts_retries_on_persistent_malformed_json` —
always-truncated body; asserts exactly `MAX_RETRIES` attempts and a
terminal error carrying `json:` + cumulative/attempt context
- `openrouter_post_retries_malformed_json_body_and_succeeds` — same
recovery through OpenRouter's separate loop
Full `cargo test -p buzz-agent` green at e7a5d7b (430 lib + all
integration targets, 0 failures); `cargo fmt` + `clippy --all-targets`
clean.
Originating conversation: buzz-benchmarking channel, thread 397a992d.
Signed-off-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>1 parent f029dea commit 5bf7867
1 file changed
Lines changed: 283 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2203 | 2203 | | |
2204 | 2204 | | |
2205 | 2205 | | |
2206 | | - | |
2207 | | - | |
| 2206 | + | |
| 2207 | + | |
| 2208 | + | |
| 2209 | + | |
| 2210 | + | |
| 2211 | + | |
| 2212 | + | |
| 2213 | + | |
| 2214 | + | |
| 2215 | + | |
| 2216 | + | |
| 2217 | + | |
| 2218 | + | |
| 2219 | + | |
| 2220 | + | |
| 2221 | + | |
| 2222 | + | |
| 2223 | + | |
| 2224 | + | |
| 2225 | + | |
| 2226 | + | |
| 2227 | + | |
| 2228 | + | |
| 2229 | + | |
| 2230 | + | |
| 2231 | + | |
| 2232 | + | |
2208 | 2233 | | |
2209 | 2234 | | |
2210 | 2235 | | |
| |||
2610 | 2635 | | |
2611 | 2636 | | |
2612 | 2637 | | |
2613 | | - | |
| 2638 | + | |
| 2639 | + | |
| 2640 | + | |
| 2641 | + | |
| 2642 | + | |
| 2643 | + | |
| 2644 | + | |
| 2645 | + | |
| 2646 | + | |
| 2647 | + | |
| 2648 | + | |
| 2649 | + | |
| 2650 | + | |
| 2651 | + | |
| 2652 | + | |
| 2653 | + | |
| 2654 | + | |
| 2655 | + | |
| 2656 | + | |
| 2657 | + | |
| 2658 | + | |
| 2659 | + | |
| 2660 | + | |
| 2661 | + | |
| 2662 | + | |
2614 | 2663 | | |
2615 | 2664 | | |
2616 | 2665 | | |
| |||
4791 | 4840 | | |
4792 | 4841 | | |
4793 | 4842 | | |
| 4843 | + | |
| 4844 | + | |
| 4845 | + | |
| 4846 | + | |
| 4847 | + | |
| 4848 | + | |
| 4849 | + | |
| 4850 | + | |
| 4851 | + | |
| 4852 | + | |
| 4853 | + | |
| 4854 | + | |
| 4855 | + | |
| 4856 | + | |
| 4857 | + | |
| 4858 | + | |
| 4859 | + | |
| 4860 | + | |
| 4861 | + | |
| 4862 | + | |
| 4863 | + | |
| 4864 | + | |
| 4865 | + | |
| 4866 | + | |
| 4867 | + | |
| 4868 | + | |
| 4869 | + | |
| 4870 | + | |
| 4871 | + | |
| 4872 | + | |
| 4873 | + | |
| 4874 | + | |
| 4875 | + | |
| 4876 | + | |
| 4877 | + | |
| 4878 | + | |
| 4879 | + | |
| 4880 | + | |
| 4881 | + | |
| 4882 | + | |
| 4883 | + | |
| 4884 | + | |
| 4885 | + | |
| 4886 | + | |
| 4887 | + | |
| 4888 | + | |
| 4889 | + | |
| 4890 | + | |
| 4891 | + | |
| 4892 | + | |
| 4893 | + | |
| 4894 | + | |
| 4895 | + | |
| 4896 | + | |
| 4897 | + | |
| 4898 | + | |
| 4899 | + | |
| 4900 | + | |
| 4901 | + | |
| 4902 | + | |
| 4903 | + | |
| 4904 | + | |
| 4905 | + | |
| 4906 | + | |
| 4907 | + | |
| 4908 | + | |
| 4909 | + | |
| 4910 | + | |
| 4911 | + | |
| 4912 | + | |
| 4913 | + | |
| 4914 | + | |
| 4915 | + | |
| 4916 | + | |
| 4917 | + | |
| 4918 | + | |
| 4919 | + | |
| 4920 | + | |
| 4921 | + | |
| 4922 | + | |
| 4923 | + | |
| 4924 | + | |
| 4925 | + | |
| 4926 | + | |
| 4927 | + | |
| 4928 | + | |
| 4929 | + | |
| 4930 | + | |
| 4931 | + | |
| 4932 | + | |
| 4933 | + | |
| 4934 | + | |
| 4935 | + | |
| 4936 | + | |
| 4937 | + | |
| 4938 | + | |
| 4939 | + | |
| 4940 | + | |
| 4941 | + | |
| 4942 | + | |
| 4943 | + | |
| 4944 | + | |
| 4945 | + | |
| 4946 | + | |
| 4947 | + | |
| 4948 | + | |
| 4949 | + | |
| 4950 | + | |
| 4951 | + | |
| 4952 | + | |
| 4953 | + | |
| 4954 | + | |
| 4955 | + | |
| 4956 | + | |
| 4957 | + | |
| 4958 | + | |
| 4959 | + | |
| 4960 | + | |
| 4961 | + | |
| 4962 | + | |
| 4963 | + | |
| 4964 | + | |
| 4965 | + | |
| 4966 | + | |
| 4967 | + | |
| 4968 | + | |
| 4969 | + | |
| 4970 | + | |
| 4971 | + | |
| 4972 | + | |
| 4973 | + | |
| 4974 | + | |
| 4975 | + | |
| 4976 | + | |
| 4977 | + | |
| 4978 | + | |
| 4979 | + | |
| 4980 | + | |
| 4981 | + | |
| 4982 | + | |
| 4983 | + | |
| 4984 | + | |
| 4985 | + | |
| 4986 | + | |
| 4987 | + | |
| 4988 | + | |
| 4989 | + | |
| 4990 | + | |
| 4991 | + | |
| 4992 | + | |
| 4993 | + | |
| 4994 | + | |
| 4995 | + | |
| 4996 | + | |
| 4997 | + | |
| 4998 | + | |
| 4999 | + | |
| 5000 | + | |
| 5001 | + | |
| 5002 | + | |
| 5003 | + | |
| 5004 | + | |
| 5005 | + | |
| 5006 | + | |
| 5007 | + | |
| 5008 | + | |
| 5009 | + | |
| 5010 | + | |
| 5011 | + | |
| 5012 | + | |
| 5013 | + | |
| 5014 | + | |
| 5015 | + | |
| 5016 | + | |
| 5017 | + | |
| 5018 | + | |
| 5019 | + | |
| 5020 | + | |
| 5021 | + | |
| 5022 | + | |
| 5023 | + | |
| 5024 | + | |
| 5025 | + | |
| 5026 | + | |
| 5027 | + | |
| 5028 | + | |
| 5029 | + | |
| 5030 | + | |
| 5031 | + | |
| 5032 | + | |
| 5033 | + | |
| 5034 | + | |
| 5035 | + | |
| 5036 | + | |
| 5037 | + | |
| 5038 | + | |
| 5039 | + | |
| 5040 | + | |
| 5041 | + | |
| 5042 | + | |
| 5043 | + | |
| 5044 | + | |
| 5045 | + | |
| 5046 | + | |
| 5047 | + | |
| 5048 | + | |
| 5049 | + | |
| 5050 | + | |
| 5051 | + | |
| 5052 | + | |
| 5053 | + | |
| 5054 | + | |
| 5055 | + | |
| 5056 | + | |
| 5057 | + | |
| 5058 | + | |
| 5059 | + | |
| 5060 | + | |
| 5061 | + | |
| 5062 | + | |
| 5063 | + | |
| 5064 | + | |
| 5065 | + | |
| 5066 | + | |
| 5067 | + | |
| 5068 | + | |
| 5069 | + | |
| 5070 | + | |
| 5071 | + | |
| 5072 | + | |
| 5073 | + | |
4794 | 5074 | | |
4795 | 5075 | | |
4796 | 5076 | | |
| |||
0 commit comments