fix: discourage _noop tool call during LiteLLM compaction#18539
fix: discourage _noop tool call during LiteLLM compaction#18539rekram1-node merged 1 commit intoanomalyco:devfrom
Conversation
github-actions
bot
added
the
needs:compliance
|
The following comment was made by an LLM, it may be inaccurate: Found one potentially related PR: #8497 - "fix: handle dangling tool_use blocks for LiteLLM proxy compatibility" This PR also addresses LiteLLM proxy compatibility issues, which is related to the current PR's focus on fixing LiteLLM/Bedrock tool handling. However, it appears to be addressing a different specific issue (dangling tool_use blocks) rather than the empty tools array and inputSchema validation problem that PR #18539 fixes. |
github-actions
bot
removed
the
needs:compliance
|
Thanks for updating your PR! It now meets our contributing guidelines. 👍 |
KnutZuidema
force-pushed
the
opencode/shiny-eagle
branch
2 times, most recently
from
89489b0 to
2a9f61b
Compare
| input.model.api.id.toLowerCase().includes("litellm") | ||
|
|
||
| if (isLiteLLMProxy && Object.keys(tools).length === 0 && hasToolCalls(input.messages)) { | ||
| tools["_noop"] = tool({ |
There was a problem hiding this comment.
is a read tool better than just adding an input schema for noop tool?
I dont use litellm
There was a problem hiding this comment.
to be honest, this worked for me once, but now upon retrying, it doesn't work anymore.
I must've gotten lucky and the compaction agent didn't make a tool call.
It appears, that when compacting, the compaction agent tends to first respond with a tool call and the first response causes the previous history to be deleted, causing the compaction to fail.
Ideally, the agent would just never make a tool call. I'm not sure yet how to fix this, will report back if I find out more.
This issue is specific to litellm, because the provider apparently requires a non-empty tool list.
Without it, I can't get any completion request to succeed.
There was a problem hiding this comment.
I think the best bet to fix this is to simply state clearly that the noop tool shouldn't be used and discourage tool use in the compaction prompt in general.
Let me know if that works for you. For me it did every time I tried it so far, but of course there's technically no guarantee.
I've also thought about actually adjusting the compaction process to simply not finish until a text response has been received, but that seemed like an unnecessarily invasive change, when this problem only affects LiteLLM/Bedrock.
KnutZuidema
force-pushed
the
opencode/shiny-eagle
branch
from
8d5f2dc to
ec865da
Compare
KnutZuidema
changed the title
When compaction runs with no active tools, LiteLLM/Bedrock rejects the request if the message history contains tool calls but the tools param is absent. The existing fix injected a _noop stub with an empty inputSchema, but Bedrock also rejected that schema and the model would call the stub instead of generating a summary. This updates the stub to have a valid inputSchema (so Bedrock accepts it) and a description that explicitly tells the model not to call it. The compaction prompt also now instructs the model to respond with text only and not call any tools.
KnutZuidema
force-pushed
the
opencode/shiny-eagle
branch
from
ec865da to
0dfa447
Compare
2 participants
Issue for this PR
Closes #14368
Closes #18053
Type of change
What does this PR do?
When compaction runs with no active tools, LiteLLM/Bedrock rejects the request if the message history contains tool calls but the
toolsparam is absent. The existing fix injected a_noopstub. The model would call the stub instead of generating a summary — producing the post-compaction amnesia. This PR changes the stubs description to explicitly say it must never be called, and adds a "Do not call any tools" instruction to the compaction prompt so the model is discouraged from invoking it via both the tool definition and the prompt.How did you verify your code works?
Ran
bun typecheckfrompackages/opencode/— no errors. I also ran opencode locally with the fix applied and triggered compaction in a live session (the session in which this PR was created). The session successfully compacted and retained context across the compaction boundary, which was not possible before the fix.Screenshots / recordings
N/A — no UI change.
Checklist