Context
GitHub discussion yologdev/yoyo-evolve#418 surfaced an Ollama/local-model compatibility issue around OpenAI-compatible tool-call transcripts. Some Ollama-served models, especially llama3.x and qwen*-coder-style models, can fail or hang when a tool message is followed directly by the next non-assistant turn.
yoagent already has the compatibility flag:
OpenAiCompat {
requires_assistant_after_tool_result: bool,
...
}
but OpenAiCompatProvider does not currently honor it when serializing messages.
Proposed change
Add first-class Ollama support as a local OpenAI-compatible preset:
- Add
ModelConfig::ollama(base_url, model_id) or equivalent.
- Default yoyo/yoyo-evolve usage should pass
http://localhost:11434/v1.
- Set
provider = "ollama".
- Set
api = ApiProtocol::OpenAiCompletions.
- Set
compat.requires_assistant_after_tool_result = true.
- Keep
ModelConfig::local(...) neutral.
Implement requires_assistant_after_tool_result in OpenAiCompatProvider:
- When serializing a
Message::ToolResult, emit the normal OpenAI-compatible tool message.
- If the compat flag is enabled, insert a synthetic assistant message immediately after the tool result.
- Add tests for the serialized sequence:
assistant(tool_calls) -> tool -> assistant(empty)
Why not change ModelConfig::local(...)?
ModelConfig::local(...) covers LM Studio, vLLM, llama.cpp, Ollama, and arbitrary OpenAI-compatible servers. The assistant-after-tool-result workaround appears useful for Ollama/model-family quirks, but making it global could regress servers that expect stricter OpenAI-compatible transcripts.
Validation
Unit tests should cover:
ModelConfig::ollama(...) has provider "ollama" and the compat flag enabled.
- generic
ModelConfig::local(...) remains provider "local" and keeps the compat flag disabled.
- OpenAI-compatible request serialization inserts the synthetic assistant message only when the flag is enabled.
Manual smoke test after release:
- Run through yoyo with Ollama
llama3.1:8b or qwen2.5-coder:7b.
- Ask it to list files in a directory and report the largest file.
- Expected behavior: the model completes cleanly after receiving the tool result instead of hanging or failing the next request.
Follow-up consumer issue: yologdev/yoyo-evolve#426
Context
GitHub discussion yologdev/yoyo-evolve#418 surfaced an Ollama/local-model compatibility issue around OpenAI-compatible tool-call transcripts. Some Ollama-served models, especially llama3.x and qwen*-coder-style models, can fail or hang when a
toolmessage is followed directly by the next non-assistant turn.yoagent already has the compatibility flag:
but
OpenAiCompatProviderdoes not currently honor it when serializing messages.Proposed change
Add first-class Ollama support as a local OpenAI-compatible preset:
ModelConfig::ollama(base_url, model_id)or equivalent.http://localhost:11434/v1.provider = "ollama".api = ApiProtocol::OpenAiCompletions.compat.requires_assistant_after_tool_result = true.ModelConfig::local(...)neutral.Implement
requires_assistant_after_tool_resultinOpenAiCompatProvider:Message::ToolResult, emit the normal OpenAI-compatibletoolmessage.Why not change
ModelConfig::local(...)?ModelConfig::local(...)covers LM Studio, vLLM, llama.cpp, Ollama, and arbitrary OpenAI-compatible servers. The assistant-after-tool-result workaround appears useful for Ollama/model-family quirks, but making it global could regress servers that expect stricter OpenAI-compatible transcripts.Validation
Unit tests should cover:
ModelConfig::ollama(...)has provider"ollama"and the compat flag enabled.ModelConfig::local(...)remains provider"local"and keeps the compat flag disabled.Manual smoke test after release:
llama3.1:8borqwen2.5-coder:7b.Follow-up consumer issue: yologdev/yoyo-evolve#426