Skip to content

Use yoagent Ollama preset for local tool-call compatibility #426

Description

@yuanhao

Context

Discussion #418 surfaced an Ollama/local-model compatibility issue around tool-call transcripts. Some Ollama-served models, especially llama3.x and qwen*-coder-style models, can fail or hang when a tool message is followed directly by the next non-assistant turn. The proposed local patch sets provider = "ollama" and enables requires_assistant_after_tool_result for Ollama configs.

This belongs primarily in yoagent, then yoyo should consume the new first-class constructor.

Proposed sequence

  1. Update yoagent:

    • Add a first-class ModelConfig::ollama(...) preset.
    • Set provider = "ollama".
    • Set compat.requires_assistant_after_tool_result = true for Ollama.
    • Make OpenAiCompatProvider actually honor requires_assistant_after_tool_result by inserting the required synthetic assistant message after tool results.
    • Add serialization tests for assistant(tool_calls) -> tool -> assistant(empty).
  2. Release a new yoagent version.

  3. Update yoyo-evolve:

    • Bump the yoagent dependency.
    • Change the Ollama branch in create_model_config to use the new constructor, e.g.:
"ollama" => {
    let url = base_url.unwrap_or("http://localhost:11434/v1");
    ModelConfig::ollama(url, model)
}
  1. Keep generic custom / local neutral:
    • Do not enable the Ollama transcript workaround for every local OpenAI-compatible endpoint.
    • Generic local endpoints can still set provider = "custom" or provider = "local" for identification.

Why not change all local models?

ModelConfig::local(...) covers LM Studio, vLLM, llama.cpp, Ollama, and arbitrary OpenAI-compatible servers. The assistant-after-tool-result workaround appears useful for Ollama/model-family quirks, but making it global could regress servers that expect stricter OpenAI-compatible transcripts.

Validation

After the yoagent release and yoyo update, test with Ollama models such as:

  • llama3.1:8b
  • qwen2.5-coder:7b

Useful smoke test: ask yoyo to list files in a directory and report the largest file. The important behavior is that it completes cleanly after receiving a tool result instead of hanging or failing the next request.

Related: #418

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent-inputIssues for yoyo to read

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions