Skip to content

Add first-class Ollama preset and honor assistant-after-tool-result compat #37

Description

@yuanhao

Context

GitHub discussion yologdev/yoyo-evolve#418 surfaced an Ollama/local-model compatibility issue around OpenAI-compatible tool-call transcripts. Some Ollama-served models, especially llama3.x and qwen*-coder-style models, can fail or hang when a tool message is followed directly by the next non-assistant turn.

yoagent already has the compatibility flag:

OpenAiCompat {
    requires_assistant_after_tool_result: bool,
    ...
}

but OpenAiCompatProvider does not currently honor it when serializing messages.

Proposed change

Add first-class Ollama support as a local OpenAI-compatible preset:

  • Add ModelConfig::ollama(base_url, model_id) or equivalent.
  • Default yoyo/yoyo-evolve usage should pass http://localhost:11434/v1.
  • Set provider = "ollama".
  • Set api = ApiProtocol::OpenAiCompletions.
  • Set compat.requires_assistant_after_tool_result = true.
  • Keep ModelConfig::local(...) neutral.

Implement requires_assistant_after_tool_result in OpenAiCompatProvider:

  • When serializing a Message::ToolResult, emit the normal OpenAI-compatible tool message.
  • If the compat flag is enabled, insert a synthetic assistant message immediately after the tool result.
  • Add tests for the serialized sequence:
assistant(tool_calls) -> tool -> assistant(empty)

Why not change ModelConfig::local(...)?

ModelConfig::local(...) covers LM Studio, vLLM, llama.cpp, Ollama, and arbitrary OpenAI-compatible servers. The assistant-after-tool-result workaround appears useful for Ollama/model-family quirks, but making it global could regress servers that expect stricter OpenAI-compatible transcripts.

Validation

Unit tests should cover:

  • ModelConfig::ollama(...) has provider "ollama" and the compat flag enabled.
  • generic ModelConfig::local(...) remains provider "local" and keeps the compat flag disabled.
  • OpenAI-compatible request serialization inserts the synthetic assistant message only when the flag is enabled.

Manual smoke test after release:

  • Run through yoyo with Ollama llama3.1:8b or qwen2.5-coder:7b.
  • Ask it to list files in a directory and report the largest file.
  • Expected behavior: the model completes cleanly after receiving the tool result instead of hanging or failing the next request.

Follow-up consumer issue: yologdev/yoyo-evolve#426

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions