Skip to content

feat(opencode): research-backed agent tuning - #225

Merged
matteo-pacini merged 7 commits into
masterfrom
opencode-research-tuning
Apr 28, 2026
Merged

matteo-pacini merged 7 commits into
masterfrom
opencode-research-tuning

Conversation

@matteo-pacini

Copy link
Copy Markdown
Owner

Summary

Research-backed pass over the OpenCode home-manager module (7 commits + docs):

  • Fix max_iterations → steps (was an unrecognized field, silently dropped → all agents were running at OpenCode's default of Infinity iterations).
  • Tune sampling per provider model cards (Kimi K2.6, DeepSeek V4 Flash, Gemini 3 Flash). All three providers explicitly recommend temperature = 1.0 and warn that lower values cause looping/repetition/degraded reasoning.
  • Add per-agent reasoning via options.reasoning.effort — the nested form OpenRouter actually consumes. Bare reasoningEffort ends up at providerOptions.openrouter.reasoningEffort and is silently ignored.
  • Switch small_model from gemini-3-flash-preview to gemini-2.5-flash-lite. The schema makes small_model a string with no per-call override, so titles would otherwise inherit Gemini 3's hardcoded thinking_level=high (~$3/Mtok output, wasteful for 5-token titles).
  • Disable runtime autoupdate (Nix manages the binary; nixpkgs derivation does not suppress this on its own).
  • Move theme from opencode.json → sibling tui.json (root schema has additionalProperties: false).
  • Drop dead permission.write blocks — write is not a canonical permission key per the schema; edit already gates writes.
  • Document all rationale, references, and open issues inline in opencode.nix so future readers can audit choices without re-running the research.

🤖 Generated with Claude Code

matteo-pacini and others added 7 commits April 28, 2026 13:49
The OpenCode agent schema uses `steps` for iteration caps; `max_iterations`
is not a recognized field and was being silently passed through to the
provider as an opaque option, leaving every agent effectively unbounded.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
- small_model: route lightweight calls (title generation) to
  gemini-3-flash-preview, already used by the review agent
- instructions: explicit AGENTS.md reference; auto-discovery would find
  it anyway, this just documents the dependency
- plan agent: temperature 0.3 → 0.1 to match the docs' recommended
  0.0-0.2 range for planning/analysis work

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Nix manages the package version via the nixpkgs derivation, so OpenCode's
runtime self-update has nothing to write to (read-only /nix/store). This
silences the update check / "update available" prompts and avoids the
network call.

The nixpkgs derivation does not suppress this on its own — it only sets
OPENCODE_DISABLE_MODELS_FETCH for the model registry, not the binary
auto-updater.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Every agent was running with temperature 0.1-0.2, but the model cards for
all three configured models (Kimi K2.6, DeepSeek V4 Flash, Gemini 3 Flash)
explicitly recommend temp=1.0 with model-specific top_p, and warn that
lower temps cause repetition / looping / degraded reasoning. These are
the same anti-patterns Qwen has below 0.3.

- build   (Kimi K2.6, primary):   1.0  / top_p 0.95   thinking-mode profile
- plan    (Kimi K2.6, primary):   0.6  / top_p 0.95   instant-mode profile
- debug   (Kimi K2.6, subagent):  0.6  / top_p 0.95   instant-mode profile
- explore (DeepSeek V4 Flash):    1.0  / top_p 1.0    V4 model card guidance
- review  (Gemini 3 Flash, prev): unset                Google strongly
                                                       recommends keeping
                                                       Gemini 3 default 1.0

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…all_model

Per-role tuning derived from each provider's model card and OpenRouter's
reasoning-tokens spec. Three parallel research agents (one per configured
model) converged on the same field convention (`reasoningEffort` camelCase
scalar) and the values below.

small_model: gemini-3-flash-preview → gemini-2.5-flash-lite
  OpenCode's schema makes small_model a string with no per-call overrides,
  so titles/summaries always run with Gemini 3 Flash's default
  thinking_level=high — wasteful for 5-token title generation. flash-lite
  has no reasoning overhead, GA not preview, ~75% cheaper for aux calls.

Per-agent changes (all bumped from previously-too-tight steps caps):

  build (Kimi K2.6, primary)
    top_p 0.95 → 1.0   (mirror Moonshot's SWE-bench harness exactly)
    steps 25  → 50     (Moonshot's published max-steps=50)
    + reasoningEffort = "high"

  plan (Kimi K2.6, primary)
    temperature 0.6 → 1.0  (was uncommitted; thinking-mode profile)
    steps 15 → 20
    + reasoningEffort = "xhigh"  (most reasoning-bound role; highest
                                  leverage knob in the whole config)

  debug (Kimi K2.6, subagent)
    temperature 0.6 → 1.0  (was uncommitted; thinking-mode profile)
    steps 15 → 30
    + reasoningEffort = "high"  (hypothesis-driven test/observe loops)

  explore (DeepSeek V4 Flash, subagent)
    steps 10 → 20
    + reasoningEffort = "high"  (NOT xhigh — Think Max requires ≥384K
                                 context and burns ~95% of budget on
                                 reasoning; wrong for a blocking subagent)

  review (Gemini 3 Flash Preview, subagent)
    steps 10 → 20
    + reasoningEffort = "high"
    Temperature stays unset by design — Google explicitly warns that any
    override below default 1.0 causes "looping or degraded performance,
    particularly with complex reasoning tasks."

Operational watch-out: anomalyco/opencode#24190 reports DeepSeek V4
multi-turn tool calls with reasoning enabled may return 400s due to
reasoning_details not being round-tripped (related: #4367). If this
fires on the explore subagent, the workaround is reasoningEffort = "none".

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…l audit

Three parallel research agents — one auditing the JSON schema, one
comparing against community configs, one reading OpenCode's source for
the reasoning passthrough — surfaced four corrections to the previous
commit. The previous values were correct in spirit, but the field names
and structure didn't actually do what they claimed.

theme → tui.json
  The root config schema has additionalProperties: false and the docs
  state theme/keybinds moved to a sibling tui.json. Runtime currently
  accepts it via fallback but strict validators reject. Fix: write
  ~/.config/opencode/tui.json with { "theme": "dracula" } and drop the
  field from opencode.json.

reasoningEffort → options.reasoning.effort
  Confirmed against packages/opencode/src/provider/transform.ts and the
  OpenRouter AI SDK provider tests: agent-level reasoningEffort is
  normalised onto agent.options.reasoningEffort, then spread into
  providerOptions.openrouter.reasoningEffort — which OpenRouter silently
  ignores (only the nested reasoning: { effort } object is recognised).
  The wrapping options = { reasoning = { effort = "..."; }; } is the
  shape that actually propagates to the request body.

  - build, plan, debug, explore: switched to options.reasoning.effort.
  - review: reasoningEffort dropped entirely. transform.ts:875-877
    hardcodes reasoning: { effort: "high" } unconditionally for any
    gemini-3* model regardless of agent setting, so any per-agent
    setting is redundant.

permission.write removed
  Per the schema and docs, "write" is not a canonical permission key.
  The wildcard block was being silently absorbed by additionalProperties
  and doing nothing. File writes are gated by the "edit" permission,
  which already denies them where applicable.

build top_p 1.0 → 0.95
  OpenCode auto-injects temp=1.0, top_p=0.95 for kimi-k2.6 (per
  transform.ts model-specific defaults), matching Moonshot's official
  K2.6 model card. The previous 1.0 mirrored the SWE-bench harness but
  overrode the smarter built-in default. 0.95 brings build into line
  with the rest of the Kimi agents and the OpenCode/Moonshot consensus.

Known limitations (not in this commit):
  - For Kimi K2.6 via OpenRouter, OpenCode has no client-side
    "thinking-on" toggle (auto-thinking is gated to @ai-sdk/anthropic
    only, transform.ts:914-922). The options.reasoning.effort value
    propagates to OpenRouter, which forwards it to Moonshot, but the
    underlying behaviour depends on Moonshot's gateway honouring it.
  - DeepSeek V4 Flash multi-turn tool calls via OpenRouter still has
    open issues anomalyco/opencode#24569 and #24722 around
    reasoning_content not being round-tripped. Workaround if 400s fire
    on the explore subagent: drop options.reasoning.effort entirely.
  - Verification command:
      OPENCODE_LOG_LEVEL=DEBUG opencode run --agent build "say hi" \
        2>&1 | grep -iE 'temperature|top_p|reasoning'
    Expect to see reasoning: { effort: "..." } in the outgoing request,
    not reasoningEffort: "..." at the top level.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Add comprehensive in-file documentation so future readers can understand
WHY each value was chosen without re-running the research.

Captures:

  * Verification recipe — OPENCODE_LOG_LEVEL=DEBUG command plus
    `opencode debug agent <name>` for inspecting resolved per-agent
    settings.

  * Primary references — schema URL, agents/config/permission docs,
    OpenCode source (transform.ts is the canonical location for model
    defaults and the reasoning passthrough whitelist), OpenRouter
    reasoning-tokens guide.

  * Open issues to monitor — DeepSeek V4 multi-turn 400s (#24569,
    #24722), Kimi/Qwen variants exclusion (#23334 / PR #23335),
    subagent variants runtime application (#21632), TUI reasoning
    visibility (#18634), with a 2026-04-28 snapshot date.

  * Conventions block — explains the four cross-cutting rules (Kimi
    sampling profile, options.reasoning.effort vs reasoningEffort,
    steps Infinity default, permission.write being a no-op key) before
    the agent blocks so each agent's comment can stay focused on its
    specifics.

  * Per-section comments — each top-level field and each agent gets a
    rationale block citing the model card / source line / community
    precedent that justifies its values, plus model-specific caveats
    (Kimi has no client-side thinking toggle; DeepSeek V4 may 400 on
    multi-turn; Gemini 3 hardcodes reasoning effort=high).

No behaviour change — only comments. Flake check passes; serialised
opencode.json is byte-identical since Nix strips comments at evaluation.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@matteo-pacini
matteo-pacini force-pushed the opencode-research-tuning branch from 1dfe554 to e35d39f Compare April 28, 2026 13:46
@matteo-pacini
matteo-pacini merged commit fa42662 into master Apr 28, 2026
14 checks passed
@matteo-pacini
matteo-pacini deleted the opencode-research-tuning branch April 28, 2026 14:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant