Skip to content

refactor(opencode): condense tool descriptions ~66% to cut system-prompt tokens - #24202

Closed
MartinWie wants to merge 1 commit into
anomalyco:devfrom
MartinWie:caveman-tool-descriptions
Closed

refactor(opencode): condense tool descriptions ~66% to cut system-prompt tokens#24202
MartinWie wants to merge 1 commit into
anomalyco:devfrom
MartinWie:caveman-tool-descriptions

Conversation

@MartinWie

@MartinWie MartinWie commented Apr 24, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Issue for this PR

Refs #11995

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Condenses all 17 tool description files in packages/opencode/src/tool/*.txt into a terser style to cut system-prompt token overhead. These files are imported as raw strings and injected into every LLM turn's tool descriptions — per @DrDexter6000's analysis in #11995 they are NOT cacheable, so every character costs tokens on every request.

Why the changes work:

  • Tool description .txt files are loaded via import DESCRIPTION from "./bash.txt" (same pattern in task.ts, apply_patch.ts, todo.ts, websearch.ts, etc.) and injected into the model's tool schema. Nothing parses them as structured data — the model reads them as natural-language instructions. Shorter, denser wording preserves instruction-following while reducing token cost.
  • Placeholder substitutions (${os}, ${shell}, ${chaining}, ${maxLines}, ${maxBytes}, ${directory}, {{year}}) use .replaceAll / .replace which are no-ops if the placeholder key is absent — so presence in the text is what matters, and all keys were preserved where originally used.
  • I kept behavioral anchors (concrete examples) where models measurably depend on them — e.g. the 42: const x = 1 → strip prefix example in edit.txt prevents a common failure mode where the model includes the N: line prefix in oldString.

Impact:

Metric Before After Cut
Total chars 33,163 11,302 66%
Approx tokens ~9,475 ~3,230 ~6,250 saved/msg

Per-file breakdown (largest first):

File Before After Cut
bash.txt 9,288 2,821 70%
todowrite.txt 8,845 1,701 81%
task.txt 3,732 1,041 72%
edit.txt 1,369 674 51%
read.txt 1,158 581 50%
apply_patch.txt 1,098 669 39%
lsp.txt 1,040 515 50%
websearch.txt 976 443 55%
codesearch.txt 802 348 57%
webfetch.txt 750 247 67%
grep.txt 689 359 48%
question.txt 657 394 40%
write.txt 623 290 53%
plan-enter.txt 613 406 34%
plan-exit.txt 579 318 45%
glob.txt 545 234 57%
skill.txt 399 261 35%

What's preserved:

  • All git safety rules (amend conditions, force-push guards, no-secrets commit, no -i flags, no empty commits, no Task/TodoWrite during commit, unclear-intent-ask-first, GitHub URL → gh routing)
  • All placeholders listed above
  • apply_patch format block verbatim (Begin/End Patch, Add/Update/Delete File, Move to, + prefix rule)
  • 9 LSP operations + required params (filePath, line, character, 1-based)
  • 4 todo states + one-in-progress rule
  • Read-before-Edit requirement
  • Full commit + PR workflow steps

Behavioral anchors retained (after a second review pass that judged the first cut too aggressive):

  • bash.txt: workdir and path-quoting good/bad pairs, parallel call example
  • edit.txt: 42: const x = 1 → strip 42: prefix example
  • read.txt: foo\n → 1: foo\n format anchor
  • todowrite.txt: 3 compressed examples (use multi-feature, use discover-scope-then-list, skip trivial)
  • task.txt: proactive code-reviewer pattern

What was dropped:

  • Repeated "you can call multiple tools in a single response..." preamble
  • Long <reasoning> blocks after each todowrite example
  • Fictional agent examples labeled "use actual agents listed above"
  • Double-stated rules (once in the instruction, once in prose)

How did you verify your code works?

  • Verified all placeholder substitution sites still have their keys present via rg across packages/opencode/src/tool/.
  • Confirmed all .txt files are consumed only as raw strings by import statements in adjacent .ts files — no schema parsing, no format-sensitive consumers.
  • Two-pass review: initial aggressive cut (88% on todowrite) produced rules-only output; audit flagged missing behavioral anchors, examples restored.
  • No code changes, no schema changes — .txt content only. .ts consumers untouched.

Screenshots / recordings

N/A — no UI change.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

@MartinWie
perf(tool): condense tool descriptions ~66% (ref anomalyco#11995)
847adfd 
Rewrite all 17 tool description files in terse style to cut
per-message system prompt overhead. Estimated savings: ~6,250
tokens per message (33,163 → 11,302 chars), compounding across
multi-turn sessions.

Preserves:
- All git safety rules (amend conditions, force-push guards,
  no-secrets, no -i flags, no empty commits, no-Task-during-commit)
- All placeholders (${os}, ${shell}, ${chaining}, ${maxLines},
  ${maxBytes}, {{year}}, ${directory})
- apply_patch format block (Begin/End Patch, Add/Update/Delete
  File, Move to, + prefix rule)
- 9 LSP operations + required params
- 4 todo states + one-in-progress rule
- Read-before-Edit requirement
- GitHub URL → gh routing

Retains behavioral anchors:
- bash.txt: workdir and quoting good/bad examples, parallel call
  example
- edit.txt: concrete oldString prefix example (42: const x = 1 →
  strip '42: ')
- read.txt: foo\n → 1: foo\n format anchor
- todowrite.txt: 3 compressed examples (use multi-feature, use
  discover-scope, skip trivial)
- task.txt: proactive code-reviewer pattern

Per-file cut:
  bash.txt:        9288 → 2821 (70%)
  todowrite.txt:   8845 → 1701 (81%)
  task.txt:        3732 → 1041 (72%)
  edit.txt:        1369 →  674 (51%)
  read.txt:        1158 →  581 (50%)
  apply_patch.txt: 1098 →  669 (39%)
  lsp.txt:         1040 →  515 (50%)
  websearch.txt:    976 →  443 (55%)
  codesearch.txt:   802 →  348 (57%)
  webfetch.txt:     750 →  247 (67%)
  grep.txt:         689 →  359 (48%)
  question.txt:     657 →  394 (40%)
  write.txt:        623 →  290 (53%)
  plan-enter.txt:   613 →  406 (34%)
  plan-exit.txt:    579 →  318 (45%)
  glob.txt:         545 →  234 (57%)
  skill.txt:        399 →  261 (35%)
@github-actions github-actions Bot added needs:compliance This means the issue will auto-close after 2 hours. contributor needs:title labels Apr 24, 2026
@github-actions

github-actions Bot commented Apr 24, 2026

Copy link
Copy Markdown
Contributor

Hey! Your PR title perf(tool): condense tool descriptions ~66% to cut system-prompt tokens doesn't follow conventional commit format.

Please update it to start with one of:

  • feat: or feat(scope): new feature
  • fix: or fix(scope): bug fix
  • docs: or docs(scope): documentation changes
  • chore: or chore(scope): maintenance tasks
  • refactor: or refactor(scope): code refactoring
  • test: or test(scope): adding or updating tests

Where scope is the package name (e.g., app, desktop, opencode).

See CONTRIBUTING.md for details.

@MartinWie MartinWie mentioned this pull request Apr 24, 2026
Closed
@MartinWie MartinWie changed the title perf(tool): condense tool descriptions ~66% to cut system-prompt tokens refactor(opencode): condense tool descriptions ~66% to cut system-prompt tokens Apr 24, 2026
@github-actions github-actions Bot removed needs:title needs:compliance This means the issue will auto-close after 2 hours. labels Apr 24, 2026
@github-actions

github-actions Bot commented Apr 24, 2026

Copy link
Copy Markdown
Contributor

Thanks for updating your PR! It now meets our contributing guidelines. 👍

@mohammadidinani

mohammadidinani commented Apr 28, 2026 •
edited
Loading

Copy link
Copy Markdown

@MartinWie when do we expect this to land and which version will it be available? the token usage is insane! Also is there a way to override these tools in current versions while this is being delivered?

@MartinWie

MartinWie commented Apr 28, 2026

Copy link
Copy Markdown
Contributor Author

@mohammadidinani atm not expecting this to land at all, but let's see if someone reviews and approves my PR, I am not an official maintainer, but thought about applying because I like the tool, let's see.

Here is the related issue I opened -> #11995

But you can check out this branch locally and build the binary yourself, maybe also merge in this one here -> #24210

It adds a "/context" command to give you some insight into what really fills your context.

I also have one PR in the pipeline that let's users view the full context of a message, to have full transparency on the context they send to providers.
But atm waiting for these 2 PRs to be approved. (Don't want to juggle too many different topics.)

@mikehardy

mikehardy commented May 20, 2026

Copy link
Copy Markdown

would love to see this merge seems like an obvious win and pretty important for local models that have tight constraints on context

@Virtox

Virtox commented May 21, 2026

Copy link
Copy Markdown

Any update on this? Seems desperately needed. With heavier models this is really killing the flow of things.

@lovehumans

lovehumans commented Jul 20, 2026

Copy link
Copy Markdown

with 3.7k open issues, probably nobody is reading this

@coderabbitai coderabbitai Bot mentioned this pull request Aug 13, 2026
Merged
@MartinWie MartinWie closed this Aug 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

contributor

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@MartinWie @mohammadidinani @mikehardy @Virtox @lovehumans