Repository navigation
feat: Enable programmatic sub-LLM calls for RLM (Recursive Language Model) pattern #8554
Description
Summary
Enable true Recursive Language Model (RLM) capabilities in OpenCode by adding a built-in tool that allows the LLM to write code that programmatically invokes sub-LLM calls in loops, rather than requiring explicit tool calls for each invocation.
Background: The RLM Paper
This is based on the research paper "Recursive Language Models" which describes a fundamentally different approach to handling large contexts and recursive decomposition.
Prior Work
I started exploring this approach by creating a Claude Code skill: https://github.com/BowTiedSwan/rlm-skill
However, Claude Code's closed tool architecture prevents true symbolic RLM - tools cannot programmatically invoke sub-LLM calls, so each sub-agent spawn requires an explicit tool call that the LLM must verbalize.
The Problem with Current Sub-Agent Approaches
Omar Khattab articulated this well:
On RLMs: The way people tend to implement recursive sub-calls or "sub-agents" don't work.
In particular, you cannot express sub-agents as tool calls.
To see why those aren't expressive enough, suppose you are given a 10M-token prompt that starts with "keep each math question below iff the answer is an even number".
A typical sub-agent strategy would fail on a few angles:
It can't write the O(N) sub-calls as tool calls, because the model can't verbalize that many explicit sub-prompts. In other words, the recursion has to be symbolic through code, not tool calls.
It also can't fit the long prompt in the first place—so it may resort to compaction. In other words, it's crucial for an RLM that its own prompts/requests (not just the typical external "environment") are accessible through pointers as an object, so it can recurse through them symbolically.
Example: If you have 10,000 chunks to analyze, the current approach requires the LLM to verbalize 10,000 task() tool calls. This is impractical. True RLM would let the LLM write:
// 3 lines instead of 10,000 tool calls
for (const chunk of chunks) {
results.push(await sub_llm(`Analyze: ${chunk}`))
}Why OpenCode is Uniquely Positioned
OpenCode's TaskTool already demonstrates that tools can invoke sub-LLMs:
// From packages/opencode/src/tool/task.ts:145
const result = await SessionPrompt.prompt({
sessionID: session.id,
agent: agent.name,
parts: promptParts,
})However, this capability is not exposed to custom tools or plugins — they only receive the limited @opencode-ai/plugin SDK context. OpenCode's open architecture makes it possible to bridge this gap.
Proposed Solution: rlm_repl Built-in Tool
Add a new built-in tool that executes LLM-generated JavaScript with sub_llm() available in scope.
Tool API
// New tool: packages/opencode/src/tool/rlm-repl.ts
{
description: "Execute JavaScript code with access to sub-LLM calls for recursive processing",
parameters: z.object({
code: z.string().describe("JavaScript code to execute. Has access to: sub_llm(prompt, agent?), context.store(key, data), context.load(key), context.chunk(key, size)"),
agent: z.string().optional().describe("Default agent for sub_llm calls"),
}),
async execute(params, ctx) {
// Sandboxed execution with sub_llm() in scope
}
}Available Functions in REPL
| Function | Purpose |
|---|---|
sub_llm(prompt, agent?) |
Invoke a sub-LLM call, returns string result |
context.store(key, data) |
Store data externally, returns pointer (key) |
context.load(key) |
Load data by pointer |
context.chunk(key, size) |
Split stored data into chunks, returns array of pointers |
Example Usage by LLM
// Store large data externally (not in LLM context)
const ptr = context.store("input", massive_10M_token_string)
// Chunk it symbolically
const chunks = context.chunk(ptr, 5000) // Returns pointers, not content
// Process in parallel with sub-LLMs
const results = await Promise.all(
chunks.map(c => sub_llm(`Is the answer even? Data: ${context.load(c)}`))
)
// Aggregate
return results.filter(r => r.includes("even")).lengthBenefits
- O(N) Scaling: LLM writes a loop, not N tool calls
- Pointer-Based Access: Large prompts stay external, accessed symbolically
- True RLM: Matches the paper's vision of recursive computation
- Backward Compatible: New tool, doesn't change existing behavior
Security Considerations
- Code execution should be sandboxed (vm2, isolated-vm, or similar)
- Rate limiting on sub_llm calls
- Context store size limits
- Timeout on code execution
Related Work
- Original RLM paper: https://arxiv.org/html/2512.24601v1
- Omar Khattab's critique: https://x.com/lateinteraction/status/2011250721681773013
- Claude Code RLM skill (limited by architecture): https://github.com/BowTiedSwan/rlm-skill
- OpenCode's existing TaskTool proves the architecture supports this
Implementation
I'm prepared to submit a PR implementing this. The core changes would be:
- New file:
packages/opencode/src/tool/rlm-repl.ts - Register in
packages/opencode/src/tool/registry.ts - Add sandboxed execution environment
- Documentation
Happy to discuss the approach before implementation.
Activity
github-actions commentedon Jan 14, 2026
This issue might be a duplicate of existing issues. Please check:
Feel free to ignore if none of these address your specific case.
feat(tool): add rlm_repl tool for recursive LLM pattern
rileyseaburg commentedon Jan 19, 2026
Hey cool! I implemented this too and I'm testing a rust repl!
Pl8tinium commentedon Feb 12, 2026
looking forward to this 🙌
mharris717 commentedon Feb 13, 2026
I would be willing to beta test.
rileyseaburg commentedon Feb 13, 2026
I'm surprised you guys haven't forked your own version.
Just compile this PR
mharris717 commentedon Feb 13, 2026
Honestly, I hadn't even seen that there was a PR since this is an issue, and I hadn't noticed that this issue linked a PR in a usable state.
rileyseaburg commentedon Feb 13, 2026
I have my own implementation but this issue refs this code: BowTiedSwan@3d1d7d0
Jbollenbacher commentedon Feb 15, 2026
I suspect this will not yield the result your'e looking for.
A key part of what makes the RLM pattern work is the fact that the agents live in a REPL and all input lives in variables in that REPL. Agents only ever see truncated previews of the prompt and of the stdout from the REPL. This forces the LM to work with their inputs symbolically, which is what makes the recursion actually valuable.
Without this restructuring of how context is handled, you're not gonna get the gains that RLMs deliver. And I think restructuring the context that way would break everything else about OpenCode.
I still think it'd be good to support recursive subcalls, but if you want a proper RLM system, it's gotta be RLM from the ground up.
rileyseaburg commentedon Feb 15, 2026
https://github.com/rileyseaburg/codetether-agent/blob/main/src/rlm/repl.rs
I managed to get it to work in rust.
fgroo commentedon Mar 16, 2026
I also built a rough version for OpenCode using RLM for agentic workflows + benchmarks about a month ago.
It has also couple extra features. https://github.com/fgroo/rlm-opencode
I wish I’d found this issue post earlier 😭
rileyseaburg commentedon Mar 16, 2026
You learn by doing friend.
TomLucidor commentedon Mar 20, 2026
Ey guys check this out (token-efficient RLM design) https://github.com/avilum/minrlm
mindplay-dk commentedon Mar 25, 2026
Does this potentially replace things like Context7? If RLM can handle context of "any size", would it make sense to just dump entire manuals into a folder and let the agent find what it needs when it needs it? 🤔
TomLucidor commentedon Mar 25, 2026
@mindplay-dk you are still paying for the input/prefill. Content7 likely has a cache of pre-checked manuals to save cost on both sides... And why not use it on
tldr/teeldeerrepos (including PR) so that community manuals can be FOSS? (separate issue)fgroo commentedon Mar 25, 2026
Yeah, that's a good idea. You could just copy the docs into the chat context file and label them. Then, in the system prompt, you could simply say, “If you need man pages for certain functions, use your RLM tools and search for ‘DOCS: XYZ’.”
TomLucidor commentedon Mar 26, 2026
@fgroo at this point a document graph and/or lightweight vector search would be good (and maybe RLM after picking the likely subset)
github-actions commentedon May 25, 2026
To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please open a new one.
Pl8tinium commentedon May 25, 2026
another idea, without narrowing down the scope first is to limit the query result that can be executed. concretely i mean some kind of hook that sits in between the command the LLM executes on the data and validates if the results are below x characters. This could prevent the LLM from accidentally querying too much of the data and forces it to narrowly peek into the data and adjust its query method
docs(integration): design notes for opencode + oh-my-openagent source…
fgroo commentedon May 29, 2026
Okay @Pl8tinium , I hope I understand your idea correctly. I will use C terminology to describe what you're might be envisioning.
When the Agent needs to traverse the Database and recursively call itself to find the best results matching the Search Query, there must be a hook that prevents Agents in the call stack from exceeding a certain character limit (e.g. X characters). This way the main Agent instance is protected from oversized responses coming back up the stack.
Is that correct?
I think this could be implemented quite easily in the style of PreToolUse hooks. The Agent would simply receive an error if it tries to pass more than X characters back to the previous frame in the call stack.
That said, I have a question for you. What is the underlying intent of your idea? If the goal is to prevent the context window from filling up, isn't that already handled well? Once the base Agent is reached and the recursive frames are popped, the tokens consumed by the recursive agents (for vector search, etc.) shouldn't be transferred back to the main context anyway.
Let me know if I'm misunderstanding anything!
Pl8tinium commentedon May 30, 2026
yeah that was totally what i was refering to.
..
but yeah, you are right. Subagents should prevent us leaking too much context into the main agent context. Thereby this wouldnt really be that handy..
The idea may still not be a throw away. This "context limit filter" could also be helpful for when subagents pass information back to the main agent. Limiting them may prevent some subagents to pass unnecessary information and may force them to just provide the essentials. This may not be that important in relatively simple queries, but I think where RLM shines is when iteratively data needs to be extracted and (LLM) processed from many different places in the data source. Aligning the "subagent data return limit" with what is expected by the subagent query could benefit the overall processing of the found data.
Though we'd probaby want the "subagent data return limit" to be individually adjusted based on the task as on certain tasks more content needs to be returned than on others. Thereby this would ideally be a parameter in the subagent spawn process that the main agent should reason abount and decide for.
Also, leaning more towards the initial concept, the benefits of a "forced to decide on" context cap for its tools could still benefit the overall extraction capability of a single agent by optimizing how much context can be thrown into its window.
Dont mind this idea here though, it should just serve as inspiration.
fgroo commentedon Jun 2, 2026
@Pl8tinium . I implemented two approaches: a global toggle for limiting the responses from RLM instances, and one where each agent can decide using an additional parameter in their tool use. You can try it out! And you're right about the subagent data return limit. I was doing experiments and, yes, sometimes it can happen that 50k tokens get transported to the main agent. Not so nice
Pl8tinium commentedon Jun 2, 2026
wow this is amazing, will for sure try it out once i find a good use case to test it with!