Repository navigation
Have you considered using embedding models? #2584
Description
Proposal: Embedding models are cheaper and faster than LLMs. I thought that opencode could first ask the embedding model which part of the code responds better to the prompt, and then send these pieces of code to the LLM.
Drawback: The only limitation is that you have to first index the whole codebase, you can either send the entire files or use the LSP to get smaller code blocks. Then, save the vectors into a database, and you are good to go, you can ask first the embedding model which parts of the code fit better.
Options: There are some embedding models that can be run locally, so you might not depend on external providers, but also there are good embedding models from google or openai.
Maybe you already do that, but I haven't seen anything quick-reviewing the code, or maybe you have already considered that and it was not such a good idea, Thanks for the incredible project!
Activity
github-actions commentedon Sep 13, 2025
This issue might be a duplicate of existing issues. Please check:
Feel free to ignore if none of these address your specific case.
rekram1-node commentedon Dec 28, 2025
[automated] Closing due to 90+ days of inactivity. Feel free to reopen if you still need this!
K0IN commentedon Dec 28, 2025
Hei even if you don't implement this, could someone explain, how the indexing works rn and why it is not needed to (schematic) fast find relevant text?
MrDoe commentedon May 28, 2026
We could save a massive amount of tokens if RAG with embedding models would be implemented correctly. Such models can easily run locally on modern CPUs and they are much faster than standard LLMs. I'm considering creating an Opencode plugin with this functionality.
MrDoe commentedon May 29, 2026
Here is my first version of the plugin: https://github.com/MrDoe/OpenCodeRAG