feat: middle-out message compression for OpenRouter#3907
feat: middle-out message compression for OpenRouter#3907kwsantiago wants to merge 1 commit intoblock:micn/ollama-bootstrapperfrom
Conversation
Signed-off-by: Kyle <[email protected]>
kwsantiago
force-pushed
the
kwsantiago/openrouter-middle-out-support
branch
from
5fac467 to
5376191
Compare
|
@kwsantiago thanks - but if I read correctly, there is no need to conditionally apply this, we can just always have it set for openrouter, it doesn't seem to do anything until it needs to, ie more of a safeguard if I read it correctly? but I am trying the simple approach first in the other branch (will report back if not good). |
1 similar comment
|
@kwsantiago thanks - but if I read correctly, there is no need to conditionally apply this, we can just always have it set for openrouter, it doesn't seem to do anything until it needs to, ie more of a safeguard if I read it correctly? but I am trying the simple approach first in the other branch (will report back if not good). |
|
Sounds good @michaelneale I'll leave this branch as is for the time being. If you need me to make any changes here or close this out, please let me know. Cheers! |
|
closing this as will be addressed in other PR, thanks again @kwsantiago this was really cool |
2 participants
Implements middle-out message compression for OpenRouter to handle context limit errors by removing messages from the middle of conversations when limits are exceeded, auto-enabling for models ≤8k context.
For reference: #3881 (comment)