Skip to content

server : fix ctx checkpoint invalidation#20671

Merged
ggerganov merged 1 commit intomasterfrom
gg/server-fix-ckpt-invalidation
Mar 17, 2026
Merged

server : fix ctx checkpoint invalidation#20671
ggerganov merged 1 commit intomasterfrom
gg/server-fix-ckpt-invalidation

Conversation

@ggerganov
Copy link
Copy Markdown
Member

fix #20532

We must not keep checkpoints that contain tokens with position beyond the position that we are about to generate next. This bug has been present for a while and mostly affects SWA models such as gpt-oss.

Copy link
Copy Markdown
Contributor

@aldehir aldehir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue is no longer reproducible and the model is coherent. Thanks for looking into this!

@ggerganov ggerganov merged commit 8cc2d81 into master Mar 17, 2026
45 of 48 checks passed
@ggerganov ggerganov deleted the gg/server-fix-ckpt-invalidation branch March 17, 2026 13:21
Ethan-a2 pushed a commit to Ethan-a2/llama.cpp that referenced this pull request Mar 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Misc. bug: Server 500 error (gpt-oss, AutoParser, cache)

2 participants