Repository navigation
Make the conversation-log shrink rewrite atomic (prevent history loss on a mid-write crash) - #70
Conversation
ConversationStore.save() appends new messages on the common path, but when a turn reduces the message count (context compaction / summarization) it rewrites the whole .jsonl with open(..., "w"), which truncates the file at open(). A crash mid-rewrite then leaves a truncated or empty log, permanently losing the conversation history. Write the reduced log to a temp file and replace() it in one atomic step -- the same tmp-then-replace pattern subscriptions.ChannelBuffer._save() already uses. Add tests/test_conversation_atomicity.py covering the crash path (history preserved when the write fails partway) and the happy path (reduced set persisted, no leftover temp file).
rajpratham1
left a comment
There was a problem hiding this comment.
This pull request improves the durability of conversation persistence by replacing the in-place rewrite during conversation shrink operations with an atomic temporary-file-and-replace workflow. This prevents conversation history loss if a crash occurs while rewriting the log. The accompanying tests verify both the crash scenario, ensuring the original history remains intact when the rewrite fails before replacement, and the successful shrink path, confirming the rewritten log contains the expected reduced history without leaving temporary files behind. The implementation is focused, follows an established atomic write pattern, and no blocking issues are visible in the current diff.
rohitprasad15
left a comment
There was a problem hiding this comment.
LGTM. Thanks for handling this case.
Problem
ConversationStore.save()appends new messages on the common path, but when a turn reduces the message count (context compaction / summarization) it rewrites the whole.jsonlin place:open(..., "w")truncates the file atopen(), so a crash (or disk-full) partway through the rewrite leaves a truncated or empty log — permanently losing conversation history, the most valuable data the app stores. The append path is safe; only this shrink branch is non-atomic.This is the write-side companion to #56 (tolerate a corrupt line on read). #56's per-line skip can't recover a whole-file truncation from this path — the messages are simply gone — so preventing the truncation closes the remaining gap. (#56's own note already observes that an interrupted mid-write corrupts the file.)
Fix
Write the reduced log to a temp file and
Path.replace()it in one atomic step — the same tmp-then-replace patternsubscriptions.ChannelBuffer._save()already uses in this repo.os.replaceis atomic on POSIX and Windows when the temp file is on the same filesystem (it is — same directory), so a concurrent reader or a crash sees either the old complete log or the new one, never a partial.Kept intentionally minimal: the atomic rename is what fixes the truncation. I left out an
fsync(power-loss durability) to match the existingChannelBufferidiom — happy to add it if you'd prefer.Tests
New
tests/test_conversation_atomicity.py:test_shrink_rewrite_preserves_history_when_write_crashes— persists a 5-message history, then simulates a crash partway through a shrink rewrite, and asserts the history survives.FAILED — AssertionError: assert 1 == 5(history truncated to a single message).test_shrink_rewrite_persists_reduced_history— the normal (non-crash) shrink still writes exactly the reduced set, with no leftover temp file.The surrounding conversation tests (autotitle, durable-resume, session-persona, attachments, memory) still pass.
This is a backend fix with no UI surface, so the "before / after" is the test result above (red
FAILED assert 1 == 5→ green2 passed) rather than a screenshot — glad to attach a terminal capture if you'd like one.