Repository navigation
fix(event_store): treat concurrent eviction as idempotent - #4171
SarthakB11 wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fb62a14a51
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| await self._event_store.delete(key=old_id) | ||
| try: | ||
| await self._event_store.delete(key=old_id) | ||
| except FileNotFoundError: |
There was a problem hiding this comment.
Reconcile the stream list after eviction conflicts
When this catches FileNotFoundError in the intended concurrent-eviction case, this store_event call has already read an old event_ids snapshot before the other task deleted the same old event and wrote its own new event ID. Continuing on to _stream_store.put(...) with that stale snapshot can overwrite the other successful call's stream list entry, so both store_event calls return successfully but one event is no longer replayable. The previous behavior aborted before writing the stale index; after swallowing the conflict, the code should refetch/merge or otherwise avoid overwriting a concurrent update.
Useful? React with 👍 / 👎.
|
tl;dr: Root Cause: At Fix: Pick one in
Option 2 is the most type-safe and matches the surrounding code's preference for precise typing over suppressions. Log excerpt``` Related files
|
|
Deleting should not raise exceptions for keys that dont exist. Please follow the contributing guide next time and do not open PRs for issues without discussing the solution first. |
Description
EventStore.store_eventkeeps a per-stream ring buffer by deleting old event keys when the list exceedsmax_events_per_stream. The caller treatsAsyncKeyValue.deleteas idempotent, but some backends (notablypy-key-value-aio'sFileTreeStore) implement delete as an unguardedexists()+unlink(). When two concurrentstore_eventcalls for the same session (thesse_writerandmessage_routertasks both run eviction) decide to evict the sameevent_id, the loser hitsFileNotFoundErrorafter the winner has already unlinked the file. That exception propagates out ofstore_eventand produces ERROR-level tracebacks on every collision. The events that did get stored are fine, but on a busy server this is constant log noise that obscures real issues.This wraps the per-key eviction
delete()intry/except FileNotFoundErrorand logs at DEBUG. Eviction is idempotent by intent: a missing key is the desired post-condition, not an error. CatchingFileNotFoundErrorspecifically (rather than a broadException) preserves visibility for genuine backend failures like permission errors or Redis disconnects.The matching
FileTreeStore.delete_entryrace inpy-key-value-aiois tracked separately upstream. The defensive guard here is the right place regardless, since any backend with non-atomic delete semantics can surface the same race.Closes #4143
Contribution type
Checklist
uv run prek run --all-filesand all checks pass