Repository navigation
Fix connection leak in utils.pipeline() on exception - #4345
Merged
vladvildanov merged 2 commits intoSep 25, 2026
Merged
vladvildanov merged 2 commits into
vladvildanov merged 2 commits into
Conversation
The pipeline() context manager only called p.execute() after the with-block body, with no try/finally - if that body raised, execute() never ran, and execute() is where Pipeline unconditionally releases its connection back to the pool (via its own internal finally: self.reset()). For a pipeline that never calls watch(), this is low-impact: commands are just buffered in an in-memory list until execute() runs, so nothing is actually checked out from the pool yet. But watch() goes through immediate_execute_command and checks out a real connection right away (WATCH has to be sent to the server immediately to establish the watch condition) - if the with-block then raises before execute() runs, that connection stays checked out until Pipeline.__del__ happens to run it during garbage collection, not deterministically or immediately. Wrapped the yield in try/finally with an unconditional p.reset() - the same call Pipeline's own __exit__ makes, and safe to call twice (reset() after a successful execute() is a no-op, since execute() already reset the state in its own finally). Added regression tests with a fake pipeline tracking execute()/reset() calls: confirms reset() always runs (success and exception paths) and execute() only runs on success. Verified both fail against the original code and pass against the fix. Full tests/test_utils.py: 64 passed. ruff check/format clean.
2 of 5 tasks
1 of 5 tasks
petyaslavova
added a commit
that referenced
this pull request
Oct 16, 2026
* Do not execute the async utils.pipeline batch after an exception The async redis.asyncio.utils.pipeline context manager called execute() in __aexit__ also when the block raised, so the commands queued before the exception were sent to Redis and part of the batch was applied. It also never called reset(). The sync utils.pipeline only executes when the block succeeds and always resets the pipeline (#4345). Do the same in the async helper, and add the async versions of the sync tests. * Test the async utils.pipeline helper with a real WATCH/MULTI transaction --------- Co-authored-by: petyaslavova <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
If the code inside
with pipeline(redis_obj) as p:raises,p.execute()never runs. That matters becausePipeline.execute()is where the connection gets unconditionally released back to the pool, via its ownfinally: self.reset().For a plain pipeline (no
watch()), this is low-impact - commands are just buffered into an in-memory list (pipeline_execute_command) untilexecute()runs, so nothing's actually checked out from the pool yet if the body raises before reaching it. Butwatch()goes throughimmediate_execute_command, which checks out a real connection right away (WATCH has to reach the server immediately to establish the watch condition). If thewithblock then raises beforeexecute()runs - aWatchErroron retry, a bug in the caller's own transaction logic, anything - that connection stays checked out untilPipeline.__del__happens to run during garbage collection, which isn't deterministic or immediate.Fix
Same call
Pipeline.__exit__already makes unconditionally. Safe to call twice -reset()right after a successfulexecute()is a no-op, sinceexecute()already reset everything in its ownfinally.Testing
Added
TestPipelineContextManagerintests/test_utils.pywith a fake pipeline trackingexecute()/reset()calls - assertsreset()runs on both the success and exception paths, andexecute()only runs on success. Verified both new tests fail against the pre-fix code and pass against the fix.Full
tests/test_utils.py: 64 passed.ruff check/ruff format --checkclean on both changed files.Note
Medium Risk
Changes connection lifecycle for callers using the
pipeline()helper; the fix aligns with existingPipelinecleanup but affects pooled connection behavior on error paths.Overview
Fixes a connection pool leak when the
pipeline()context manager inredis/utils.pyexits due to an exception inside thewithblock.The helper now wraps
yield/execute()intry/finallyand always callsp.reset()infinally, matching unconditional cleanup onPipelineteardown. On success, behavior is unchanged (execute()thenreset()). On failure,execute()is skipped butreset()still runs, so connections checked out early (e.g. afterwatch()) are returned to the pool instead of lingering until GC.Adds
TestPipelineContextManagerwith a fake pipeline/redis client to assertreset()on both success and exception paths andexecute()only on success.Reviewed by Cursor Bugbot for commit 48f0012. Bugbot is set up for automated code reviews on this repo. Configure here.