Skip to content

Fix connection leak in utils.pipeline() on exception - #4345

Merged
vladvildanov merged 2 commits into
redis:masterfrom
kratos0718:fix-pipeline-contextmanager-connection-leak
Sep 25, 2026
Merged

vladvildanov merged 2 commits into
redis:masterfrom
kratos0718:fix-pipeline-contextmanager-connection-leak

Conversation

@kratos0718

@kratos0718 kratos0718 commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

What

@contextmanager
def pipeline(redis_obj):
    p = redis_obj.pipeline()
    yield p
    p.execute()

If the code inside with pipeline(redis_obj) as p: raises, p.execute() never runs. That matters because Pipeline.execute() is where the connection gets unconditionally released back to the pool, via its own finally: self.reset().

For a plain pipeline (no watch()), this is low-impact - commands are just buffered into an in-memory list (pipeline_execute_command) until execute() runs, so nothing's actually checked out from the pool yet if the body raises before reaching it. But watch() goes through immediate_execute_command, which checks out a real connection right away (WATCH has to reach the server immediately to establish the watch condition). If the with block then raises before execute() runs - a WatchError on retry, a bug in the caller's own transaction logic, anything - that connection stays checked out until Pipeline.__del__ happens to run during garbage collection, which isn't deterministic or immediate.

Fix

@contextmanager
def pipeline(redis_obj):
    p = redis_obj.pipeline()
    try:
        yield p
        p.execute()
    finally:
        p.reset()

Same call Pipeline.__exit__ already makes unconditionally. Safe to call twice - reset() right after a successful execute() is a no-op, since execute() already reset everything in its own finally.

Testing

Added TestPipelineContextManager in tests/test_utils.py with a fake pipeline tracking execute()/reset() calls - asserts reset() runs on both the success and exception paths, and execute() only runs on success. Verified both new tests fail against the pre-fix code and pass against the fix.

Full tests/test_utils.py: 64 passed. ruff check/ruff format --check clean on both changed files.


Note

Medium Risk
Changes connection lifecycle for callers using the pipeline() helper; the fix aligns with existing Pipeline cleanup but affects pooled connection behavior on error paths.

Overview
Fixes a connection pool leak when the pipeline() context manager in redis/utils.py exits due to an exception inside the with block.

The helper now wraps yield / execute() in try/finally and always calls p.reset() in finally, matching unconditional cleanup on Pipeline teardown. On success, behavior is unchanged (execute() then reset()). On failure, execute() is skipped but reset() still runs, so connections checked out early (e.g. after watch()) are returned to the pool instead of lingering until GC.

Adds TestPipelineContextManager with a fake pipeline/redis client to assert reset() on both success and exception paths and execute() only on success.

Reviewed by Cursor Bugbot for commit 48f0012. Bugbot is set up for automated code reviews on this repo. Configure here.

kratos0718 and others added 2 commits September 24, 2026 14:29
The pipeline() context manager only called p.execute() after the
with-block body, with no try/finally - if that body raised, execute()
never ran, and execute() is where Pipeline unconditionally releases
its connection back to the pool (via its own internal finally:
self.reset()).

For a pipeline that never calls watch(), this is low-impact: commands
are just buffered in an in-memory list until execute() runs, so
nothing is actually checked out from the pool yet. But watch() goes
through immediate_execute_command and checks out a real connection
right away (WATCH has to be sent to the server immediately to
establish the watch condition) - if the with-block then raises before
execute() runs, that connection stays checked out until Pipeline.__del__
happens to run it during garbage collection, not deterministically or
immediately.

Wrapped the yield in try/finally with an unconditional p.reset() -
the same call Pipeline's own __exit__ makes, and safe to call twice
(reset() after a successful execute() is a no-op, since execute()
already reset the state in its own finally).

Added regression tests with a fake pipeline tracking execute()/reset()
calls: confirms reset() always runs (success and exception paths) and
execute() only runs on success. Verified both fail against the
original code and pass against the fix. Full tests/test_utils.py:
64 passed. ruff check/format clean.
@vladvildanov
vladvildanov merged commit ba6976b into redis:master Sep 25, 2026
366 of 368 checks passed
@petyaslavova petyaslavova added the maintenance Maintenance (CI, Releases, etc) label Sep 28, 2026
petyaslavova added a commit that referenced this pull request Oct 16, 2026
* Do not execute the async utils.pipeline batch after an exception

The async redis.asyncio.utils.pipeline context manager called
execute() in __aexit__ also when the block raised, so the commands
queued before the exception were sent to Redis and part of the batch
was applied. It also never called reset(). The sync utils.pipeline
only executes when the block succeeds and always resets the pipeline
(#4345). Do the same in the async helper, and add the async versions
of the sync tests.

* Test the async utils.pipeline helper with a real WATCH/MULTI transaction

---------

Co-authored-by: petyaslavova <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

maintenance Maintenance (CI, Releases, etc)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants