fix : [ClusterPipeline] ExecutorService/thread is created and destroyed too frequently in ClusterPipeline by ggivo · Pull Request #4479 · redis/jedis

ggivo · 2026-04-01T11:06:35Z

Problem

Multi-node cluster pipelines create a new ExecutorService on every sync() call, causing significant overhead in high-throughput scenarios.

Solution

Added support for externally managed ExecutorService in cluster pipelines:

New API: cluster.pipelined(ExecutorService) - allows users to provide a shared executor
Backward compatible: cluster.pipelined() retains existing auto-managed behavior

Implementation

Modified MultiNodePipelineBase to accept optional ExecutorService
Added pipelined(ExecutorService) methods to RedisClusterClient and JedisCluster

Usage Example:

ExecutorService executor = Executors.newFixedThreadPool(3);
try (RedisClusterClient cluster = ...) {
    // Reuse executor across multiple pipelines
    try (ClusterPipeline pipeline = cluster.pipelined(executor)) {
        pipeline.set("key", "value");
        pipeline.sync();
    }
} finally {
    executor.shutdown(); // User controls lifecycle
}

Closes #4141

Note

Medium Risk
Changes the concurrency/executor lifecycle for multi-node cluster pipelining; incorrect executor management or shared-executor contention could impact performance or cause hangs, though behavior is covered by new integration tests.

Overview
Cluster pipelining can now reuse a caller-provided ExecutorService instead of creating/shutting down a thread pool on every multi-node sync().

MultiNodePipelineBase accepts an optional shared executor and only shuts down internally-created executors, while JedisCluster and RedisClusterClient add pipelined(ExecutorService) (with pipelined() delegating to it) to pass the executor through to ClusterPipeline. New integration tests verify the shared executor is not shut down and that many concurrent pipelines sharing a small executor complete without deadlock.

^{Written by Cursor Bugbot for commit 38fa2a5. This will update automatically on new commits. Configure here.}

…ed too frequently in ClusterPipeline

github-actions · 2026-04-01T11:06:58Z

Test Results

176 files ± 0 176 suites ±0 11m 1s ⏱️ +45s
8 527 tests + 9 8 316 ✅ +1 272 211 💤 - 1 263 0 ❌ ±0
3 233 runs +15 3 170 ✅ + 445 63 💤 - 430 0 ❌ ±0

Results for commit 38fa2a5. ± Comparison against base commit a3cfede.

This pull request removes 1 and adds 10 tests. Note that renamed tests count towards both.

redis.clients.jedis.commands.unified.search.FTHybridCommandsTestBase$SupportedScorersTest ‑ testScorer(Scorer, double, double)

redis.clients.jedis.ClusterPipeliningTest ‑ sharedExecutorConcurrentPipelinesNoDeadlock
redis.clients.jedis.ClusterPipeliningTest ‑ sharedExecutorPipelineDoesNotShutdownSharedExecutor
redis.clients.jedis.ClusterPipeliningTest ‑ sharedExecutorPipelineKeysAtSameNode
redis.clients.jedis.commands.unified.search.FTHybridCommandsTestBase$SupportedScorersTest ‑ testScorer(Scorer, double, double)[1]
redis.clients.jedis.commands.unified.search.FTHybridCommandsTestBase$SupportedScorersTest ‑ testScorer(Scorer, double, double)[2]
redis.clients.jedis.commands.unified.search.FTHybridCommandsTestBase$SupportedScorersTest ‑ testScorer(Scorer, double, double)[3]
redis.clients.jedis.commands.unified.search.FTHybridCommandsTestBase$SupportedScorersTest ‑ testScorer(Scorer, double, double)[4]
redis.clients.jedis.commands.unified.search.FTHybridCommandsTestBase$SupportedScorersTest ‑ testScorer(Scorer, double, double)[5]
redis.clients.jedis.commands.unified.search.FTHybridCommandsTestBase$SupportedScorersTest ‑ testScorer(Scorer, double, double)[6]
redis.clients.jedis.commands.unified.search.FTHybridCommandsTestBase$SupportedScorersTest ‑ testScorer(Scorer, double, double)[7]

♻️ This comment has been updated with latest results.

jit-ci · 2026-04-01T11:08:15Z

🛡️ Jit Security Scan Results

✅ No security findings were detected in this PR

^{Security scan by Jit}

Copilot

Pull request overview

This PR adds an option for Redis Cluster pipelining to reuse a caller-provided ExecutorService, avoiding the overhead of creating/shutting down a thread pool on every multi-node sync().

Changes:

Added pipelined(ExecutorService) overloads to RedisClusterClient and JedisCluster.
Updated MultiNodePipelineBase/ClusterPipeline to accept and use an optional shared executor for multi-node sync().
Added tests validating shared-executor lifecycle behavior and concurrent pipeline usage.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
`src/main/java/redis/clients/jedis/RedisClusterClient.java`	Adds `pipelined(ExecutorService)` API and delegates existing `pipelined()` to it.
`src/main/java/redis/clients/jedis/JedisCluster.java`	Adds `pipelined(ExecutorService)` API (deprecated client) and delegates existing `pipelined()` to it.
`src/main/java/redis/clients/jedis/MultiNodePipelineBase.java`	Implements shared-vs-dedicated executor selection and avoids shutting down shared executors.
`src/main/java/redis/clients/jedis/ClusterPipeline.java`	Wires the optional executor through to the base pipeline implementation.
`src/test/java/redis/clients/jedis/ClusterPipeliningTest.java`	Adds coverage for shared-executor behavior and concurrency/deadlock regression.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

atakavci

MultiNodePipelineBase is somewhat in need of care in general.
but approving if you like to proceed with this PR as is.

…troyed too frequently in ClusterPipeline #4479 (cherry picked from commit 7744cd4)

#4480) backport: [ClusterPipeline] ExecutorService/thread is created and destroyed too frequently in ClusterPipeline #4479 (cherry picked from commit 7744cd4)

fix : [ClusterPipeline] ExecutorService/thread is created and destroy…

f18b185

…ed too frequently in ClusterPipeline

ggivo added the enhancement label Apr 1, 2026

ggivo added 2 commits April 1, 2026 14:09

acquireExecutorService based on sharedExecutor flag

60439bf

clean up

00d5eef

ggivo requested a review from Copilot April 1, 2026 12:11

ggivo marked this pull request as ready for review April 1, 2026 12:12

ggivo requested a review from atakavci April 1, 2026 12:12

Copilot started reviewing on behalf of ggivo April 1, 2026 12:12 View session

Copilot AI reviewed Apr 1, 2026

View reviewed changes

cursor Bot reviewed Apr 1, 2026

View reviewed changes

Comment thread src/test/java/redis/clients/jedis/ClusterPipeliningTest.java

address co-pilot review comments

38fa2a5

atakavci approved these changes Apr 2, 2026

View reviewed changes

Comment thread src/main/java/redis/clients/jedis/MultiNodePipelineBase.java

Comment thread src/main/java/redis/clients/jedis/MultiNodePipelineBase.java

ggivo merged commit 7744cd4 into master Apr 2, 2026
31 of 32 checks passed

ggivo added this to the 8.0.0 milestone Apr 2, 2026

ggivo added the backport required label Apr 2, 2026

ggivo added a commit that referenced this pull request Apr 2, 2026

backport: [ClusterPipeline] ExecutorService/thread is created and des…

bf1711b

…troyed too frequently in ClusterPipeline #4479 (cherry picked from commit 7744cd4)

ggivo mentioned this pull request Apr 2, 2026

backport: [ClusterPipeline] ExecutorService/thread is created and destroyed too frequently in ClusterPipeline #4480

Merged

ggivo deleted the topic/ggivo/4141-cluster-pipeline-with-external-executor branch April 3, 2026 12:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix : [ClusterPipeline] ExecutorService/thread is created and destroyed too frequently in ClusterPipeline#4479

fix : [ClusterPipeline] ExecutorService/thread is created and destroyed too frequently in ClusterPipeline#4479
ggivo merged 4 commits intomasterfrom
topic/ggivo/4141-cluster-pipeline-with-external-executor

ggivo commented Apr 1, 2026 •

edited by cursor Bot

Loading

Uh oh!

github-actions Bot commented Apr 1, 2026 •

edited

Loading

Uh oh!

jit-ci Bot commented Apr 1, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment

Uh oh!

Uh oh!

atakavci left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ggivo commented Apr 1, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Solution

Implementation

Uh oh!

github-actions Bot commented Apr 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Uh oh!

jit-ci Bot commented Apr 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🛡️ Jit Security Scan Results

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

atakavci left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ggivo commented Apr 1, 2026 •

edited by cursor Bot

Loading

github-actions Bot commented Apr 1, 2026 •

edited

Loading

jit-ci Bot commented Apr 1, 2026 •

edited

Loading