09:27 10/9/25 Using Redis · sidekiq/sidekiq Wiki

sidekiq sidekiq

Code Issues 40 Pull requests 7 Discussions Actions Wiki

Using Redis
Jump to bottom

Mike Perham edited this page on Mar 18 · 57 revisions

Sidekiq uses Redis to store all of its job and operational data.

By default, Sidekiq tries to connect to Redis at localhost:6379 . This typically works great
during development but needs tuning in production.

Using an ENV variable

You can set the Redis URL using an environment variable. The easiest option is to set
REDIS_URL . Sidekiq will pick it up and use it. A Redis URL looks like redis://[hostname]:
[port]/[dbnumber] , e.g. redis://redis.example.com:7777/11 .

If you're on Heroku, set the REDIS_PROVIDER env var to the name of the env var containing the
Redis server URL. (Example with RedisCloud: set REDIS_PROVIDER=REDISCLOUD_URL and
Sidekiq will use the value of the REDISCLOUD_URL env var when connecting to Redis.)

heroku config:set REDIS_PROVIDER=REDISCLOUD_URL

Using an initializer

Prefer to use the ENV variables above. If you have to use logic in Ruby, you can configure Redis
in the initializer. Note that to configure the location of Redis, you must define both the
Sidekiq.configure_server and Sidekiq.configure_client blocks. To do this put the
following code into config/initializers/sidekiq.rb .

Sidekiq.configure_server do |config|
config.redis = { url: 'redis://:[email protected]:6379/0' }
end

Sidekiq.configure_client do |config|
config.redis = { url: 'redis://:[email protected]:6379/0' }
end

https://github.com/sidekiq/sidekiq/wiki/Using-Redis 1/5
09:27 10/9/25 Using Redis · sidekiq/sidekiq Wiki

Unknown parameters are passed to the underlying Redis client so any parameters supported by
the driver can go in the Hash. Keys can be strings or symbols and will be unified before being
passed on to the Redis client.

In 7.0, Sidekiq does not allow you to directly create your own connection pool anymore. Sidekiq
creates and manages any connection pools internally. See the redis-client docs for the
configuration options you can add to the config.redis hash.

Life in the Cloud

One problem with cloud-based systems like EC2 and Heroku can be multi-tenancy and
unpredictable network performance. You should tune your network and pool timeouts to be a little
more lenient if you are seeing occasional timeout errors, it defaults to 1 second.

config.redis = { url: 'redis://...', network_timeout: 5, pool_timeout: 5 }

REMEMBER: THIS IS A BANDAID You are not solving the actual cause of the slow
performance.

If you are seeing Redis timeout errors, you should check your Redis latency by using the redis-
cli --latency and --latency-history flags:

$ redis-cli --latency-history localhost

min: 0, max: 1, avg: 0.20 (1359 samples) -- 15.01 seconds range
min: 0, max: 1, avg: 0.18 (1356 samples) -- 15.01 seconds range
min: 0, max: 1, avg: 0.18 (1355 samples) -- 15.01 seconds range
min: 0, max: 1, avg: 0.17 (1359 samples) -- 15.01 seconds range
min: 0, max: 1, avg: 0.19 (1358 samples) -- 15.00 seconds range
min: 0, max: 1, avg: 0.18 (1353 samples) -- 15.01 seconds range
min: 0, max: 1, avg: 0.19 (1357 samples) -- 15.01 seconds range

This says my average latency to localhost is 0.2ms or 200 microseconds: excellent. With users
seeing odd Redis behavior, I regularly see setups with latency over 5 seconds: terrible. You can
move to a different Redis provider or run your own Redis server on a dedicated machine but
there's nothing Sidekiq can do if the network performance is terrible. Contact your Redis provider
and ask about your available options.

Lockdown
If your IT organization insists on locking down Sidekiq's Redis user, you can use this
configuration as a starting point:

acl setuser sidekiquser +@all -@admin -@dangerous +info allkeys

https://github.com/sidekiq/sidekiq/wiki/Using-Redis 2/5
09:27 10/9/25 Using Redis · sidekiq/sidekiq Wiki

I believe those restrictions work with all Sidekiq functionality today.

Client naming
If your Redis server supports it, you may instruct Sidekiq to use CLIENT SETNAME when
establishing connections. This will make it easier to identify Sidekiq clients when debugging with
CLIENT LIST .

config.redis = { id: "Sidekiq-server-PID-#{::Process.pid}" }

This is disabled by default because many SaaS Redis hosts do not support it.

Architecture
Redis offers many different topologies:

Single node -- offers no fault tolerence

Redis Sentinel -- offers fault tolerence, fails over to a replica in case of primary failure
Redis Cluster -- multi-master keyspace spread across many instances

Cluster is designed for large-scale datasets, like caches, that can spread evenly across
machines. Cluster is NOT appropriate for Sidekiq as Sidekiq has a few very hot keys which
are constantly changing (aka queues) and Cluster cannot guarantee high-performance
transactions, necessary to keep your job system fast and consistent. I recommend using Sentinel
or use a Redis SaaS which has built-in support for failover.

Tuning
You can see Redis's config variables with the command redis-cli info .

Scale
Customers have reported processing 20,000+ jobs/sec with a single Redis instance. Ensure you
are running a recent version; Redis 6 has significant performance improvements with the --io-
threads option.

Memory

Redis runs best when all data fits in memory. You should set maxmemory-policy noeviction in
redis.conf so Redis doesn't drop Sidekiq's data silently.

https://github.com/sidekiq/sidekiq/wiki/Using-Redis 3/5
09:27 10/9/25 Using Redis · sidekiq/sidekiq Wiki

Multiple Redis instances

Many people use Redis as a cache (it works great as a Rails cache store) but it's important that
Sidekiq be run against a Redis instance that is not configured as a cache but as a persistent
store. I recommend using two separate Redis instances, each configured appropriately, if you
wish to use Redis for caching and Sidekiq. Redis namespaces do not allow for this configuration
and come with many other problems, so using discrete Redis instances is always preferred.

Timeouts
The most common reasons for Redis networking timeouts are:

0. CPU overload - you've configured your concurrency way too high and threads are starving
for CPU time. Reduce your concurrency, create more processes.
1. swapping - you are running out of RAM and disk swapping is causing massive latency
spikes. Using Redis to hold cache data can take a lot of RAM, configure Sidekiq to use a
separate Redis instance if necessary.
2. command latency - you are running a Redis command which is taking a large amount of
time. Read the Monitoring blog post below.

Notes

redis-cli --help shows several useful options, including --latency-history and --

bigkeys .

redis-rdb-tools has a few options to introspect a dumped RDB file.

Monitoring Redis covers the most important things you should monitor when using Redis.
Datadog has a nice whitepaper on Redis internals and monitoring.
BRPOPLPUSH and other B* commands are blocking; large latency spikes with them are
normal and not a cause for concern.

Previous: Best Practices Next: Error Handling

Home | The Basics | Best Practices | Using Redis | Error Handling | Advanced Options | Problems?

Pages 60

Find a page…

Home

Active Job

https://github.com/sidekiq/sidekiq/wiki/Using-Redis 4/5
09:27 10/9/25 Using Redis · sidekiq/sidekiq Wiki

Advanced Options

API

Batches

Best Practices

Build vs Buy

Bulk Queueing

Comm Installation

Commercial collaboration

Commercial FAQ

Commercial Support

Complex Job Workflows with Batches

Delayed extensions

Deployment

Show 45 more pages…

Clone this wiki locally

https://github.com/sidekiq/sidekiq.wiki.git

https://github.com/sidekiq/sidekiq/wiki/Using-Redis 5/5