Version: 4.5.5
Platform: Python 3.10.10, Ubuntu 22.04
Description:
When the Redis client is initialized with retry_on_timeout set to True or retry_on_error set to [redis.exceptions.TimeoutError, socket.timeout] and connection_pool is not provided, the timeout retrying does not work in client's pipeline execution.
For example:
my_redis_client = redis.StrictRedis(
host='...',
port=6380,
password='...',
ssl=True,
socket_timeout=30,
socket_connect_timeout=30,
retry=retry,
retry_on_timeout=True,
)
pipe = my_redis_client.pipeline()
pipe.incr(name='key')
pipe.execute() # Retrying on timeout won't work here
Problem found:
Inside Redis init method only retry_on_error is propagated through newly created kwargs into the created ConnectionPool (line 1043 in redis/client.py) with retry_on_timeout causing TimeoutError to be added to retry_on_error (line 988 in redis/client.py).
Subsequently, when a Redis Pipeline is initialized from the client using the pipeline method (1096 in redis/client.py), it is initialized using the ConnectionPool created in the Redis constructor.
When we execute the pipeline using the execute method (line 2100 in redis/client.py), conn.retry.call_with_retry will be called, where conn is a Connection object initiated from the ConnectionPool with retry_on_error set, and retry_on_timeout not set as it was not propagated into ConnectionPool and thus to Connection.
When a timeout occurs while executing the pipeline, _disconnect_raise_reset(conn, error) method of the Redis Pipeline is called (line 2080 in redis/client.py). The problem with this method is that it will always re-raise the TimeoutError due to conn.retry_on_timeoutalways being False. The reason is that conn was created from the ConnectionPool, which was initialized without retry_on_timeout being set (in the Redis constructor).
Possible fix:
Use retry_on_error instead of retry_on_timeout inside _disconnect_raise_reset.
Version: 4.5.5
Platform: Python 3.10.10, Ubuntu 22.04
Description:
When the
Redisclient is initialized withretry_on_timeoutset toTrueorretry_on_errorset to[redis.exceptions.TimeoutError, socket.timeout]andconnection_poolis not provided, the timeout retrying does not work in client's pipeline execution.For example:
Problem found:
Inside
Redisinit method onlyretry_on_erroris propagated through newly createdkwargsinto the createdConnectionPool(line 1043 inredis/client.py) withretry_on_timeoutcausingTimeoutErrorto be added toretry_on_error(line 988 inredis/client.py).Subsequently, when a Redis Pipeline is initialized from the client using the
pipelinemethod (1096 inredis/client.py), it is initialized using theConnectionPoolcreated in theRedisconstructor.When we execute the pipeline using the
executemethod (line 2100 inredis/client.py),conn.retry.call_with_retrywill be called, whereconnis aConnectionobject initiated from theConnectionPoolwithretry_on_errorset, andretry_on_timeoutnot set as it was not propagated intoConnectionPooland thus toConnection.When a timeout occurs while executing the pipeline,
_disconnect_raise_reset(conn, error)method of the Redis Pipeline is called (line2080inredis/client.py). The problem with this method is that it will always re-raise theTimeoutErrordue toconn.retry_on_timeoutalways beingFalse. The reason is thatconnwas created from theConnectionPool, which was initialized withoutretry_on_timeoutbeing set (in theRedisconstructor).Possible fix:
Use
retry_on_errorinstead ofretry_on_timeoutinside_disconnect_raise_reset.