Skip to content

LOADING Redis is loading the dataset in memory #1144

@turbo124

Description

@turbo124

Describe the bug
When using redis-sentinel, if the slave network cannot maintain throughput to stay within the client-output-buffer limits, the slave will disconnect with the following

Predis\Response\ServerException
LOADING Redis is loading the dataset in memory

This issue has been in the library since 1.x
#475

The above PR provided a graceful fallback to the master in these scenarios where the slaves were temporarily unable to return read requests. When evaluating 2.x of this library, we are still able to reproduce these scenarios resulting in exceptions rather than graceful fallback to master

Expected behavior
There are a couple of options for this scenario.

  • Provide a configuration option to only serve requests to the master. Load balancing is not the primary goal of using a Redis-sentinel.
  • Patch using a similar approach where in the catch, if the exception matches, we serve the request from the master
    private function retryCommandOnFailure(CommandInterface $command, $method)
    {
        $retries = 0;

        while ($retries <= $this->retryLimit) {
            try {
                $response = $this->getConnectionByCommand($command)->$method($command);
                break;
            } catch(\Predis\Response\ServerException $exception){

                if($exception->getMessage() == 'LOADING Redis is loading the dataset in memory'){
                    $response = $this->getMaster()->$method($command);
                }

            } catch (CommunicationException $exception) {

                
                $this->wipeServerList();
                $exception->getConnection()->disconnect();

                if ($retries === $this->retryLimit) {
                    throw $exception;
                }

                usleep($this->retryWait * 1000);

                ++$retries;
            }
        }

        return $response;
    }

Would a solution like this be considered? Or is there another approach recommended so that we can get this issue resolved and merged into 2.x ?

Versions (please complete the following information):

  • Predis: [e.g. 1.x 2.x]
  • PHP [e.g. 8.1.14]
  • Redis Server [e.g. 7.*]
  • OS [e.g. Ubuntu 20.04]

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions