Skip to content

Exception when using KeysAsync over a huge database #1926

@alexandrutatagit

Description

@alexandrutatagit

Hi!
I have an issue: I want to check all the keys for a specific condition, once per day. In the last days, on PROD, the size of the DB has increased to 50M. I am doing the following:
await foreach (var item in server.KeysAsync(_db)){
...
}
I have updated the connection string to use a 10 min timeout:
allowAdmin=true,connectTimeout=600000,syncTimeout=600000,asyncTimeout=600000,keepAlive=600,$INFO=,$PING=,name=...

But still after 100 sec I get this error:

StackExchange.Redis.RedisConnectionException: InternalFailure (None, last-recv: 434) on XXX:6359/Interactive, Idle/Faulted, last: TIME, origin: ReadFromPipe, outstanding: 2, last-read: 18s ago, last-write: 42s ago, unanswered-write: 103s ago, keep-alive: 60s, state: ConnectedEstablished, mgr: 7 of 10 available, in: 0, in-pipe: 396870066, out-pipe: 0, last-heartbeat: 0s ago, last-mbeat: 0s ago, global: 0s ago, v: 2.2.88.56325 ---> System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values. (Parameter 'offset') at System.ThrowHelper.ThrowArgumentOutOfRangeException_OffsetOutOfRange() at StackExchange.Redis.BufferReader.SnapshotPosition() in //src/StackExchange.Redis/BufferReader.cs:line 117 at StackExchange.Redis.PhysicalConnection.ReadBulkString(BufferReader& reader, Boolean includeDetailInExceptions, ServerEndPoint server) in //src/StackExchange.Redis/PhysicalConnection.cs:line 1770 at StackExchange.Redis.PhysicalConnection.TryParseResult(Arena1 arena, ReadOnlySequence1& buffer, BufferReader& reader, Boolean includeDetilInExceptions, ServerEndPoint server, Boolean allowInlineProtocol) in //src/StackExchange.Redis/PhysicalConnection.cs:line 1841 at StackExchange.Redis.PhysicalConnection.ReadArray(Arena1 arena, ReadOnlySequence1& buffer, BufferReader& reader, Boolean includeDetailInExceptions, ServerEndPoint server) in //src/StackExchange.Redis/PhysicalConnection.cs:line 1756 at StackExchange.Redis.PhysicalConnection.TryParseResult(Arena1 arena, ReadOnlySequence1& buffer, BufferReader& reader, Boolean includeDetilInExceptions, ServerEndPoint server, Boolean allowInlineProtocol) in //src/StackExchange.Redis/PhysicalConnection.cs:line 1841 at StackExchange.Redis.PhysicalConnection.ProcessBuffer(ReadOnlySequence1& buffer) in /_/src/StackExchange.Redis/PhysicalConnection.cs:line 1660 at StackExchange.Redis.PhysicalConnection.ReadFromPipe() in /_/src/StackExchange.Redis/PhysicalConnection.cs:line 1607 --- End of inner exception stack trace --- at StackExchange.Redis.CursorEnumerable1.SingleBlockEnumerable.AwaitedGetNextPageAsync() in //src/StackExchange.Redis/CursorEnumerable.cs:line 352 at StackExchange.Redis.CursorEnumerable`1.Enumerator.AwaitedNextAsync(Boolean isInitial) in /_/src/StackExchange.Redis/CursorEnumerable.cs:line 272 at

Server version is 5+, client version is latest, I am using dotnet core 6.0
What can I do?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions