Optimize aggregation buffer cleanup in CommandHandler #906
Description
When use pressure test to get data from redis, the redis data value is 16KB. It will cause the out of direct memory.
Saw the source code, I found:
Now the method decode in io.lettuce.core.protocol.CommandHandler will try to decode message received from redis server. When faced partial response message(most under BULK scene), will try to continue receive more and then decode more. While current buffer end with a complete message, it will discard the read bytes.
This behavior will keep read bytes always until input buffer end with a complete message. Under my pressure test sense, will always get partial message.
Reproduction setup
- first set a small direct memory to make it easy to reach the direct memory limit, like 50M with
-XX:MaxDirectMemorySize=20m - then set the logger level for
io.lettuce.core.protocoltoDEBUG
Use below test code:
package io.lettuce.core.protocol;
import io.lettuce.core.RedisClient;
import io.lettuce.core.api.StatefulRedisConnection;
import io.lettuce.core.api.async.RedisAsyncCommands;
import java.util.concurrent.TimeUnit;
public class MemoryLeakTest {
public static void main(String[] args) throws InterruptedException {
RedisClient redisClient = RedisClient.create("redis://localhost:6379/0");
StatefulRedisConnection<String, String> connection = redisClient.connect();
RedisAsyncCommands<String, String> syncCommands = connection.async();
syncCommands.set("key", "quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string,"
+ "quiet long string,quiet long string,quiet long string,quiet long string,quiet long string")
.await(5, TimeUnit.SECONDS);
long startTime = System.currentTimeMillis();
while (System.currentTimeMillis() - startTime <= 30_000) {
syncCommands.get("key");
}
connection.close();
redisClient.shutdown();
}
}Then the exception occur:
2018-10-31 10:40:51 [DEBUG] [lettuce-kqueueEventLoop-4-1] [channel=0x2ac75c7d, /127.0.0.1:64284 -> localhost/127.0.0.1:6379, chid=0x1] Unexpected exception during request: io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 4194304 byte(s) of direct memory (used: 16845793, max: 20971520) (CommandHandler:216)
Environment
- redis : 4.0.11
- lettuce : 5.1.2-RELEASE