Skip to content

redis 6.0.8 crashing when reloading from aof file #7869

@ambarsarkar

Description

@ambarsarkar

The .aof file is not huge (~54MB)
The command used is:
redis-6.0.8/bin/redis-server --appendonly yes --appendfilename test_bdc.aof --loglevel debug

Help appreciated!!!

=== REDIS BUG REPORT START: Cut & paste starting from here ===
23506:M 30 Sep 2020 06:30:43.434 # Redis 6.0.8 crashed by signal: 11
23506:M 30 Sep 2020 06:30:43.434 # Crashed running the instruction at: 0x4480b5
23506:M 30 Sep 2020 06:30:43.434 # Accessing address: (nil)
23506:M 30 Sep 2020 06:30:43.434 # Failed assertion: (:0)

------ STACK TRACE ------
EIP:
/home/utils/redis-6.0.8/bin/redis-server *:6379(protectClient+0x15)[0x4480b5]

Backtrace:
/home/utils/redis-6.0.8/bin/redis-server *:6379(logStackTrace+0x32)[0x47bd92]
/home/utils/redis-6.0.8/bin/redis-server *:6379(sigsegvHandler+0xa6)[0x47c466]
/lib64/libpthread.so.0(+0xf7e0)[0x7ffff753f7e0]
/home/utils/redis-6.0.8/bin/redis-server *:6379(protectClient+0x15)[0x4480b5]
/home/utils/redis-6.0.8/bin/redis-server *:6379(luaMaskCountHook+0x91)[0x489781]
/home/utils/redis-6.0.8/bin/redis-server *:6379[0x4d0cf0]
/home/utils/redis-6.0.8/bin/redis-server *:6379[0x4da451]
/home/utils/redis-6.0.8/bin/redis-server *:6379[0x4d14fd]
/home/utils/redis-6.0.8/bin/redis-server *:6379[0x4d0848]
/home/utils/redis-6.0.8/bin/redis-server *:6379[0x4d165b]
/home/utils/redis-6.0.8/bin/redis-server *:6379(lua_pcall+0x46)[0x4cee66]
/home/utils/redis-6.0.8/bin/redis-server *:6379(evalGenericCommand+0x18e)[0x48aebe]
/home/utils/redis-6.0.8/bin/redis-server *:6379(loadAppendOnlyFile+0x537)[0x4772f7]
/home/utils/redis-6.0.8/bin/redis-server *:6379(loadDataFromDisk+0xcc)[0x43e96c]
/home/utils/redis-6.0.8/bin/redis-server *:6379(main+0x3b8)[0x430488]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x7ffff71bad1d]
/home/utils/redis-6.0.8/bin/redis-server *:6379[0x43084d]

------ INFO OUTPUT ------

Server

redis_version:6.0.8
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:240e25d97efd0115
redis_mode:standalone
os:Linux 4.1.15.pnotify x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:atomic-builtin
gcc_version:5.5.0
process_id:23506
run_id:ea7429b07e9403177cb648c7193157b9d8941f9f
tcp_port:6379
uptime_in_seconds:5
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:7637118
executable:/home/utils/redis-6.0.8/bin/redis-server
config_file:
io_threads_active:0

Clients

connected_clients:0
client_recent_max_input_buffer:0
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0

Memory

used_memory:1018552
used_memory_human:994.68K
used_memory_rss:0
used_memory_rss_human:0B
used_memory_peak:1018552
used_memory_peak_human:994.68K
used_memory_peak_perc:101.18%
used_memory_overhead:869128
used_memory_startup:786680
used_memory_dataset:149424
used_memory_dataset_perc:64.44%
allocator_allocated:0
allocator_active:0
allocator_resident:0
total_system_memory:135069868032
total_system_memory_human:125.79G
used_memory_lua:67202048
used_memory_lua_human:64.09M
used_memory_scripts:80016
used_memory_scripts_human:78.14K
number_of_cached_scripts:34
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:-nan
allocator_frag_bytes:0
allocator_rss_ratio:-nan
allocator_rss_bytes:0
rss_overhead_ratio:-nan
rss_overhead_bytes:0
mem_fragmentation_ratio:-nan
mem_fragmentation_bytes:0
mem_not_counted_for_evict:0
mem_replication_backlog:0
mem_clients_slaves:0
mem_clients_normal:0
mem_aof_buffer:0
mem_allocator:jemalloc-5.1.0
active_defrag_running:0
lazyfree_pending_objects:0

Persistence

loading:1
rdb_changes_since_last_save:1541
rdb_bgsave_in_progress:0
rdb_last_save_time:1601472638
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0
aof_current_size:0
aof_base_size:0
aof_pending_rewrite:0
aof_buffer_length:0
aof_rewrite_buffer_length:0
aof_pending_bio_fsync:0
aof_delayed_fsync:0
loading_start_time:1601472638
loading_total_bytes:56626161
loading_loaded_bytes:1155844
loading_loaded_perc:2.04
loading_eta_seconds:239

Stats

total_connections_received:0
total_commands_processed:3956699
instantaneous_ops_per_sec:0
total_net_input_bytes:0
total_net_output_bytes:0
instantaneous_input_kbps:0.00
instantaneous_output_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:0
evicted_keys:0
keyspace_hits:3955433
keyspace_misses:180
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:2
total_reads_processed:0
total_writes_processed:0
io_threaded_reads_processed:0
io_threaded_writes_processed:0

Replication

role:master
connected_slaves:0
master_replid:3d32f9cca76797332a33a0ee857b2e965d75a84b
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

CPU

used_cpu_sys:0.024000
used_cpu_user:4.992000
used_cpu_sys_children:0.000000
used_cpu_user_children:0.000000

Modules

Commandstats

Cluster

cluster_enabled:0

Keyspace

db0:keys=48,expires=0,avg_ttl=0

------ CLIENT LIST OUTPUT ------

------ REGISTERS ------
23506:M 30 Sep 2020 06:30:43.435 #
RAX:0000000000000000 RBX:00007ffff6d18b00
RCX:000000000043741a RDX:0000000000000000
RDI:0000000000000000 RSI:0000000000000000
RBP:0000000000000170 RSP:00007fffffffc8f0
R8 :00007ffff7fb1f40 R9 :64373866203a7369
R10:6332323164333537 R11:0000000000000000
R12:0000000000000170 R13:00000000004896f0
R14:00000000009fa5b0 R15:0000000000a0977c
RIP:00000000004480b5 EFL:0000000000010246
CSGSFS:0000000000000033
23506:M 30 Sep 2020 06:30:43.435 # (00007fffffffc8ff) -> 00000000004d8f7d
23506:M 30 Sep 2020 06:30:43.435 # (00007fffffffc8fe) -> 0000000000a1fa90
23506:M 30 Sep 2020 06:30:43.435 # (00007fffffffc8fd) -> 0000000000000064
23506:M 30 Sep 2020 06:30:43.435 # (00007fffffffc8fc) -> 0000000000a1fab0
23506:M 30 Sep 2020 06:30:43.435 # (00007fffffffc8fb) -> 0000000000a1faa0
23506:M 30 Sep 2020 06:30:43.435 # (00007fffffffc8fa) -> 00000000009fa5b0
23506:M 30 Sep 2020 06:30:43.435 # (00007fffffffc8f9) -> 00000000ffffffff
23506:M 30 Sep 2020 06:30:43.435 # (00007fffffffc8f8) -> 0000000000a1faa0
23506:M 30 Sep 2020 06:30:43.435 # (00007fffffffc8f7) -> 00000000004deff5
23506:M 30 Sep 2020 06:30:43.435 # (00007fffffffc8f6) -> 00000000009fa5b0
23506:M 30 Sep 2020 06:30:43.435 # (00007fffffffc8f5) -> 00000000009fa5b0
23506:M 30 Sep 2020 06:30:43.435 # (00007fffffffc8f4) -> 0000000000000003
23506:M 30 Sep 2020 06:30:43.435 # (00007fffffffc8f3) -> 00000000004d0cf0
23506:M 30 Sep 2020 06:30:43.435 # (00007fffffffc8f2) -> 00000000009fa5b0
23506:M 30 Sep 2020 06:30:43.435 # (00007fffffffc8f1) -> 0000000000489781
23506:M 30 Sep 2020 06:30:43.435 # (00007fffffffc8f0) -> 00000000009fa5b0

------ MODULES INFO OUTPUT ------

------ FAST MEMORY TEST ------
23506:M 30 Sep 2020 06:30:43.435 # Bio thread for job type #0 terminated
23506:M 30 Sep 2020 06:30:43.435 # Bio thread for job type #1 terminated
23506:M 30 Sep 2020 06:30:43.435 # Bio thread for job type #2 terminated
*** Preparing to test memory region 7ca000 (2633728 bytes)
*** Preparing to test memory region 7fffe6912000 (67112960 bytes)
*** Preparing to test memory region 7ffff0753000 (33787904 bytes)
*** Preparing to test memory region 7ffff6800000 (8388608 bytes)
*** Preparing to test memory region 7ffff752c000 (16384 bytes)
*** Preparing to test memory region 7ffff7749000 (16384 bytes)
*** Preparing to test memory region 7ffff7fb0000 (24576 bytes)
*** Preparing to test memory region 7ffff7fed000 (45056 bytes)
*** Preparing to test memory region 7ffff7ffe000 (4096 bytes)
.O.O.O.O.O.O.O.O.O
Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.

------ DUMPING CODE AROUND EIP ------
Symbol: protectClient (base: 0x4480a0)
Module: /home/utils/redis-6.0.8/bin/redis-server *:6379 (base 0x400000)
$ xxd -r -p /tmp/dump.hex /tmp/dump.bin
$ objdump --adjust-vma=0x4480a0 -D -b binary -m i386:x86-64 /tmp/dump.bin

23506:M 30 Sep 2020 06:30:43.749 # dump of function (hexdump of 149 bytes):
534889fb48818fb000000000000010488b7f0831f6488b07ff5038488b7b0831d231f65b488b07488b4030ffe00f1f00488b87b0000000a9000000107502c3904825ffffffef534889fb488987b0000000488b7f08bed0c54400488b07ff50388b835402000085c075164889dfe8aee5ffff85c0750a5bc30f1f8400000000004889df5be9b7e9ffff0f1f800000000055534889fb

=== REDIS BUG REPORT END. Make sure to include from START to END. ===

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions