Jan 30 23:22:32.637122 str2-xxxx-lc1-3 INFO acms#start.py: start: main: Waiting for bootstrap cert
Jan 30 23:23:00.293090 str2-xxxx-lc1-3 DEBUG syncd0#saidump: :> main: enter
Jan 30 23:23:00.307986 str2-xxxx-lc1-3 INFO syncd0#saidump: :- loadRedisScript: lua script local keys = redis.call("KEYS", KEYS[1] .. ":*")#012local res = {}#012#012for i,k in pairs(keys) do#012 local sres={}#012#012 local flat_map = redis.call('HGETALL', k)#012 for j = 1, #flat_map, 2 do#012 sres[flat_map[j]] = flat_map[j + 1]#012 end#012#012 res[k] = sres#012end#012#012return cjson.encode(res)#012 loaded, sha: 654245aafba722f7b601e1fc414c3647058c053c
Jan 30 23:23:05.329309 str2-xxxx-lc1-3 INFO database0#supervisord: redis 39:M 30 Jan 2023 23:23:05.328 # Lua slow script detected: still in execution after 5021 milliseconds. You can try killing the script using the SCRIPT KILL command. Script SHA1 is: 654245aafba722f7b601e1fc414c3647058c053c
Jan 30 23:23:05.329391 str2-xxxxx-lc1-3 ERR gbsyncd0#GBSAI[18]: :- checkReplyType: Expected to get redis type 5 got type 6, err: BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE.
On T2 with more route scale we see LUA script: https://github.com/sonic-net/sonic-swss-common/blob/master/common/table_dump.lua invoked as part of saidump commands can take more than 5 sec and thus all other process start failing as no/incorrect reply from Redis.
Command
generate_dump -s yesterday