-
Notifications
You must be signed in to change notification settings - Fork 1.8k
S6100-T0-64-Continuous Orchagent crash is seen in the latest Jenkins image HEAD.187-dirty-20200130.072127. #4089
Description
Continuous Orchagent crash is seen in the latest Jenkins image HEAD.187-dirty-20200130.072127.
Though all the containers were initially up, few of them stopped running after a few minutes
Attached the syslog.
root@sonic-s6100-07:/var/log# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a530841b280b docker-lldp-sv2:latest "/usr/bin/supervisord" 23 hours ago Up 12 minutes lldp
9b221b9263c2 docker-platform-monitor:latest "/usr/bin/docker_ini…" 23 hours ago Up 12 minutes pmon
eee8566ea817 docker-fpm-frr:latest "/usr/bin/supervisord" 23 hours ago Up 12 minutes bgp
d6265bc8235e docker-database:latest "/usr/local/bin/dock…" 23 hours ago Up 28 minutes database
root@sonic-s6100-07:/var/log#
root@sonic-s6100-07:/var/log# show ip bgp summ
IPv4 Unicast Summary:
BGP router identifier 10.1.0.32, local AS number 64601 vrf-id 0
BGP table version 2
RIB entries 3, using 552 bytes of memory
Peers 4, using 82 KiB of memory
Peer groups 4, using 256 bytes of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd NeighborName
10.0.0.1 4 64802 0 0 0 0 0 never Active ARISTA01T1
10.0.0.5 4 64802 0 0 0 0 0 never Active ARISTA02T1
10.0.0.9 4 64802 0 0 0 0 0 never Active ARISTA03T1
10.0.0.13 4 64802 0 0 0 0 0 never Active ARISTA04T1
Log snipp
Jan 31 08:10:09.998647 sonic-s6100-07 ERR syncd#syncd: [none] brcm_sai_set_lag_member_attribute:1103 trunk set failed with error Invalid parameter (0xfffffffc).
Jan 31 08:10:09.998647 sonic-s6100-07 ERR syncd#syncd: :- processEvent: VID: oid:0x1b000000000b1e RID: oid:0x69001b00000000
Jan 31 08:10:09.998647 sonic-s6100-07 ERR syncd#syncd: :- processEvent: attr: SAI_LAG_MEMBER_ATTR_INGRESS_DISABLE: true
Jan 31 08:10:09.998647 sonic-s6100-07 ERR syncd#syncd: :- processEvent: failed to execute api: set, key: SAI_OBJECT_TYPE_LAG_MEMBER:oid:0x1b000000000b1e, status: SAI_STATUS_INVALID_PARAMETER
Jan 31 08:10:09.998647 sonic-s6100-07 ERR syncd#syncd: :- syncd_main: Runtime error: :- processEvent: failed to execute api: set, key: SAI_OBJECT_TYPE_LAG_MEMBER:oid:0x1b000000000b1e, status: SAI_STATUS_INVALID_PARAMETER
Jan 31 08:10:59.784594 sonic-s6100-07 ERR monit[542]: 'telemetry' process is not running
Jan 31 08:10:59.797896 sonic-s6100-07 ERR monit[542]: 'dialout_client' process is not running
Jan 31 08:10:59.995597 sonic-s6100-07 ERR monit[542]: 'sflowmgrd' process is not running
Jan 31 08:11:01.436965 sonic-s6100-07 INFO systemd[1]: swss.service: Service hold-off time over, scheduling restart.
Jan 31 08:11:01.438464 sonic-s6100-07 INFO systemd[1]: Stopped switch state service.
Jan 31 08:11:01.439095 sonic-s6100-07 WARNING systemd[1]: swss.service: Start request repeated too quickly.
Jan 31 08:11:01.439722 sonic-s6100-07 ERR systemd[1]: Failed to start switch state service.
Jan 31 08:12:00.023480 sonic-s6100-07 ERR monit[542]: 'telemetry' process is not running
Jan 31 08:12:00.037042 sonic-s6100-07 ERR monit[542]: 'dialout_client' process is not running
Jan 31 08:12:00.233619 sonic-s6100-07 ERR monit[542]: 'sflowmgrd' process is not running
Jan 31 08:13:00.263916 sonic-s6100-07 ERR monit[542]: 'telemetry' process is not running
Jan 31 08:13:00.278011 sonic-s6100-07 ERR monit[542]: 'dialout_client' process is not running
Jan 31 08:13:00.474411 sonic-s6100-07 ERR monit[542]: 'sflowmgrd' process is not running
Jan 31 08:14:00.587865 sonic-s6100-07 ERR monit[542]: 'telemetry' process is not running
Jan 31 08:14:00.600532 sonic-s6100-07 ERR monit[542]: 'dialout_client' process is not running
Jan 31 08:14:00.793517 sonic-s6100-07 ERR monit[542]: 'sflowmgrd' process is not running
Jan 31 08:15:00.845085 sonic-s6100-07 ERR monit[542]: 'telemetry' process is not running
Thanks
Mini