Skip to content

S6100-T0-64-Continuous Orchagent crash is seen in the latest Jenkins image HEAD.187-dirty-20200130.072127. #4089

@mini-nair-dell

Description

@mini-nair-dell

Continuous Orchagent crash is seen in the latest Jenkins image HEAD.187-dirty-20200130.072127.

Though all the containers were initially up, few of them stopped running after a few minutes

Attached the syslog.

root@sonic-s6100-07:/var/log# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a530841b280b docker-lldp-sv2:latest "/usr/bin/supervisord" 23 hours ago Up 12 minutes lldp
9b221b9263c2 docker-platform-monitor:latest "/usr/bin/docker_ini…" 23 hours ago Up 12 minutes pmon
eee8566ea817 docker-fpm-frr:latest "/usr/bin/supervisord" 23 hours ago Up 12 minutes bgp
d6265bc8235e docker-database:latest "/usr/local/bin/dock…" 23 hours ago Up 28 minutes database
root@sonic-s6100-07:/var/log#

root@sonic-s6100-07:/var/log# show ip bgp summ

IPv4 Unicast Summary:
BGP router identifier 10.1.0.32, local AS number 64601 vrf-id 0
BGP table version 2
RIB entries 3, using 552 bytes of memory
Peers 4, using 82 KiB of memory
Peer groups 4, using 256 bytes of memory

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd NeighborName
10.0.0.1 4 64802 0 0 0 0 0 never Active ARISTA01T1
10.0.0.5 4 64802 0 0 0 0 0 never Active ARISTA02T1
10.0.0.9 4 64802 0 0 0 0 0 never Active ARISTA03T1
10.0.0.13 4 64802 0 0 0 0 0 never Active ARISTA04T1

Log snipp

Jan 31 08:10:09.998647 sonic-s6100-07 ERR syncd#syncd: [none] brcm_sai_set_lag_member_attribute:1103 trunk set failed with error Invalid parameter (0xfffffffc).
Jan 31 08:10:09.998647 sonic-s6100-07 ERR syncd#syncd: :- processEvent: VID: oid:0x1b000000000b1e RID: oid:0x69001b00000000
Jan 31 08:10:09.998647 sonic-s6100-07 ERR syncd#syncd: :- processEvent: attr: SAI_LAG_MEMBER_ATTR_INGRESS_DISABLE: true
Jan 31 08:10:09.998647 sonic-s6100-07 ERR syncd#syncd: :- processEvent: failed to execute api: set, key: SAI_OBJECT_TYPE_LAG_MEMBER:oid:0x1b000000000b1e, status: SAI_STATUS_INVALID_PARAMETER
Jan 31 08:10:09.998647 sonic-s6100-07 ERR syncd#syncd: :- syncd_main: Runtime error: :- processEvent: failed to execute api: set, key: SAI_OBJECT_TYPE_LAG_MEMBER:oid:0x1b000000000b1e, status: SAI_STATUS_INVALID_PARAMETER
Jan 31 08:10:59.784594 sonic-s6100-07 ERR monit[542]: 'telemetry' process is not running
Jan 31 08:10:59.797896 sonic-s6100-07 ERR monit[542]: 'dialout_client' process is not running
Jan 31 08:10:59.995597 sonic-s6100-07 ERR monit[542]: 'sflowmgrd' process is not running
Jan 31 08:11:01.436965 sonic-s6100-07 INFO systemd[1]: swss.service: Service hold-off time over, scheduling restart.
Jan 31 08:11:01.438464 sonic-s6100-07 INFO systemd[1]: Stopped switch state service.
Jan 31 08:11:01.439095 sonic-s6100-07 WARNING systemd[1]: swss.service: Start request repeated too quickly.
Jan 31 08:11:01.439722 sonic-s6100-07 ERR systemd[1]: Failed to start switch state service.
Jan 31 08:12:00.023480 sonic-s6100-07 ERR monit[542]: 'telemetry' process is not running
Jan 31 08:12:00.037042 sonic-s6100-07 ERR monit[542]: 'dialout_client' process is not running
Jan 31 08:12:00.233619 sonic-s6100-07 ERR monit[542]: 'sflowmgrd' process is not running
Jan 31 08:13:00.263916 sonic-s6100-07 ERR monit[542]: 'telemetry' process is not running
Jan 31 08:13:00.278011 sonic-s6100-07 ERR monit[542]: 'dialout_client' process is not running
Jan 31 08:13:00.474411 sonic-s6100-07 ERR monit[542]: 'sflowmgrd' process is not running
Jan 31 08:14:00.587865 sonic-s6100-07 ERR monit[542]: 'telemetry' process is not running
Jan 31 08:14:00.600532 sonic-s6100-07 ERR monit[542]: 'dialout_client' process is not running
Jan 31 08:14:00.793517 sonic-s6100-07 ERR monit[542]: 'sflowmgrd' process is not running
Jan 31 08:15:00.845085 sonic-s6100-07 ERR monit[542]: 'telemetry' process is not running

Thanks
Mini

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions