Skip to content

[Soaking Test] In long term test with DMAC mobility, the memory runs out. #4357

@tim-rj

Description

@tim-rj

Description

Steps to reproduce the issue:

  1. Port 1 and Port 2 are members of Vlan2, and Port 1 and Port 2 are connected to the tester respectively.
  2. Send the same traffic with 2k DMAC to the two ports.

Describe the results you received:
Memory continues to grow until memory runs out.

Describe the results you expected:
Memory shouldn't run out.

Additional information you deem important (e.g. issue happens only occasionally):

**Output of `show version`:**

```
SONiC Software Version: SONiC.201811_rj.0-85995e0-20200307.004601
```

From a dump out of memory information, the cause is that the SYNCD put the message of MAC address changes in the message queue, and start a thread to retrieve messages from the queue to updates Redis database. While Redis is busy, which leads to slow update Redis database. A backlog of messages in the message queue causes memory usage increasement. Shutdown the interface or stop the stream, and memory usage will slowly decrease to normal level.
Solution: This problem can be solved by neutralizing MAC addition and aging messages.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions