Skip to content

Continuous memory growth in containerd v2.1.4 #12738

@spy16x

Description

@spy16x

Description

We are seeing continuous memory growth on containerd version github.com/containerd/containerd/v2 2.1.4 in our AWS EKS v1.32 cluster.

The containerd process memory grows constantly and reaches maximum capacity and causes the node to become non-responsive and also causes OS to kill other things abruptly due to memory pressure causing node to go into NotReady status.

Steps to reproduce the issue

We simply keep it running for a few days and memory grows.

(Note: We have duplicate metrics. So two same value lines are shown. Ignore one)

Image

Describe the results you received and expected

Memory should not grow unbounded like this.

What version of containerd are you using?

containerd github.com/containerd/containerd/v2 2.1.4 75cb2b7

Any other relevant information

We also use these nodes for running periodic Argo jobs (short-lived jobs run periodically every 2mins, etc.). I am not sure if this has any relation.

Show configuration if it is related to CRI plugin.

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions