Skip to content

libcontainerd: work around exec start bug in c8d#45624

Merged
thaJeztah merged 1 commit intomoby:masterfrom
corhere:libc8d/serialize-exec-starts-workaround
May 25, 2023
Merged

libcontainerd: work around exec start bug in c8d#45624
thaJeztah merged 1 commit intomoby:masterfrom
corhere:libc8d/serialize-exec-starts-workaround

Conversation

@corhere
Copy link
Copy Markdown
Contributor

@corhere corhere commented May 25, 2023

- What I did
It turns out that the unnecessary serialization removed in b752462 happened to work around a bug in containerd. When many exec processes are started concurrently in the same containerd task, it takes seconds to minutes for them all to start. Add the workaround back in, only deliberately this time.

- How I did it
✨ mutex ✨

- How to verify it

# docker run --rm -d --name pause1 alpine sleep infinity
# time sh -c 'myjobs=""; for i in $(seq 1 50); do docker exec pause1 true & myjobs="$myjobs $!"; done; for pid in $myjobs; do wait $pid; done'

Takes ~3 seconds on my machine, compared to around a minute without the workaround.

- Description for the changelog

  • Work around an issue in containerd which causes execs to take an excessive amount of time to start

- A picture of a cute animal (not mandatory but encouraged)

It turns out that the unnecessary serialization removed in
b752462 happened to work around a bug
in containerd. When many exec processes are started concurrently in the
same containerd task, it takes seconds to minutes for them all to start.
Add the workaround back in, only deliberately this time.

Signed-off-by: Cory Snider <[email protected]>
@corhere corhere added this to the 25.0.0 milestone May 25, 2023
Copy link
Copy Markdown
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@thaJeztah
Copy link
Copy Markdown
Member

cherry-pick in #45625

@thaJeztah thaJeztah merged commit 88f6a92 into moby:master May 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Many parallel calls to docker exec make Docker unresponsive

4 participants