Skip to content

signal: do not print message when dealing with SIG_PIPE#4918

Merged
crosbymichael merged 1 commit intocontainerd:masterfrom
liusdu:sig_bus
Jan 12, 2021
Merged

signal: do not print message when dealing with SIG_PIPE#4918
crosbymichael merged 1 commit intocontainerd:masterfrom
liusdu:sig_bus

Conversation

@liusdu
Copy link
Copy Markdown

@liusdu liusdu commented Jan 8, 2021

If we print message when SIG_PIPE occuers in signal handler.
There is a loop {print->SIG_PIPE->print->SIG_PIPE...}, which consume
a lot of cpu time. So do not print message in this situaiton.

Signed-off-by: Liu Hua [email protected]

@k8s-ci-robot
Copy link
Copy Markdown

Hi @liusdu. Thanks for your PR.

I'm waiting for a containerd member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@theopenlab-ci
Copy link
Copy Markdown

theopenlab-ci Bot commented Jan 8, 2021

Build succeeded.

Comment thread cmd/containerd/command/main_unix.go Outdated
@theopenlab-ci
Copy link
Copy Markdown

theopenlab-ci Bot commented Jan 11, 2021

Build succeeded.

@fuweid
Copy link
Copy Markdown
Member

fuweid commented Jan 11, 2021

I only see this issue when I shutdown the integration testing manually, because the TestMain uses pipe to read the debug-log-mode testing-containerd process. The testing-containerd process will be leaky and receive the SIG_PIPE broken event and repeat it...

Comment thread cmd/containerd/command/main_unix.go Outdated
@liusdu
Copy link
Copy Markdown
Author

liusdu commented Jan 12, 2021

I only see this issue when I shutdown the integration testing manually, because the TestMain uses pipe to read the debug-log-mode testing-containerd process. The testing-containerd process will be leaky and receive the SIG_PIPE broken event and repeat it...

Hi @fuweid , I encounter this issue when system journal has been rotated, with messageWarning: Journal has been rotated since unit was started. Log output is incomplete or unavailable. in output of systemctl status dockerd. In this situation containerd writes to the breakpipe, which cause nested signals. The strace output is as following

3084323 12:31:24 write(2, "time=\"2021-01-12T12:31:24.841183276+08:00\" level=debug msg=\"received signal\" signal=broken pipe \n", 97 <unfinished ...>
3081094 12:31:24 futex(0x55b8417a2680, FUTEX_WAIT, 0, NULL <unfinished ...>
3084323 12:31:24 <... write resumed> )  = -1 EPIPE (Broken pipe) <0.000021>
3084323 12:31:24 --- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=1298888, si_uid=0} ---
3084323 12:31:24 futex(0x55b8417a2680, FUTEX_WAKE, 1) = 1 <0.000029>
3081094 12:31:24 <... futex resumed> )  = 0 <0.000111>
3084323 12:31:24 rt_sigreturn( <unfinished ...>
3081094 12:31:24 futex(0x55b8417a2680, FUTEX_WAIT, 0, NULL <unfinished ...>
3084323 12:31:24 <... rt_sigreturn resumed> ) = -1 EPIPE (Broken pipe) <0.000021>
3084323 12:31:24 futex(0x55b8417a2680, FUTEX_WAKE, 1) = 1 <0.000033>
3081094 12:31:24 <... futex resumed> )  = 0 <0.000097>
3084323 12:31:24 write(2, "Failed to write to log, write /dev/stderr: broken pipe\n", 55 <unfinished ...>
3081094 12:31:24 futex(0x55b8417a2680, FUTEX_WAIT, 0, NULL <unfinished ...>
3084323 12:31:24 <... write resumed> )  = -1 EPIPE (Broken pipe) <0.000020>

If we print message when SIG_PIPE occuers in signal handler.
There is a loop {print->SIG_PIPE->print->SIG_PIPE...}, which consume
a lot of cpu time. So do not print message in this situaiton.

Signed-off-by: Liu Hua <[email protected]>
@fuweid
Copy link
Copy Markdown
Member

fuweid commented Jan 12, 2021

@liusdu thanks for the information. What version of docker are you using right now? Is it that the dockerd starts containerd instead of containerd.service?

@theopenlab-ci
Copy link
Copy Markdown

theopenlab-ci Bot commented Jan 12, 2021

Build succeeded.

  • containerd-build-arm64 : RETRY_LIMIT in 1m 00s (non-voting)

@liusdu
Copy link
Copy Markdown
Author

liusdu commented Jan 12, 2021

Yes, I use dockerd as container engine.

docker version
Client:
 Version:           18.06.3-ce
 API version:       1.38
 Go version:        go1.10.4
 Git commit:        d7080c1
 Built:             Wed Feb 20 02:24:22 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server:
 Engine:
  Version:          18.06.3-ce
  API version:      1.38 (minimum version 1.12)
  Go version:       go1.10.3
  Git commit:       d7080c1
  Built:            Wed Feb 20 02:25:33 2019
  OS/Arch:          linux/amd64
  Experimental:     false

@fuweid
Copy link
Copy Markdown
Member

fuweid commented Jan 12, 2021

@liusdu The change is LGTM. Could you add some comment about the case mentioned by @thaJeztah ? Thanks

Copy link
Copy Markdown
Member

@crosbymichael crosbymichael left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@crosbymichael crosbymichael merged commit abc0041 into containerd:master Jan 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants