Skip to content

docker kill does not execute "direct kill" if initial sigkill fails #41587

Description

@sparrc

Description

If killPossiblyDeadProcess within the Kill function fails here: https://github.com/moby/moby/blob/master/daemon/kill.go#L141, then docker waits for 2 seconds, and if the container is not killed then dockerd returns here: https://github.com/moby/moby/blob/master/daemon/kill.go#L160

What docker should do is fall-through to the "killProcessDirectly" function here: https://github.com/moby/moby/blob/master/daemon/kill.go#L165

so if killPossiblyDeadProcess fails, the user should expect to see a log message like "failed to exit within 10 seconds of kill - trying direct SIGKILL" coming from here:

logrus.Infof("Container %s failed to exit within 10 seconds of kill - trying direct SIGKILL", stringid.TruncateID(cntr.ID))

Instead, the return at https://github.com/moby/moby/blob/master/daemon/kill.go#L160 prevents killProcessDirectly from being called.

Steps to reproduce the issue:

  1. add return fmt.Errorf("foo error") here:
    err := daemon.killWithSignal(container, sig)
  2. Build a docker container that captures SIGTERM:
cat << EOF > Dockerfile
FROM bash:5.0

COPY killtrapper.sh /

ENTRYPOINT ["/killtrapper.sh"]
EOF
cat << EOF > killtrapper.sh
#!/usr/bin/env bash

sigterm() {
    echo "caught a signal"
}

trap 'sigterm' QUIT
trap 'sigterm' INT
trap 'sigterm' HUP
trap 'sigterm' TERM

echo "test script started. My PID is $$"
while :; do
    sleep 0.5
done
EOF
  1. build and run the container chmod +x killtrapper.sh && docker build -t "killtrapper" . && docker run -it --rm --name "kt" killtrapper
  2. attempt to kill the container: docker kill kt
  3. command fails without trying "direct kill"
% docker kill kt
Error response from daemon: Cannot kill container: kt: foo error

Describe the results you expected:

if initial docker kill signal fails, fall back to "direct kill" (ie unix kill)

Output of docker version:

dockerd engine is current head of master:
Client:
 Version:           19.03.6-ce
 API version:       1.40
 Go version:        go1.13.4
 Git commit:        369ce74
 Built:             Fri May 29 04:01:26 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server:
 Engine:
  Version:          dev
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       8a4671fb1f
  Built:            Fri Oct 23 04:07:23 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.3.2
  GitCommit:        ff48f57fc83a8c44cf4ad5d672424a98ba37ded6
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Output of docker info:

Client:
 Debug Mode: false

Server:
 Containers: 2
  Running: 1
  Paused: 0
  Stopped: 1
 Images: 7
 Server Version: dev
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: ff48f57fc83a8c44cf4ad5d672424a98ba37ded6
 runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
 init version: fec3683 (expected: de40ad0)
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 4.14.198-152.320.amzn2.x86_64
 Operating System: Amazon Linux 2
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 30.96GiB
 Name: ip-10-0-0-176.us-west-2.compute.internal
 ID: J7BA:BTKN:ZD5S:FWGT:2A2Y:NKTF:DDAJ:QSNA:UE23:L6S2:W7RG:BXBK
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.):

this test was run on an AWS m5.xlarge instance

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions