Skip to content

30s HealthStartInterval being applied to containers created with API 1.43 client #46733

@tboddyspargo

Description

@tboddyspargo

Description

If any non-zero health-start-period is provided to docker run, no health-cmd outcome will affect the container's health status (regardless of success or failure) until the first run after 30s has elapsed or the container is stopped/exits.

Reproduce

Case 1

  • --health-interval=2s
  • --health-start-period=2s
  • container is "running" for 35s (sleep 35)
  • EXPECTED: "healthy" after ~2-4s
  • ACTUAL: "healthy" after ~30-32s
docker run --name=case-1 --health-cmd 'echo hello' --health-interval=2s --health-start-period=2s busybox sleep 35

Case 2

  • --health-interval=2s
  • --health-start-period=15s
  • container is "running" for 35s (sleep 35)
  • EXPECTED: "healthy" after ~2-4s
  • ACTUAL: "healthy" after 32s
docker run --name=case-2 --health-cmd 'echo hello' --health-interval=2s --health-start-period=15s busybox sleep 35

Case 3

  • --health-interval=2s
  • container is "running" for 5s (sleep 5)
  • EXPECTED: "healthy" after ~2s
  • ACTUAL: "healthy" after ~2s (works as expected)
docker run --name=case-3 --health-cmd 'echo hello' --health-interval=2s busybox sleep 5

Case 4

  • --health-interval=2s
  • --health-start-period=15s
  • container is "running" for 20s (sleep 20)
  • EXPECTED: "unhealthy" after ~15-17s
  • ACTUAL: "unhealthy" w/o any health logs after 20s, so we don't know when or how the status changed.
    • In this case, inspecting the container doesn't show any log of healthcheck results, despite the fact that there should have been ~5s during which the container was running outside of the start-period.
      • Extend the sleep command to 35 and you will see that some healthcheck test outcomes are logged, but only after 30s (which is not what the start-period was configured as).
    • I believe this case demonstrates that some 30s threshold is being respected as the start-period instead of the user-provided value.
docker run --name=case-4 --health-cmd 'fake' --health-interval=2s --health-start-period=15s 
busybox sleep 20

Expected behavior

Based on the docs I expected:

  • Any successful health command execution within a specified start-period to allow the container to be marked "healthy" (and any subsequent failure, even if within the start-period, would mark it as "unhealthy")
  • The first health-cmd failure result to occur after the specified start-period should result in the container being immediately marked as unhealthy.

docker version

Client:
 Cloud integration: v1.0.35+desktop.5
 Version:           24.0.6
 API version:       1.43
 Go version:        go1.20.7
 Git commit:        ed223bc
 Built:             Mon Sep  4 12:28:49 2023
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.24.2 (124339)
 Engine:
  Version:          dev
  API version:      1.44 (minimum version 1.12)
  Go version:       go1.20.8
  Git commit:       HEAD
  Built:            Tue Sep 26 11:52:32 2023
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.22
  GitCommit:        8165feabfdfe38c65b599c4993d227328c231fca
 runc:
  Version:          1.1.8
  GitCommit:        v1.1.8-0-g82f18fe
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Version:    24.0.6
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2-desktop.5
    Path:     /Users/tyler/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.22.0-desktop.2
    Path:     /Users/tyler/.docker/cli-plugins/docker-compose
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /Users/tyler/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.20
    Path:     /Users/tyler/.docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.8
    Path:     /Users/tyler/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/tyler/.docker/cli-plugins/docker-sbom
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     /Users/tyler/.docker/cli-plugins/docker-scan
  scout: Docker Scout (Docker Inc.)
    Version:  v1.0.7
    Path:     /Users/tyler/.docker/cli-plugins/docker-scout

Server:
 Containers: 30
  Running: 24
  Paused: 0
  Stopped: 6
 Images: 77
 Server Version: dev
 Storage Driver: stargz
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 8165feabfdfe38c65b599c4993d227328c231fca
 runc version: v1.1.8-0-g82f18fe
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.4.16-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 8
 Total Memory: 19.52GiB
 Name: docker-desktop
 ID: 36252aca-a483-4b9d-b39c-ea24c2c0d207
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

Additional Info

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/apiAPIarea/runtimeRuntimekind/bugBugs are bugs. The cause may or may not be known at triage time so debugging may be needed.priority/P0Urgent: Security, critical bugs, blocking issues. drop everything until this issue is addressed.version/master

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions