Skip to content

/containers/{id}/stop doesn't commit state of stopped container #50133

Closed
#50136
Closed
/containers/{id}/stop doesn't commit state of stopped container#50133
#50136
Labels
kind/bugBugs are bugs. The cause may or may not be known at triage time so debugging may be needed.status/0-triage
@mabrarov

Description

@mabrarov
mabrarov
opened on Jun 2, 2025

Description

/containers/{id}/stop Docker Engine API implementation doesn't commit state of stopped container in Docker Engine memory database (e.g. using container.Container.CommitInMemory method like container healthcheck monitoring does, or using container.Container.CheckpointTo method like /containers/{id}/pause API does) which causes subsequent Docker Engine API calls reporting wrong state (running) of already stopped container. Due to this issue is of race condition nature, it can be hard to reproduce (the pause b/w Docker Engine API calls should be small).

Refer to testcontainers/testcontainers-go#3177 for original issue reported for Testcontainers for Go and to testcontainers/testcontainers-go#3177 (comment) for the test reproducing this issue with curl.

Reproduce

container_id="$(docker run -d --name 'my-nginx' 'nginx:alpine')" && \
sleep 10s && \
docker_unix_socket='/var/run/docker.sock' && \
curl_unix_socket_host='http://localhost' && \
containers_list_url="${curl_unix_socket_host}/containers/json?all=1&filters=$(jq -rn --arg x '{"name":{"^my-nginx$":true}}' '$x|@uri')" && \
curl -v --unix-socket "${docker_unix_socket}" -X POST "${curl_unix_socket_host}/containers/${container_id}/stop" \
  --next -v --unix-socket "${docker_unix_socket}" "${containers_list_url}" && \
docker ps -a && \
docker rm -fv "${container_id}"

Expected behavior

State/status of container reported by curl (by /containers/json Docker Engine API) is the same (stopped) as state/status reported by "docker ps" command.

docker version

Client: Docker Engine - Community
 Version:           28.2.2
 API version:       1.50
 Go version:        go1.24.3
 Git commit:        e6534b4
 Built:             Fri May 30 12:07:27 2025
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          28.2.2
  API version:      1.50 (minimum version 1.24)
  Go version:       go1.24.3
  Git commit:       45873be
  Built:            Fri May 30 12:07:27 2025
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.27
  GitCommit:        05044ec0a9a75232cad458027ca83437aae3f4da
 runc:
  Version:          1.2.5
  GitCommit:        v1.2.5-0-g59923ef
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client: Docker Engine - Community
 Version:    28.2.2
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.24.0
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.36.2
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 17
 Server Version: 28.2.2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 05044ec0a9a75232cad458027ca83437aae3f4da
 runc version: v1.2.5-0-g59923ef
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.8.0-60-generic
 Operating System: Ubuntu 24.04.2 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 15.57GiB
 Name: ubuntu-24
 ID: f2e6309d-6ed9-4c1d-8413-d1a7fde46d84
 Docker Root Dir: /data/docker
 Debug Mode: false
 Username: abrarov
 Experimental: false
 Insecure Registries:
  192.168.0.0/16
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false

Additional Info

  1. Possible fix: fix: daemon: state of stopped container visible to other queries when container is stopped #50136. It was successfully (manually, within debugger session) tested with the same test as provided in description of this issue. This issue was also reproduced under the same conditions, but without the fix.
  2. Workaround - using API which acquires lock of container state (like /containers/{id}/json API does) to check container state after container was stopped: fix: workaround for moby/moby#50133 when reusing container testcontainers/testcontainers-go#3197

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugBugs are bugs. The cause may or may not be known at triage time so debugging may be needed.status/0-triage

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions