Description
Compose watch mode may currently stop services that link other services without ever restarting:
- The idea is that a service (think a front proxy) links any another service (think a backend).
- Compose watch mode will correctly identify that the backend service needs to be recreated/restarted (e.g. using
action: rebuild).
- It will correctly stop the front service before stopping the backend service.
- It will correctly restart the backend, but fails to restart the front service afterwards.
- The application is now in a broken state ("half up"), the backend can no longer be reached without a front proxy.
I've specifically encountered this when using legacy links between services, but the same behavior can be reproduced when using depends_on with restart: true. It's my understanding legacy links would basically act like depends_on with restart: true, despite restart: false being the default otherwise.
I think it's debatable if the front proxy service needs to be restarted in this example. In either case, it should either be restarted completely or not restarted at all. I don't think stopping the service without restarting it again would ever be intended behavior.
The problem occurred as part of a bigger application using multiple services, but I've been able to reproduce this in a most basic compose.yaml file with no external dependencies except a public image. The resulting file may look a bit contrived, but I hope this should be enough to reproduce this without any specific service configurations.
Possibly related, I've checked docker compose up -a reports the affected services as "exited". Using docker compose restart backend does restart the backend and the affected services as expected. However, the running docker compose up --watch session will not pick up the affected services again. This appears to work fine if I do not trigger a rebuild first.
I've checked for similar reports and stumbled upon #11695 and #11813, but believe they may point to different, but possibly related, issues.
The recent changes regarding watch mode and watching multiple services as implemented in #12469 and others have been a real game-changer for me personally and a larger number of projects I'm working on, so keep up the great work! 👍
Steps To Reproduce
Basic reproducer:
services:
nok:
image: httpd:2.4-alpine
links:
- backend
backend:
build:
dockerfile_inline: |
FROM httpd:2.4-alpine
develop:
watch:
- path: compose.yaml
action: rebuild
Reproducer showing two broken services + 2 working ones as possible workarounds:
services:
nok:
image: httpd:2.4-alpine
links:
- backend
same:
image: httpd:2.4-alpine
depends_on:
backend:
condition: service_started
restart: true
ok:
image: httpd:2.4-alpine
depends_on:
- backend
alright:
image: httpd:2.4-alpine
depends_on:
backend:
condition: service_started
restart: false
backend:
build:
dockerfile_inline: |
FROM httpd:2.4-alpine
develop:
watch:
- path: compose.yaml
action: rebuild
Start watch mode in foreground:
docker compose up --watch
Change compose.yaml file to trigger rebuild:
Log output shows backend service is correctly rebuilt, but affected front proxy services stop:
⦿ Rebuilding service(s) ["backend"] after changes were detected...
⦿ service(s) ["backend"] successfully built
nok-1 | [Thu May 08 13:30:52.184586 2025] [mpm_event:notice] [pid 1:tid 123450472430344] AH00492: caught SIGWINCH, shutting down gracefully
same-1 | [Thu May 08 13:30:52.189356 2025] [mpm_event:notice] [pid 1:tid 130455114877704] AH00492: caught SIGWINCH, shutting down gracefully
nok-1 exited with code 0
same-1 exited with code 0
backend-1 | [Thu May 08 13:30:53.516278 2025] [mpm_event:notice] [pid 1:tid 128908057438984] AH00492: caught SIGWINCH, shutting down gracefully
backend-1 exited with code 0
backend-1 has been recreated
You can see both nok and same are stopped without restarting.
Compose Version
Docker Compose version v2.36.0
Docker Environment
Client:
Version: 26.1.3
Context: default
Debug Mode: false
Plugins:
compose: Docker Compose (Docker Inc.)
Version: v2.36.0
Path: /usr/local/libexec/docker/cli-plugins/docker-compose
Server:
Containers: 33
Running: 11
Paused: 0
Stopped: 22
Images: 1537
Server Version: 26.1.3
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version:
runc version:
init version:
Security Options:
apparmor
seccomp
Profile: builtin
cgroupns
Kernel Version: 6.8.0-59-generic
Operating System: Ubuntu 24.04.2 LTS
OSType: linux
Architecture: x86_64
CPUs: 16
Total Memory: 30.54GiB
Name: me
ID: 519acebf-0404-4fc7-a89d-afcb3bcf9b15
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Anything else?
No response
Description
Compose watch mode may currently stop services that link other services without ever restarting:
action: rebuild).I've specifically encountered this when using legacy
linksbetween services, but the same behavior can be reproduced when usingdepends_onwithrestart: true. It's my understanding legacylinkswould basically act likedepends_onwithrestart: true, despiterestart: falsebeing the default otherwise.I think it's debatable if the front proxy service needs to be restarted in this example. In either case, it should either be restarted completely or not restarted at all. I don't think stopping the service without restarting it again would ever be intended behavior.
The problem occurred as part of a bigger application using multiple services, but I've been able to reproduce this in a most basic
compose.yamlfile with no external dependencies except a public image. The resulting file may look a bit contrived, but I hope this should be enough to reproduce this without any specific service configurations.Possibly related, I've checked
docker compose up -areports the affected services as "exited". Usingdocker compose restart backenddoes restart thebackendand the affected services as expected. However, the runningdocker compose up --watchsession will not pick up the affected services again. This appears to work fine if I do not trigger a rebuild first.I've checked for similar reports and stumbled upon #11695 and #11813, but believe they may point to different, but possibly related, issues.
The recent changes regarding watch mode and watching multiple services as implemented in #12469 and others have been a real game-changer for me personally and a larger number of projects I'm working on, so keep up the great work! 👍
Steps To Reproduce
Basic reproducer:
Reproducer showing two broken services + 2 working ones as possible workarounds:
Start watch mode in foreground:
Change
compose.yamlfile to trigger rebuild:Log output shows backend service is correctly rebuilt, but affected front proxy services stop:
You can see both
nokandsameare stopped without restarting.Compose Version
Docker Environment
Anything else?
No response