Description
With the last upgrade of our 2 docker swarm environments, we have the bad surprise to see that the external networking of our containers does not work anymore.
working version :
❯ docker version
Client: Docker Engine - Community
Version: 28.1.1
API version: 1.49
Go version: go1.23.8
Git commit: 4eba377
Built: Fri Apr 18 09:52:57 2025
OS/Arch: linux/amd64
Context: default
Server: Docker Engine - Community
Engine:
Version: 28.1.1
API version: 1.49 (minimum version 1.24)
Go version: go1.23.8
Git commit: 01f442b
Built: Fri Apr 18 09:52:57 2025
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.7.27
GitCommit: 05044ec0a9a75232cad458027ca83437aae3f4da
runc:
Version: 1.2.5
GitCommit: v1.2.5-0-g59923ef
docker-init:
Version: 0.19.0
GitCommit: de40ad0
ill version :
❯ docker version
Client: Docker Engine - Community
Version: 28.2.2
API version: 1.50
Go version: go1.24.3
Git commit: e6534b4
Built: Fri May 30 12:07:26 2025
OS/Arch: linux/amd64
Context: default
Server: Docker Engine - Community
Engine:
Version: 28.2.2
API version: 1.50 (minimum version 1.24)
Go version: go1.24.3
Git commit: 45873be
Built: Fri May 30 12:07:26 2025
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.7.27
GitCommit: 05044ec0a9a75232cad458027ca83437aae3f4da
runc:
Version: 1.2.5
GitCommit: v1.2.5-0-g59923ef
docker-init:
Version: 0.19.0
GitCommit: de40ad0
in the "ill" environment containers answer with gateway timeout (504)
I can see there are many differences between the two iptables (ill/normal) and there are issues arround this but it claims to be fixed in 28.2.1+ version (tried this version with no luck)
Happy to help to solve this issue that we do not have in our standalone docker environnments (on a working 28.2.2 version)
Reproduce
curl -vvv https://:443/my-api-path/actuator/health => 504 Gateway timeout
Expected behavior
curl -vvv https://:443/my-api-path/actuator/health => 200
docker version
❯ docker version
Client: Docker Engine - Community
Version: 28.2.2
API version: 1.50
Go version: go1.24.3
Git commit: e6534b4
Built: Fri May 30 12:07:26 2025
OS/Arch: linux/amd64
Context: default
Server: Docker Engine - Community
Engine:
Version: 28.2.2
API version: 1.50 (minimum version 1.24)
Go version: go1.24.3
Git commit: 45873be
Built: Fri May 30 12:07:26 2025
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.7.27
GitCommit: 05044ec0a9a75232cad458027ca83437aae3f4da
runc:
Version: 1.2.5
GitCommit: v1.2.5-0-g59923ef
docker-init:
Version: 0.19.0
GitCommit: de40ad0
docker info
❯ docker info
Client: Docker Engine - Community
Version: 28.2.2
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.24.0
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.36.2
Path: /usr/libexec/docker/cli-plugins/docker-compose
Server:
Containers: 65
Running: 18
Paused: 0
Stopped: 47
Images: 60
Server Version: 28.2.2
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
CDI spec directories:
/etc/cdi
/var/run/cdi
Swarm: active
NodeID: lguz1823tpttyi6qhs1rdj52z
Is Manager: true
ClusterID: difcxb8iji4qias88qea4inqj
Managers: 7
Nodes: 7
Default Address Pool: 192.168.208.0/21
SubnetSize: 25
Data Path Port: 4789
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
Election Tick: 10
Dispatcher:
Heartbeat Period: 5 seconds
CA Configuration:
Expiry Duration: 3 months
Force Rotate: 0
Autolock Managers: false
Root Rotation In Progress: false
Node Address: 172.30.x.x
Manager Addresses:
172.30.x.x:2377
172.30.x.x:2377
172.30.x.x:2377
172.30.x.x:2377
172.30.x.x:2377
172.30.x.x:2377
172.30.x.x:2377
Runtimes: runc io.containerd.runc.v2
Default Runtime: runc
Init Binary: docker-init
containerd version: 05044ec0a9a75232cad458027ca83437aae3f4da
runc version: v1.2.5-0-g59923ef
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: builtin
cgroupns
Kernel Version: 6.1.0-37-amd64
Operating System: Debian GNU/Linux 12 (bookworm)
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 19.53GiB
Name: srv-swarm-xx-lab
ID: d9161163-6842-4e59-a6f9-197b06e64df1
Docker Root Dir: /var/lib/docker
Debug Mode: false
Username: [email protected]
Experimental: false
Insecure Registries:
::1/128
127.0.0.0/8
Live Restore Enabled: false
Default Address Pools:
Base: 192.168.216.0/22, Size: 26Additional Info
No response
Description
With the last upgrade of our 2 docker swarm environments, we have the bad surprise to see that the external networking of our containers does not work anymore.
working version :
ill version :
in the "ill" environment containers answer with gateway timeout (504)
I can see there are many differences between the two iptables (ill/normal) and there are issues arround this but it claims to be fixed in 28.2.1+ version (tried this version with no luck)
Happy to help to solve this issue that we do not have in our standalone docker environnments (on a working 28.2.2 version)
Reproduce
curl -vvv https://:443/my-api-path/actuator/health => 504 Gateway timeout
Expected behavior
curl -vvv https://:443/my-api-path/actuator/health => 200
docker version
❯ docker version Client: Docker Engine - Community Version: 28.2.2 API version: 1.50 Go version: go1.24.3 Git commit: e6534b4 Built: Fri May 30 12:07:26 2025 OS/Arch: linux/amd64 Context: default Server: Docker Engine - Community Engine: Version: 28.2.2 API version: 1.50 (minimum version 1.24) Go version: go1.24.3 Git commit: 45873be Built: Fri May 30 12:07:26 2025 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.7.27 GitCommit: 05044ec0a9a75232cad458027ca83437aae3f4da runc: Version: 1.2.5 GitCommit: v1.2.5-0-g59923ef docker-init: Version: 0.19.0 GitCommit: de40ad0docker info
Additional Info
No response