Add chain DOCKER-FORWARD by robmry · Pull Request #49518 · moby/moby

robmry · 2025-02-21T12:31:59Z

- What I did

In 28.0.0, Docker appended to the FORWARD chain - breaking other applications that had appended their own rules that needed to execute after Docker's rules.

fix 28.0.0 broke networking for Discourse #49511
fix Docker version 28.0.0, build af898ab configures host network with no network access: docker/for-linux#1521
Docker 28 stops containers communicating with tailscale network. #49498
- fixes the issues with Tailscale and Oracle Cloud
- likely fixes issues with swag, but tbc.
fix 28.0.0 containers can't reach external hosts when UFW DEFAULT_FORWARD_POLICY="REJECT" is set #49528

- How I did it

Move most of Docker's rules out of the filter-FORWARD chain into a new DOCKER-FORWARD chain, so that Docker can append to DOCKER-FORWARD without affecting the order of rules in the FORWARD chain.

After daemon startup inserts jumps to DOCKER-USER and DOCKER-FORWARD, the bridge driver will not touch the FORWARD chain again. DOCKER-INGRESS is still added to the FORWARD chain, if used, as it was in 27.x and earlier.

(Links to code in the generated iptables docs are all out of date ... I'll update them once this is merged and there's something to link to - haven't thought of a better way to deal with that, but I think the links are probably worth keeping.)

- How to verify it

Existing and updated tests.

New test TestDropInForwardChain - reliably fails without this change.

To check removal of rules from the FORWARD chain on upgrade from 27.5.1 ...

flushed iptables/ip6tables
started 27.5.1
joined swarm, created some networks, started some containers ...
stashed the output of iptables-save, ip6tables-save
stopped dockerd, started docker built from this PR and restarted the containers
stashed the iptables rules again
stopped the dev-docker, flushed iptables/ip6tables, started everything back up
stashed the iptables rules again, and compared them with the post-upgrade set
- the only difference between the rules for upgrade/fresh start was due to creation order of networks (per-network rules in the DOCKER-FORWARD that can safely be swapped)

Upgrade test detail ...

Networks and containers ...

docker network create --ipv6 b46
docker network create --internal --ipv6 internalnw
docker network create -o com.docker.network.bridge.enable_icc=false --ipv6 noicc

docker run --rm -d --network b46 -p 10000:1000 alpine sleep infinity
docker run --rm -d --network noicc -p 10001:1001 alpine sleep infinity
docker run --rm -d --network internalnw alpine sleep infinity
docker service create alpine sleep infinity

27.5.1 iptables

# Generated by iptables-save v1.8.9 (nf_tables) on Fri Feb 21 15:12:36 2025
*filter
:INPUT ACCEPT [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
:DOCKER - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -i br-233f92e88674 -o br-233f92e88674 -j ACCEPT
-A FORWARD -o br-f2cfb6c855a7 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o br-f2cfb6c855a7 -j DOCKER
-A FORWARD -i br-f2cfb6c855a7 ! -o br-f2cfb6c855a7 -j ACCEPT
-A FORWARD -o docker_gwbridge -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker_gwbridge -j DOCKER
-A FORWARD -i docker_gwbridge ! -o docker_gwbridge -j ACCEPT
-A FORWARD -o br-3cdb1cb92a36 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o br-3cdb1cb92a36 -j DOCKER
-A FORWARD -i br-3cdb1cb92a36 ! -o br-3cdb1cb92a36 -j ACCEPT
-A FORWARD -i br-3cdb1cb92a36 -o br-3cdb1cb92a36 -j ACCEPT
-A FORWARD -i docker_gwbridge -o docker_gwbridge -j DROP
-A FORWARD -i br-f2cfb6c855a7 -o br-f2cfb6c855a7 -j DROP
-A DOCKER -d 172.18.0.2/32 ! -i br-3cdb1cb92a36 -o br-3cdb1cb92a36 -p tcp -m tcp --dport 1000 -j ACCEPT
-A DOCKER -d 172.20.0.2/32 ! -i br-f2cfb6c855a7 -o br-f2cfb6c855a7 -p tcp -m tcp --dport 1001 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 ! -s 172.19.0.0/16 -o br-233f92e88674 -j DROP
-A DOCKER-ISOLATION-STAGE-1 ! -d 172.19.0.0/16 -i br-233f92e88674 -j DROP
-A DOCKER-ISOLATION-STAGE-1 -i br-f2cfb6c855a7 ! -o br-f2cfb6c855a7 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i docker_gwbridge ! -o docker_gwbridge -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i br-3cdb1cb92a36 ! -o br-3cdb1cb92a36 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o br-f2cfb6c855a7 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o docker_gwbridge -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o br-3cdb1cb92a36 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
COMMIT
# Completed on Fri Feb 21 15:12:36 2025
# Generated by iptables-save v1.8.9 (nf_tables) on Fri Feb 21 15:12:36 2025
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -s 172.20.0.0/16 ! -o br-f2cfb6c855a7 -j MASQUERADE
-A POSTROUTING -s 172.21.0.0/16 ! -o docker_gwbridge -j MASQUERADE
-A POSTROUTING -s 172.18.0.0/16 ! -o br-3cdb1cb92a36 -j MASQUERADE
-A POSTROUTING -s 172.18.0.2/32 -d 172.18.0.2/32 -p tcp -m tcp --dport 1000 -j MASQUERADE
-A POSTROUTING -s 172.20.0.2/32 -d 172.20.0.2/32 -p tcp -m tcp --dport 1001 -j MASQUERADE
-A DOCKER -i docker0 -j RETURN
-A DOCKER -i br-f2cfb6c855a7 -j RETURN
-A DOCKER -i docker_gwbridge -j RETURN
-A DOCKER -i br-3cdb1cb92a36 -j RETURN
-A DOCKER ! -i br-3cdb1cb92a36 -p tcp -m tcp --dport 10000 -j DNAT --to-destination 172.18.0.2:1000
-A DOCKER ! -i br-f2cfb6c855a7 -p tcp -m tcp --dport 10001 -j DNAT --to-destination 172.20.0.2:1001
COMMIT
# Completed on Fri Feb 21 15:12:36 2025

27.5.1 - ip6tables

# Generated by ip6tables-save v1.8.9 (nf_tables) on Fri Feb 21 15:12:29 2025
*filter
:INPUT ACCEPT [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
:DOCKER - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -i br-233f92e88674 -o br-233f92e88674 -j ACCEPT
-A FORWARD -o br-f2cfb6c855a7 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o br-f2cfb6c855a7 -j DOCKER
-A FORWARD -i br-f2cfb6c855a7 ! -o br-f2cfb6c855a7 -j ACCEPT
-A FORWARD -o br-3cdb1cb92a36 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o br-3cdb1cb92a36 -j DOCKER
-A FORWARD -i br-3cdb1cb92a36 ! -o br-3cdb1cb92a36 -j ACCEPT
-A FORWARD -i br-3cdb1cb92a36 -o br-3cdb1cb92a36 -j ACCEPT
-A FORWARD -i br-f2cfb6c855a7 -o br-f2cfb6c855a7 -j DROP
-A DOCKER -d fd5b:823e:5b1a::2/128 ! -i br-3cdb1cb92a36 -o br-3cdb1cb92a36 -p tcp -m tcp --dport 1000 -j ACCEPT
-A DOCKER -d fd5b:823e:5b1a:2::2/128 ! -i br-f2cfb6c855a7 -o br-f2cfb6c855a7 -p tcp -m tcp --dport 1001 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 ! -s fd5b:823e:5b1a:1::/64 ! -i br-233f92e88674 -o br-233f92e88674 -j DROP
-A DOCKER-ISOLATION-STAGE-1 ! -d fd5b:823e:5b1a:1::/64 -i br-233f92e88674 ! -o br-233f92e88674 -j DROP
-A DOCKER-ISOLATION-STAGE-1 -i br-f2cfb6c855a7 ! -o br-f2cfb6c855a7 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i docker_gwbridge ! -o docker_gwbridge -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i br-3cdb1cb92a36 ! -o br-3cdb1cb92a36 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o br-f2cfb6c855a7 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o docker_gwbridge -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o br-3cdb1cb92a36 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
COMMIT
# Completed on Fri Feb 21 15:12:29 2025
# Generated by ip6tables-save v1.8.9 (nf_tables) on Fri Feb 21 15:12:29 2025
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d ::1/128 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s fd5b:823e:5b1a:2::/64 ! -o br-f2cfb6c855a7 -j MASQUERADE
-A POSTROUTING -s fd5b:823e:5b1a::/64 ! -o br-3cdb1cb92a36 -j MASQUERADE
-A POSTROUTING -s fd5b:823e:5b1a::2/128 -d fd5b:823e:5b1a::2/128 -p tcp -m tcp --dport 1000 -j MASQUERADE
-A POSTROUTING -s fd5b:823e:5b1a:2::2/128 -d fd5b:823e:5b1a:2::2/128 -p tcp -m tcp --dport 1001 -j MASQUERADE
-A DOCKER -i br-f2cfb6c855a7 -j RETURN
-A DOCKER -i br-3cdb1cb92a36 -j RETURN
-A DOCKER ! -i br-3cdb1cb92a36 -p tcp -m tcp --dport 10000 -j DNAT --to-destination [fd5b:823e:5b1a::2]:1000
-A DOCKER ! -i br-f2cfb6c855a7 -p tcp -m tcp --dport 10001 -j DNAT --to-destination [fd5b:823e:5b1a:2::2]:1001
COMMIT
# Completed on Fri Feb 21 15:12:29 2025

Dev build after upgrade from 27.5.1 - iptables

# Generated by iptables-save v1.8.9 (nf_tables) on Fri Feb 21 15:14:15 2025
*raw
:PREROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A PREROUTING -d 172.18.0.2/32 ! -i br-3cdb1cb92a36 -p tcp -m tcp --dport 1000 -j DROP
-A PREROUTING -d 172.20.0.2/32 ! -i br-f2cfb6c855a7 -p tcp -m tcp --dport 1001 -j DROP
COMMIT
# Completed on Fri Feb 21 15:14:15 2025
# Generated by iptables-save v1.8.9 (nf_tables) on Fri Feb 21 15:14:15 2025
*filter
:INPUT ACCEPT [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
:DOCKER - [0:0]
:DOCKER-FORWARD - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-FORWARD
-A DOCKER -d 172.20.0.2/32 ! -i br-f2cfb6c855a7 -o br-f2cfb6c855a7 -p tcp -m tcp --dport 1001 -j ACCEPT
-A DOCKER -d 172.18.0.2/32 ! -i br-3cdb1cb92a36 -o br-3cdb1cb92a36 -p tcp -m tcp --dport 1000 -j ACCEPT
-A DOCKER ! -i br-3cdb1cb92a36 -o br-3cdb1cb92a36 -j DROP
-A DOCKER ! -i docker_gwbridge -o docker_gwbridge -j DROP
-A DOCKER ! -i br-f2cfb6c855a7 -o br-f2cfb6c855a7 -j DROP
-A DOCKER ! -i docker0 -o docker0 -j DROP
-A DOCKER-FORWARD -m set --match-set docker-ext-bridges-v4 dst -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-FORWARD -j DOCKER-ISOLATION-STAGE-1
-A DOCKER-FORWARD -m set --match-set docker-ext-bridges-v4 dst -j DOCKER
-A DOCKER-FORWARD -i br-233f92e88674 -o br-233f92e88674 -j ACCEPT
-A DOCKER-FORWARD -i br-3cdb1cb92a36 -j ACCEPT
-A DOCKER-FORWARD -i docker_gwbridge -o docker_gwbridge -j DROP
-A DOCKER-FORWARD -i docker_gwbridge ! -o docker_gwbridge -j ACCEPT
-A DOCKER-FORWARD -i br-f2cfb6c855a7 -o br-f2cfb6c855a7 -j DROP
-A DOCKER-FORWARD -i br-f2cfb6c855a7 ! -o br-f2cfb6c855a7 -j ACCEPT
-A DOCKER-FORWARD -i docker0 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 ! -s 172.19.0.0/16 -o br-233f92e88674 -j DROP
-A DOCKER-ISOLATION-STAGE-1 ! -d 172.19.0.0/16 -i br-233f92e88674 -j DROP
-A DOCKER-ISOLATION-STAGE-1 -i br-3cdb1cb92a36 ! -o br-3cdb1cb92a36 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i docker_gwbridge ! -o docker_gwbridge -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i br-f2cfb6c855a7 ! -o br-f2cfb6c855a7 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o br-f2cfb6c855a7 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o docker_gwbridge -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o br-3cdb1cb92a36 -j DROP
-A DOCKER-USER -j RETURN
COMMIT
# Completed on Fri Feb 21 15:14:15 2025
# Generated by iptables-save v1.8.9 (nf_tables) on Fri Feb 21 15:14:15 2025
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -s 172.20.0.0/16 ! -o br-f2cfb6c855a7 -j MASQUERADE
-A POSTROUTING -s 172.21.0.0/16 ! -o docker_gwbridge -j MASQUERADE
-A POSTROUTING -s 172.18.0.0/16 ! -o br-3cdb1cb92a36 -j MASQUERADE
-A DOCKER -i docker0 -j RETURN
-A DOCKER -i br-f2cfb6c855a7 -j RETURN
-A DOCKER -i docker_gwbridge -j RETURN
-A DOCKER -i br-3cdb1cb92a36 -j RETURN
-A DOCKER ! -i br-3cdb1cb92a36 -p tcp -m tcp --dport 10000 -j DNAT --to-destination 172.18.0.2:1000
-A DOCKER ! -i br-f2cfb6c855a7 -p tcp -m tcp --dport 10001 -j DNAT --to-destination 172.20.0.2:1001
COMMIT
# Completed on Fri Feb 21 15:14:15 2025

Dev build after upgrade from 27.5.1- ip6tables

# Generated by ip6tables-save v1.8.9 (nf_tables) on Fri Feb 21 15:14:25 2025
*raw
:PREROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A PREROUTING -d fd5b:823e:5b1a::2/128 ! -i br-3cdb1cb92a36 -p tcp -m tcp --dport 1000 -j DROP
-A PREROUTING -d fd5b:823e:5b1a:2::2/128 ! -i br-f2cfb6c855a7 -p tcp -m tcp --dport 1001 -j DROP
COMMIT
# Completed on Fri Feb 21 15:14:25 2025
# Generated by ip6tables-save v1.8.9 (nf_tables) on Fri Feb 21 15:14:25 2025
*filter
:INPUT ACCEPT [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
:DOCKER - [0:0]
:DOCKER-FORWARD - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-FORWARD
-A DOCKER -d fd5b:823e:5b1a:2::2/128 ! -i br-f2cfb6c855a7 -o br-f2cfb6c855a7 -p tcp -m tcp --dport 1001 -j ACCEPT
-A DOCKER -d fd5b:823e:5b1a::2/128 ! -i br-3cdb1cb92a36 -o br-3cdb1cb92a36 -p tcp -m tcp --dport 1000 -j ACCEPT
-A DOCKER ! -i br-3cdb1cb92a36 -o br-3cdb1cb92a36 -j DROP
-A DOCKER ! -i br-f2cfb6c855a7 -o br-f2cfb6c855a7 -j DROP
-A DOCKER-FORWARD -m set --match-set docker-ext-bridges-v6 dst -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-FORWARD -j DOCKER-ISOLATION-STAGE-1
-A DOCKER-FORWARD -m set --match-set docker-ext-bridges-v6 dst -j DOCKER
-A DOCKER-FORWARD -i br-233f92e88674 -o br-233f92e88674 -j ACCEPT
-A DOCKER-FORWARD -i br-3cdb1cb92a36 -j ACCEPT
-A DOCKER-FORWARD -i br-f2cfb6c855a7 -o br-f2cfb6c855a7 -j DROP
-A DOCKER-FORWARD -i br-f2cfb6c855a7 ! -o br-f2cfb6c855a7 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 ! -s fd5b:823e:5b1a:1::/64 ! -i br-233f92e88674 -o br-233f92e88674 -j DROP
-A DOCKER-ISOLATION-STAGE-1 ! -d fd5b:823e:5b1a:1::/64 -i br-233f92e88674 ! -o br-233f92e88674 -j DROP
-A DOCKER-ISOLATION-STAGE-1 -i br-3cdb1cb92a36 ! -o br-3cdb1cb92a36 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i docker_gwbridge ! -o docker_gwbridge -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i br-f2cfb6c855a7 ! -o br-f2cfb6c855a7 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o br-f2cfb6c855a7 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o docker_gwbridge -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o br-3cdb1cb92a36 -j DROP
-A DOCKER-USER -j RETURN
COMMIT
# Completed on Fri Feb 21 15:14:25 2025
# Generated by ip6tables-save v1.8.9 (nf_tables) on Fri Feb 21 15:14:25 2025
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d ::1/128 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s fd5b:823e:5b1a:2::/64 ! -o br-f2cfb6c855a7 -j MASQUERADE
-A POSTROUTING -s fd5b:823e:5b1a::/64 ! -o br-3cdb1cb92a36 -j MASQUERADE
-A DOCKER -i br-f2cfb6c855a7 -j RETURN
-A DOCKER -i br-3cdb1cb92a36 -j RETURN
-A DOCKER ! -s fe80::/10 ! -i br-3cdb1cb92a36 -p tcp -m tcp --dport 10000 -j DNAT --to-destination [fd5b:823e:5b1a::2]:1000
-A DOCKER ! -s fe80::/10 ! -i br-f2cfb6c855a7 -p tcp -m tcp --dport 10001 -j DNAT --to-destination [fd5b:823e:5b1a:2::2]:1001
COMMIT
# Completed on Fri Feb 21 15:14:25 2025

Dev build started after iptables flush - iptables

# Generated by iptables-save v1.8.9 (nf_tables) on Fri Feb 21 15:18:20 2025
*raw
:PREROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A PREROUTING -d 172.18.0.2/32 ! -i br-3cdb1cb92a36 -p tcp -m tcp --dport 1000 -j DROP
-A PREROUTING -d 172.20.0.2/32 ! -i br-f2cfb6c855a7 -p tcp -m tcp --dport 1001 -j DROP
COMMIT
# Completed on Fri Feb 21 15:18:20 2025
# Generated by iptables-save v1.8.9 (nf_tables) on Fri Feb 21 15:18:20 2025
*filter
:INPUT ACCEPT [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
:DOCKER - [0:0]
:DOCKER-FORWARD - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-FORWARD
-A DOCKER -d 172.20.0.2/32 ! -i br-f2cfb6c855a7 -o br-f2cfb6c855a7 -p tcp -m tcp --dport 1001 -j ACCEPT
-A DOCKER -d 172.18.0.2/32 ! -i br-3cdb1cb92a36 -o br-3cdb1cb92a36 -p tcp -m tcp --dport 1000 -j ACCEPT
-A DOCKER ! -i br-3cdb1cb92a36 -o br-3cdb1cb92a36 -j DROP
-A DOCKER ! -i docker_gwbridge -o docker_gwbridge -j DROP
-A DOCKER ! -i br-f2cfb6c855a7 -o br-f2cfb6c855a7 -j DROP
-A DOCKER ! -i docker0 -o docker0 -j DROP
-A DOCKER-FORWARD -m set --match-set docker-ext-bridges-v4 dst -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-FORWARD -j DOCKER-ISOLATION-STAGE-1
-A DOCKER-FORWARD -m set --match-set docker-ext-bridges-v4 dst -j DOCKER
-A DOCKER-FORWARD -i br-3cdb1cb92a36 -j ACCEPT
-A DOCKER-FORWARD -i docker_gwbridge -o docker_gwbridge -j DROP
-A DOCKER-FORWARD -i docker_gwbridge ! -o docker_gwbridge -j ACCEPT
-A DOCKER-FORWARD -i br-f2cfb6c855a7 -o br-f2cfb6c855a7 -j DROP
-A DOCKER-FORWARD -i br-f2cfb6c855a7 ! -o br-f2cfb6c855a7 -j ACCEPT
-A DOCKER-FORWARD -i br-233f92e88674 -o br-233f92e88674 -j ACCEPT
-A DOCKER-FORWARD -i docker0 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 ! -s 172.19.0.0/16 -o br-233f92e88674 -j DROP
-A DOCKER-ISOLATION-STAGE-1 ! -d 172.19.0.0/16 -i br-233f92e88674 -j DROP
-A DOCKER-ISOLATION-STAGE-1 -i br-3cdb1cb92a36 ! -o br-3cdb1cb92a36 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i docker_gwbridge ! -o docker_gwbridge -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i br-f2cfb6c855a7 ! -o br-f2cfb6c855a7 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o br-f2cfb6c855a7 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o docker_gwbridge -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o br-3cdb1cb92a36 -j DROP
-A DOCKER-USER -j RETURN
COMMIT
# Completed on Fri Feb 21 15:18:20 2025
# Generated by iptables-save v1.8.9 (nf_tables) on Fri Feb 21 15:18:20 2025
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -s 172.20.0.0/16 ! -o br-f2cfb6c855a7 -j MASQUERADE
-A POSTROUTING -s 172.21.0.0/16 ! -o docker_gwbridge -j MASQUERADE
-A POSTROUTING -s 172.18.0.0/16 ! -o br-3cdb1cb92a36 -j MASQUERADE
-A DOCKER -i docker0 -j RETURN
-A DOCKER -i br-f2cfb6c855a7 -j RETURN
-A DOCKER -i docker_gwbridge -j RETURN
-A DOCKER -i br-3cdb1cb92a36 -j RETURN
-A DOCKER ! -i br-3cdb1cb92a36 -p tcp -m tcp --dport 10000 -j DNAT --to-destination 172.18.0.2:1000
-A DOCKER ! -i br-f2cfb6c855a7 -p tcp -m tcp --dport 10001 -j DNAT --to-destination 172.20.0.2:1001
COMMIT
# Completed on Fri Feb 21 15:18:20 2025

Dev build started after ip6tables flush - ip6tables

# Generated by ip6tables-save v1.8.9 (nf_tables) on Fri Feb 21 15:18:13 2025
*raw
:PREROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A PREROUTING -d fd5b:823e:5b1a::2/128 ! -i br-3cdb1cb92a36 -p tcp -m tcp --dport 1000 -j DROP
-A PREROUTING -d fd5b:823e:5b1a:2::2/128 ! -i br-f2cfb6c855a7 -p tcp -m tcp --dport 1001 -j DROP
COMMIT
# Completed on Fri Feb 21 15:18:13 2025
# Generated by ip6tables-save v1.8.9 (nf_tables) on Fri Feb 21 15:18:13 2025
*filter
:INPUT ACCEPT [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
:DOCKER - [0:0]
:DOCKER-FORWARD - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-FORWARD
-A DOCKER -d fd5b:823e:5b1a:2::2/128 ! -i br-f2cfb6c855a7 -o br-f2cfb6c855a7 -p tcp -m tcp --dport 1001 -j ACCEPT
-A DOCKER -d fd5b:823e:5b1a::2/128 ! -i br-3cdb1cb92a36 -o br-3cdb1cb92a36 -p tcp -m tcp --dport 1000 -j ACCEPT
-A DOCKER ! -i br-3cdb1cb92a36 -o br-3cdb1cb92a36 -j DROP
-A DOCKER ! -i br-f2cfb6c855a7 -o br-f2cfb6c855a7 -j DROP
-A DOCKER-FORWARD -m set --match-set docker-ext-bridges-v6 dst -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-FORWARD -j DOCKER-ISOLATION-STAGE-1
-A DOCKER-FORWARD -m set --match-set docker-ext-bridges-v6 dst -j DOCKER
-A DOCKER-FORWARD -i br-3cdb1cb92a36 -j ACCEPT
-A DOCKER-FORWARD -i br-f2cfb6c855a7 -o br-f2cfb6c855a7 -j DROP
-A DOCKER-FORWARD -i br-f2cfb6c855a7 ! -o br-f2cfb6c855a7 -j ACCEPT
-A DOCKER-FORWARD -i br-233f92e88674 -o br-233f92e88674 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 ! -s fd5b:823e:5b1a:1::/64 ! -i br-233f92e88674 -o br-233f92e88674 -j DROP
-A DOCKER-ISOLATION-STAGE-1 ! -d fd5b:823e:5b1a:1::/64 -i br-233f92e88674 ! -o br-233f92e88674 -j DROP
-A DOCKER-ISOLATION-STAGE-1 -i br-3cdb1cb92a36 ! -o br-3cdb1cb92a36 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i docker_gwbridge ! -o docker_gwbridge -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i br-f2cfb6c855a7 ! -o br-f2cfb6c855a7 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o br-f2cfb6c855a7 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o docker_gwbridge -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o br-3cdb1cb92a36 -j DROP
-A DOCKER-USER -j RETURN
COMMIT
# Completed on Fri Feb 21 15:18:13 2025
# Generated by ip6tables-save v1.8.9 (nf_tables) on Fri Feb 21 15:18:13 2025
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d ::1/128 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s fd5b:823e:5b1a:2::/64 ! -o br-f2cfb6c855a7 -j MASQUERADE
-A POSTROUTING -s fd5b:823e:5b1a::/64 ! -o br-3cdb1cb92a36 -j MASQUERADE
-A DOCKER -i br-f2cfb6c855a7 -j RETURN
-A DOCKER -i br-3cdb1cb92a36 -j RETURN
-A DOCKER ! -s fe80::/10 ! -i br-3cdb1cb92a36 -p tcp -m tcp --dport 10000 -j DNAT --to-destination [fd5b:823e:5b1a::2]:1000
-A DOCKER ! -s fe80::/10 ! -i br-f2cfb6c855a7 -p tcp -m tcp --dport 10001 -j DNAT --to-destination [fd5b:823e:5b1a:2::2]:1001
COMMIT

- Human readable description for the release notes

Move most of Docker's iptables rules out of the filter-FORWARD chain, so that other applications are free to append rules that must follow Docker's rules.

akerouanton

I took a look and it seems okay but I'd like to take another look and play with it a bit on Monday before approving.

BJReplay · 2025-02-21T21:25:22Z

Will this fix fix the problem described in docker/for-linux#1521? This does not appear to be directly linked to Tailscale, though I note in the comments to the Tailscale issue that one user notes OCI as a contributing factor and 1521 was discovered and repro'd on OCI.

robmry · 2025-02-21T22:04:07Z

Will this fix fix the problem described in docker/for-linux#1521?

Yes, it looks the same.

robmry · 2025-02-22T13:00:16Z

Added a regression test.

In 28.0.0, Docker appended to the FORWARD chain - breaking other applications that had appended their own rules that needed to execute after Docker's rules. Move most of Docker's rules out of the filter-FORWARD chain into a new DOCKER-FORWARD chain, so that Docker can append to DOCKER-FORWARD without affecting the order of rules in the FORWARD chain. After daemon startup inserts jumps to DOCKER-USER and DOCKER-FORWARD, the bridge driver will not touch the FORWARD chain again. DOCKER-INGRESS is still added to the FORWARD chain, if used, as it was in 27.x and earlier. Signed-off-by: Rob Murray <[email protected]>

Signed-off-by: Rob Murray <[email protected]>

robmry · 2025-02-24T17:26:26Z

Rebased to resolve conflict.

akerouanton

LGTM

@tag

## Description - Describe new iptables chain `DOCKER-FORWARD`, which splits Docker's rules out of the main `FORWARD` chain where they were being antisocial - related to moby/moby#49518 - Update notes about IP Forwarding and the default DROP policy - Only engine < 28.0.0 allows remote access to ports published to the localhost address ## Related issues or tickets ## Reviews   - [ ] Technical review - [ ] Editorial review - [ ] Product review --------- Signed-off-by: Rob Murray <[email protected]>

robmry added area/networking Networking impact/changelog kind/bugfix PR's that fix bugs area/networking/firewalling Networking area/networking/d/bridge Networking version/28.0 labels Feb 21, 2025

robmry added this to the 28.0.1 milestone Feb 21, 2025

robmry self-assigned this Feb 21, 2025

robmry force-pushed the docker_forward_chain branch from 9a041f2 to c07435c Compare February 21, 2025 14:42

valentinBoch approved these changes Feb 21, 2025

View reviewed changes

robmry force-pushed the docker_forward_chain branch from c07435c to 6da51b8 Compare February 21, 2025 15:33

robmry marked this pull request as ready for review February 21, 2025 16:32

robmry requested review from akerouanton, thaJeztah and vvoland February 21, 2025 16:32

thaJeztah reviewed Feb 21, 2025

View reviewed changes

Comment thread integration/network/bridge/iptablesdoc/generated/new-daemon.md

akerouanton reviewed Feb 21, 2025

View reviewed changes

thaJeztah mentioned this pull request Feb 21, 2025

Docker version 28.0.0, build af898ab configures host network with no network access: docker/for-linux#1521

Closed

irbekrm mentioned this pull request Feb 21, 2025

Docker 28 stops containers communicating with tail scale network. tailscale/tailscale#15072

Closed

This was referenced Feb 23, 2025

Rootless slirp4netns network broken in v28 docker/cli#5860

Closed

Updates for moby 28.0.1 docker/docs#22086

Merged

28.0.0 containers can't reach external hosts when UFW DEFAULT_FORWARD_POLICY="REJECT" is set #49528

Closed

robmry added 2 commits February 24, 2025 17:09

Test with a drop rule appended to filter-FORWARD

341cafa

Signed-off-by: Rob Murray <[email protected]>

robmry mentioned this pull request Feb 24, 2025

Drop the use of ipset #49530

Merged

robmry force-pushed the docker_forward_chain branch from d04ff16 to 341cafa Compare February 24, 2025 17:26

robmry mentioned this pull request Feb 25, 2025

Random networking failures in bridged network #49537

Open

vvoland approved these changes Feb 25, 2025

View reviewed changes

vvoland requested a review from akerouanton February 25, 2025 10:37

akerouanton approved these changes Feb 25, 2025

View reviewed changes

robmry merged commit 0274c63 into moby:master Feb 25, 2025

thaJeztah mentioned this pull request Feb 25, 2025

28.0.0 broke networking for Discourse #49511

Closed

vvoland added process/cherry-pick process/cherry-pick/28.x process/cherry-picked and removed process/cherry-pick process/cherry-pick/28.x labels Feb 26, 2025

robmry mentioned this pull request Feb 26, 2025

Docker 28 stops containers communicating with tailscale network. #49498

Closed

robmry deleted the docker_forward_chain branch February 26, 2025 14:46

jtgasper3 mentioned this pull request Mar 9, 2025

Bump github.com/docker/docker from 27.5.1+incompatible to 28.0.1+incompatible jtgasper3/swarm-visualizer#9

Merged

anoadragon453 mentioned this pull request May 7, 2025

Docker v28 update causes failed to listen on TCP socket: address already in use matrix-org/complement#775

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add chain DOCKER-FORWARD#49518

Add chain DOCKER-FORWARD#49518
robmry merged 2 commits into
moby:masterfrom
robmry:docker_forward_chain

robmry commented Feb 21, 2025 •

edited by thaJeztah

Loading

Uh oh!

Uh oh!

akerouanton left a comment

Uh oh!

BJReplay commented Feb 21, 2025

Uh oh!

robmry commented Feb 21, 2025

Uh oh!

robmry commented Feb 22, 2025

Uh oh!

robmry commented Feb 24, 2025

Uh oh!

akerouanton left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

robmry commented Feb 21, 2025 • edited by thaJeztah Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Networks and containers ...

Uh oh!

Uh oh!

akerouanton left a comment

Choose a reason for hiding this comment

Uh oh!

BJReplay commented Feb 21, 2025

Uh oh!

robmry commented Feb 21, 2025

Uh oh!

robmry commented Feb 22, 2025

Uh oh!

robmry commented Feb 24, 2025

Uh oh!

akerouanton left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

robmry commented Feb 21, 2025 •

edited by thaJeztah

Loading