Skip to content

Can not set MTU above 1500 for docker default bridge device #47308

Closed
Closed
Can not set MTU above 1500 for docker default bridge device#47308
Labels
area/networkingNetworkingarea/networking/d/bridgeNetworkingkind/bugBugs are bugs. The cause may or may not be known at triage time so debugging may be needed.status/confirmedversion/25.0
Milestone
25.0.3
@ivasilyev-mxr

Description

@ivasilyev-mxr
ivasilyev-mxr
opened on Feb 2, 2024

Description

After upgrade to version 25, attempting to set the mtu value above 1500 in /etc/docker/daemon.json results in this error being thrown: https://github.com/moby/moby/pull/46849/files#diff-77245372071f1b23e0b41e6ac4bd8212a33512dce6159146fff873d3da252c75R50

Even attempting to delete the device and any network information in docker data dir results in same error.

Setting a value less than 1500 works as expected and as described in #46849 (the device has the correct value even if no containers are attached).

➜  ~ sudo ip link delete docker0
➜  ~ sudo rm -rf /var/lib/docker/network
➜  ~ sudo ip link show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1600 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether fa:16:3e:63:50:c6 brd ff:ff:ff:ff:ff:ff
    

➜  ~ sudo dockerd --mtu 1501 -D
INFO[2024-02-02T12:41:37.575765119Z] Starting up                                  
DEBU[2024-02-02T12:41:37.577609096Z] Listener created for HTTP on unix (/var/run/docker.sock) 
DEBU[2024-02-02T12:41:37.756685501Z] Golang's threads limit set to 56070          
DEBU[2024-02-02T12:41:37.757971951Z] metrics API listening on /var/run/docker/metrics.sock 
DEBU[2024-02-02T12:41:37.793784202Z] Using default logging driver json-file       
DEBU[2024-02-02T12:41:37.794156567Z] processing event stream                       module=libcontainerd namespace=plugins.moby
DEBU[2024-02-02T12:41:37.794288490Z] No quota support for local volumes in /var/lib/docker/volumes: Filesystem does not support, or has not enabled quotas 
DEBU[2024-02-02T12:41:37.837618598Z] [graphdriver] priority list: [overlay2 fuse-overlayfs btrfs zfs vfs] 
DEBU[2024-02-02T12:41:37.862174483Z] successfully detected metacopy status         storage-driver=overlay2 usingMetacopy=false
DEBU[2024-02-02T12:41:37.879104888Z] backingFs=xfs, projectQuotaSupported=false, usingMetacopy=false, indexOff="index=off,", userxattr=""  storage-driver=overlay2
INFO[2024-02-02T12:41:37.879190019Z] [graphdriver] using prior storage driver: overlay2 
DEBU[2024-02-02T12:41:37.879210700Z] Initialized graph driver overlay2            
DEBU[2024-02-02T12:41:39.783188765Z] Max Concurrent Downloads: 3                  
DEBU[2024-02-02T12:41:39.783241726Z] Max Concurrent Uploads: 5                    
DEBU[2024-02-02T12:41:39.783256036Z] Max Download Attempts: 5                     
INFO[2024-02-02T12:41:39.783298216Z] Loading containers: start.                   
DEBU[2024-02-02T12:41:39.783408948Z] Option DefaultDriver: bridge                 
DEBU[2024-02-02T12:41:39.783422698Z] Option DefaultNetwork: bridge                
DEBU[2024-02-02T12:41:39.783432199Z] Network Control Plane MTU: 1500              
DEBU[2024-02-02T12:41:39.783677922Z] processing event stream                       module=libcontainerd namespace=moby
INFO[2024-02-02T12:41:39.803737077Z] Firewalld: docker zone already exists, returning 
DEBU[2024-02-02T12:41:39.836776204Z] Firewalld passthrough: ipv4, [-t filter -C FORWARD -j DOCKER-ISOLATION] 
DEBU[2024-02-02T12:41:39.849317590Z] Firewalld passthrough: ipv4, [-t nat -D PREROUTING -m addrtype --dst-type LOCAL -j DOCKER] 
DEBU[2024-02-02T12:41:39.862420576Z] Firewalld passthrough: ipv4, [-t nat -D OUTPUT -m addrtype --dst-type LOCAL ! --dst 127.0.0.0/8 -j DOCKER] 
DEBU[2024-02-02T12:41:39.872397902Z] Firewalld passthrough: ipv4, [-t nat -D OUTPUT -m addrtype --dst-type LOCAL -j DOCKER] 
DEBU[2024-02-02T12:41:39.883982353Z] Firewalld passthrough: ipv4, [-t nat -D PREROUTING] 
DEBU[2024-02-02T12:41:39.894377456Z] Firewalld passthrough: ipv4, [-t nat -D OUTPUT] 
DEBU[2024-02-02T12:41:39.904517865Z] Firewalld passthrough: ipv4, [-t nat -F DOCKER] 
DEBU[2024-02-02T12:41:39.914930898Z] Firewalld passthrough: ipv4, [-t nat -X DOCKER] 
DEBU[2024-02-02T12:41:39.928427809Z] Firewalld passthrough: ipv4, [-t filter -F DOCKER] 
DEBU[2024-02-02T12:41:39.940257565Z] Firewalld passthrough: ipv4, [-t filter -X DOCKER] 
DEBU[2024-02-02T12:41:39.951720524Z] Firewalld passthrough: ipv4, [-t filter -F DOCKER-ISOLATION-STAGE-1] 
DEBU[2024-02-02T12:41:39.961257423Z] Firewalld passthrough: ipv4, [-t filter -X DOCKER-ISOLATION-STAGE-1] 
DEBU[2024-02-02T12:41:39.970111752Z] Firewalld passthrough: ipv4, [-t filter -F DOCKER-ISOLATION-STAGE-2] 
DEBU[2024-02-02T12:41:39.979666842Z] Firewalld passthrough: ipv4, [-t filter -X DOCKER-ISOLATION-STAGE-2] 
DEBU[2024-02-02T12:41:39.990484350Z] Firewalld passthrough: ipv4, [-t filter -F DOCKER-ISOLATION] 
DEBU[2024-02-02T12:41:40.000524318Z] Firewalld passthrough: ipv4, [-t filter -X DOCKER-ISOLATION] 
DEBU[2024-02-02T12:41:40.009488559Z] Firewalld passthrough: ipv4, [-t nat -n -L DOCKER] 
DEBU[2024-02-02T12:41:40.017724368Z] Firewalld passthrough: ipv4, [-t nat -N DOCKER] 
DEBU[2024-02-02T12:41:40.026659756Z] Firewalld passthrough: ipv4, [-t filter -n -L DOCKER] 
DEBU[2024-02-02T12:41:40.036242738Z] Firewalld passthrough: ipv4, [-t filter -n -L DOCKER-ISOLATION-STAGE-1] 
DEBU[2024-02-02T12:41:40.045497731Z] Firewalld passthrough: ipv4, [-t filter -n -L DOCKER-ISOLATION-STAGE-2] 
DEBU[2024-02-02T12:41:40.054704017Z] Firewalld passthrough: ipv4, [-t filter -N DOCKER-ISOLATION-STAGE-2] 
DEBU[2024-02-02T12:41:40.062958146Z] Firewalld passthrough: ipv4, [-t filter -C DOCKER-ISOLATION-STAGE-1 -j RETURN] 
DEBU[2024-02-02T12:41:40.070953191Z] Firewalld passthrough: ipv4, [-A DOCKER-ISOLATION-STAGE-1 -j RETURN] 
DEBU[2024-02-02T12:41:40.078731612Z] Firewalld passthrough: ipv4, [-t filter -C DOCKER-ISOLATION-STAGE-2 -j RETURN] 
DEBU[2024-02-02T12:41:40.087589791Z] Firewalld passthrough: ipv4, [-A DOCKER-ISOLATION-STAGE-2 -j RETURN] 
DEBU[2024-02-02T12:41:40.155279152Z] Firewalld passthrough: ipv4, [-t filter -n -L DOCKER-USER] 
DEBU[2024-02-02T12:41:40.164695118Z] Firewalld passthrough: ipv4, [-t filter -C DOCKER-USER -j RETURN] 
DEBU[2024-02-02T12:41:40.173922974Z] Firewalld passthrough: ipv4, [-t filter -C FORWARD -j DOCKER-USER] 
DEBU[2024-02-02T12:41:40.182493528Z] Firewalld passthrough: ipv4, [-D FORWARD -j DOCKER-USER] 
DEBU[2024-02-02T12:41:40.191471038Z] Firewalld passthrough: ipv4, [-I FORWARD -j DOCKER-USER] 
DEBU[2024-02-02T12:41:40.274351696Z] Firewalld passthrough: ipv4, [-t filter -n -L DOCKER-USER] 
DEBU[2024-02-02T12:41:40.284534576Z] Firewalld passthrough: ipv4, [-t filter -C DOCKER-USER -j RETURN] 
DEBU[2024-02-02T12:41:40.294126566Z] Firewalld passthrough: ipv4, [-t filter -C FORWARD -j DOCKER-USER] 
DEBU[2024-02-02T12:41:40.302748301Z] Firewalld passthrough: ipv4, [-D FORWARD -j DOCKER-USER] 
DEBU[2024-02-02T12:41:40.313711923Z] Firewalld passthrough: ipv4, [-I FORWARD -j DOCKER-USER] 
DEBU[2024-02-02T12:41:40.440678600Z] Allocating IPv4 pools for network bridge (c3185d870889cc0cd1ad11b0542d5b4beb2ab0212ecaa3e582841d71fd6cd169) 
DEBU[2024-02-02T12:41:40.440728832Z] RequestPool(LocalDefault, , , _, false)      
DEBU[2024-02-02T12:41:40.441222008Z] RequestAddress(LocalDefault/172.17.0.0/16, <nil>, map[RequestAddressType:com.docker.network.gateway]) 
DEBU[2024-02-02T12:41:40.441287260Z] Request address PoolID:172.17.0.0/16 Bits: 65536, Unselected: 65534, Sequence: (0x80000000, 1)->(0x0, 2046)->(0x1, 1)->end Curr:0 Serial:false PrefAddress:invalid IP  
DEBU[2024-02-02T12:41:40.441389682Z] Did not find any interface with name docker0: Link not found 
DEBU[2024-02-02T12:41:40.441424332Z] Setting bridge mac address to 02:42:75:6f:84:38 
ERRO[2024-02-02T12:41:40.442017412Z] Failed to set bridge MTU docker0 via netlink  error="invalid argument"
DEBU[2024-02-02T12:41:40.442071133Z] releasing IPv4 pools from network bridge (c3185d870889cc0cd1ad11b0542d5b4beb2ab0212ecaa3e582841d71fd6cd169) 
DEBU[2024-02-02T12:41:40.442107653Z] ReleaseAddress(LocalDefault/172.17.0.0/16, 172.17.0.1) 
DEBU[2024-02-02T12:41:40.442129964Z] Released address Address:172.17.0.1 Sequence:Bits: 65536, Unselected: 65534, Sequence: (0x80000000, 1)->(0x0, 2046)->(0x1, 1)->end Curr:2 
DEBU[2024-02-02T12:41:40.442140844Z] ReleasePool(LocalDefault/172.17.0.0/16)      
DEBU[2024-02-02T12:41:40.442159903Z] daemon configured with a 15 seconds minimum shutdown timeout 
DEBU[2024-02-02T12:41:40.442176364Z] start clean shutdown of all containers with a 15 seconds timeout... 
DEBU[2024-02-02T12:41:40.443544736Z] Unix socket /var/run/docker/libnetwork/af367d88d83b.sock was closed. The external key listener will stop. 
DEBU[2024-02-02T12:41:40.443971402Z] Cleaning up old mountid : start.             
DEBU[2024-02-02T12:41:40.444282437Z] Cleaning up old mountid : done.              
failed to start daemon: Error initializing network controller: error creating default "bridge" network: invalid argument


# Note: device still created with MTU = 1500, even tho different value passed on command line above
➜  ~ ip link show docker0
25: docker0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default 
    link/ether 02:42:75:6f:84:38 brd ff:ff:ff:ff:ff:ff

Error Invalid Argument is same as trying to set value directly like:

➜  ~ sudo ip link set docker0 mtu 1501
RTNETLINK answers: Invalid argument

Any value mtu <=1500 works ok and the daemon starts

➜  ~ sudo dockerd --mtu 1234 -D 
INFO[2024-02-02T12:46:58.344400932Z] Starting up                                  
...               
INFO[2024-02-02T12:46:59.447297038Z] Docker daemon                                 commit=fce6e0c containerd-snapshotter=false storage-driver=overlay2 version=25.0.2
INFO[2024-02-02T12:46:59.447384729Z] Daemon has completed initialization          


# Value set to MTU = 1234 like passed on command line
➜  ~ ip link show docker0
25: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1234 qdisc noqueue state DOWN mode DEFAULT group default 
    link/ether 02:42:75:6f:84:38 brd ff:ff:ff:ff:ff:ff

Reproduce

  1. Stop docker daemon
  2. (Optional) Delete docker0 device to be sure it is created fresh
  3. Set a value mtu > 1500
  4. Start docker daemon

Expected behavior

Docker daemon should start and set the docker0 device MTU equal to whatever value was given, even if above 1500

docker version

Client: Docker Engine - Community
 Version:           25.0.2
 API version:       1.44
 Go version:        go1.21.6
 Git commit:        29cf629
 Built:             Thu Feb  1 00:26:25 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          25.0.2
  API version:      1.44 (minimum version 1.24)
  Go version:       go1.21.6
  Git commit:       fce6e0c
  Built:            Thu Feb  1 00:25:25 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.25
  GitCommit:        d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f
 runc:
  Version:          1.1.10
  GitCommit:        v1.1.10-0-g18a0cb0
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client: Docker Engine - Community
 Version:    25.0.2
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.21.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 14
 Server Version: 25.0.2
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f
 runc version: v1.1.10-0-g18a0cb0
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
 Kernel Version: 3.10.0-1160.83.1.el7.x86_64
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 7.638GiB
 Name: xxx
 ID: 152d8db8-141c-4e42-83eb-c573bb5c1de3
 Docker Root Dir: /var/lib/docker
 Debug Mode: true
  File Descriptors: 25
  Goroutines: 48
  System Time: 2024-02-02T12:49:58.542495393Z
  EventsListeners: 0
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional Info

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/networkingNetworkingarea/networking/d/bridgeNetworkingkind/bugBugs are bugs. The cause may or may not be known at triage time so debugging may be needed.status/confirmedversion/25.0

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions