@selansen @arkodg

@euanh @chiragtayal

@suwang48404 would help if you shared some examples using the moby dev shell

Some questions;

I think SwarmKit and "regular" containers use different ranges for the ephemeral port range; IIRC, regular containers use the range that's configured on the host (net.ipv4.ip_local_port_range), whereas SwarmKit (currently) has a fixed port range.

With this change;

• what is the default range?
• is there something in place to prevent SwarmKit from allocating ports that were already used by "regular" containers (and vice-versa)?

Given that this affects both the ephemeral port range for docker run, and for swarm services (docker service create / docker stack deploy); when, and by "who", is the port selected?

Given a cluster with (e.g.) 3 managers and several workers;

If I create a service with a port published through the ingress network;

• will the manager I'm connected to pick the port?
• will the worker? (likely not)

Same, but when creating a service with host mode publishing;

• will the manager I'm connected to pick the port?
• will the worker? (likely not)

Give that this is a configuration on the daemon, we must assume that each daemon can have a different configuration, which means that if the manager picks the port, that the selected port range can be different depending on which manager I'm connected to.

So, should we store this configuration in the swarm raft store instead? (similar to what we do for --default-addr-pool, which is configured for the cluster)

setting and updating this configuration

Per my earlier comment; validation is done in the libnetwork code; I still have to build and test this change, but;

• at what point is the given value validated?
• is the value validated when reloading the configuration (SIGHUP); in other words; do we have guards in place to prevent reloading with an invalid configuration?

(Without having given this much thought, so just thinking out loud); Are there possible complications if I update this port-range? What happens to existing containers if the daemon restarts? I assume containers (and services) preserve the port, but does the daemon / swarm continue to keep track of ports that were allocated in the old range?

Hi Sebastiaan,

thx for review, and spending time think about it.

reply inline please

Some questions;

I think SwarmKit and "regular" containers use different ranges for the ephemeral port range; IIRC, regular containers use the range that's configured on the host (net.ipv4.ip_local_port_range), whereas SwarmKit (currently) has a fixed port range.

With this change;

• what is the default range?
  ====> on linux based system, 49153-60999, on windows 60000-65000

• is there something in place to prevent SwarmKit from allocating ports that were already used by "regular" containers (and vice-versa)?

===> swarm allocating range is hard coded to 30000-32767, and they are not yet tunable

Given that this affects both the ephemeral port range for docker run, and for swarm services (docker service create / docker stack deploy); when, and by "who", is the port selected?

=====> docker engine port allocator is used when "docker run -p ..." and "docker service create --publish mode=host", swarm port allocator is used when "docker service create --publish .."

Given a cluster with (e.g.) 3 managers and several workers;

If I create a service with a port published through the ingress network;

• will the manager I'm connected to pick the port?
  =====> if "--publish mode" setting is not host, swarm manager pick a port from its port allocator, and this port is reserved on all nodes in cluster

• will the worker? (likely not)
  ======> correct. docker engine will not allocate port

Same, but when creating a service with host mode publishing;

• will the manager I'm connected to pick the port?
  ====> swarm defers to docker engine to allocate port

• will the worker? (likely not)
  ==========> docker engine allocate port if "docker service create --publish mode=host,...",the allocated port has only local node scope.

Give that this is a configuration on the daemon, we must assume that each daemon can have a different configuration, which means that if the manager picks the port, that the selected port range can be different depending on which manager I'm connected to.

So, should we store this configuration in the swarm raft store instead? (similar to what we do for --default-addr-pool, which is configured for the cluster)

========> each docker engine picks different port independently as port allocated by docker engine has local scope only, does not impact other node? That is, the same port allocated by docker engines can be allocated and assigned to same/different service instances on different nodes. Do we need raft in this use case?

setting and updating this configuration

Per my earlier comment; validation is done in the libnetwork code; I still have to build and test this change, but;

• at what point is the given value validated?
  ====> validation take place when config is read via daeomon.json or dockerd parameter

• is the value validated when reloading the configuration (SIGHUP); in other words; do we have guards in place to prevent reloading with an invalid configuration?

=========> yes

(Without having given this much thought, so just thinking out loud); Are there possible complications if I update this port-range? What happens to existing containers if the daemon restarts? I assume containers (and services) preserve the port, but does the daemon / swarm continue to keep track of ports that were allocated in the old range?
=======> The follows refers to services instances created with "--publish mode=host", or via "docker run -p", if port range is changed, config reloaded, existing container will continue to use old port allocated even if it is outside of new range; if dockerd rebooted, and containers restarted, they will be given ports from the new range.
=======> for port allocated by swarm, i.e. "docker service create --publish ....", I believe port persists across docker engine restart.

Thyx, Su

For the record, the libnetwork changes needed by this PR have been introduced in moby/libnetwork#2461 and is being reverted by #43066. If someone wants to work on this PR, they'd have to re-apply the reverted commit and fix the issue described in #43054.

@thaJeztah There's been no activity on this PR since two years and there's no linked issue so I'm not sure if that's really something people want. Moreover, the port range can already by configured through sysctl, so this PR only brings the config reload part. Maybe this change should be closed and mark as "won't implement" (unless there're requests for it).

Agreed, IMO we should close this for now in favor of #43066.

@neersighted @corhere @evol262 FYI - I recall the motivation for these changes was related to (at the time) Docker EE, to be able to reserve separate "ephemeral" port ranges for SwarmKit, "docker run", and k8s (?). Perhaps you're already carrying patches for that, but if not, perhaps this needs to be carried at some point