Skip to content

Commit d8ba1e2

Browse files
mrjanamrjana
committed
Driver api refactor
Refactored the driver api so that is aligns well with the design of endpoint lifecycle becoming decoupled from the container lifecycle. Introduced go interfaces to obtain address information during CreateEndpoint. Go interfaces are also used to get data from driver during join. This sort of deisgn hides the libnetwork specific type details from drivers. Another adjustment is to provide a list of interfaces during CreateEndpoint. The goal of this is many-fold: * To indicate to the driver that IP address has been assigned by some other entity (like a user wanting to use their own static IP for an endpoint/container) and asking the driver to honor this. Driver may reject this configuration and return an error but it may not try to allocate an IP address and override the passed one. * To indicate to the driver that IP address has already been allocated once for this endpoint by an instance of the same driver in some docker host in the cluster and this is merely a notification about that endpoint and the allocated resources. * In case the list of interfaces is empty the driver is required to allocate and assign IP addresses for this endpoint. Signed-off-by: Jana Radhakrishnan <[email protected]>
1 parent 7242f8f commit d8ba1e2

18 files changed

Lines changed: 624 additions & 222 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@ There are many networking solutions available to suit a broad range of use-cases
5959
}
6060

6161
// libentwork client can check the endpoint's operational data via the Info() API
62-
epInfo, err := ep.Info()
62+
epInfo, err := ep.DriverInfo()
6363
mapData, ok := epInfo[netlabel.PortMap]
6464
if ok {
6565
portMapping, ok := mapData.([]netutils.PortBinding)

‎api/api.go‎

Lines changed: 1 addition & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,6 @@ import (
88

99
"github.com/docker/libnetwork"
1010
"github.com/docker/libnetwork/netutils"
11-
"github.com/docker/libnetwork/sandbox"
1211
"github.com/gorilla/mux"
1312
)
1413

@@ -143,7 +142,6 @@ type endpointResource struct {
143142
Name string
144143
ID string
145144
Network string
146-
Info sandbox.Info
147145
}
148146

149147
func buildNetworkResource(nw libnetwork.Network) *networkResource {
@@ -168,11 +166,6 @@ func buildEndpointResource(ep libnetwork.Endpoint) *endpointResource {
168166
r.Name = ep.Name()
169167
r.ID = ep.ID()
170168
r.Network = ep.Network()
171-
172-
i := ep.SandboxInfo()
173-
if i != nil {
174-
r.Info = *i
175-
}
176169
}
177170
return r
178171
}
@@ -430,7 +423,7 @@ func procLeaveEndpoint(c libnetwork.NetworkController, vars map[string]string, b
430423
return nil, errRsp
431424
}
432425

433-
err := ep.Leave(vars[urlCnID], nil)
426+
err := ep.Leave(vars[urlCnID])
434427
if err != nil {
435428
return nil, convertNetworkError(err)
436429
}

‎cmd/readme_test/readme.go‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ func main() {
5555
}
5656

5757
// libentwork client can check the endpoint's operational data via the Info() API
58-
epInfo, err := ep.Info()
58+
epInfo, err := ep.DriverInfo()
5959
mapData, ok := epInfo[netlabel.PortMap]
6060
if ok {
6161
portMapping, ok := mapData.([]netutils.PortBinding)

‎docs/design.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -73,14 +73,14 @@ Consumers of the CNM, like Docker for example, interact through the CNM Objects
7373

7474
3. `controller.NewNetwork()` API also takes in optional `options` parameter which carries Driver-specific options and `Labels`, which the Drivers can make use for its purpose.
7575

76-
4. `network.CreateEndpoint()` can be called to create a new Endpoint in a given network. This API also accepts optional `options` parameter which drivers can make use of. These 'options' carry both well-known labels and driver-specific labels. Drivers will inturn be called with `driver.CreateEndpoint` and it can choose to reserve any required resources when an `Endpoint` is created in a `Network`. The `Driver` must return the reserved resources via the `sandbox.Info` return object. LibNetwork will make use of the `SandboxInfo` when a Container is attached later. The reason we get the `sandbox.Info` at the time of endpoint creation and not during the `Join()` is that, `Endpoint` represents a Service endpoint and not neccessarily the container that attaches later.
76+
4. `network.CreateEndpoint()` can be called to create a new Endpoint in a given network. This API also accepts optional `options` parameter which drivers can make use of. These 'options' carry both well-known labels and driver-specific labels. Drivers will in turn be called with `driver.CreateEndpoint` and it can choose to reserve IPv4/IPv6 addresses when an `Endpoint` is created in a `Network`. The `Driver` will assign these addresses using `InterfaceInfo` interface defined in the `driverapi`. The IP/IPv6 are needed to complete the endpoint as service definition along with the ports the endpoint exposes since essentially a service endpoint is nothing but a network address and the port number that the application container is listening on.
7777

7878
5. `endpoint.Join()` can be used to attach a container to a `Endpoint`. The Join operation will create a `Sandbox` if it doesnt exist already for that container. The Drivers can make use of the Sandbox Key to identify multiple endpoints attached to a same container. This API also accepts optional `options` parameter which drivers can make use of.
7979
* Though it is not a direct design issue of LibNetwork, it is highly encouraged to have users like `Docker` to call the endpoint.Join() during Container's `Start()` lifecycle that is invoked *before* the container is made operational. As part of Docker integration, this will be taken care of.
8080
* one of a FAQ on endpoint join() API is that, why do we need an API to create an Endpoint and another to join the endpoint.
8181
- The answer is based on the fact that Endpoint represents a Service which may or may not be backed by a Container. When an Endpoint is created, it will have its resources reserved so that any container can get attached to the endpoint later and get a consistent networking behaviour.
8282

83-
6. `endpoint.Leave()` can be invoked when a container is stopped. The `Driver` can cleanup the states that it allocated during the `Join()` call. LibNetwork will delete the `Sandbox` when the last referencing endpoint leaves the network. But LibNetwork keeps hold of the `sandbox.Info` and will be reused when the container joins again. This ensures that the container's resources are reused when they are Stopped and Started again.
83+
6. `endpoint.Leave()` can be invoked when a container is stopped. The `Driver` can cleanup the states that it allocated during the `Join()` call. LibNetwork will delete the `Sandbox` when the last referencing endpoint leaves the network. But LibNetwork keeps hold of the IP addresses as long as the endpoint is still present and will be reused when the container(or any container) joins again. This ensures that the container's resources are reused when they are Stopped and Started again.
8484

8585
7. `endpoint.Delete()` is used to delete an endpoint from a network. This results in deleting an endpoint and cleaning up the cached `sandbox.Info`.
8686

‎driverapi/driverapi.go‎

Lines changed: 70 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@ package driverapi
33
import (
44
"errors"
55
"fmt"
6+
"net"
67

7-
"github.com/docker/libnetwork/sandbox"
88
"github.com/docker/libnetwork/types"
99
)
1010

@@ -37,32 +37,91 @@ type Driver interface {
3737
DeleteNetwork(nid types.UUID) error
3838

3939
// CreateEndpoint invokes the driver method to create an endpoint
40-
// passing the network id, endpoint id and driver
41-
// specific config. The config mechanism will eventually be replaced
42-
// with labels which are yet to be introduced.
43-
CreateEndpoint(nid, eid types.UUID, options map[string]interface{}) (*sandbox.Info, error)
40+
// passing the network id, endpoint id endpoint information and driver
41+
// specific config. The endpoint information can be either consumed by
42+
// the driver or populated by the driver. The config mechanism will
43+
// eventually be replaced with labels which are yet to be introduced.
44+
CreateEndpoint(nid, eid types.UUID, epInfo EndpointInfo, options map[string]interface{}) error
4445

4546
// DeleteEndpoint invokes the driver method to delete an endpoint
4647
// passing the network id and endpoint id.
4748
DeleteEndpoint(nid, eid types.UUID) error
4849

49-
// EndpointInfo retrieves from the driver the operational data related to the specified endpoint
50-
EndpointInfo(nid, eid types.UUID) (map[string]interface{}, error)
50+
// EndpointOperInfo retrieves from the driver the operational data related to the specified endpoint
51+
EndpointOperInfo(nid, eid types.UUID) (map[string]interface{}, error)
5152

5253
// Join method is invoked when a Sandbox is attached to an endpoint.
53-
Join(nid, eid types.UUID, sboxKey string, options map[string]interface{}) (*JoinInfo, error)
54+
Join(nid, eid types.UUID, sboxKey string, jinfo JoinInfo, options map[string]interface{}) error
5455

5556
// Leave method is invoked when a Sandbox detaches from an endpoint.
56-
Leave(nid, eid types.UUID, options map[string]interface{}) error
57+
Leave(nid, eid types.UUID) error
5758

5859
// Type returns the the type of this driver, the network type this driver manages
5960
Type() string
6061
}
6162

63+
// EndpointInfo provides a go interface to fetch or populate endpoint assigned network resources.
64+
type EndpointInfo interface {
65+
// Interfaces returns a list of interfaces bound to the endpoint.
66+
// If the list is not empty the driver is only expected to consume the interfaces.
67+
// It is an error to try to add interfaces to a non-empty list.
68+
// If the list is empty the driver is expected to populate with 0 or more interfaces.
69+
Interfaces() []InterfaceInfo
70+
71+
// AddInterface is used by the driver to add an interface to the interface list.
72+
// This method will return an error if the driver attempts to add interfaces
73+
// if the Interfaces() method returned a non-empty list.
74+
// ID field need only have significance within the endpoint so it can be a simple
75+
// monotonically increasing number
76+
AddInterface(ID int, mac net.HardwareAddr, ipv4 net.IPNet, ipv6 net.IPNet) error
77+
}
78+
79+
// InterfaceInfo provides a go interface for drivers to retrive
80+
// network information to interface resources.
81+
type InterfaceInfo interface {
82+
// MacAddress returns the MAC address.
83+
MacAddress() net.HardwareAddr
84+
85+
// Address returns the IPv4 address.
86+
Address() net.IPNet
87+
88+
// AddressIPv6 returns the IPv6 address.
89+
AddressIPv6() net.IPNet
90+
91+
// ID returns the numerical id of the interface and has significance only within
92+
// the endpoint.
93+
ID() int
94+
}
95+
96+
// InterfaceNameInfo provides a go interface for the drivers to assign names
97+
// to interfaces.
98+
type InterfaceNameInfo interface {
99+
// SetNames method assigns the srcName and dstName for the interface.
100+
SetNames(srcName, dstName string) error
101+
102+
// ID returns the numerical id that was assigned to the interface by the driver
103+
// CreateEndpoint.
104+
ID() int
105+
}
106+
62107
// JoinInfo represents a set of resources that the driver has the ability to provide during
63108
// join time.
64-
type JoinInfo struct {
65-
HostsPath string
109+
type JoinInfo interface {
110+
// InterfaceNames returns a list of InterfaceNameInfo go interface to facilitate
111+
// setting the names for the interfaces.
112+
InterfaceNames() []InterfaceNameInfo
113+
114+
// SetGateway sets the default IPv4 gateway when a container joins the endpoint.
115+
SetGateway(net.IP) error
116+
117+
// SetGatewayIPv6 sets the default IPv6 gateway when a container joins the endpoint.
118+
SetGatewayIPv6(net.IP) error
119+
120+
// SetHostsPath sets the overriding /etc/hosts path to use for the container.
121+
SetHostsPath(string) error
122+
123+
// SetResolvConfPath sets the overriding /etc/resolv.conf path to use for the container.
124+
SetResolvConfPath(string) error
66125
}
67126

68127
// ErrActiveRegistration represents an error when a driver is registered to a networkType that is previously registered

0 commit comments

Comments
 (0)