Skip to content

Commit f46bd9c

Browse files
committed
feat: Support configuring local IP address for N9 interface
Until now, eupf only allowed configuring the local IP address for the N3 interface, which actually ended up being used for both N3 and N9 (GTP-U forwarding towards other uplink UPFs). However, that limitation poses so problems under some production scenarios. Some users require running an UPF to forward GTPv1U traffic from network A to network B and viceversa, where those 2 networks are not reachable to each other, only through the UPF host. This happens for instance when cellular operators don't control/own the entirety of their networks, e.g. a network which runs their NAT under a satellite link with private addresses (eg VPN) which end up at a single host. As a result, the UPF requires managing at least 2 separate local IP addresses for GTP-U traffic: One (usually private) IP address towards the RAN, and one (usually public, or another private network) IP address towards the Corei. As a result, when a PFCP client (an SMF, or osmo-hnbgw, or osmo-s1gw to mention some other examples) tries to setup a session to forward GTPU traffic from network A (RAN) to network B (Core), will result in eUPF forwarding the GTP-U packet to the Core but using the RAN configured IP as the GTPU packet local IP address. Hence, either local routing will be wrong, or the Core will drop the received packet, or won't know how to route traffic back to it. This patch allows eupf to be used co-located with 3G osmo-hnbgw [1] and 4G osmo-s1gw [2] under the mentioned network setups, both supporting PFCP to manage a UPF to forward data plane also in GTP1U protocol. [1] https://osmocom.org/projects/osmohnbgw [2] https://gitea.osmocom.org/erlang/osmo-s1gw Related: edgecomllc#577
1 parent 44cfbf6 commit f46bd9c

21 files changed

Lines changed: 60 additions & 10 deletions

File tree

‎README.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@ Use following command to mount it: `sudo mount bpffs /sys/fs/bpf -t bpf`
4444

4545
- UPF_INTERFACE_NAME=lo *Network interfaces handling N3 (GTP) & N6 (SGi) traffic.*
4646
- UPF_N3_ADDRESS=127.0.0.1 *IPv4 address for N3 interface*
47+
- UPF_N9_ADDRESS=127.0.0.1 *IPv4 address for N9 interface*
4748
- UPF_XDP_ATTACH_MODE=generic *XDP attach mode. Generic-only at the moment*
4849
- UPF_API_ADDRESS=:8080 *Local host:port for serving [REST API](api.md) server*
4950
- UPF_PFCP_ADDRESS=:8805 *Local host:port that PFCP server will listen to*
@@ -63,7 +64,7 @@ sudo docker run -d --rm -v --privileged \
6364
-e UPF_INTERFACE_NAME=[eth0,n6] -e UPF_XDP_ATTACH_MODE=generic \
6465
-e UPF_API_ADDRESS=:8081 -e UPF_PFCP_ADDRESS=:8806 \
6566
-e UPF_METRICS_ADDRESS=:9091 -e UPF_PFCP_NODE_ID=10.100.50.241 \
66-
-e UPF_N3_ADDRESS=10.100.50.233 \
67+
-e UPF_N3_ADDRESS=10.100.50.233 -e UPF_N9_ADDRESS=10.100.50.233 \
6768
ghcr.io/edgecomllc/eupf:main
6869
```
6970

‎cmd/config/config.go‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ type UpfConfig struct {
2020
AssociationSetupTimeout uint32 `mapstructure:"association_setup_timeout" json:"association_setup_timeout"`
2121
MetricsAddress string `mapstructure:"metrics_address" validate:"hostname_port" json:"metrics_address"`
2222
N3Address string `mapstructure:"n3_address" validate:"ipv4" json:"n3_address"`
23+
N9Address string `mapstructure:"n9_address" validate:"ipv4" json:"n9_address"`
2324
GtpPeer []string `mapstructure:"gtp_peer" validate:"omitempty,dive,hostname_port" json:"gtp_peer"`
2425
GtpEchoInterval uint32 `mapstructure:"gtp_echo_interval" validate:"min=1" json:"gtp_echo_interval"`
2526
QerMapSize uint32 `mapstructure:"qer_map_size" validate:"min=1" json:"qer_map_size"`
@@ -47,6 +48,7 @@ func init() {
4748
pflag.String("nodeid", "127.0.0.1", "PFCP Server Node ID")
4849
pflag.String("maddr", ":9090", "Address to bind metrics server to")
4950
pflag.String("n3addr", "127.0.0.1", "Address for communication over N3 interface")
51+
pflag.String("n9addr", "127.0.0.1", "Address for communication over N9 interface")
5052
pflag.StringArray("peer", []string{}, "Address of GTP peer")
5153
pflag.Uint32("echo", 10, "Interval of sending echo requests in seconds")
5254
pflag.Uint32("qersize", 1024, "Size of the QER ebpf map")
@@ -76,6 +78,7 @@ func init() {
7678
_ = v.BindPFlag("association_setup_timeout", pflag.Lookup("astimeout"))
7779
_ = v.BindPFlag("metrics_address", pflag.Lookup("maddr"))
7880
_ = v.BindPFlag("n3_address", pflag.Lookup("n3addr"))
81+
_ = v.BindPFlag("n9_address", pflag.Lookup("n9addr"))
7982
_ = v.BindPFlag("gtp_peer", pflag.Lookup("peer"))
8083
_ = v.BindPFlag("gtp_echo_interval", pflag.Lookup("echo"))
8184
_ = v.BindPFlag("qer_map_size", pflag.Lookup("qersize"))
@@ -100,6 +103,7 @@ func init() {
100103
v.SetDefault("association_setup_timeout", 5)
101104
v.SetDefault("metrics_address", ":9090")
102105
v.SetDefault("n3_address", "127.0.0.1")
106+
v.SetDefault("n9_address", "127.0.0.1")
103107
v.SetDefault("gtp_echo_interval", 10)
104108
v.SetDefault("qer_map_size", 1024)
105109
v.SetDefault("far_map_size", 1024)

‎cmd/core/pfcp_connection.go‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@ type PfcpConnection struct {
3939
nodeId string
4040
nodeAddrV4 net.IP
4141
n3Address net.IP
42+
n9Address net.IP
4243
mapOperations ebpf.ForwardingPlaneController
4344
RecoveryTimestamp time.Time
4445
featuresOctets []uint8
@@ -54,7 +55,7 @@ func (connection *PfcpConnection) GetAssociation(assocAddr string) *NodeAssociat
5455
return nil
5556
}
5657

57-
func NewPfcpConnection(addr string, nodeId string, n3Ip string, mapOperations ebpf.ForwardingPlaneController, resourceManager *service.ResourceManager) (*PfcpConnection, error) {
58+
func NewPfcpConnection(addr string, nodeId string, n3Ip string, n9Ip string, mapOperations ebpf.ForwardingPlaneController, resourceManager *service.ResourceManager) (*PfcpConnection, error) {
5859
udpAddr, err := net.ResolveUDPAddr("udp", addr)
5960
if err != nil {
6061
log.Warn().Msgf("Can't resolve UDP address: %s", err.Error())
@@ -70,7 +71,12 @@ func NewPfcpConnection(addr string, nodeId string, n3Ip string, mapOperations eb
7071
if n3Addr == nil {
7172
return nil, fmt.Errorf("failed to parse N3 IP address ID: %s", n3Ip)
7273
}
73-
log.Info().Msgf("Starting PFCP connection: %v with Node ID: %v and N3 address: %v", udpAddr, nodeId, n3Addr)
74+
n9Addr := net.ParseIP(n9Ip)
75+
if n9Addr == nil {
76+
return nil, fmt.Errorf("failed to parse N9 IP address ID: %s", n9Ip)
77+
}
78+
79+
log.Info().Msgf("Starting PFCP connection: %v with Node ID: %v, N3 address: %v, N9 address: %v", udpAddr, nodeId, n3Addr, n9Addr)
7480

7581
featuresOctets := []uint8{0, 0, 0}
7682
featuresOctets[1] = setBit(featuresOctets[1], 0)
@@ -89,6 +95,7 @@ func NewPfcpConnection(addr string, nodeId string, n3Ip string, mapOperations eb
8995
nodeId: nodeId,
9096
nodeAddrV4: udpAddr.IP,
9197
n3Address: n3Addr,
98+
n9Address: n9Addr,
9299
mapOperations: mapOperations,
93100
RecoveryTimestamp: time.Now(),
94101
featuresOctets: featuresOctets,

‎cmd/core/pfcp_session_handlers.go‎

Lines changed: 20 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ func HandlePfcpSessionEstablishmentRequest(conn *PfcpConnection, msg message.Mes
4646
err = func() error {
4747
mapOperations := conn.mapOperations
4848
for _, far := range req.CreateFAR {
49-
farInfo, err := composeFarInfo(far, conn.n3Address.To4(), ebpf.FarInfo{})
49+
farInfo, err := composeFarInfo(far, conn.n3Address.To4(), conn.n9Address.To4(), ebpf.FarInfo{})
5050
if err != nil {
5151
log.Info().Msgf("Error extracting FAR info: %s", err.Error())
5252
continue
@@ -256,7 +256,7 @@ func HandlePfcpSessionModificationRequest(conn *PfcpConnection, msg message.Mess
256256
mapOperations := conn.mapOperations
257257

258258
for _, far := range req.CreateFAR {
259-
farInfo, err := composeFarInfo(far, conn.n3Address.To4(), ebpf.FarInfo{})
259+
farInfo, err := composeFarInfo(far, conn.n3Address.To4(), conn.n9Address.To4(), ebpf.FarInfo{})
260260
if err != nil {
261261
log.Info().Msgf("Error extracting FAR info: %s", err.Error())
262262
continue
@@ -278,7 +278,7 @@ func HandlePfcpSessionModificationRequest(conn *PfcpConnection, msg message.Mess
278278
return err
279279
}
280280
sFarInfo := session.GetFar(farid)
281-
sFarInfo.FarInfo, err = composeFarInfo(far, conn.n3Address.To4(), sFarInfo.FarInfo)
281+
sFarInfo.FarInfo, err = composeFarInfo(far, conn.n3Address.To4(), conn.n9Address.To4(), sFarInfo.FarInfo)
282282
if err != nil {
283283
log.Info().Msgf("Error extracting FAR info: %s", err.Error())
284284
continue
@@ -553,14 +553,14 @@ func cloneIP(ip net.IP) net.IP {
553553
return dup
554554
}
555555

556-
func composeFarInfo(far *ie.IE, localIp net.IP, farInfo ebpf.FarInfo) (ebpf.FarInfo, error) {
557-
farInfo.LocalIP = binary.LittleEndian.Uint32(localIp)
556+
func composeFarInfo(far *ie.IE, localN3Ip net.IP, localN9Ip net.IP, farInfo ebpf.FarInfo) (ebpf.FarInfo, error) {
558557
if applyAction, err := far.ApplyAction(); err == nil {
559558
farInfo.Action = applyAction[0]
560559
}
561560
var forward []*ie.IE
562561
var err error
563562
if far.Type == ie.CreateFAR {
563+
farInfo.LocalIP = binary.LittleEndian.Uint32(localN3Ip)
564564
forward, err = far.ForwardingParameters()
565565
} else if far.Type == ie.UpdateFAR {
566566
forward, err = far.UpdateForwardingParameters()
@@ -574,6 +574,7 @@ func composeFarInfo(far *ie.IE, localIp net.IP, farInfo ebpf.FarInfo) (ebpf.FarI
574574
} else {
575575
outerHeaderCreation, _ := forward[outerHeaderCreationIndex].OuterHeaderCreation()
576576
farInfo.OuterHeaderCreation = uint8(outerHeaderCreation.OuterHeaderCreationDescription >> 8)
577+
577578
farInfo.Teid = outerHeaderCreation.TEID
578579
if outerHeaderCreation.HasIPv4() {
579580
farInfo.RemoteIP = binary.LittleEndian.Uint32(outerHeaderCreation.IPv4Address)
@@ -582,6 +583,20 @@ func composeFarInfo(far *ie.IE, localIp net.IP, farInfo ebpf.FarInfo) (ebpf.FarI
582583
log.Info().Msg("WARN: IPv6 not supported yet, ignoring")
583584
return ebpf.FarInfo{}, fmt.Errorf("IPv6 not supported yet")
584585
}
586+
587+
destInterfaceIndex := findIEindex(forward, 42) // IE Destination Interface
588+
if destInterfaceIndex == -1 {
589+
log.Info().Msg("WARN: No Destination Interface IE")
590+
} else {
591+
destInterface, _ := forward[destInterfaceIndex].DestinationInterface()
592+
// OuterHeaderCreation == GTP-U/UDP/IPv4 && DestinationInterface == Core:
593+
if (farInfo.OuterHeaderCreation&0x01) == 0x01 && (destInterface == 0x01) {
594+
log.Info().Msgf("PESPIN: Using N9 local IP address %v", localN9Ip)
595+
farInfo.LocalIP = binary.LittleEndian.Uint32(localN9Ip)
596+
} else {
597+
farInfo.LocalIP = binary.LittleEndian.Uint32(localN3Ip)
598+
}
599+
}
585600
}
586601
}
587602
transportLevelMarking, err := GetTransportLevelMarking(far)

‎cmd/main.go‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,9 @@ func main() {
8181
}
8282

8383
// Create PFCP connection
84-
pfcpConn, err := core.NewPfcpConnection(config.Conf.PfcpAddress, config.Conf.PfcpNodeId, config.Conf.N3Address, bpfObjects, resourceManager)
84+
pfcpConn, err := core.NewPfcpConnection(config.Conf.PfcpAddress, config.Conf.PfcpNodeId,
85+
config.Conf.N3Address, config.Conf.N9Address,
86+
bpfObjects, resourceManager)
8587
if err != nil {
8688
log.Fatal().Msgf("Could not create PFCP connection: %s", err.Error())
8789
}

‎docs/Configuration.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ Currently UPF have several config parameters shown below.<br>Parameters can be c
88
|--------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------|-----------------------|-------------|-----------------|
99
| Interface name<br>`Mandatory` | List of network interfaces handling N3 (GTP) & N6 (SGi) traffic. eUPF attaches XDP hook to every interface in this list. Format: `[ifnameA, ifnameB, ...]`. | `interface_name` | `UPF_INTERFACE_NAME` | `--iface` | `lo` |
1010
| N3 address <br>`Mandatory` | IPv4 address for N3 interface | `n3_address` | `UPF_N3_ADDRESS` | `--n3addr` | `127.0.0.1` |
11+
| N9 address <br>`Mandatory` | IPv4 address for N9 interface | `n9_address` | `UPF_N9_ADDRESS` | `--n9addr` | `127.0.0.1` |
1112
| XDP mode <br>`Optional` | XDP attach mode: <br> ∘ **generic** – Kernel-level implementation. For evaluation purpose. <br> ∘ **native** – Driver-level implenemntaion <br> ∘ **offload** – NIC-level implementation. XDP can be loaded and executed directly on the NIC. <br> Refer to [How XDP Works](https://www.tigera.io/learn/guides/ebpf/ebpf-xdp/#How-XDP-Works) | `xdp_attach_mode` | `UPF_XDP_ATTACH_MODE` | `--attach` | `generic` |
1213
| API address <br>`Optional` | Local address for serving [REST API](api.md) server | `api_address` | `UPF_API_ADDRESS` | `--aaddr` | `:8080` |
1314
| PFCP address <br>`Optional` | Local address that PFCP server will listen to | `pfcp_address` | `UPF_PFCP_ADDRESS` | `--paddr` | `:8805` |
@@ -46,6 +47,7 @@ pfcp_address: :8805
4647
pfcp_node_id: 127.0.0.1
4748
metrics_address: :9090
4849
n3_address: 127.0.0.1
50+
n9_address: 127.0.0.1
4951
qer_map_size: 1024
5052
far_map_size: 1024
5153
pdr_map_size: 1024
@@ -65,6 +67,7 @@ UPF_PFCP_ADDRESS=:8806
6567
UPF_METRICS_ADDRESS=:9091
6668
UPF_PFCP_NODE_ID: 10.100.50.241 # address on n4 interface
6769
UPF_N3_ADDRESS: 10.100.50.233
70+
UPF_N9_ADDRESS: 10.100.50.233
6871
```
6972
7073
### CLI

‎docs/api.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@ In addition to prometheus metrics the eUPF API provides a set of endpoints for m
4949
"pfcp_node_id": "10.100.200.14",
5050
"metrics_address": ":9090",
5151
"n3_address": "10.100.200.14",
52+
"n9_address": "10.100.200.14",
5253
"qer_map_size": 1024,
5354
"far_map_size": 1024,
5455
"pdr_map_size": 1024,

‎docs/deployments/free5gc-ulcl/eupf-b.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,11 +43,13 @@ configMaps:
4343
pfcp_address: 10.100.50.241:8805
4444
metrics_address: :9090
4545
n3_address: 10.100.50.233
46+
n9_address: 10.100.50.233
4647
gtp_peer: [10.100.50.226:2152, 10.100.50.227:2152]
4748
4849
env:
4950
UPF_PFCP_NODE_ID: 10.100.50.241 # address on n4 interface
5051
# UPF_N3_ADDRESS: 10.100.50.233
52+
# UPF_N9_ADDRESS: 10.100.50.233
5153

5254
volumes:
5355
- name: sys

‎docs/deployments/free5gc-with-bgp/values/eupf.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ args:
1616
env:
1717
UPF_PFCP_NODE_ID: 10.100.50.241
1818
UPF_N3_ADDRESS: 10.100.50.233
19+
UPF_N9_ADDRESS: 10.100.50.233
1920

2021
configMaps:
2122
config:

‎docs/deployments/open5gs-compose/docker-compose.eupf.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ services:
1616
UPF_METRICS_ADDRESS: ":9091"
1717
UPF_PFCP_NODE_ID: "172.20.0.100"
1818
UPF_N3_ADDRESS: "172.20.0.100"
19+
UPF_N9_ADDRESS: "172.20.0.100"
1920
ulimits:
2021
memlock: -1
2122
cap_add:

0 commit comments

Comments
 (0)