P4 Language Tutorial

Slides: [Link]
Software Setup
◦ Download VM or copy from USB stick
◦ Import VM into VirtualBox or VMware Fusion
◦ Boot VM and login as user "p4" with passwd "p4"
◦ Open Terminal
◦ cd ~/tutorials; git pull
Copyright © 2018 – [Link]
Goals
• Learn P4 Language
◦ Traditional applications
◦ Novel applications
• Learn P4 software tools
◦ P4 Compiler
◦ BMv2
◦ P4Runtime
• Learn about future technology trends
◦ Keynote by Arvind Krishnamurthy (Washington)
◦ Panel with representatives from ONF, Kaloom, and Keysight
• Networking (the other kind)
• Have fun!
Copyright © 2018 – [Link]
Introduction to Data Plane Programming
Language Basics
Break
Software Tools & P4 Runtime
Lunch
Keynote (Arvind Krishnamurthy)
Monitoring & Debugging
Break
Advanced Data Structures
Panel Discussion
Reception
Copyright © 2018 – [Link]
Instructors

Stephen Ibanez Brian O'Connor Mina Arashloo

Stanford ONF Princeton

Copyright © 2018 – [Link]

Thank you

• Sedef Ozcana ([Link])

• Rachel Everman (Barefoot)
• Arvind Krishnamurthy (Washington)
• Panelists
◦ Uyen Chau (ONF)
◦ Bochra Boughzala (Kaloom)
◦ Ben Pfaff (VMware)
◦ Chris Sommers (Keysight)
• TAs
◦ Santiago Bautista (ENS/Cornell)
◦ Sean Choi (Stanford)
◦ Theo Jepsen (Lugano)
◦ Sarah Tollman (Stanford)
Copyright © 2018 – [Link]
Copyright © 2018 – [Link] 6
What is Data Plane Programming?
• Why program the Data Plane?

Copyright © 2018 – [Link] 7

Software Defined Networking (SDN)
• Main contributions Control Function (“App”) xm
◦ OpenFlow = standardized protocol to interact with
switch
■ download flow table entries, query statistics, etc. Northbound I/F
◦ OpenFlow = standardized model
■ match/action abstraction
◦ Concept of logically centralized control via a SDN
single entity (“SDN controller”) x1
■ Simplifies control plane Controller

• Issues Southbound
◦ Data-plane protocol evolution requires changes to I/F
standards (12 → 40 OpenFlow match fields)
◦ Limited interoperability between vendors
(OpenFlow / netconf / JSON / XML variants) xn
◦ Limited programmability

Copyright © 2018 – [Link]

Switch 8
Status Quo: Bottom-up design

Switch OS
Network Demands

?
“This is how I know to
Run-time API
Driver

process packets”
(i.e. the ASIC datasheet
makes the rules)
Fixed-function ASIC
Copyright © 2018 – [Link] 9
A Better Approach: Top-down design

Switch OS
Network Demands

Feedback

Run-time API
Driver

P4

“This is how I want the

network to behave and how to
switch packets…”
(the user / controller
makes the rules)
P4 Programmable Device
Copyright © 2018 – [Link] 10
Benefits of Data Plane Programmability
• New Features – Add new protocols
• Reduce complexity – Remove unused protocols
• Efficient use of resources – flexible use of tables
• Greater visibility – New diagnostic techniques, telemetry, etc.
• SW style development – rapid design cycle, fast innovation, fix data
plane bugs in the field
• You keep your own ideas

Think programming rather than protocols…

Copyright © 2018 – [Link] 11

Programmable Network Devices
• PISA: Flexible Match+Action ASICs
◦ Intel Flexpipe, Cisco Doppler, Cavium (Xpliant), Barefoot Tofino, …
• NPU
◦ EZchip, Netronome, …
• CPU
◦ Open Vswitch, eBPF, DPDK, VPP…
• FPGA
◦ Xilinx, Altera, …

These devices let us tell them how to process packets.

Copyright © 2018 – [Link] 12

What can you do with P4?
• Layer 4 Load Balancer – SilkRoad[1]
• Low Latency Congestion Control – NDP[2]
• In-band Network Telemetry – INT[3]
• In-Network caching and coordination – NetCache[4] / NetChain[5]
• Aggregation for MapReduce Applications [7]
• … and much more

[1] Miao, Rui, et al. "SilkRoad: Making Stateful Layer-4 Load Balancing Fast and Cheap Using Switching ASICs." SIGCOMM, 2017.
[2] Handley, Mark, et al. "Re-architecting datacenter networks and stacks for low latency and high performance.” SIGCOMM, 2017.
[3] Kim, Changhoon, et al. "In-band network telemetry via programmable dataplanes.” SIGCOMM. 2015.
[4] Xin Jin et al. “NetCache: Balancing Key-Value Stores with Fast In-Network Caching.” To appear at SOSP 2017
[5] Jin, Xin, et al. "NetChain: Scale-Free Sub-RTT Coordination." NSDI, 2018.
[6] Dang, Huynh Tu, et al. "NetPaxos: Consensus at network speed.” SIGCOMM, 2015.
[7] Sapio, Amedeo, et al. "In-Network Computation is a Dumb Idea Whose Time Has Come." Hot Topics in Networks. ACM, 2017.

Copyright © 2018 – [Link] 13

Brief History and Trivia
• May 2013: Initial idea and the name “P4”
• July 2014: First paper (SIGCOMM CCR)
• Aug 2014: First P414 Draft Specification (v0.9.8)
• Sep 2014: P414 Specification released (v1.0.0)
• Jan 2015: P414 v1.0.1
• Mar 2015: P414 v1.0.2
• Nov 2016: P414 v1.0.3
• May 2017: P414 v1.0.4

• Apr 2016: P416 – first commits

• Dec 2016: First P416 Draft Specification
• May 2017: P416 Specification released

Copyright © 2018 – [Link] 14

P4_16 Data Plane Model

Copyright © 2018 – [Link] 15

PISA: Protocol-Independent Switch Architecture

Programmer defines the

tables and the exact Programmer declares
Programmer declares the processing algorithm how the output packet
headers that should be will look on the wire
recognized and their order in
the packet

Programmable Match-Action Pipeline

Programmable Programmable
Parser Deparser

Copyright © 2018 – [Link] 16

PISA in Action
• Packet is parsed into individual headers (parsed representation)
• Headers and intermediate results can be used for matching and
actions
• Headers can be modified, added or removed
• Packet is deparsed (serialized)
Programmable Match-Action Pipeline
Programmable Programmable
Parser Deparser

Copyright © 2017 – [Link] 17

P416 Language Elements

State machine,
Parsers bitfield extraction

Tables, Actions,
Controls control flow
statements

Basic operations
Expressions and operators

Bistrings, headers,
Data Types structures, arrays

Architecture Programmable blocks

Description and their interfaces

Support for specialized

Extern Libraries components

Copyright © 2017 – [Link] 18

P4_16 Approach

Term Explanation

P4 Target An embodiment of a specific hardware implementation

Provides an interface to program a target via some set of P4-programmable components,

P4 Architecture externs, fixed components

Community-Developed Vendor-supplied

P416 P416 Core Extern Architecture

Language Library
Libraries Definition

Copyright © 2018 – [Link] 19

Example Architectures and Targets
V1Model

TM

SimpleSumeSwitch

TM

Portable Switch Architecture (PSA)

TM Anything

Copyright © 2018 – [Link] 20

Programming a P4 Target
User supplied

Control Plane

RUNTIME
P4 Program P4 Compiler Add/remove Extern Packet-in/out
table entries control

CPU port
P4 Architecture Target-specific Extern
configuration Load Tables Data Plane
Model objects
binary

Target
Vendor supplied

Copyright © 2018 – [Link] 21

Lab 1: Basics

Copyright © 2018 – [Link] 22

Before we start...
•Install VM image (Look for instructor with USB sticks)
•Please make sure that your VM is up to date
◦$ cd ~/tutorials && git pull
•We’ll be using several software tools pre-installed on the VM
◦Bmv2: a P4 software switch
◦p4c: the reference P4 compiler
◦Mininet: a lightweight network emulation environment
•Each directory contains a few scripts
◦$ make : compiles P4 program, execute on Bmv2 in Mininet, populate tables
◦*.py: send and receive test packets in Mininet
•Exercises
◦Each example comes with an incomplete implementation; your job is to finish it!
◦Look for “TODOs” (or peek at the P4 code in solution/ if you must)
Copyright © 2018 – [Link] 23
V1Model Architecture
• Implemented on top of Bmv2’s simple_switch target

Parser Checksum Verification / Checksum Update / Deparser

Ingress Match-Action Egress Match-Action

Traffic
Manager

Copyright © 2017 – [Link] 24

V1Model Standard Metadata
struct standard_metadata_t { • ingress_port - the port on which
bit<9> ingress_port;
bit<9> egress_spec; the packet arrived
bit<9> egress_port;
bit<32> clone_spec; • egress_spec - the port to which
bit<32> instance_type;
bit<1> drop;
bit<16> recirculate_port;
the packet should be sent to
bit<32> packet_length;
bit<32> enq_timestamp;
• egress_port - the port that the
bit<19> enq_qdepth;
bit<32> deq_timedelta;
packet will be sent out of (read
bit<19> deq_qdepth;
bit<48> ingress_global_timestamp; only in egress pipeline)
bit<32> lf_field_list;
bit<16> mcast_grp;
bit<1> resubmit_flag;
bit<16> egress_rid;
bit<1> checksum_error;
}

Copyright © 2018 – [Link] 25

P416 Program Template (V1Model)
#include <core.p4> /* EGRESS PROCESSING */
#include <v1model.p4> control MyEgress(inout headers hdr,
/* HEADERS */ inout metadata meta,
struct metadata { ... } inout standard_metadata_t std_meta) {
struct headers { ...
ethernet_t ethernet; }
ipv4_t ipv4; /* CHECKSUM UPDATE */
} control MyComputeChecksum(inout headers hdr,
/* PARSER */ inout metadata meta) {
parser MyParser(packet_in packet, ...
out headers hdr, }
inout metadata meta, /* DEPARSER */
inout standard_metadata_t smeta) { control MyDeparser(inout headers hdr,
... inout metadata meta) {
} ...
/* CHECKSUM VERIFICATION */ }
control MyVerifyChecksum(in headers hdr, /* SWITCH */
inout metadata meta) { V1Switch(
... MyParser(),
} MyVerifyChecksum(),
/* INGRESS PROCESSING */ MyIngress(),
control MyIngress(inout headers hdr, MyEgress(),
inout metadata meta, MyComputeChecksum(),
inout standard_metadata_t std_meta) { MyDeparser()
... ) main;
}

Copyright © 2018 – [Link] 26

P416 Hello World (V1Model)
#include <core.p4>
#include <v1model.p4> control MyEgress(inout headers hdr,
struct metadata {} inout metadata meta,
struct headers {} inout standard_metadata_t standard_metadata) {
apply { }
parser MyParser(packet_in packet, }
out headers hdr,
inout metadata meta, control MyComputeChecksum(inout headers hdr, inout metadata
inout standard_metadata_t standard_metadata) { meta) {
apply { }
state start { transition accept; } }
}
control MyDeparser(packet_out packet, in headers hdr) {
control MyVerifyChecksum(inout headers hdr, inout metadata apply { }
meta) { apply { } } }

control MyIngress(inout headers hdr, V1Switch(

inout metadata meta, MyParser(),
inout standard_metadata_t standard_metadata) { MyVerifyChecksum(),
apply { MyIngress(),
if (standard_metadata.ingress_port == 1) { MyEgress(),
standard_metadata.egress_spec = 2; MyComputeChecksum(),
} else if (standard_metadata.ingress_port == 2) { MyDeparser()
standard_metadata.egress_spec = 1; ) main;
}
}
}
Copyright © 2018 – [Link] 27
P416 Hello World (V1Model)
#include <core.p4>
#include <v1model.p4> control MyEgress(inout headers hdr,
struct metadata {} inout metadata meta,
struct headers {} inout standard_metadata_t standard_metadata) {
apply { }
parser MyParser(packet_in packet, out headers hdr, }
inout metadata meta,
inout standard_metadata_t standard_metadata) { control MyVerifyChecksum(inout headers hdr, inout metadata
state start { transition accept; } meta) { apply { } }
}
control MyComputeChecksum(inout headers hdr, inout metadata
control MyIngress(inout headers hdr, inout metadata meta, meta) { apply { } }
inout standard_metadata_t standard_metadata) {
action set_egress_spec(bit<9> port) { control MyDeparser(packet_out packet, in headers hdr) {
standard_metadata.egress_spec = port; apply { }
} }
table forward {
key = { standard_metadata.ingress_port: exact; } V1Switch( MyParser(), MyVerifyChecksum(), MyIngress(),
actions = { MyEgress(), MyComputeChecksum(), MyDeparser() ) main;
set_egress_spec;
NoAction; Key Action ID Action Data
}
size = 1024; 1 set_egress_spec ID 2
default_action = NoAction();
}
apply { [Link](); } 2 set_egress_spec ID 1
}
Copyright © 2018 – [Link] 28
Running Example: Basic Forwarding
•We’ll use a simple application as a running example—a basic
router—to illustrate the main features of P416

•Basic router functionality:

◦Parse Ethernet and IPv4 headers from packet
◦Find destination in IPv4 routing table
◦Update source / destination MAC addresses
◦Decrement time-to-live (TTL) field
◦Set the egress port
◦Deparse headers back into a packet

•We’ve written some starter code for you (basic.p4) and

implemented a static control plane
Copyright © 2018 – [Link] 29
Basic Forwarding: Topology

h1 h2
([Link]) ([Link])
s1 s2

s3

h3
([Link])
Copyright © 2018 – [Link] 30
P416 Types (Basic and Header Types)
typedef bit<48> macAddr_t; Basic Types
typedef bit<32> ip4Addr_t;
header ethernet_t { • bit<n>: Unsigned integer (bitstring) of size n
macAddr_t r; dstAdd • bit is the same as bit<1>
macAddr_t srcAddr;
• int<n>: Signed integer of size n (>=2)
bit<16> etherType;
} • varbit<n>: Variable-length bitstring
header ipv4_t {
bit<4> version;
Header Types: Ordered collection of members
bit<4> ihl;
bit<8> diffserv; • Can contain bit<n>, int<n>, and varbit<n>
bit<16> totalLen; • Byte-aligned
bit<16> identification;
bit<3> flags;
• Can be valid or invalid
bit<13> fragOffset; • Provides several operations to test and set validity bit:
bit<8> ttl; isValid(), setValid(), and setInvalid()
bit<8> protocol;
bit<16> hdrChecksum;
ip4Addr_t srcAddr; Typedef: Alternative name for a type
ip4Addr_t dstAddr;
}

Copyright © 2018 – [Link] 31

P416 Types (Other Types)
/* Architecture */
struct standard_metadata_t {
Other useful types
bit<9> ingress_port;
bit<9> egress_spec; • Struct: Unordered collection of members (with no
bit<9> egress_port;
bit<32> clone_spec; alignment restrictions)
bit<32> instance_type;
bit<1> drop; • Header Stack: array of headers
bit<16> recirculate_port;
bit<32> packet_length; • Header Union: one of several headers
...
}

/* User program */
struct metadata {
...
}
struct headers {
ethernet_t ethernet;
ipv4_t ipv4;
}

Copyright © 2018 – [Link] 32

P416 Parsers
• Parsers are functions that map packets
into headers and metadata, written in a start
state machine style
• Every parser has three predefined states
◦ start
◦ accept
◦ reject
• Other states may be defined by the
programmer
• In each state, execute zero or more
statements, and then transition to
accept reject
another state (loops are OK)

Copyright © 2018 – [Link] 33

Parsers (V1Model)
/* From core.p4 */
extern packet_in {
void extract<T>(out T hdr); MyParser
void extract<T>(out T variableSizeHeader,
in bit<32> variableFieldSizeInBits); packet_in hdr
T lookahead<T>();
void advance(in bit<32> sizeInBits);
bit<32> length();
} meta meta
/* User Program */
parser MyParser(packet_in packet,
The platform Initializes
out headers hdr, User Metadata to 0
inout metadata meta,
inout standard_metadata_t std_meta) {

state start { standard_meta

[Link]([Link]);
transition accept;
}

Copyright © 2018 – [Link] 34

Select Statement
state start { P416 has a select statement that can be
transition parse_ethernet; used to branch in a parser
}
Similar to case statements in C or Java,
state parse_ethernet { but without “fall-through behavior”—i.e.,
[Link]([Link]);
break statements are not needed
transition select([Link]) {
0x800: parse_ipv4;
default: accept;
In parsers it is often necessary to branch
} based on some of the bits just parsed
}
For example, etherType determines the
format of the rest of the packet

Match patterns can either be literals or

simple computations such as masks

Copyright © 2018 – [Link] 35

Coding Break

Copyright © 2018 – [Link] 36

P416 Controls
• Similar to C functions (without loops)

• Can declare variables, create tables, instantiate externs, etc.

• Functionality specified by code in apply statement

• Represent all kinds of processing that are expressible as DAG:

◦ Match-Action Pipelines
◦ Deparsers
◦ Additional forms of packet processing (updating checksums)

• Interfaces with other blocks are governed by user- and

architecture-specified types (typically headers and metadata)
Copyright © 2018 – [Link] 37
Example: Reflector (V1Model)
control MyIngress(inout headers hdr, Desired Behavior:
inout metadata meta,
inout standard_metadata_t std_meta) {
/* Declarations region */ • Swap source and
bit<48> tmp; destination MAC
apply { addresses
/* Control Flow */
tmp = [Link];
[Link] = [Link];
• Bounce the packet back
[Link] = tmp; out on the physical port
std_meta.egress_spec = std_meta.ingress_port; that it came into the
}
}
switch on

Copyright © 2018 – [Link]

Example: Simple Actions
control MyIngress(inout headers hdr, • Very similar to C functions
inout metadata meta, • Can be declared inside a control or
inout standard_metadata_t std_meta) { globally
• Parameters have type and direction
action swap_mac(inout bit<48> src,
inout bit<48> dst) {
• Variables can be instantiated inside
bit<48> tmp = src; • Many standard arithmetic and
src = dst; logical operations are supported
dst = tmp; ◦ +, -, *
} ◦ ~, &, |, ^, >>, <<
◦ ==, !=, >, >=, <, <=
apply { ◦ No division/modulo
swap_mac([Link], • Non-standard operations:
[Link]); ◦ Bit-slicing: [m:l] (works as l-value too)
std_meta.egress_spec = std_meta.ingress_port; ◦ Bit Concatenation: ++
}
}

Copyright © 2018 – [Link]

P416 Tables
• The fundamental unit of a Match-Action Pipeline
◦ Specifies what data to match on and match kind
◦ Specifies a list of possible actions
◦ Optionally specifies a number of table properties
■ Size
■ Default action
■ Static entries
■ etc.
• Each table contains one or more entries (rules)
• An entry contains:
◦ A specific key to match on
◦ A single action that is executed when a packet matches the entry
◦ Action data (possibly empty)

Copyright © 2018 – [Link] 40

Tables: Match-Action Processing
Control Plane

(DataPlane)
Parameters

Directional
Headers and Metadata
Lookup Key

Hit
Key Action ID Action Data
Action

ID
Action ID
Code

Selector
Hit/Miss

(Action Data)
Directionless

Parameters
Data

Headers and Headers and

Metadata
Default Default
Action Metadata
(Input) Action ID Action Data
Execution (Output)

Copyright © 2018 – [Link]

Example: IPv4_LPM Table
• Data Plane (P4) Program
◦ Defines the format of the table
[Link] [Link] ■ Key Fields
■ Actions
■ Action Data
◦ Performs the lookup
◦ Executes the chosen action
• Control Plane (IP stack,
Key Action Action Data Routing protocols)
[Link]/32 ipv4_forward dstAddr=[Link] ◦ Populates table entries with
port=1 specific information
[Link]/32 drop ■ Based on the configuration
*` NoAction ■ Based on automatic discovery
■ Based on protocol calculations

Copyright © 2018 – [Link] 42

IPv4_LPM Table
table ipv4_lpm {
key = {
[Link]: lpm;
}
actions = {
ipv4_forward;
drop;
NoAction;
}
size = 1024;
default_action = NoAction();
}

Copyright © 2018 – [Link] 43

Match Kinds
/* core.p4 */ • The type match_kind is special in P4
match_kind {
exact, • The standard library (core.p4) defines
ternary, three standard match kinds
lpm ◦ Exact match
} ◦ Ternary match
◦ LPM match
/* v1model.p4 */
match_kind { • The architecture (v1model.p4) defines
range, two additional match kinds:
selector ◦ range
} ◦ selector

/* Some other architecture */ • Other architectures may define (and

match_kind { provide implementation for) additional
regexp, match kinds
fuzzy
}
Copyright © 2018 – [Link] 44
Defining Actions for L3 forwarding
/* core.p4 */ • Actions can have two different

(DataPlane)
Parameters

Directional
action NoAction() { types of parameters
} ◦ Directional (from the Data Plane)
◦ Directionless (from the Control
/* basic.p4 */ Plane)
action drop() { • Actions that are called directly:
mark_to_drop(); ◦ Only use directional parameters Action
} • Actions used in tables: Code

◦ Typically use directionless

/* basic.p4 */ parameters

(Action Data)
Directionless

Parameters
action ipv4_forward(macAddr_t dstAddr, ◦ May sometimes use directional
bit<9> port) { parameters too
...
}
Action
Execution

Copyright © 2018 – [Link] 45

Applying Tables in Controls

control MyIngress(inout headers hdr,

inout metadata meta,
inout standard_metadata_t standard_metadata) {
table ipv4_lpm {
...
}
apply {
...
ipv4_lpm.apply();
...
}
}

Copyright © 2018 – [Link] 46

P416 Deparsing
/* From core.p4 */ • Assembles the headers back
extern packet_out {
void emit<T>(in T hdr); into a well-formed packet
}
• Expressed as a control function
/* User Program */
control DeparserImpl(packet_out packet, ◦ No need for another construct!
in headers hdr) {
apply {
... • packet_out extern is defined in
[Link]([Link]); core.p4: emit(hdr): serializes
... header if it is valid
}
}
• Advantages:
• Makes deparsing explicit...
...but decouples from parsing
Copyright © 2018 – [Link] 47
Coding Break

Copyright © 2018 – [Link] 48

Basic Tunneling
•Add support for basic tunneling to the basic IP router

•Define a new header type (myTunnel) to encapsulate the IP packet

•myTunnel header includes:

○ proto_id : type of packet being encapsulated
○ dst_id : ID of destination host

•Modify the switch to perform routing using the myTunnel header

Copyright © 2018 – [Link] 49

Basic Tunneling TODO List
• Define myTunnel_t header type and add to headers struct
• Update parser
• Define myTunnel_forward action
• Define myTunnel_exact table
• Update table application logic in MyIngress apply statement
• Update deparser
• Adding forwarding rules

Copyright © 2018 – [Link] 50

Basic Forwarding: Topology

h1 h2
1 1
([Link]) 2 2 ([Link])
(dst_id: 1) s1 3 3 s2 (dst_id: 2)

2 3

1
s3

h3
([Link]) (dst_id: 3)
Copyright © 2018 – [Link] 51
Coding Break

Copyright © 2018 – [Link] 52

FAQs
• Can I apply a table multiple times in my P4 Program?
◦ No (except via resubmit / recirculate)
• Can I modify table entries from my P4 Program?
◦ No (except for direct counters)
• What happens upon reaching the reject state of the parser?
◦ Architecture dependent
• How much of the packet can I parse?
◦ Architecture dependent

Copyright © 2018 – [Link] 53

Fin!

Copyright © 2018 – [Link] 54

P4→NetFPGA
• Prototype and evaluate P4 programs in real hardware!
• 4x10G network interfaces
• Special price for academic users :)
• [Link]

Copyright © 2018 – [Link] 55

Debugging
• Bmv2 maintains logs that keep track of
control MyIngress(...) { how packets are processed in detail
table debug { • /tmp/[Link]
key = { • /tmp/[Link]
std_meta.egress_spec : exact; • /tmp/[Link]
}
actions = { } • Can manually add information to the
}
logs by using a dummy debug table that
apply {
reads headers and metadata of interest
...
[Link]();
} •[Link].145] [bmv2] [D]
} [thread 4090] [96.0] [cxt 0]
Looking up key:
* std_meta.egress_spec : 2

Copyright © 2018 – [Link] 56

Questions and Answers

[Link]
Join #d2-2018-spring channel
} Questions for TAs

[Link]
Event code: P4D2
Copyright © 2018 – [Link]
} Questions for Panel

57
Lab 2: P4Runtime

58
Copyright © 2018 – [Link], ONF
P4 Software Tools

59
Copyright © 2018 – [Link], ONF
Makefile: under the hood
P4Runtime static controller
[Link] P4Runtime client
test.p4

P4Runtime server
[Link]
[Link]
p4c-bm2-ss L

simple_switch_grpc (BMv2)
[Link] o
TM g

Egress
Ingress
P4
D Debugger
[Link] test.p4info e
b
u
g

Parser Deparser
Packet Packet
generator Port Interface sniffer
veth0..n
Linux Kernel
60
Copyright © 2018 – [Link], ONF
Makefile: under the hood (in pseudocode)
P4C_ARGS = --p4runtime-file $(basename $@).p4info
--p4runtime-format text
RUN_SCRIPT = ../../utils/run_exercise.py
TOPO = [Link]
dirs:
mkdir -p build pcaps logs
build: for each P4 program, generate BMv2 json file
p4c-bm2-ss --p4v 16 $(P4C_ARGS) -o $@ $<
run: build, then [default target]
sudo python $(RUN_SCRIPT) -t $(TOPO)
stop: sudo mn -c
clean: stop, then
rm -f *.pcap
rm -rf build pcaps logs
61
Copyright © 2018 – [Link], ONF
Step 1: P4 Program compilation [build phase]

test.p4 $ p4c-bm2-ss --p4v 16 \

-o [Link] \
--p4runtime-file test.p4info \
--p4runtime-format text \
p4c-bm2-ss
test.p4

[Link] test.p4info

62
Copyright © 2018 – [Link], ONF
Step 2: run_exercise.py [run phase]
P4Runtime static controller
a. Create network based [Link]
P4Runtime client (gRPC)
on [Link]
b. Start
simple_switch_grpc
P4Runtime server (gRPC)
● P4Runtime server
address and port
instance for each L

simple_switch_grpc (BMv2)
TM o ● Path to BMv2 JSON
g
switch ●
and P4Info to deploy
Table entries
c. Use P4Runtime to

Egress
Ingress
D
push the P4 program e
b
u
(P4Info and BMv2 g

JSON) Parser Deparser

d. Add the static rules Port Interface

defined in [Link] veth0..n
Linux Kernel
63
Copyright © 2018 – [Link], ONF
Step 3: Run the traffic generator and sniffer
P4Runtime static controller
In some exercises, this is
P4Runtime client
[Link] and [Link]
P4Runtime server
In others, we use standard
Linux programs, like ping
L

simple_switch_grpc (BMv2)
o
TM g

Egress
Ingress
In the p4runtime exercise, D
P4
Debugger
e
we also run our own controller: b
u

[Link] instead of
g

Parser Deparser
the static one Packet Packet
generator Port Interface sniffer
veth0..n
Linux Kernel
64
Copyright © 2018 – [Link], ONF
P4Runtime
• API overview
• Workflow
• Exercise - Tunneling
• Why P4Runtime?

65
Copyright © 2018 – [Link], ONF
Runtime control of P4 data planes
User supplied

Control Plane
P4 Program P4 Compiler Add/remove Extern Packet-in/out
table entries control

CPU port
P4 Architecture Target-specific Extern
configuration Load Tables Data Plane
Model objects
binary

Vendor supplied Focus of this

session
66
Copyright © 2018 – [Link], ONF
Existing approaches to runtime control
• P4 compiler auto-generated runtime APIs
◦ Program-dependent -- hard to provision new P4 program without restarting
the control plane!

• BMv2 CLI
◦ Program-independent, but target-specific -- control plane not portable!

• OpenFlow
◦ Target-independent, but protocol-dependent -- protocol headers and
actions baked in the specification!

• OCP Switch Abstraction Interface (SAI)

◦ Target-independent, but protocol-dependent
67
Copyright © 2018 – [Link], ONF
Why do we need another data plane control API?

68
Copyright © 2018 – [Link], ONF
Properties of a runtime control API

API Target-independent Protocol-independent

P4 compiler
auto-generated ✔ ✘
BMv2 CLI ✘ ✔
OpenFlow ✔ ✘
SAI ✔ ✘
P4Runtime ✔ ✔

69
Copyright © 2018 – [Link], ONF
What is P4Runtime?
• Framework for runtime control of P4 targets
◦ Open-source API + server implementation
■ [Link]
◦ Initial contribution by Google and Barefoot Control plane

• Work-in-progress by the [Link] API WG

[Link]
◦ Draft of version 1.0 available (API)

• Protobuf-based API definition

◦ [Link]
◦ gRPC transport Program-independent
server (e.g. gRPC)

• P4 program-independent Target driver

◦ API doesn’t change with the P4 program

• Enables field-reconfigurability P4 target

◦ Ability to push new P4 program without recompiling the
software stack of target switches
70
Copyright © 2018 – [Link], ONF
Protocol Buffers Basics
• Language for describing data for syntax = "proto3";
serialization in a structured way message Person {
string name = 1;
• Common binary wire-format int32 id = 2;
string email = 3;
• Language-neutral
enum PhoneType {
◦ Code generators for: Action Script, C, MOBILE = 0;
C++, C#, Clojure, Lisp, D, Dart, Erlang, HOME = 1;
Go, Haskell, Java, Javascript, Lua, WORK = 2;
}
Objective C, OCaml, Perl, PHP, Python,
Ruby, Rust, Scala, Swift, Visual Basic, ... message PhoneNumber {
string number = 1;
• Platform-neutral PhoneType type = 2;
}
• Extensible and backwards compatible repeated PhoneNumber phone = 4;
}
• Strongly typed

71
Copyright © 2018 – [Link], ONF
gRPC Basics
● Use Protocol Buffers to define service API and messages
● Automatically generate native stubs in:
○ C / C++
○ C#
○ Dart
○ Go
○ Java
○ [Link]
○ PHP
○ Python
○ Ruby
● Transport over HTTP/2.0 and TLS
○ Efficient single TCP connection implementation that supports
bidirectional streaming
72
Copyright © 2018 – [Link], ONF
gRPC Service Example
// The greeter service definition.
service Greeter {
// Sends a greeting
rpc SayHello (HelloRequest) returns (HelloReply) {}
}

// The request message containing the user's name.

message HelloRequest {
string name = 1;
}

// The response message containing the greetings

message HelloReply {
string message = 1;
}

More details here: [Link]

73
Copyright © 2018 – [Link], ONF
P4Runtime Service
Enables a local or remote entity to load the pipeline/program,
send/receive packets, and read and write forwarding table entries,
counters, and other chip features.
service P4Runtime {
rpc Write(WriteRequest) returns (WriteResponse) {}
rpc Read(ReadRequest) returns (stream ReadResponse) {}
rpc SetForwardingPipelineConfig(SetForwardingPipelineConfigRequest)
returns (SetForwardingPipelineConfigResponse) {}
rpc GetForwardingPipelineConfig(GetForwardingPipelineConfigRequest)
returns (GetForwardingPipelineConfigResponse) {}
rpc StreamChannel(stream StreamMessageRequest)
returns (stream StreamMessageResponse) {}
}

74
Copyright © 2018 – [Link], ONF
P4Runtime Service

Protobuf Definition:
[Link]

Service Specification:
Working draft of version 1.0 is available now
[Link]

75
Copyright © 2018 – [Link], ONF
P4Runtime Write Request
message WriteRequest { message Entity {
uint64 device_id = 1; oneof entity {
uint64 role_id = 2; ExternEntry extern_entry = 1;
Uint128 election_id = 3; TableEntry table_entry = 2;
repeated Update updates = 4; ActionProfileMember
} action_profile_member = 3;
ActionProfileGroup
action_profile_group = 4;
message Update { MeterEntry meter_entry = 5;
enum Type { DirectMeterEntry direct_meter_entry = 6;
UNSPECIFIED = 0; CounterEntry counter_entry = 7;
INSERT = 1; DirectCounterEntry direct_counter_entry = 8;
MODIFY = 2; PacketReplicationEngineEntry
DELETE = 3; packet_replication_engine_entry = 9;
} ValueSetEntry value_set_entry = 10;
Type type = 1; RegisterEntry register_entry = 11;
Entity entity = 2; }
} }

76
Copyright © 2018 – [Link], ONF
P4Runtime Table Entry
[Link] simplified excerpts:
message TableEntry { message FieldMatch {
To add a table entry, the control
uint32 table_id; uint32 field_id; plane needs to know:
repeated FieldMatch match; message Exact {
Action action; bytes value;
int32 priority; } • IDs of P4 entities
... message Ternary { ◦ Tables, field matches, actions,
} bytes value; params, etc.
bytes mask;
}
message Action { ... • Field matches for the
uint32 action_id; oneof field_match_type { particular table
message Param { Exact exact;
◦ Match type, bitwidth, etc.
uint32 param_id; Ternary ternary;
bytes value; ...
}
} • Parameters for the particular
}
repeated Param params; action
}

Full protobuf definition: • Other P4 program attributes

[Link]

77
Copyright © 2018 – [Link], ONF
P4Runtime workflow
P4Info test.p4

• Captures P4 program attributes

p4c-bm2-ss
needed at runtime (compiler)
Control plane
◦ IDs for tables, actions, params, etc.
◦ Table structure, action parameters, etc.

test.p4info
[Link]
• Protobuf-based format

P4Runtime server
• Target-independent compiler output [Link]
BMv2 driver
◦ Same P4Info for BMv2, ASIC, etc.
simple_switch

Full P4Info protobuf specification:

BMv2
[Link]
78
Copyright © 2018 – [Link], ONF
P4Info example
basic_router.p4 basic_router.p4info
... actions {
id: 16786453
action ipv4_forward(bit<48> dstAddr, name: "ipv4_forward"
bit<9> port) { params {
/* Action implementation */ id: 1
} name: "dstAddr"
bitwidth: 48
... ...
id: 2
table ipv4_lpm { name: "port"
key = { bitwidth: 9
[Link]: lpm; }
} }
actions = { ...
ipv4_forward; tables {
... P4 compiler id: 33581985
} name: "ipv4_lpm"
... match_fields {
} id: 1
name: "[Link]"
bitwidth: 32
match_type: LPM
}
action_ref_id: 16786453
}
79
Copyright © 2018 – [Link], ONF
P4Runtime Table Entry Example
basic_router.p4 Protobuf message
action ipv4_forward(bit<48> dstAddr, table_entry {
bit<9> port) { table_id: 33581985
/* Action implementation */ match {
} field_id: 1
table ipv4_lpm { lpm {
key = { value: "\n\000\001\001"
[Link]: lpm; Control plane prefix_len: 32
} generates }
actions = { }
ipv4_forward; action {
... action_id: 16786453
} params {
... param_id: 1
} value: "\000\000\000\000\000\n"
}
params {
param_id: 2
Logical view of table entry value: "\000\007"
}
[Link]=[Link]/32 }
-> ipv4_forward([Link], 7) }

80
Copyright © 2018 – [Link], ONF
P4Runtime SetPipelineConfig
test.
message SetForwardingPipelineConfigRequest {
p4
enum Action {
UNSPECIFIED = 0; Pipeline config
VERIFY = 1; p4info json Control plane
VERIFY_AND_SAVE = 2;
VERIFY_AND_COMMIT = 3; p4c-bm2-ss SetPipelineConfig()
COMMIT = 4; (compiler)
Pipeline config bits
RECONCILE_AND_COMMIT = 5;
} P4Runtime server
uint64 device_id = 1;
BMv2 driver
uint64 role_id = 2;
Uint128 election_id = 3; BMv2 simple_switch_grpc
Action action = 4;
ForwardingPipelineConfig config = 5;
} message ForwardingPipelineConfig {
config.P4Info p4info = 1;
// Target-specific P4 configuration.
bytes p4_device_config = 2;
}
81
Copyright © 2018 – [Link], ONF
P4Runtime StreamChannel
// Packet sent from the controller to the switch.
message StreamMessageRequest { message PacketOut {
oneof update { bytes payload = 1;
MasterArbitrationUpdate // This will be based on P4 header annotated as
arbitration = 1; // @controller_header("packet_out").
PacketOut packet = 2; // At most one P4 header can have this annotation.
} repeated PacketMetadata metadata = 2;
} }

// Packet sent from the switch to the controller.

message StreamMessageResponse { message PacketIn {
oneof update { bytes payload = 1;
MasterArbitrationUpdate // This will be based on P4 header annotated as
arbitration = 1; // @controller_header("packet_in").
PacketIn packet = 2; // At most one P4 header can have this annotation.
} repeated PacketMetadata metadata = 2;
} }

82
Copyright © 2018 – [Link], ONF
P4Runtime Common Parameters
• device_id
◦ Specifies the specific forwarding chip or software bridge
◦ Set to 0 for single chip platforms
• role_id
◦ Corresponds to a role with specific capabilities (i.e. what operations, P4
entities, behaviors, etc. are in the scope of a given role)
◦ Role definition is currently agreed upon between control and data planes
offline
◦ Default role_id (0) has full pipeline access
• election_id
◦ P4Runtime supports mastership on a per-role basis
◦ Client with the highest election ID is referred to as the "master", while all
other clients are referred to as "slaves"
◦ Set to 0 for single instance controllers
83
Copyright © 2018 – [Link], ONF
Mastership Arbitration
• Upon connecting to the device, the client (e.g. controller)
needs to open a StreamChannel
• The client must advertise its role_id and election_id using a
MasterArbitrationUpdate message
◦ If role_id is not set, it implies the default role and will be granted full
pipeline access
◦ The election_id is opaque to the server and determined by the control
plane (can be omitted for single-instance control plane)
• The switch marks the client for each role with the highest
election_id as master
• Master can:
◦ Perform Write requests
◦ Receive PacketIn messages
◦ Send PacketOut messages 84
Copyright © 2018 – [Link], ONF
Remote control
table_entry {
table_id: 33581985
Target-independent P4-defined
match { protobuf format OSPF BGP etc.
field_id: 1
lpm {
custom protocol
value: "\f\000\...
prefix_len: 8
p4info
}
}
action {
Remote control plane
action_id: 16786453
params {
param_id: 1
value: "\000\0...
}
params {
param_id: 2
value: 7
}
}
}

p4info p4info p4info

P4Runtime P4Runtime P4Runtime
control server control server control server
Target driver Target driver Target driver

Vendor A Vendor B Vendor C

85
Copyright © 2018 – [Link], ONF
Local control
table_entry {
Same table_id: 33581985
match {
P4-defined
etc.
target-independent field_id: 1
OSPF BGP lpm {
protocol protobuf format value: "\f\000\...
prefix_len: 8
p4info }
Local control plane }
action {
action_id: 16786453
params {
param_id: 1
p4info value: "\000\0...
P4Runtime }
params {
control server param_id: 2
value: 7
}
}
Target driver }

P4 target

P4Runtime can be used equally well

by a remote or local control plane 86
Copyright © 2018 – [Link], ONF
P4Runtime API recap
Things we covered: What we didn’t cover:
• P4Runtime definition • How to control other P4 entities
• P4Info ◦ Externs, counters, meters
• Table entries • Batched reads/writes
• Set pipeline config • Switch configuration
• Packet-in/out support ◦ Outside the P4Runtime scope
◦ Achieved with other mechanisms
• Controller replication ■ e.g., OpenConfig and gNMI
◦Via master-slave arbitration

87
Copyright © 2018 – [Link], ONF
P4Runtime exercise

88
Copyright © 2018 – [Link], ONF
Exercise Overview
Controller’s responsibilities:
1. Establish a gRPC connection to the
switches for the P4Runtime service [Link]
2. Push the P4 program to each switch
3. Write the tunnel forwarding rules:
a. myTunnel_ingress rule to
encapsulate packets on the ingress
switch
b. myTunnel_forward rule to
forward packets on the ingress
switch
c. myTunnel_egress rule to
decapsulate and forward packets
on the egress switch
4. Read the tunnel ingress and egress
counters every 2 seconds

89
Copyright © 2018 – [Link], ONF
Getting started
The source code has already been downloaded on your VM:
~/tutorials/exercises/p4runtime

You should start by reading the [Link]

In this exercise, you will need to complete the

implementation of writeTunnelRules
in [Link]

You will need two Terminal windows: one for

your dataplane network (Mininet) that you will
start using make, and the other is for your
controller program.

To find the source code:

[Link]

90
Copyright © 2018 – [Link], ONF
P4Runtime in the wild

91
Copyright © 2018 – [Link], ONF
P4Runtime-enabled Open Source SDN Stack
Inventory Global Orchestrator OSS / BSS

dhcp SR
Control and Management Plane
Monitoring & Admin &
SDN Control Configuration
Telemetry Orchestration
Services Services
Services Services

P4Runtime OpenConfig
gNOI
& gNMI
spine.p4 spine.p4
Stratum Stratum

leaf.p4 leaf.p4 leaf.p4

Stratum Stratum Stratum
92
Copyright © 2018 – [Link], ONF
Some Pointers
[Link]

[Link]

[Link]

93
Copyright © 2018 – [Link], ONF
Lab 3: Monitoring & Debugging

Copyright © 2018 – [Link]

Monitoring & Debugging

Congestion!
Fine … h2
h1
([Link]) ([Link])

h11 h22
([Link]) s1 s2 ([Link])

Hosts can infer

congestion: Switches can
● RTT help!
● Packet drops s3
● Dup Acks

h3
([Link])

Copyright © 2018 – [Link] 95

Monitoring & Debugging

s1
Copyright © 2018 – [Link] 96
Explicit Congestion Notification

TOS

Copyright © 2018 – [Link] 97

Explicit Congestion Notification
•Explicit Congestion Notification
◦00: Non ECN-Capable Transport, Non-ECT
◦10: ECN Capable Transport, ECT(0)
◦01: ECN Capable Transport, ECT(1)
◦11: Congestion Encountered, CE

•For packets originating from ECT, ECN-capable switches set the

CE bit upon congestion
◦E.g., observed queue depth > threshold

Copyright © 2018 – [Link] 98

Explicit Congestion Notification in P4
•The standard data for the V1Model includes the queue depth:
bit<19> standard_metadata.enq_qdepth

Measured here Available here

Traffic
Manager

Ingress Egress

Copyright © 2018 – [Link] 99

Coding Break

Copyright © 2018 – [Link] 100

Multi-Route Inspection
Switch ID, queue h2
h1 depth, … (anything)
([Link])
([Link]) pkt

s1 s2
h22
h11 ([Link])
([Link])

The most verbose ECN s3

Packets carry their own logs! h3
([Link])

Copyright © 2018 – [Link] 101

Multi-Route Inspect: Packet Format
header mri_t {
• Header validity opertions:
bit<16> count;
o [Link](): add_header
}
o [Link](): remove_header
o [Link]():test validity
header switch_t {
switchID_t swid;
• Header Stacks
qdepth_t qdepth;
o hdr[CNT] stk;
}
• Header Stacks in Parsers
o [Link]
struct headers {
o [Link]
ethernet_t ethernet;
o [Link]
ipv4_t ipv4; • Header Stacks in Controls
ipv4_option_t ipv4_option; o stk[i]
mri_t mri; o [Link]
switch_t[MAX_HOPS] swtraces; o stk.push_front(int count)
} o stk.pop_front(int count)

Copyright © 2018 – [Link] 102

Header verification
/* Standard errors, defined in core.p4 */
error {
NoError, // no error
PacketTooShort, // not enough bits in packet for extract
NoMatch, // match expression has no matches
StackOutOfBounds, // reference to invalid element of a header stack
OverwritingHeader, // one header is extracted twice
HeaderTooShort, // extracting too many bits in a varbit field
ParserTimeout // parser execution time limit exceeded
}
/* Additional error added by the programmer */
error { IPv4BadHeader }
...
state parse_ipv4 {
[Link](hdr.ipv4);
verify([Link] == 4, error.IPv4BadHeader);
transition accept;
}

Copyright © 2018 – [Link] 103

Coding Break

Copyright © 2018 – [Link] 104

Questions and Answers

[Link]
Join #d2-2018-spring channel
} Questions for TAs

[Link]
Event code: P4D2
Copyright © 2018 – [Link]
} Questions for Panel

105
Lab 4: Advanced Behavior

Copyright © 2018 – [Link] 106

Source Routing
2,3,2,2,1, {payload}

h1 2 2 1 h2
([Link]) 1
([Link])
s1 3 3 s2

2 3

1 s3

107
h3
([Link])
Copyright © 2018 – [Link]
Source Routing: Packet Format
#define MAX_HOPS 9 • Parse source routes only if etherType is
0x1234
const bit<16> TYPE_IPV4 = 0x800;
const bit<16> TYPE_SRCROUTING = 0x1234;
header srcRoute_t { • The special value bos == 1 indicates the
bit<1> bos; “bottom of stack”
bit<15> port;
}
• Forward packets using source routes,
struct headers { and also decrement IPv4 TTL
ethernet_t ethernet;
srcRoute_t[MAX_HOPS] srcRoutes; • Drop the packet if source routes are not
ipv4_t ipv4; valid
}

• Hint: Use the next, pop_front primitives

[Link]([Link])
[Link].pop_front(1)
Copyright © 2018 – [Link] 108
Coding Break

Copyright © 2018 – [Link] 109

Calculator

2+4=?

h1 s1
([Link]) 2+4=6

Copyright © 2018 – [Link] 110

Calculator: Packet Format
0 1 2 3
+----------------+----------------+----------------+---------------+
| P | 4 | Version | Op |
+----------------+----------------+----------------+---------------+
| Operand A |
+----------------+----------------+----------------+---------------+
| Operand B |
+----------------+----------------+----------------+---------------+
| Result |
+----------------+----------------+----------------+---------------+

Copyright © 2018 – [Link] 111

Table Initializers
table tbl { Can initialize tables with constant
key = { hdr.h.f : exact } entries
actions = { a1; a2; a3 }
entries = { Must fully specify the value of all
{ 0x01 } : a1(1); action data, including values that are
{ 0x02 } : a1(2); normally supplied by the
control-plane
{ _ } : NoAction();
}
Hint: for the calculator, use a table
}
that matches on the op-code

Copyright © 2018 – [Link] 112

Coding Break

Copyright © 2018 – [Link] 113

Simple Load Balancing
h1 → [Link]

h1 h2
([Link]) ([Link])
s1 s2

Virtual IP ([Link])
shared by h2 and h3

s3

h3
([Link])
Copyright © 2018 – [Link] 114
Hashing (V1Model)
enum HashAlgorithm {
csum16,
Computes the hash of data
xor16, (using algo) modulo max
crc32,
crc32_custom, and adds it to base
crc16,
crc16_custom,
random, Uses type variables (like
identity
}
C++ templates / Java
extern void hash<O, T, D, M>( Generics) to allow hashing
out O result,
in HashAlgorithm algo, primitive to be used with
in T base, many different types.
in D data,
in M max);

Copyright © 2018 – [Link] 115

Wrapping up & Next Steps

Copyright © 2018 – [Link] 116

Why P416?
• Clearly defined semantics
◦ You can describe what your data plane program is doing
• Expressive
◦ Supports a wide range of architectures through standard methodology
• High-level, Target-independent
◦ Uses conventional constructs
◦ Compiler manages the resources and deals with the hardware
• Type-safe
◦ Enforces good software design practices and eliminates “stupid” bugs
• Agility
◦ High-speed networking devices become as flexible as any software
• Insight
◦ Freely mixing packet headers and intermediate results

Copyright © 2018 – [Link] 117

Things we covered
• The P4 ”world view”
◦ Protocol-Independent Packet Processing
◦ Language/Architecture Separation
◦ If you can interface with it, it can be used
• Key data types
• Constructs for packet parsing
◦ State machine-style programming
• Constructs for packet processing
◦ Actions, tables and controls
• Packet deparsing
• Architectures & Programs

Copyright © 2018 – [Link] 118

Things we didn’t cover
• Mechanisms for modularity
◦ Instantiating and invoking parsers or controls
• Details of variable-length field processing
◦ Parsing and deparsing of options and TLVs
• Architecture definition constructs
◦ How these “templated” definitions are created
• Advanced features
◦ How to do learning, multicast, cloning, resubmitting
◦ Header unions
• Other architectures
• Control plane interface

Copyright © 2018 – [Link] 119

 The P4 Language Consortium

• Consortium of academic
and industry members

• Open source, evolving,

domain-specific language

• Permissive Apache license,

code on GitHub today

• Membership is free:
contributions are welcome

• Independent, set up as a
California nonprofit

Copyright © 2018 – [Link] 120