DMZ Exercises
TCP Congestion Control, Buffer Sizing
Elie Kfoury, Jorge Crichigno
University of South Carolina
2021 NSF Campus Cyberinfrastructure (CC*) Workshop
April 15, 2021
1
Content
• NTP lab series
• Organization of lab manuals
• Exercise 1: TCP congestion control
• Exercise 2: Buffer sizing
DMZ exercises 2
NTP Lab Series
• The lab series provides learners an emulated WAN infrastructure operating at high
speeds and devices running real protocol stacks
• It helps students to acquire hands-on skills on
Performance and measurement tools
Configuration of devices for high-speed networks
Emulate scenarios using real protocol stacks
DMZ exercises 3
NTP Lab Series
• The lab series can be partitioned into three parts
Measurement (throughput, latency, packet loss) and emulation (link bandwidth, buffer size, delay)
tools
TCP features for high-speed transfers, router buffer size
Active Queue Management (AQM) algorithms
DMZ exercises 4
NTP Lab Series
• Lab experiments
Lab 1: Introduction to Mininet Lab 11: Router’s Buffer Size
Lab 2: Introduction to iPerf Lab 12: TCP Rate Control with Pacing
Lab 3: WANs with latency, Jitter Lab 13: Impact of Maximum Segment Size on Throughput
Lab 4: WANs with Packet Loss, Duplication, Corruption Lab 14: Router’s Bufferbloat
Lab 5: Setting WAN Bandwidth with Token Bucket Filter (TBF) Lab 15: Hardware Offloading on TCP Performance
Lab 6: Traditional TCP Congestion Control (HTCP, Cubic, Reno) Lab 16: Random Early Detection
Lab 7: Rate-based TCP Congestion Control (BBR) Lab 17: Stochastic Fair Queueing
Lab 8: Bandwidth-delay Product and TCP Buffer Size Lab 18: Controlled Delay (CoDel) Active Queue Management
Lab 9: Enhancing TCP Throughput with Parallel Streams Lab 19: Proportional Integral Controller-Enhanced (PIE)
Lab 10: Measuring TCP Fairness Lab 20: Classifying TCP traffic using Hierarchical Token Bucket (HTB)
DMZ exercises 5
Organization of Lab Manuals
• Each lab starts with a section Overview
Objectives
Lab settings: passwords, device names
Roadmap: organization of the lab
• Section 1
Background information of the topic being covered (e.g., fundamentals of TCP congestion control)
Section 1 is optional (i.e., the reader can skip this section and move to lab directions)
• Section 2… n
Step-by-step directions
DMZ exercises 6
Exercise 1: TCP Congestion
Control
TCP Congestion Control 7
TCP Traditional Congestion Control
• The principles of window-based CC were described in the 1980s 1
• Traditional CC algorithms follow the additive-increase multiplicative-decrease (AIMD)
form of congestion control
Sender Receiver
Seq
= 11
0, 10
Seq bytes
= 12 Packet loss
0, 10
Seq bytes Sending rate Additive increase
= 13
0, 10 Multiplicative decrease
bytes
Out-of-order
Seq
segments
= 14
0, 10
0 bytes
Triple duplicate ACK
= 11
A ck
0
= 11
A ck
0 Time
= 11
A ck
Seq
= 11
0, 10 b
ytes
Time
1. V. Jacobson, M. Karels, Congestion avoidance and control, ACM SIGCOMM Computer Communication Review 18 (4) (1988).
TCP Congestion Control 8
BBR: Model-based CC
• TCP Bottleneck Bandwidth and RTT (BBR) is a rate-based congestion-control
algorithm1
• BBR represented a disruption to the traditional CC algorithms:
is not governed by AIMD control law
does not the use packet loss as a signal of congestion
• At any time, a TCP connection has one slowest link bottleneck bandwidth (btlbw)
probe
Sending rate
125
Sender Router Receiver btlbw
100
75
drain
Bottleneck
(btlbw)
Output port buffer cycle 1 cycle 2 ... Time
8 RTTs
1. N. Cardwell et al. "BBR v2, A Model-based Congestion Control." IETF 104, March 2019.
TCP Congestion Control 9
Lab Goal and Topology
• Modify the TCP congestion control algorithm in Linux using sysctl tool
• Deploy emulated WANs in Mininet
• Compare the performance of TCP Reno and TCP BBR in high-throughput high-latency
networks
• Lab topology:
TCP Congestion Control 10
TCP Buffer Size
• In many WANs, the round-trip time (RTT) is dominated by the propagation delay
• To keep the sender busy while ACKs are received, the TCP buffer must be:
Traditional congestion controls: TCP buffer size ≥ 2BDP
BBRv1 and BBRv2: TCP buffer size must be considerable larger than 2BDP
TCP Congestion Control 11
Exercise 2: Buffer Sizing
Buffer Sizing 12
Buffer Size
• The router’s buffer plays an important role in absorbing traffic fluctuations
• Buffers avoid losses by momentarily buffering packets as transitory bursts dissipate
Sender Router Receiver
Bottleneck
(btlbw)
Output port buffer
Buffer Sizing 13
Buffer Size
• The rule of thumb has been that the amount of buffering (in bits) in a router’s port
should equal the RTT (in seconds) multiplied by the capacity C (in bits per seconds) of
the port1:
Router’s buffer size = C ⋅ RTT
1. C. Villamizar, C. Song, “High performance TCP in ansnet,” ACM Computer Communications Review, vol. 24, no. 5, pp. 45-60, Oct.
1994.
Buffer Sizing 14
Buffer Size
• The rule of thumb has been that the amount of buffering (in bits) in a router’s port
should equal the RTT (in seconds) multiplied by the capacity C (in bits per seconds) of
the port1:
Router’s buffer size = C ⋅ RTT
• When there is a large number of TCP flows passing through a link, say N, the amount of
buffering can be reduced to2:
Router’s buffer size = C ⋅ RTT / √ (N)
1. C. Villamizar, C. Song, “High performance TCP in ansnet,” ACM Computer Communications Review, vol. 24, no. 5, pp. 45-60, Oct.
1994.
2. G. Appenzeller, I. Keslassy, N. McKeown, “Sizing router buffers,” in Proceedings of the 2004 conference on Applications, technologies,
architectures, and protocols for computer communications, pp. 281-292, Oct. 2004.
Buffer Sizing 15
Bufferbloat
• Bufferbloat is a condition that occurs when the router buffers too much data, leading to
excessive delays
1. N. Cardwell, Y. Cheng, C. Gunn, S. Yeganeh, V. Jacobson, “BBR: congestion-based congestion control,” Communications of the ACM,
vol 60, no. 2, pp. 58-66, Feb. 2017.
Buffer Sizing 16
Lab Goal and Topology
• Understand the buffering process in a router and buffer sizing
• Explain the concept of Bufferbloat
• Modify routers’ buffer size to solve the bufferbloat problem
• Lab topology:
Buffer Sizing 17
