2BRobust -- Overcoming TCP BBR Performance Degradation in Virtual Machines under CPU Contention

TL;DR

This work investigates the robustness of TCP BBR under CPU contention in cloud-based virtual machines, revealing extensive throughput degradation that undermines Internet robustness if BBR displaces loss-based congestion control without safeguards. The authors introduce a controlled testbed using Linux deadline scheduling to emulate CPU contention with fine granularity, enabling precise study of BBR behavior under varied network and scheduling conditions. They show that BBR variants can be severely CPU-limited, with throughput often capped below 10–20 Mbps, and propose a minimal inflight-deficit-driven patch that increases pacing during CPU contention, shifting the throughput root-mean-square CPU-share threshold to lower values. The results demonstrate that the patch improves performance in many critical cases while maintaining fairness, and highlight the need for cautious deployment of BBR in VM/cloud environments. Overall, the paper offers a practical methodology and a concrete mitigation to enhance BBR robustness in CPU-constrained virtualized networks, with implications for large-scale Internet deployments.

Abstract

Motivated by the recent introduction and large-scale deployment of BBR congestion control algorithms, multiple studies have investigated the performance and fairness implications of this shift from loss-based to delay-based congestion control. Given the potential Internet-wide adoption of BBR, we must also consider its robustness in network and system scenarios. One such scenario is Cloud-based Virtual Machine (VM) networking - highly relevant in today's CDN-centric Internet. Interestingly, previous work has shown significant performance problems of BBRv1-2 running in Xen VMs, with BBR performance dropping to almost zero when CPU credit is low. In this paper, we develop a framework for measuring TCP throughput under fully controlled CPU contention, which uses Linux deadline scheduling to emulate generalized CPU contention conditions. Our measurements reveal that - in stark contrast to Cubic! - BBR throughput can break down during CPU contention under any hypervisor and all tested BDP conditions. Characterizing this performance degradation on a fine-granular level, we show that CPU limited BBR senders are capped at very low throughput levels below 10-20 Mbps. This finding implies that an Internet-wide shift from Cubic to BBR could harm the Internet's overall robustness, if not deployed with caution. To detect and overcome CPU-limited throughput, we propose a minimal BBR patch which detects the problematic situation by monitoring inflight bytes and reacts by increasing the pacing rate to make better use of the available CPU time. We show that our BBR patch overcomes the throughput problem for the most critical cases.

Figures (15)

• Figure 1: Motivating observation: Degrading BBRv3 throughput at decreasing VM CPU shares - Cubic robust.
• Figure 2: Evaluation framework using a simple TCP Setup with link emulation, and CPU contention modeled as deadline scheduling of the sender VM's vCPU.
• Figure 3: BBR vs. Cubic throughput (medians) for varying SCHEDULING conditions and RTT. Bottleneck bandwidth is fixed to 100 Mbps.
• Figure 4: Throughput over time for BBRv1-3 compared to Cubic, with 25% CPU shares, at 100 Mbps, 10 ms RTT. Left: 1ms timeslice, right: 10ms timeslice.
• Figure 5: Pacing rate, delivery rate, and BBR bandwidth estimate of individual flow samples for different BBR versions and CPU shares.