QUIC Steps: Evaluating Pacing Strategies in QUIC Implementations

TL;DR

This paper addresses the challenge of pacing in QUIC, a user-space transport protocol, by evaluating pacing behavior across three libraries (quiche, picoquic, ngtcp2) and system features such as qdiscs, GSO, and hardware offloading. The authors develop a reproducible measurement framework with a passive sniffer and network emulation to quantify inter-packet timing and packet trains, and they compare baseline behavior to configurations using fq, GSO, and ETF with LaunchTime. A kernel patch enabling pacing within GSO buffers is introduced and evaluated, alongside a comparison of kernel-assisted and pure user-space pacing approaches. The findings show that pacing can be achieved through diverse methods—picoquic with BBR, quiche with fq, and paced GSO with kernel support—though gains depend on the combination of library, CC, and kernel features, with hardware offloading offering limited improvements in the tested scenarios. The work provides practical guidance for optimizing QUIC pacing in real-world applications like video streaming and real-time communications.

Abstract

Pacing is a key mechanism in modern transport protocols, used to regulate packet transmission timing to minimize traffic burstiness, lower latency, and reduce packet loss. Standardized in 2021, QUIC is a UDP-based protocol designed to improve upon the TCP / TLS stack. While the QUIC protocol recommends pacing, and congestion control algorithms like BBR rely on it, the user-space nature of QUIC introduces unique challenges. These challenges include coarse-grained timers, system call overhead, and OS scheduling delays, all of which complicate precise packet pacing. This paper investigates how pacing is implemented differently across QUIC stacks, including quiche, picoquic, and ngtcp2, and evaluates the impact of system-level features like GSO and Linux qdiscs on pacing. Using a custom measurement framework and a passive optical fiber tap, we establish a baseline with default settings and systematically explore the effects of qdiscs, hardware offloading using the ETF qdisc, and GSO on pacing precision and network performance. We also extend and evaluate a kernel patch to enable pacing of individual packets within GSO buffers, combining batching efficiency with precise pacing. Kernel-assisted and purely user-space pacing approaches are compared. We show that pacing with only user-space timers can work well, as demonstrated by picoquic with BBR. With quiche, we identify FQ as a qdisc well-suited for pacing QUIC traffic, as it is relatively easy to use and offers precise pacing based on packet timestamps. Our findings provide new insights into the trade-offs involved in implementing pacing in QUIC and highlight potential optimizations for real-world applications like video streaming and video calls.

Figures (7)

• Figure 1: Topology of the measurement setup with an optical fiber tap. Dashed links only carry management traffic, solid links only carry measurement traffic. Delay and rate limiting are denoted in blue and orange.
• Figure 2: Distribution of gaps between packets sent by the server during the download for the baseline measurements. All implementations use CUBIC.
• Figure 3: Distribution of packets across packet trains with specific lengths (x-axis) for the baseline. Packet trains are sequences of packets with at most 0.1ms between two packets.
• Figure 4: Comparison of the QUIC libraries picoquic, quiche and ngtcp2 configured with different cca. For each library, the top subfigure shows the inter-packet gaps while the bottom subfigure shows the packet train lengths (PTLs).
• Figure 5: The impact of the fq qdisc on quiche pacing. SF denotes our quiche patch to change the behavior of the spurious loss detection. Without the patch, pacing can be drastically impacted in case of loss.