SwiftQueue: Optimizing Low-Latency Applications with Swift Packet Queuing

TL;DR

SwiftQueue tackles tail latency in L4S networks by introducing a per-packet latency predictor based on a compact Transformer and a prediction-driven per-packet queue selector. It demonstrates that focusing on sharp latency changes yields 45–65% accuracy gains over state-of-the-art baselines and 36–45% P99 tail-latency reductions for L4S flows. The design supports online fine-tuning with low overhead and batched inference to meet line-speed requirements up to $200$ Mbps. Evaluations on real ISP traces and NS-3 simulations show practical viability, though limitations remain in extreme conditions and integration with CC algorithms.

Abstract

Low Latency, Low Loss, and Scalable Throughput (L4S), as an emerging router-queue management technique, has seen steady deployment in the industry. An L4S-enabled router assigns each packet to the queue based on the packet header marking. Currently, L4S employs per-flow queue selection, i.e. all packets of a flow are marked the same way and thus use the same queues, even though each packet is marked separately. However, this may hurt tail latency and latency-sensitive applications because transient congestion and queue buildups may only affect a fraction of packets in a flow. We present SwiftQueue, a new L4S queue-selection strategy in which a sender uses a novel per-packet latency predictor to pinpoint which packets likely have latency spikes or drops. The insight is that many packet-level latency variations result from complex interactions among recent packets at shared router queues. Yet, these intricate packet-level latency patterns are hard to learn efficiently by traditional models. Instead, SwiftQueue uses a custom Transformer, which is well-studied for its expressiveness on sequential patterns, to predict the next packet's latency based on the latencies of recently received ACKs. Based on the predicted latency of each outgoing packet, SwiftQueue's sender dynamically marks the L4S packet header to assign packets to potentially different queues, even within the same flow. Using real network traces, we show that SwiftQueue is 45-65% more accurate in predicting latency and its variations than state-of-art methods. Based on its latency prediction, SwiftQueue reduces the tail latency for L4S-enabled flows by 36-45%, compared with the existing L4S queue-selection method.

Figures (17)

• Figure 1: To reduce tail latency, SwiftQueue predicts latency per packet and choose the appropriate queue for each packet.
• Figure 2: L4S by default marks all packets from the flow into the same queues, and once marked, packets traverse the chosen queues at all hops.
• Figure 3: A Transformer maps the inputs of low-latency packets and those of high-latency packets to separate clusters.
• Figure 4: SwiftQueue consists of a custom Transformer latency predictor and a prediction-driven per-packet queue selector.
• Figure 5: SwiftQueue's architecture with showing its context window, multiple attention blocks and linear layers.