Combining Heuristic and Reinforcement Learning to Achieve the Low-latency and High-throughput Receiver-side Congestion Control

TL;DR

The paper addresses the challenge of achieving low latency and high throughput in wireless networks by decoupling congestion control from the sender and implementing it at the receiver. It introduces Nuwa, a receiver-driven framework that uses one-way delay measurements and Kalman filtering to estimatequeuing delay, updates the sending window via a per-application target delay, and communicates this window to the sender. To further enhance performance across dynamic environments, the authors integrate reinforcement learning with PPO-LSTM to adapt the key Nuwa parameter $k$, optimizing a multi-objective reward that balances throughput, delay, and loss with alpha-fairness. Experimental results in 5G and related networks show that Nuwa improves TCP throughput by about 4–15.4% and reduces average queueing delay by about 6.9–29.4%, while Nuwa-RL improves environmental adaptability and fairness across diverse network scenarios. The work has practical implications for DASH bandwidth estimation and video streaming over HTTP by providing more accurate network state information at the receiver and enabling adaptive control through learning.

Abstract

Traditional congestion control algorithms struggle to maintain the consistent and satisfactory data transmission performance over time-varying networking condition. Simultaneously, as video traffic becomes dominant, the loose coupling between the DASH framework and TCP congestion control results in the un-matched bandwidth usage, thereby limiting video streaming performance. To address these issues, this paper proposes a receiver-driven congestion control framework named Nuwa. Nuwa deploys the congestion avoidance phase at the receiver-side, utilizing one-way queueing delay detection to monitor network congestion and setting specific target delays for different applications. Experimental results demonstrate that, in most cases, with appropriate parameter configuration, Nuwa can improve the throughput of TCP flows 4% to 15.4% and reduce average queueing delay by 6.9% to 29.4%. Furthermore, we also introduce the use of reinforcement learning to dynamically adjust Nuwa's key parameter , enhancing Nuwa's adaptability to the unpredictable environment.

Figures (17)

• Figure 1: The network topology adopted to verify the performance of traditional congestion control algorithms in 5G networks.
• Figure 2: Comparison of DASH predicted bandwidth with real bandwidth.
• Figure 3: The overall architecture of Nuwa.
• Figure 4: Trend of tanh function.
• Figure 5: Topology of the emulation experiment of Nuwa.