A Deep Reinforcement Learning-Based TCP Congestion Control Algorithm: Design, Simulation, and Evaluation

TL;DR

Traditional TCP congestion control struggles in dynamic networks. The study develops a Deep Q-Network–based congestion control that learns to optimize cwnd via MD P formulation and NS-3/OpenGym simulation. It defines state/action/reward constructs, trains the agent with standard DQN techniques, and demonstrates substantial gains over TCP New Reno in latency and throughput, with strong adaptability. The results suggest RL-based congestion control as a practical approach for modern networks, including mobile and IoT environments, where conditions vary rapidly.

Abstract

This paper presents a novel TCP congestion control algorithm based on Deep Reinforcement Learning. The proposed approach utilizes Deep Q-Networks to optimize the congestion window (cWnd) by observing key network parameters and taking real-time actions. The algorithm is trained and evaluated within the NS-3 network simulator using the OpenGym interface. The results demonstrate significant improvements over traditional TCP New Reno in terms of latency and throughput, with better adaptability to changing network conditions. This study emphasizes the potential of reinforcement learning techniques for solving complex congestion control problems in modern networks.

Figures (5)

• Figure 1: Dumbbell Network Topology Visual
• Figure 2: Comparison of Round-Trip Time (RTT) for Deep Reinforcement Learning-based TCP Algorithm and TCP New Reno.
• Figure 3: Comparison of Average Throughput for Deep Reinforcement Learning-based TCP Algorithm and TCP New Reno.
• Figure 4: Comparison of Congestion Window (cWnd) for Deep Reinforcement Learning-based TCP Algorithm and TCP New Reno.
• Figure 5: Reward Change During Training for the Deep Reinforcement Learning-based TCP Algorithm.