Deep Reinforcement Learning for Multi-flow Routing in Heterogeneous Wireless Networks

TL;DR

This work tackles routing in heterogeneous wireless networks (HWNs) supporting multiple data flows, where links vary by technology and experience interference. It introduces a deep reinforcement learning framework using a single shared dueling DDQN agent that jointly selects communication resources and next-hop relays to maximize the end-to-end sum-rate $R_{tot}$. Key contributions include (i) frequency-aware neighbor selection strategies, (ii) a compact state-action design for multi-technology routing, and (iii) extensive robustness and generalizability evaluations under mobility, topology changes, and varying flow counts. The results demonstrate improved scalability, adaptability, and throughput over traditional and heuristic schemes, suggesting DRL-based routing as a viable solution for dynamic HWNs in practical deployments.

Abstract

Due to the rapid growth of heterogeneous wireless networks (HWNs), where devices with diverse communication technologies coexist, there is increasing demand for efficient and adaptive multi-hop routing with multiple data flows. Traditional routing methods, designed for homogeneous environments, fail to address the complexity introduced by links consisting of multiple technologies, frequency-dependent fading, and dynamic topology changes. In this paper, we propose a deep reinforcement learning (DRL)-based routing framework using deep Q-networks (DQN) to establish routes between multiple source-destination pairs in HWNs by enabling each node to jointly select a communication technology, a subband, and a next hop relay that maximizes the rate of the route. Our approach incorporates channel and interference-aware neighbor selection approaches to improve decision-making beyond conventional distance-based heuristics. We further evaluate the robustness and generalizability of the proposed method under varying network dynamics, including node mobility, changes in node density, and the number of data flows. Simulation results demonstrate that our DRL-based routing framework significantly enhances scalability, adaptability, and end-to-end throughput in complex HWN scenarios.

Figures (12)

• Figure 1: Overall framework of DRL-based HWN routing for multi-flow. The frontier node decide the next hop and communication resource through a two-stage process: 1) neighbor node set selection, 2) route decision and resource allocation.
• Figure 2: States that DRL agent at the frontier node gathers such as distance, angle, channel gain, interference, and rate.
• Figure 3: Training process of DDQN with experience replay. The route is established using $\epsilon$-greedy policy. After the route is established, (state, action, reward) tuple is saved and used during training.
• Figure 4: 3D ray tracing-based simulation environment with 36 nodes.
• Figure 5: Performance of DRL agent with 2 flows compared to other benchmark schemes using path loss model-based environment (top) and ray tracing-based environment (bottom).