Deep Q-Network for Optimizing NOMA-Aided Resource Allocation in Smart Factories with URLLC Constraints

TL;DR

This work targets URLLC-enabled wireless operation in dense smart factories by optimizing NOMA-based resource allocation with a Deep Q-Network (DQN). The authors formulate the problem as an MDP and introduce a tunable parameter $\lambda$ to balance throughput and latency, enabling device-specific performance trade-offs between robots, sensors, and controllers. Through extensive simulations, the proposed approach demonstrates high robot throughput while ensuring low-latency, reliable communication for URLLC-critical devices, and highlights the effectiveness of the DQN framework with a carefully designed reward. The study advances industrial wireless networking by providing a learning-driven, adaptable resource allocator suitable for Industry 4.0 deployments and points to future work in multi-agent and energy-efficient extensions.

Abstract

This paper presents a Deep Q-Network (DQN)- based algorithm for NOMA-aided resource allocation in smart factories, addressing the stringent requirements of Ultra-Reliable Low-Latency Communication (URLLC). The proposed algorithm dynamically allocates sub-channels and optimizes power levels to maximize throughput while meeting strict latency constraints. By incorporating a tunable parameter λ, the algorithm balances the trade-off between throughput and latency, making it suitable for various devices, including robots, sensors, and controllers, each with distinct communication needs. Simulation results show that robots achieve higher throughput, while sensors and controllers meet the low-latency requirements of URLLC, ensuring reliable communication for real-time industrial applications.

Figures (5)

• Figure 1: Deep Q-Network (DQN) Architecture
• Figure 2: Reward vs. Steps for different learning rates (lr).
• Figure 3: Throughput (Mbps) for Robots and Sensors across different $\lambda$.
• Figure 4: Latency (ms) for Robots, Sensors, and Controllers at different $\lambda$.
• Figure :