Q-Learning-Based Time-Critical Data Aggregation Scheduling in IoT
Van-Vi Vo, Tien-Dung Nguyen, Duc-Tai Le, Hyunseung Choo
TL;DR
The paper tackles latency-critical data aggregation in IoT by unifying aggregation-tree construction and scheduling into a single Q-learning framework. It models the problem as an MDP with hashed states to scale to larger networks and uses a greedy, collision-free batch mechanism guided by a reward $r=|T|^2$ to maximize throughput per time slot. Empirical results on static networks with up to 300 nodes show latency reductions up to $10.87\%$ compared to a state-of-the-art heuristic, with notable gains in asymmetric topologies. This approach offers a scalable, low-latency path for time-sensitive IoT data aggregation, while future work aims to extend to dynamic topologies and energy-aware objectives.
Abstract
Time-critical data aggregation in Internet of Things (IoT) networks demands efficient, collision-free scheduling to minimize latency for applications like smart cities and industrial automation. Traditional heuristic methods, with two-phase tree construction and scheduling, often suffer from high computational overhead and suboptimal delays due to their static nature. To address this, we propose a novel Q-learning framework that unifies aggregation tree construction and scheduling, modeling the process as a Markov Decision Process (MDP) with hashed states for scalability. By leveraging a reward function that promotes large, interference-free batch transmissions, our approach dynamically learns optimal scheduling policies. Simulations on static networks with up to 300 nodes demonstrate up to 10.87% lower latency compared to a state-of-the-art heuristic algorithm, highlighting its robustness for delay-sensitive IoT applications. This framework enables timely insights in IoT environments, paving the way for scalable, low-latency data aggregation.