Q-learning-based Opportunistic Communication for Real-time Mobile Air Quality Monitoring Systems

TL;DR

This work tackles the problem of real-time air quality monitoring with bus-mounted devices by minimizing 4G transmission costs while keeping data latency under a target. It proposes a distributed Q-learning based opportunistic offloading framework (OCMA) that leverages three communication paths: direct 4G to the cloud, Wi-Fi to Road Side Units, and device-to-device relaying. Key contributions include defining a state space that captures timing and resource availability, an action set for offloading options, and a reward design that balances latency and 4G usage, implemented as per-device learning. Experiments on urban bus-route data show a 40–50% reduction in 4G usage while ensuring that about 99.5% of packets meet the latency target, demonstrating practical potential for cost-effective real-time monitoring.

Abstract

We focus on real-time air quality monitoring systems that rely on devices installed on automobiles in this research. We investigate an opportunistic communication model in which devices can send the measured data directly to the air quality server through a 4G communication channel or via Wi-Fi to adjacent devices or the so-called Road Side Units deployed along the road. We aim to reduce 4G costs while assuring data latency, where the data latency is defined as the amount of time it takes for data to reach the server. We propose an offloading scheme that leverages Q-learning to accomplish the purpose. The experiment results show that our offloading method significantly cuts down around 40-50% of the 4G communication cost while keeping the latency of 99.5% packets smaller than the required threshold.

Figures (6)

• Figure 1: Network model.
• Figure 2: Q-learning overview.
• Figure 3: The real vehicle trajectory.
• Figure 4: Relative breakdown of the simulation packets.
• Figure 5: Impacts of the packet generation interval ($\lambda_d$). $\delta$ is fixed to 5 time steps.