A Plug-in Tiny AI Module for Intelligent and Selective Sensor Data Transmission

TL;DR

The paper tackles IoT sensor networks' data deluge by proposing a plug-in intelligent sensing module that performs near-sensor FOI detection and uses a switch to regulate data transmission, reducing transmitted frames while maintaining essential information. The approach deploys a lightweight YOLO-based detector near the sensor, simplifies inference by removing bounding boxes and NMS, and applies quantization with a custom loss to enable aggressive int4 precision; a non-zero minimum transmission frequency $f_{min}$ and a lazy sensor deactivation scheme further reduce misses and energy use, while dual-camera collaboration and FPGA acceleration enable practical deployment. Evaluations on MS COCO show up to about 87% system-energy reduction and substantial storage savings with minimal drop in FOI recall, especially when FOIs are rare. The work presents a modular, orthogonal paradigm that can be readily inserted into existing IoT sensing pipelines, with broad potential in surveillance, environmental monitoring, and security applications.

Abstract

Applications in the Internet of Things (IoT) utilize machine learning to analyze sensor-generated data. However, a major challenge lies in the lack of targeted intelligence in current sensing systems, leading to vast data generation and increased computational and communication costs. To address this challenge, we propose a novel sensing module to equip sensing frameworks with intelligent data transmission capabilities by integrating a highly efficient machine learning model placed near the sensor. This model provides prompt feedback for the sensing system to transmit only valuable data while discarding irrelevant information by regulating the frequency of data transmission. The near-sensor model is quantized and optimized for real-time sensor control. To enhance the framework's performance, the training process is customized and a "lazy" sensor deactivation strategy utilizing temporal information is introduced. The suggested method is orthogonal to other IoT frameworks and can be considered as a plugin for selective data transmission. The framework is implemented, encompassing both software and hardware components. The experiments demonstrate that the framework utilizing the suggested module achieves over 85% system efficiency in terms of energy consumption and storage, with negligible impact on performance. This methodology has the potential to significantly reduce data output from sensors, benefiting a wide range of IoT applications.

Figures (6)

• Figure 1: Motivation and design of our proposed intelligent sensing module.a Application scenarios of an intelligent system. b General system framework of conventional systems and our system. c Visualization of the data transmission in our system. d Illustration of minimum data transmission frequency (denoted by $f_{min}$) in our system. $f_r$ denotes the camera's refresh rate. e Illustration of lazy sensor deactivation scheme in our system, $N$ is the number for deactivation count. f Energy consumption breakdown on the sensor for the framework utilizing the proposed module using dual-camera collaboration.
• Figure 2: In-sensor model comparison.a ROC curves of three lightweight models. b ROC curves of the models with different quantization trained by original loss. c The ROC curves of the model subjected to int4 quantization, trained with our adapted loss function and the original loss function.
• Figure 3: Performance evaluation.a Heatmaps that display the miss rate $P_{miss}$ with different parameter combinations (threshold ($T$), the ratio ($M$) of the number of background frames to the number of FOIs, minimum transmission frequency ($f_{min}$), and the count ($N$) at which the sensor deactivates). b Heatmaps that display the percentage of transmission $P_{trans}$ with different parameter combinations. c Spearman coefficient of the parameters.
• Figure 4: Experiment results.a System built for experiments. b Accelerator placement layout on AMD Xilinx ZCU104 FPGA. c Energy consumption comparison with $M = 20$ (the total number of the frames $n_{total}=21336$, $P_{miss}=3\%\pm0.6\%$). The conventional system implements Fast R-CNN on the server. d Energy consumption comparison with various $M$ values. All servers are equipped with GeForce RTX 4090.
• Figure 5: Energy consumption comparison.a The average energy consumption breakdown for the conventional system. b The average energy consumption breakdown for the framework utilizing the proposed module compared to the conventional framework. c Energy consumption comparison of our proposed method and the baseline.