Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Deep Learning-based Pest Insect Monitoring System for Ultra-low Power Pocket-sized Drones

Luca Crupi, Luca Butera, Alberto Ferrante, Daniele Palossi

TL;DR

The paper presents a hardware–software co-design for on-device pest detection using two ultra-low-power platforms (STM32H74 and GAP9) and two CNNs (FOMO-MobileNetV2 and SSDLite-MobileNetV3). After COCO pre-training, models are fine-tuned on a 3-class pest dataset and quantized to 8-bit (and float16 where applicable) for on-board inference, achieving mAPs of 0.66 and 0.79 with frame rates in the 6.8–16.1 fps range and power envelopes from tens to hundreds of milliwatts. The work demonstrates large reductions in memory and compute relative to RetinaNet baselines, enabling autonomous palm-sized drones and smart traps with up to several months of battery life. This approach offers practical, low-power, on-device pest detection suitable for targeted interventions in precision agriculture, reducing unnecessary chemical use and enabling scalable deployment on lightweight aerial platforms.

Abstract

Smart farming and precision agriculture represent game-changer technologies for efficient and sustainable agribusiness. Miniaturized palm-sized drones can act as flexible smart sensors inspecting crops, looking for early signs of potential pest outbreaking. However, achieving such an ambitious goal requires hardware-software codesign to develop accurate deep learning (DL) detection models while keeping memory and computational needs under an ultra-tight budget, i.e., a few MB on-chip memory and a few 100s mW power envelope. This work presents a novel vertically integrated solution featuring two ultra-low power System-on-Chips (SoCs), i.e., the dual-core STM32H74 and a multi-core GWT GAP9, running two State-of-the-Art DL models for detecting the Popillia japonica bug. We fine-tune both models for our image-based detection task, quantize them in 8-bit integers, and deploy them on the two SoCs. On the STM32H74, we deploy a FOMO-MobileNetV2 model, achieving a mean average precision (mAP) of 0.66 and running at 16.1 frame/s within 498 mW. While on the GAP9 SoC, we deploy a more complex SSDLite-MobileNetV3, which scores an mAP of 0.79 and peaks at 6.8 frame/s within 33 mW. Compared to a top-notch RetinaNet-ResNet101-FPN full-precision baseline, which requires 14.9x more memory and 300x more operations per inference, our best model drops only 15\% in mAP, paving the way toward autonomous palm-sized drones capable of lightweight and precise pest detection.

A Deep Learning-based Pest Insect Monitoring System for Ultra-low Power Pocket-sized Drones

TL;DR

The paper presents a hardware–software co-design for on-device pest detection using two ultra-low-power platforms (STM32H74 and GAP9) and two CNNs (FOMO-MobileNetV2 and SSDLite-MobileNetV3). After COCO pre-training, models are fine-tuned on a 3-class pest dataset and quantized to 8-bit (and float16 where applicable) for on-board inference, achieving mAPs of 0.66 and 0.79 with frame rates in the 6.8–16.1 fps range and power envelopes from tens to hundreds of milliwatts. The work demonstrates large reductions in memory and compute relative to RetinaNet baselines, enabling autonomous palm-sized drones and smart traps with up to several months of battery life. This approach offers practical, low-power, on-device pest detection suitable for targeted interventions in precision agriculture, reducing unnecessary chemical use and enabling scalable deployment on lightweight aerial platforms.

Abstract

Smart farming and precision agriculture represent game-changer technologies for efficient and sustainable agribusiness. Miniaturized palm-sized drones can act as flexible smart sensors inspecting crops, looking for early signs of potential pest outbreaking. However, achieving such an ambitious goal requires hardware-software codesign to develop accurate deep learning (DL) detection models while keeping memory and computational needs under an ultra-tight budget, i.e., a few MB on-chip memory and a few 100s mW power envelope. This work presents a novel vertically integrated solution featuring two ultra-low power System-on-Chips (SoCs), i.e., the dual-core STM32H74 and a multi-core GWT GAP9, running two State-of-the-Art DL models for detecting the Popillia japonica bug. We fine-tune both models for our image-based detection task, quantize them in 8-bit integers, and deploy them on the two SoCs. On the STM32H74, we deploy a FOMO-MobileNetV2 model, achieving a mean average precision (mAP) of 0.66 and running at 16.1 frame/s within 498 mW. While on the GAP9 SoC, we deploy a more complex SSDLite-MobileNetV3, which scores an mAP of 0.79 and peaks at 6.8 frame/s within 33 mW. Compared to a top-notch RetinaNet-ResNet101-FPN full-precision baseline, which requires 14.9x more memory and 300x more operations per inference, our best model drops only 15\% in mAP, paving the way toward autonomous palm-sized drones capable of lightweight and precise pest detection.
Paper Structure (12 sections, 9 figures, 4 tables)

This paper contains 12 sections, 9 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. System implementation
  4. Platforms
  5. Neural networks
  6. Dataset
  7. Deployment
  8. Results
  9. Object detection performance
  10. Embedded systems performance
  11. Discussion
  12. Conclusions

Figures (9)

  • Figure 1: Use case: a pocket-sized nano-drone inspecting a plant relying only on onboard sensing and computational capabilities.
  • Figure 2: Arduino Portenta H7 block diagram (A) and picture of the board (B).
  • Figure 3: GAP9 evaluation kit block diagram (A) and picture of the board (B).
  • Figure 4: SSDLite with MobileNetV3 backbone architecture.
  • Figure 5: FOMO with MobileNetV2 backbone architecture.
  • ...and 4 more figures