Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BlankSkip: Early-exit Object Detection onboard Nano-drones

Carlo Marra, Beatrice Alessandra Motetti, Alessio Burrello, Enrico Macii, Massimo Poncino, Daniele Jahier Pagliari

Abstract

Deploying tiny computer vision Deep Neural Networks (DNNs) on-board nano-sized drones is key for achieving autonomy, but is complicated by the extremely tight constraints of their computational platforms (approximately 10 MiB memory, 1 W power budget). Early-exit adaptive DNNs that dial down the computational effort for "easy-to-process" input frames represent a promising way to reduce the average inference latency. However, while this approach is extensively studied for classification, its application to dense tasks like object detection (OD) is not straightforward. In this paper, we propose BlankSkip, an adaptive network for on-device OD that leverages a simple auxiliary classification task for early exit, i.e., identifying frames with no objects of interest. With experiments using a real-world nano-drone platform, the Bitcraze Crazyflie 2.1, we achieve up to 24% average throughput improvement with a limited 0.015 mean Average Precision (mAP) drop compared to a static MobileNet-SSD detector, on a state-of-the-art nano-drones OD dataset.

BlankSkip: Early-exit Object Detection onboard Nano-drones

Abstract

Deploying tiny computer vision Deep Neural Networks (DNNs) on-board nano-sized drones is key for achieving autonomy, but is complicated by the extremely tight constraints of their computational platforms (approximately 10 MiB memory, 1 W power budget). Early-exit adaptive DNNs that dial down the computational effort for "easy-to-process" input frames represent a promising way to reduce the average inference latency. However, while this approach is extensively studied for classification, its application to dense tasks like object detection (OD) is not straightforward. In this paper, we propose BlankSkip, an adaptive network for on-device OD that leverages a simple auxiliary classification task for early exit, i.e., identifying frames with no objects of interest. With experiments using a real-world nano-drone platform, the Bitcraze Crazyflie 2.1, we achieve up to 24% average throughput improvement with a limited 0.015 mean Average Precision (mAP) drop compared to a static MobileNet-SSD detector, on a state-of-the-art nano-drones OD dataset.

Paper Structure

This paper contains 16 sections, 7 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background and Related Works
  3. Nano-drone computational platforms
  4. Object detection on constrained devices
  5. Methodology
  6. Problem formulation
  7. Network architecture
  8. Training strategy
  9. Architectural exploration
  10. Deployment details
  11. Experimental Results
  12. Experimental setup & datasets
  13. Himax EE results
  14. Deployment results
  15. Cityscapes results
  16. ...and 1 more sections

Figures (2)

  • Figure 1: Overview of the proposed BlankSkip architecture. The backbone processes the input frame through layers $1$ to $\ell$. Then, the branch predicts the "emptyness" probability $P_i(1)$. If the scene is classified as empty, inference stops early (green path), avoiding the remaining backbone layers and the detection head. Otherwise, the full pipeline runs to produce bounding box predictions (red path).
  • Figure 2: Accuracy-efficiency trade-off for with threshold $\tau$ sweep (0.5-0.99) on the Himax EE dataset. Stars mark optimal thresholds according to Eq. \ref{['eq:hpo']}. Dashed line: static SSD at different widths. Stage 1 is omitted due to prohibitively high exit costs from large feature maps (120$\times$160).