GAP9Shield: A 150GOPS AI-capable Ultra-low Power Module for Vision and Ranging Applications on Nano-drones
Hanna Müller, Victor Kartsch, Luca Benini
TL;DR
GAP9Shield addresses the limited onboard sensing and compute for nano-drones by integrating a GAP9 SoC with a high-definition OV5647 camera, a 5D VL53L1 ranging array, and a WiFi-BLE module in a compact shield. The approach demonstrates sub-100 mW energy envelopes while supporting on-board object detection, localization, and mapping, with RGB streaming at multiple FPS and VGA streaming capabilities. Key contributions include a compact 6-layer PCB integration, weight and size reductions over prior decks, and open-source platform schematics to accelerate development of next-generation nano-drones. This work enables practical, autonomous indoor nano-drones for tasks such as inspection and cluttered-environment navigation, expanding the potential of AI-enabled tiny aerial systems.
Abstract
The evolution of AI and digital signal processing technologies, combined with affordable energy-efficient processors, has propelled the development of both hardware and software for drone applications. Nano-drones, which fit into the palm of the hand, are suitable for indoor environments and safe for human interaction; however, they often fail to deliver the required performance for complex tasks due to the lack of hardware providing sufficient sensing and computing performance. Addressing this gap, we present the GAP9Shield, a nano-drone-compatible module powered by the GAP9, a 150GOPS-capable SoC. The system also includes a 5MP OV5647 camera for high-definition imaging, a WiFi-BLE NINA module, and a 5D VL53L1-based ranging subsystem, which enhances obstacle avoidance capabilities. In comparison with similarly targeted state-of-the-art systems, GAP9Shield provides a 20% higher sample rate (RGB images) while offering a 20% weight reduction. In this paper, we also highlight the energy efficiency and processing power capabilities of GAP9 for object detection (YOLO), localization, and mapping, which can run within a power envelope of below 100 mW and at low latency (as 17 ms for object detection), highlighting the transformative potential of GAP9 for the new generation of nano-drone applications.