AI and Vision based Autonomous Navigation of Nano-Drones in Partially-Known Environments

TL;DR

This work tackles safe autonomous navigation of nano-drones in partially known environments by implementing an ISCC framework that offloads vision-based obstacle detection to an edge processor while performing reactive planning onboard. The system uses a SSD MobileNet V2 detector trained on a custom four-obstacle dataset and a 2D, cue-based planning algorithm that yields velocity and yaw commands by blending repulsive and attractive actions with a smoothed safety factor. Key contributions include the edge-onboard split architecture, a minimal data-collection pipeline for nano-drones, an implemented reactive planner with bounded actuation, and extensive latency and detection evaluations that demonstrate real-time operation at about 8 FPS and robust obstacle avoidance in field tests. The approach provides a practical alternative to fully onboard AI on nano-drones and lays groundwork for autonomous exploration under ISCC constraints, enabling scalable deployment of vision-based navigation in micro- and nano-UAVs.

Abstract

The miniaturisation of sensors and processors, the advancements in connected edge intelligence, and the exponential interest in Artificial Intelligence are boosting the affirmation of autonomous nano-size drones in the Internet of Robotic Things ecosystem. However, achieving safe autonomous navigation and high-level tasks such as exploration and surveillance with these tiny platforms is extremely challenging due to their limited resources. This work focuses on enabling the safe and autonomous flight of a pocket-size, 30-gram platform called Crazyflie 2.1 in a partially known environment. We propose a novel AI-aided, vision-based reactive planning method for obstacle avoidance under the ambit of Integrated Sensing, Computing and Communication paradigm. We deal with the constraints of the nano-drone by splitting the navigation task into two parts: a deep learning-based object detector runs on the edge (external hardware) while the planning algorithm is executed onboard. The results show the ability to command the drone at $\sim8$ frames-per-second and a model performance reaching a COCO mean-average-precision of $60.8$. Field experiments demonstrate the feasibility of the solution with the drone flying at a top speed of $1$ m/s while steering away from an obstacle placed in an unknown position and reaching the target destination. The outcome highlights the compatibility of the communication delay and the model performance with the requirements of the real-time navigation task. We provide a feasible alternative to a fully onboard implementation that can be extended to autonomous exploration with nano-drones.

Figures (13)

• Figure 1: High-level pipeline representation. Nano-drone flies from point A to B and avoids obstacles using an AI-aided, vision-based reactive planning. The navigation task is split: obstacle detection on the external HW, planning on the UAV.
• Figure 2: Dataset Example. The original images (left) are collected from the drone. The same images after labelling (right).
• Figure 3: Reference frames and Variables of Interest. The image illustrates the global (black) and body fixed (blue) reference frames. The variables of interest are the two waypoints $P_A$ and $P_B$, the drone's estimated pose in the global frame $(\hat{x},\hat{y},\hat{\psi})$, the coordinates of the distance from $P_B$ in the global frame $(x_g,y_g)$ and in the body fixed frame $(x_r,y_r)$, and the heading offset $\psi_r$.
• Figure 4: Safety margin and critical FOV. (left) Inflation of the detected BB by a safety margin of 20px on each side. (right) Concept of critical FOV, i.e. the FOV enclosing a window of 40px centered in the middle of the image plane.
• Figure 5: (a) Rescaling of the Collision Risk. The Collision risk is derived from the width of the BB. $W$ represents the width of the image plane. (b) Repulsive Velocity. The repulsive velocity $v_{rep}$ is directed towards the less-occupied halfplane by the inflated BB. Its module is proportional to the width of interest, i.e. the width of the portion of the BB lying in the target halfplane.