YOPOv2-Tracker: An End-to-End Agile Tracking and Navigation Framework from Perception to Action

TL;DR

The paper introduces YOPOv2-Tracker, an end-to-end agile tracking and navigation framework for quadrotors that directly maps sensory observations to attitude and thrust commands. It fuses perception, detection, and navigation through a grid of motion-primitives and trains via differentiable trajectory costs backpropagated from privileged environment information, achieving millisecond-scale latency on a compact onboard platform. A disturbance observer compensates model uncertainties and disturbances, enabling robust real-world deployment in cluttered forests and buildings. The approach demonstrates superior real-time performance, high-speed tracking, and resilience to occlusions, validated through extensive simulation and on-board real-world experiments, with open-source hardware and software released for community use.

Abstract

Traditional target tracking pipelines including detection, mapping, navigation, and control are comprehensive but introduce high latency, limitting the agility of quadrotors. On the contrary, we follow the design principle of "less is more", striving to simplify the process while maintaining effectiveness. In this work, we propose an end-to-end agile tracking and navigation framework for quadrotors that directly maps the sensory observations to control commands. Importantly, leveraging the multimodal nature of navigation and detection tasks, our network maintains interpretability by explicitly integrating the independent modules of the traditional pipeline, rather than a crude action regression. In detail, we adopt a set of motion primitives as anchors to cover the searching space regarding the feasible region and potential target. Then we reformulate the trajectory optimization as regression of primitive offsets and associated costs considering the safety, smoothness, and other metrics. For tracking task, the trajectories are expected to approach the target and additional objectness scores are predicted. Subsequently, the predictions, after compensation for the estimated lumped disturbance, are transformed into thrust and attitude as control commands for swift response. During training, we seamlessly integrate traditional motion planning with deep learning by directly back-propagating the gradients of trajectory costs to the network, eliminating the need for expert demonstration in imitation learning and providing more direct guidance than reinforcement learning. Finally, we deploy the algorithm on a compact quadrotor and conduct real-world validations in both forest and building environments to demonstrate the efficiency of the proposed method.

Figures (19)

• Figure 1: Overview of the proposed aerial tracking and navigation system. (A-B) show the agile tracking in dense forests and cluttered buildings, while (C-D) depict the first-person view (FPV) of the tracker. The system is fully autonomous and onboard, with further details provided in Section \ref{['Sec_Experiment']} and attached video.
• Figure 2: Comparison with existing tracking pipeline.
• Figure 3: System Overview. The proposed framework takes RGB-D images and state observations as inputs, and outputs attitude and thrust as control commands. The ground-truth map is available in training while only the image observation is accessible during inference. It can easily switch between different tasks by modifying the modules marked in red and purple, which are required only for navigation and tracking, respectively.
• Figure 4: We divide the RGB-D image into grids, each of which corresponds to a primitive and a prediction. In the tracking task, the predictions are further divided into positive, negative, and ignored samples based on the target's location in the image.
• Figure 5: We predict offsets and derivatives to improve the primitives (red) and then filter out negative predictions (white) based on the objectness score. Finally, we select the best prediction from the remainings (green) based on the predicted safety and smoothness cost.