A real-time dynamic obstacle tracking and mapping system for UAV navigation and collision avoidance with an RGB-D camera

TL;DR

The paper tackles real-time navigation for UAVs in dynamic environments using an RGB-D camera by presenting a 3D hybrid map that preserves static structures with an occupancy voxel map while tracking dynamic obstacles as velocity-bearing regions. A lightweight pipeline combines depth-image region proposals refined by map information, Kalman-filter–based obstacle tracking with a continuity filter, dynamic-region cleaning to update the static map, and an environment-aware Markov-chain trajectory predictor to anticipate obstacle motion. The approach achieves real-time performance (under 40 ms per iteration, >25 Hz) and demonstrates effective obstacle tracking, map refinement, and collision-free planning in both simulation and physical flight experiments, with open-source ROS implementation. The results suggest significant practical value for safe UAV navigation in crowded and dynamic spaces, offering a computation-efficient alternative to heavier learning-based detectors while providing robust trajectory prediction through environment-aware modeling.

Abstract

The real-time dynamic environment perception has become vital for autonomous robots in crowded spaces. Although the popular voxel-based mapping methods can efficiently represent 3D obstacles with arbitrarily complex shapes, they can hardly distinguish between static and dynamic obstacles, leading to the limited performance of obstacle avoidance. While plenty of sophisticated learning-based dynamic obstacle detection algorithms exist in autonomous driving, the quadcopter's limited computation resources cannot achieve real-time performance using those approaches. To address these issues, we propose a real-time dynamic obstacle tracking and mapping system for quadcopter obstacle avoidance using an RGB-D camera. The proposed system first utilizes a depth image with an occupancy voxel map to generate potential dynamic obstacle regions as proposals. With the obstacle region proposals, the Kalman filter and our continuity filter are applied to track each dynamic obstacle. Finally, the environment-aware trajectory prediction method is proposed based on the Markov chain using the states of tracked dynamic obstacles. We implemented the proposed system with our custom quadcopter and navigation planner. The simulation and physical experiments show that our methods can successfully track and represent obstacles in dynamic environments in real-time and safely avoid obstacles. Our software is available on GitHub as an open-source ROS package.

Figures (10)

• Figure 1: A physical flight experiment using the proposed mapping system. (a) A person walks toward the robot in the top view. (b) The robot generates the avoidance trajectory based on the detected dynamic obstacles and the voxel map. (c) The side view of the experiment. (d) The 2D bounding boxes of obstacles from the depth image during the flight.
• Figure 2: System framework. The system contains three dynamic modules with one static occupancy map module. It takes the depth images with the robot poses as the inputs and outputs cleaned static maps and dynamic obstacle states with their predicted trajectories.
• Figure 3: Illustration of detecting the raw bounding box of obstacles. (a) The RGB image. (b) The depth map with a 2D detected obstacle. (c) The 3D bounding box in the map frame. (d) The U-map with the 2D bounding box.
• Figure 4: Illustration of the map refinement. a) The red points represent the voxel in the static map, and the blue box is the raw box generated from the detector, which might have a misestimated shape and size. The brown box is inflated by $C_{inflate}$ b) After searching for occupied voxels, the green box, which is a refined box, gives an improved result.
• Figure 5: Illustration of the continuity filter. The k=6 frames' history of positions of a dynamic obstacle and a static obstacle are shown in the top and bottom, respectively. When the obstacle is dynamic, the angles $\theta_{1}$ and $\theta_{2}$ are smaller.