3D Multi-Object Tracking Employing MS-GLMB Filter for Autonomous Driving

TL;DR

An improved approach that integrates an additional sensor, such as LiDAR, into the MS-GLMB framework for 3D multi-object tracking is introduced, along with a multi-camera and LiDAR multi-object measurement model.

Abstract

The MS-GLMB filter offers a robust framework for tracking multiple objects through the use of multi-sensor data. Building on this, the MV-GLMB and MV-GLMB-AB filters enhance the MS-GLMB capabilities by employing cameras for 3D multi-sensor multi-object tracking, effectively addressing occlusions. However, both filters depend on overlapping fields of view from the cameras to combine complementary information. In this paper, we introduce an improved approach that integrates an additional sensor, such as LiDAR, into the MS-GLMB framework for 3D multi-object tracking. Specifically, we present a new LiDAR measurement model, along with a multi-camera and LiDAR multi-object measurement model. Our experimental results demonstrate a significant improvement in tracking performance compared to existing MS-GLMB-based methods. Importantly, our method eliminates the need for overlapping fields of view, broadening the applicability of the MS-GLMB filter. Our source code for nuScenes dataset is available at https://github.com/linh-gist/ms-glmb-nuScenes.

Figures (2)

• Figure 1: An illustration for the requirement of overlapped camera fields of view. Suppose an object is only observed on one camera. In that case, the uncertainty of the 3D object state distribution cannot be resolved along the 3D projection line of the observed 2D measurement, since given a 2D measurement, there are many 3D object states that could generate this measurement with a high likelihood (without any prior information, e.g., 3D training data), hence the high uncertainty in the updated 3D object state distribution. Suppose the object is observed on multiple cameras. In that case, the 3D object state uncertainty is resolved by the complementary information from different views, hence the lower uncertainty in the updated 3D object state distribution.
• Figure 2: Overlapping camera Field of Views in nuScenes dataset (Obtained from https://scale.com/open-av-datasets/nuscenes). Cameras are set up to look in six different directions (Front, Front Left, Front Right, Back, Back Left, and Back Right). One camera has very little overlapping FoVs with at most two cameras. The angle of view of CAM Back is 110 degrees, the other five cameras are 70 degrees.