Bayesian Multiobject Tracking With Neural-Enhanced Motion and Measurement Models

TL;DR

This paper tackles MOT by bridging model-based Bayesian estimation and data-driven learning through neural enhancements. It introduces a neural-enhanced non-Markovian motion model that accounts for historical object states and neighbor interactions, and a neural-enhanced measurement model featuring affinity and false-positive rejection factors learned from data. The framework leverages belief propagation with sigma-point methods for prediction and particle-based updates, achieving state-of-the-art results on nuScenes with LiDAR and camera inputs. The work demonstrates that integrating neural features into statistical MOT models can improve both robustness and accuracy while maintaining tractable inference for real-world sensing scenarios.

Abstract

Multiobject tracking (MOT) is an important task in applications including autonomous driving, ocean sciences, and aerospace surveillance. Traditional MOT methods are model-based and combine sequential Bayesian estimation with data association and an object birth model. More recent methods are fully data-driven and rely on the training of neural networks. Both approaches offer distinct advantages in specific settings. In particular, model-based methods are generally applicable across a wide range of scenarios, whereas data-driven MOT achieves superior performance in scenarios where abundant labeled data for training is available. A natural thought is whether a general framework can integrate the two approaches. This paper introduces a hybrid method that utilizes neural networks to enhance specific aspects of the statistical model in Bayesian MOT that have been identified as overly simplistic. By doing so, the performance of the prediction and update steps of Bayesian MOT is improved. To ensure tractable computation, our framework uses belief propagation to avoid high-dimensional operations combined with sequential Monte Carlo methods to perform low-dimensional operations efficiently. The resulting method combines the flexibility and robustness of model-based approaches with the capability to learn complex information from data of neural networks. We evaluate the performance of the proposed method based on the nuScenes autonomous driving dataset and demonstrate that it has state-of-the-art performance.

Figures (8)

• Figure 1: Autonomous driving scenario used for performance evaluation. LiDAR measurements, ground truth object states, and object state estimates provided by the proposed method are shown. The orange dashed rectangles indicate the object state estimates and the black rectangles indicate ground truth object states.
• Figure 2: Block diagram of a single time step of the proposed MOT method that uses a neural network to enhance statistical motion and measurement models.
• Figure 3: Proposed neural network structure as used for the neural-enhanced motion model.
• Figure 4: Extraction of an object-oriented shape feature for the case where there is $L=1$ feature map. Shape features are used in the proposed neural-enhanced measurement model .
• Figure 5: Block diagram of the proposed Bayesian MOT method for a single time step $k$. The PDF computation in the proposed method is based on the indicated factor graph and corresponding message-passing procedure KscFreLoe:J01-SPAMeyThoWil:J18-BP. Different parts of the proposed method are indicated by different colors (1) green: input and output PDF related to time step $k$; (2) blue: computations related to the prediction step (3) red: computations related to the evaluation of the likelihood function (4) black/gray: probabilistic data association. The following shorthand notations are used: $f^i = f(\underline{\bm{\mathbf{y}}}^i_{k}|{\bm{\mathbf{y}}}^i_{k-1}, {\mathbf s}^i_{k-1}, {\mathbf h}^i_{k-1})$, $\hat{q}^i = \hat{q}(\underline{\bm{\mathbf{y}}}_k^i, a_k^i; \bm{\mathbf{z}}_k)$, $\hat{v}^j = \hat{v}(\overline{\bm{\mathbf{y}}}_k^j, b_k^j; \bm{\mathbf{z}}_k)$, $\phi^{i,j} = \phi^{i,j}(a_k^i, b_k^j)$.