Rethinking Gaussian Trajectory Predictors: Calibrated Uncertainty for Safe Planning

TL;DR

A novel loss function for calibrating prediction uncertainty which leverages Kernel Density Estimation to estimate the empirical distribution of confidence levels and enforces consistency with the properties of a Gaussian assumption by explicitly matching the estimated empirical distribution to the Chi-squared distribution.

Abstract

Accurate trajectory prediction is critical for safe autonomous navigation in crowded environments. While many trajectory predictors output Gaussian distributions to represent the multi-modal distribution over future pedestrian positions, the reliability of their confidence levels often remains unaddressed. This limitation can lead to unsafe or overly conservative motion planning when the predictor is integrated with an uncertainty-aware planner. Existing Gaussian trajectory predictors primarily rely on the Negative Log-Likelihood loss, which is prone to predict over- or under-confident distributions, and may compromise downstream planner safety. This paper introduces a novel loss function for calibrating prediction uncertainty which leverages Kernel Density Estimation to estimate the empirical distribution of confidence levels. The proposed formulation enforces consistency with the properties of a Gaussian assumption by explicitly matching the estimated empirical distribution to the Chi-squared distribution. To ensure accurate mean prediction, a Mean Squared Error term is also incorporated in the final loss formulation. Experimental results on real-world trajectory datasets show that our method significantly improves the reliability of confidence levels predicted by different State-Of-The-Art Gaussian trajectory predictors. We also demonstrate the importance of providing planners with reliable probabilistic insights (i.e. calibrated confidence levels) for collision-free navigation in complex scenarios. For this purpose, we integrate Gaussian trajectory predictors trained with our loss function with an uncertainty-aware Model Predictive Control on scenarios extracted from real-world datasets, achieving improved planning performance through calibrated confidence levels.

Figures (4)

• Figure 1: Empirical (solid lines) and estimated (transparent lines) CDFs of the squared Mahalanobis distance predicted by three SLSTM variants on UNIV dataset. The black curve denotes the theoretical $\chi^2_2$ distribution. Among the compared models, $\text{SLSTM}^*$ exhibits the closest alignment with the true distribution, indicating superior uncertainty calibration.
• Figure 2: Difference in Mean Absolute Delta Empirical Sigma Value ($\overline{|\Delta \mathbb{ESV}|}$) and Difference in Average Distance Error (ADE, unit:m) computed as: "model trained with our loss - baseline model trained with NLL loss" for each model pair, on (a) ETH, (b) HOTEL, (c) ZARA, (d) UNIV. More negative values indicate that the model trained with our loss obtains lower $\overline{|\Delta \mathbb{ESV}|}$ and ADE compared to the model trained with NLL. ($\Delta \overline{|\Delta \mathbb{ESV}|}$) is hatched for cases where the trajectory predictor trained with our loss achieves a higher success rate in planning, indicating that gains in planning performance often coincide with significant improvement in $\overline{|\Delta \mathbb{ESV}|}$ resulting from our proposed loss function.
• Figure 3: Predicted trajectories and confidence levels corresponding to $1\sigma$ of (a) SLSTM+MHD and (b) $\text{SLSTM}^*$ evaluated on ZARA dataset. SLSTM+MHD has large positive $\Delta \mathbb{ESV}_1$ which indicates under-confident behavior of the model and results in larger confidence levels than ideal; while $\Delta \mathbb{ESV}_1$ of $\text{SLSTM}^*$ is the closer to the zero which makes its confidence levels to be the most similar to confidence levels of an ideal bivariate Gaussian.
• Figure 4: Planning result of DSTIGCN on a scenario from ZARA dataset. Planner integrated with the model trained with our loss successfully reaches the goal by taking a longer collision-free path due to larger calibrated confidence level.