A Multi-Loss Strategy for Vehicle Trajectory Prediction: Combining Off-Road, Diversity, and Directional Consistency Losses

TL;DR

This work tackles the safety and diversity gaps in multimodal vehicle trajectory prediction by replacing the standard winner-takes-all objective with three differentiable, all-mode losses: Offroad Loss, Road Direction Consistency, and Mode Diversity. These losses enforce drivable-area adherence, correct heading alignment with road centerlines, and diverse yet feasible trajectory futures, and are compatible with any prediction model within the UniTraj framework. Empirical results on nuScenes and Argoverse 2 using Wayformer and AutoBots show substantial reductions in off-road predictions and improved robustness to scene perturbations, with a notable 47% reduction on original scenes and 37% on attacked scenes reported in the abstract. The approach offers a scalable, architecture-agnostic means to enhance realism and safety in autonomous driving predictions and provides a basis for future extensions, including additional differentiable losses and adaptive weighting.

Abstract

Trajectory prediction is essential for the safety and efficiency of planning in autonomous vehicles. However, current models often fail to fully capture complex traffic rules and the complete range of potential vehicle movements. Addressing these limitations, this study introduces three novel loss functions: Offroad Loss, Direction Consistency Error, and Diversity Loss. These functions are designed to keep predicted paths within driving area boundaries, aligned with traffic directions, and cover a wider variety of plausible driving scenarios. As all prediction modes should adhere to road rules and conditions, this work overcomes the shortcomings of traditional "winner takes all" training methods by applying the loss functions to all prediction modes. These loss functions not only improve model training but can also serve as metrics for evaluating the realism and diversity of trajectory predictions. Extensive validation on the nuScenes and Argoverse 2 datasets with leading baseline models demonstrates that our approach not only maintains accuracy but significantly improves safety and robustness, reducing offroad errors on average by 47% on original and by 37% on attacked scenes. This work sets a new benchmark for trajectory prediction in autonomous driving, offering substantial improvements in navigating complex environments. Our code is available at https://github.com/vita-epfl/stay-on-track .

Figures (4)

• Figure 1: Trajectory predictions by Wayformer nayakanti_wayformer_2022, a state-of-the-art model, highlighting errors such as off-road movements, incorrect traffic direction adherence, and missed predictions for other plausible maneuvers at the intersection, such as continuing straight. Our proposed loss functions aim to correct these prediction errors.
• Figure 2: An illustration for our proposed loss functions. The colors in (a) show the Offroad Loss values for areas around the blue vehicle, with the red offroad trajectory having a high penalty. Panel (b) illustrates Road Direction Consistency, showing centerline points and directions; the incorrect red trajectory fails to align with the proper road direction. In (c), we compare three prediction sets: red trajectories demonstrate diversity but are infeasible due to straying off the drivable area; yellow trajectories are feasible but lack diversity, missing a potential right turn; green trajectories successfully combine diversity with feasibility, accurately reflecting viable path options.
• Figure 3: Comparative visualization of model predictions on the Argoverse 2 dataset using Wayformer as the baseline. Each panel illustrates enhancements from applying a distinct auxiliary loss function to the baseline model: (a) Offroad Loss corrects four off-road predictions by enhancing adherence to drivable areas; (b) Direction Error adjusts a potentially hazardous right turn against traffic flow; (c) Mode Diversity introduces a new left turn and increases the spacing between trajectories.
• Figure 4: Performance impact of integrating our loss functions on Wayformer's minADE across the Argoverse 2 dataset. The blue curves demonstrate the trade-off between increasing auxiliary loss weights and prediction accuracy, with red dots marking the baseline performance. Yellow dots represent the selected models, showing regions where enhanced losses maintain similar accuracy compared to the baseline. Similar patterns are observed across different models and dataset configurations.