Uncertainty-Aware Autonomous Vehicles: Predicting the Road Ahead

TL;DR

The paper addresses perceptual uncertainty in autonomous vehicles, particularly under edge cases where traditional models are overconfident. It proposes Random-Set Neural Networks (RS-NNs) that output belief functions over class sets to quantify and signal uncertainty in real time, and demonstrates their integration into a ROS-based autonomous racing stack. Through the TrackDrive dataset and live vehicle experiments, RS-NN achieves higher accuracy than CNNs and Bayesian baselines and provides calibrated uncertainty via entropy and belief masses. The authors show how entropy-based uncertainty can dynamically modulate vehicle speed, improving safety without sacrificing performance, and they validate data-efficient active learning for uncertainty-aware perception. This work suggests uncertainty-aware perception as a practical route to safer, more robust autonomous driving.

Abstract

Autonomous Vehicle (AV) perception systems have advanced rapidly in recent years, providing vehicles with the ability to accurately interpret their environment. Perception systems remain susceptible to errors caused by overly-confident predictions in the case of rare events or out-of-sample data. This study equips an autonomous vehicle with the ability to 'know when it is uncertain', using an uncertainty-aware image classifier as part of the AV software stack. Specifically, the study exploits the ability of Random-Set Neural Networks (RS-NNs) to explicitly quantify prediction uncertainty. Unlike traditional CNNs or Bayesian methods, RS-NNs predict belief functions over sets of classes, allowing the system to identify and signal uncertainty clearly in novel or ambiguous scenarios. The system is tested in a real-world autonomous racing vehicle software stack, with the RS-NN classifying the layout of the road ahead and providing the associated uncertainty of the prediction. Performance of the RS-NN under a range of road conditions is compared against traditional CNN and Bayesian neural networks, with the RS-NN achieving significantly higher accuracy and superior uncertainty calibration. This integration of RS-NNs into Robot Operating System (ROS)-based vehicle control pipeline demonstrates that predictive uncertainty can dynamically modulate vehicle speed, maintaining high-speed performance under confident predictions while proactively improving safety through speed reductions in uncertain scenarios. These results demonstrate the potential of uncertainty-aware neural networks - in particular RS-NNs - as a practical solution for safer and more robust autonomous driving.

Figures (18)

• Figure 1: Real-time uncertainty awareness in the autonomous racing car.
• Figure 2: The IMechE Autonomous Racing Car on track at Silverstone during Formula Student UK Artificial Intelligence.
• Figure 3: Illustration of curvature classes: (a) Straight, (b) Left-Easy, (c) Left-Medium, (d) Left-Hard; right turns follow the same scheme. The orange arrow at the car's nose shows the forward axis, and the blue dashed line shows track deviation. Classes are defined by deviation: Straight$<15^\circ$, Easy $15^\circ$–$35^\circ$, Medium $35^\circ$–$60^\circ$, Hard $>60^\circ$. Images are black-and-white to highlight geometry.
• Figure 4: Entropy distributions: RS-NN, CNN and LB-BNN on regular and uncertain (hatched) test data of Trackdrive dataset.
• Figure 5: Confusion matrices for RS-NN. (a) Test sample counts: frequency of correct and incorrect predictions across seven classes, with highlighted rectangles indicating within-group confusion (green, yellow) and between-group confusion (red, purple). (b) Average entropy: mean uncertainty for each true--predicted pair, where higher off-diagonal entropy reflects the inherent ambiguity and similarity between certain directional classes.