PRIBOOT: A New Data-Driven Expert for Improved Driving Simulations

TL;DR

This work tackles data scarcity in CARLA Leaderboard 2.0 by introducing PRIBOOT, an expert agent that leverages privileged BEV information and limited human logs. It designs an RGB-encoded BEV representation and a transfer-learning–driven architecture (EfficientNet + MLP + GRU) to predict future waypoints, followed by PID controllers, with data augmentation to improve out-of-distribution robustness. A novel Infraction Rate Score, $IRS$, complements the Driving Score, enabling fair evaluation over long routes. PRIBOOT outperforms Autopilot across most metrics, achieving a Route Completion around $75\%$ and demonstrating substantial potential for scalable data generation and benchmarking in autonomous driving simulations. These findings advance data-efficient learning for challenging driving scenarios and can extend to other simulators beyond CARLA.

Abstract

The development of Autonomous Driving (AD) systems in simulated environments like CARLA is crucial for advancing real-world automotive technologies. To drive innovation, CARLA introduced Leaderboard 2.0, significantly more challenging than its predecessor. However, current AD methods have struggled to achieve satisfactory outcomes due to a lack of sufficient ground truth data. Human driving logs provided by CARLA are insufficient, and previously successful expert agents like Autopilot and Roach, used for collecting datasets, have seen reduced effectiveness under these more demanding conditions. To overcome these data limitations, we introduce PRIBOOT, an expert agent that leverages limited human logs with privileged information. We have developed a novel BEV representation specifically tailored to meet the demands of this new benchmark and processed it as an RGB image to facilitate the application of transfer learning techniques, instead of using a set of masks. Additionally, we propose the Infraction Rate Score (IRS), a new evaluation metric designed to provide a more balanced assessment of driving performance over extended routes. PRIBOOT is the first model to achieve a Route Completion (RC) of 75% in Leaderboard 2.0, along with a Driving Score (DS) and IRS of 20% and 45%, respectively. With PRIBOOT, researchers can now generate extensive datasets, potentially solving the data availability issues that have hindered progress in this benchmark.

Figures (6)

• Figure 1: bev used in PRIBOOT. This representation was built upon Roach and LBC, with critical adaptations to tailor it for the complexities of Leaderboard 2.0. (a) Differentiates emergency vehicles (dark blue) from regular vehicles (blue). (b) Introduces an additional class for construction objects, illustrated in orange. Additionally, both images depict a simple method to represent motion with directional arrows, illustrated in green. The images with a white frame provide a zoomed-in view to highlight specific details.
• Figure 2: Architecture of PRIBOOT. The system receives two types of inputs: a bev image and a vector of vehicle measurements. These inputs are processed independently— the BEV through a pretrained EfficientNet model and the vehicle measurements via a MLP. The resultant feature vectors from both models are concatenated to form a comprehensive feature vector, which is then fed into a GRU-based waypoint decoder, similar to the approach used by Transfuser chitta2022transfuser. The final stage involves processing the waypoints through both longitudinal and lateral PID controllers to generate the vehicle control commands.
• Figure 3: Data augmentation techniques used to expose the agent to a broader range of driving scenarios. In these illustrations, white dots indicate the future waypoints that were followed by the human driver, and the black dot represents the target point. (a) Displays a sample with no augmentation, showing the standard scenario. (b) Shows a sample where both translation and rotation augmentations have been applied to the ego vehicle, illustrating a situation where the agent needs to recover to the center of the lane.
• Figure 4: Qualitative comparison in a parking exit scenario between Autopilot and PRIBOOT. The first row depicts a sequence of keyframes from Autopilot, while the second row shows the keyframes from PRIBOOT.
• Figure 5: Qualitative comparison in a lane obstacle scenario between Autopilot and PRIBOOT. The first row depicts a sequence of keyframes from Autopilot, while the second row shows the keyframes from PRIBOOT.