Towards Collaborative Autonomous Driving: Simulation Platform and End-to-End System

TL;DR

This work addresses how to harness V2X communications to enhance system-level autonomous driving performance. It introduces V2Xverse, a CARLA-based platform for offline benchmarking and online closed-loop evaluation, and CoDriving, an end-to-end collaborative driving system that uses driving-oriented communication via a driving-request map and multi-scale BEV feature fusion. The paper provides extensive evaluations across perception and system-level driving tasks, demonstrating substantial gains over state-of-the-art single-agent methods and robust performance under bandwidth, latency, and pose disturbances. It highlights the practical impact of enabling scalable, bandwidth-adaptive collaborative driving and lays groundwork for rapid prototyping of V2X-enabled autonomous systems.

Abstract

Vehicle-to-everything-aided autonomous driving (V2X-AD) has a huge potential to provide a safer driving solution. Despite extensive researches in transportation and communication to support V2X-AD, the actual utilization of these infrastructures and communication resources in enhancing driving performances remains largely unexplored. This highlights the necessity of collaborative autonomous driving: a machine learning approach that optimizes the information sharing strategy to improve the driving performance of each vehicle. This effort necessitates two key foundations: a platform capable of generating data to facilitate the training and testing of V2X-AD, and a comprehensive system that integrates full driving-related functionalities with mechanisms for information sharing. From the platform perspective, we present V2Xverse, a comprehensive simulation platform for collaborative autonomous driving. This platform provides a complete pipeline for collaborative driving. From the system perspective, we introduce CoDriving, a novel end-to-end collaborative driving system that properly integrates V2X communication over the entire autonomous pipeline, promoting driving with shared perceptual information. The core idea is a novel driving-oriented communication strategy. Leveraging this strategy, CoDriving improves driving performance while optimizing communication efficiency. We make comprehensive benchmarks with V2Xverse, analyzing both modular performance and closed-loop driving performance. Experimental results show that CoDriving: i) significantly improves the driving score by 62.49% and drastically reduces the pedestrian collision rate by 53.50% compared to the SOTA end-to-end driving method, and ii) achieves sustaining driving performance superiority over dynamic constraint communication conditions.

Figures (17)

• Figure 1: Platform overview. V2Xverse simulates the complete V2X-AD driving pipeline, incorporating various driving functionalities and delivering extensive driving annotations. It facilitates both the offline benchmark generation and online closed-loop driving performance evaluation.
• Figure 2: Safety-critical scenarios caused by occlusion, including crazy pedestrians from occluded vehicles, unexpected bicyclists from occluded buildings, and vehicles rapidly coming from occluded road infrastructures.
• Figure 3: System overview. CoDriving comprises two components: end-to-end single-agent autonomous driving, which transforms the sensor inputs into driving actions, and driving-oriented collaboration, which enhances the single-agent features by aggregating the driving-critical perceptual features shared through communication. The benefits propagate from the perception module to the entire driving pipeline, enhancing all driving signals.
• Figure 4: Benchmark CoDriving and previous collaborative perception methods on commonly used collaborative perception real-world and simulation datasets under homogeneous setting.
• Figure 5: Benchmark CoDriving and previous collaborative perception methods on commonly used collaborative perception real-world and simulation datasets under heterogeneous settings.