V2X-QA: A Comprehensive Reasoning Dataset and Benchmark for Multimodal Large Language Models in Autonomous Driving Across Ego, Infrastructure, and Cooperative Views

Abstract

Multimodal large language models (MLLMs) have shown strong potential for autonomous driving, yet existing benchmarks remain largely ego-centric and therefore cannot systematically assess model performance in infrastructure-centric and cooperative driving conditions. In this work, we introduce V2X-QA, a real-world dataset and benchmark for evaluating MLLMs across vehicle-side, infrastructure-side, and cooperative viewpoints. V2X-QA is built around a view-decoupled evaluation protocol that enables controlled comparison under vehicle-only, infrastructure-only, and cooperative driving conditions within a unified multiple-choice question answering (MCQA) framework. The benchmark is organized into a twelve-task taxonomy spanning perception, prediction, and reasoning and planning, and is constructed through expert-verified MCQA annotation to enable fine-grained diagnosis of viewpoint-dependent capabilities. Benchmark results across ten representative state-of-the-art proprietary and open-source models show that viewpoint accessibility substantially affects performance, and infrastructure-side reasoning supports meaningful macroscopic traffic understanding. Results also indicate that cooperative reasoning remains challenging since it requires cross-view alignment and evidence integration rather than simply additional visual input. To address these challenges, we introduce V2X-MoE, a benchmark-aligned baseline with explicit view routing and viewpoint-specific LoRA experts. The strong performance of V2X-MoE further suggests that explicit viewpoint specialization is a promising direction for multi-view reasoning in autonomous driving. Overall, V2X-QA provides a foundation for studying multi-perspective reasoning, reliability, and cooperative physical intelligence in connected autonomous driving. The dataset and V2X-MoE resources are publicly available at: https://github.com/junwei0001/V2X-QA.

Figures (8)

• Figure 1: Overview of V2X-QA. Left: representative examples of the twelve viewpoint-aligned tasks under vehicle-side, infrastructure-side, and cooperative settings. Right: MCQA-based training and evaluation pipeline of V2X-MoE on the V2X-QA training and testing splits.
• Figure 2: Construction pipeline of V2X-QA. The pipeline starts from the twelve viewpoint-aligned tasks under three settings, then proceeds to MCQA task bank construction, model-assisted answer selection with human verification, data split generation, and standardized benchmarking across proprietary and open-source MLLMs.
• Figure 3: Statistics of V2X-QA. Left: overall task and functional distribution across the twelve viewpoint-aligned tasks. Right: distribution of perception, prediction, and reasoning & planning samples under vehicle-side, infrastructure-side, and cooperative views.
• Figure 4: Answer-distribution diagnostics of V2X-QA. Left: distribution of correct option positions across the twelve tasks. Middle: majority-class ratio for each task. Right: answer entropy for each task.
• Figure 5: Overall architecture and staged training pipeline of V2X-MoE. The model takes viewpoint-aligned visual evidence and MCQA prompts as input, uses a Qwen3-VL multimodal processor and a shared frozen backbone, activates one viewpoint-specific LoRA expert through an explicit router, and is trained by full MCQA training followed by CO-focused and IS-focused refinement.