A Survey of Reasoning in Autonomous Driving Systems: Open Challenges and Emerging Paradigms

TL;DR

A novel Cognitive Hierarchy is proposed to decompose the monolithic driving task according to its cognitive and interactive complexity and identifies a fundamental and unresolved tension between the high-latency, deliberative nature of LLM-based reasoning and the millisecond-scale, safety-critical demands of vehicle control.

Abstract

The development of high-level autonomous driving (AD) is shifting from perception-centric limitations to a more fundamental bottleneck, namely, a deficit in robust and generalizable reasoning. Although current AD systems manage structured environments, they consistently falter in long-tail scenarios and complex social interactions that require human-like judgment. Meanwhile, the advent of large language and multimodal models (LLMs and MLLMs) presents a transformative opportunity to integrate a powerful cognitive engine into AD systems, moving beyond pattern matching toward genuine comprehension. However, a systematic framework to guide this integration is critically lacking. To bridge this gap, we provide a comprehensive review of this emerging field and argue that reasoning should be elevated from a modular component to the system's cognitive core. Specifically, we first propose a novel Cognitive Hierarchy to decompose the monolithic driving task according to its cognitive and interactive complexity. Building on this, we further derive and systematize seven core reasoning challenges, such as the responsiveness-reasoning trade-off and social-game reasoning. Furthermore, we conduct a dual-perspective review of the state-of-the-art, analyzing both system-centric approaches to architecting intelligent agents and evaluation-centric practices for their validation. Our analysis reveals a clear trend toward holistic and interpretable "glass-box" agents. In conclusion, we identify a fundamental and unresolved tension between the high-latency, deliberative nature of LLM-based reasoning and the millisecond-scale, safety-critical demands of vehicle control. For future work, a primary objective is to bridge the symbolic-to-physical gap by developing verifiable neuro-symbolic architectures, robust reasoning under uncertainty, and scalable models for implicit social negotiation.

Figures (9)

• Figure 1: Motivation: why explicit reasoning matters in autonomous driving. The left panel summarizes seven recurring reasoning challenges in our taxonomy. The right panel presents three illustrative scenarios (E1--E3) that contrast a representative current AD system driven by rule-based heuristics with a reasoning-in-AD approach that integrates contextual signals, traffic rules, and multi-agent interaction cues via explicit inference (dashed boxes). The comparison highlights how brittle policies can yield unsafe or overly conservative actions (red), while structured reasoning supports context-appropriate decisions (green).
• Figure 2: The outline of the survey on reasoning in autonomous driving systems.
• Figure 3: The proposed Cognitive Hierarchy for reasoning in autonomous driving. This framework deconstructs the monolithic "driving" task into three distinct levels of increasing cognitive and interactive complexity: (1) the Sensorimotor Level, (2) the Egocentric Reasoning Level, and (3) the Social-Cognitive Level.
• Figure 4: The taxonomy of seven core reasoning challenges in autonomous driving. These challenges are categorized by their corresponding cognitive level: C1--C4 at the Egocentric Reasoning level, while C5--C7 are at the Social-Cognitive level. Each numbered scenario illustrates a specific challenge analyzed in the text.
• Figure 5: The chronological evolution of major methods in autonomous driving. This timeline highlights the rapid progression of the field and provides historical context for the thematic complexity taxonomy introduced in \ref{['ss:system-centric-app']}.