Road Rage Reasoning with Vision-language Models (VLMs): Task Definition and Evaluation Dataset

TL;DR

Proactively regulating road rage using Vision-Language Models is challenging. The authors define a road rage reasoning task, release a richly annotated dashcam dataset, and benchmark leading VLMs to assess scene understanding, event recognition, and textual reasoning. Findings show substantial gaps in visual-scene comprehension and spatial reasoning in text, even when using manual descriptions to decouple understanding from reasoning. The contributions include an 81-video dataset with 22,226 annotations and a three-task evaluation framework that informs fine-tuning and model development for antecedent-focused regulation in driving. This work lays groundwork for dialog-based calming interventions and safer driving by enabling VLMs to anticipate and mitigate road rage triggers before they escalate.

Abstract

Road rage, triggered by driving-related stimuli such as traffic congestion and aggressive driving, poses a significant threat to road safety. Previous research on road rage regulation has primarily focused on response suppression, lacking proactive prevention capabilities. With the advent of Vision-Language Models (VLMs), it has become possible to reason about trigger events visually and then engage in dialog-based comforting before drivers' anger escalates. To this end, we propose the road rage reasoning task, along with a finely annotated test dataset and evaluation metrics, to assess the capabilities of current mainstream VLMs in scene understanding, event recognition, and road rage reasoning. The results indicate that current VLMs exhibit significant shortcomings in scene understanding within the visual modality, as well as in comprehending the spatial relationships between objects in the textual modality. Improving VLMs' performance in these areas will greatly benefit downstream tasks like antecedent-focused road rage regulation.

Figures (9)

• Figure 1: We propose the road rage reasoning task to evaluate VLMs' capabilities in road rage visual understanding, behavior recognition, and scenario reasoning. Provides prior knowledge for downstream emotion regulation tasks, enabling antecedent-focused regulation.
• Figure 2: We design three tasks to evaluate VLMs' road rage reasoning abilities. The Main Task uses dashcam footage to identify road rage scenarios, testing overall reasoning. Due to poor performance, we introduce two subtasks. Sub-task 1 uses dashcam footage to assess scene understanding but lacks complete responses for quantitative analysis. Thus, Sub-task 2 uses manual descriptions, decoupling visual understanding from reasoning, and assesses textual reasoning and scene understanding capabilities.
• Figure 3: The statistics (a) and an annotation example (b) of our dataset. The dataset includes 81 videos, 2,299 frames, and 22,226 annotations. The annotations cover both overall labels (environment descriptions, road rage events and road rage scenarios) and detailed labels (lane count, ego car, and critical objects).
• Figure 4: An experimental result from sub-task 1. Under the given constraints, VLMs still fail to describe all frames. In the details, VLMs show some incorrect descriptions. This result prevents us from performing a quantitative analysis of VLMs' visual understanding ability. Therefore, we introduce sub-task 2.
• Figure 5: The experimental results for the main task are shown in the figure. We use video frames as input and ask the VLMs to identify dangerous driving, aggressive driving, and obstructive driving in the video. To simplify result analysis, VLMs are required to output a binary response (0 or 1).