DriveCritic: Towards Context-Aware, Human-Aligned Evaluation for Autonomous Driving with Vision-Language Models

TL;DR

DriveCritic tackles the context-agnostic nature of state-of-the-art open-loop metrics like EPDMS by introducing a context-aware evaluation framework that aligns with human preferences. It combines a NAVSIM-derived dataset of challenging trajectory pairs with a vision-language-model evaluator trained in a two-stage pipeline (supervised fine-tuning followed by RLVR-based reinforcement learning) to adjudicate between candidate trajectories. The approach achieves 76.0% accuracy on human-preference alignment, outperforms rule-based metrics and generic VLMs, and demonstrates robustness to input-order perturbations. This work provides a practical, human-aligned foundation for evaluating autonomous driving systems and suggests a path toward scalable, context-sensitive evaluation in real-world deployment.

Abstract

Benchmarking autonomous driving planners to align with human judgment remains a critical challenge, as state-of-the-art metrics like the Extended Predictive Driver Model Score (EPDMS) lack context awareness in nuanced scenarios. To address this, we introduce DriveCritic, a novel framework featuring two key contributions: the DriveCritic dataset, a curated collection of challenging scenarios where context is critical for correct judgment and annotated with pairwise human preferences, and the DriveCritic model, a Vision-Language Model (VLM) based evaluator. Fine-tuned using a two-stage supervised and reinforcement learning pipeline, the DriveCritic model learns to adjudicate between trajectory pairs by integrating visual and symbolic context. Experiments show DriveCritic significantly outperforms existing metrics and baselines in matching human preferences and demonstrates strong context awareness. Overall, our work provides a more reliable, human-aligned foundation to evaluating autonomous driving systems.

Figures (3)

• Figure 1: Example from NAVSIM Dauner2024NEURIPS illustrating the need for context-aware evaluation in autonomous driving. In this pairwise comparison task, trajectory A briefly nudges left to maintain a safe lateral buffer - an action that is contextually appropriate and not a true lane departure. Our DriveCritic model not only prefers A but also generates similar reasoning, demonstrating its contextual understanding capability. By contrast, the SOTA rule-based metric EPDMS Cao2025CORL assigns a lower score to A and favors B simply because B remains within a fixed lane-keeping threshold despite its unnecessarily low progress. Key BEV legend: Ego vehicle - red rectangle at the center (0 m, 0 m) oriented upward; Trajectory waypoints - green dotted line with circular points (8 waypoints for a 4 s horizon, spaced 0.5 s apart) starting at the ego vehicle’s rear-axle center. Best viewed zoomed in and in color.
• Figure 2: An overview of the DriveCritic framework. The DriveCritic dataset is a pilot benchmark focusing on context-aware evaluation. The DriveCritic model integrates rich multi-modal inputs and is fine-tuned with a two-stage training procedure, enabling it to generate human-aligned evaluation decisions in challenging driving scenarios.
• Figure 3: Qualitative examples illustrating DriveCritic’s contextual understanding and evaluation ability. Only representative reasoning steps are shown due to space constraints. Key BEV legend: Ego vehicle - red rectangle at the center (0 m, 0 m) oriented upward; Trajectory waypoints - green dotted line with circular points (8 waypoints for a 4 s horizon, spaced 0.5 s apart) starting at the ego vehicle’s rear-axle center. Best viewed zoomed in and in color.