Can Users Specify Driving Speed? Bench2Drive-Speed: Benchmark and Baselines for Desired-Speed Conditioned Autonomous Driving

Abstract

End-to-end autonomous driving (E2E-AD) has achieved remarkable progress. However, one practical and useful function has been long overlooked: users may wish to customize the desired speed of the policy or specify whether to allow the autonomous vehicle to overtake. To bridge this gap, we present Bench2Drive-Speed, a benchmark with metrics, dataset, and baselines for desired-speed conditioned autonomous driving. We introduce explicit inputs of users' desired target-speed and overtake/follow instructions to driving policy models. We design quantitative metrics, including Speed-Adherence Score and Overtake Score, to measure how faithfully policies follow user specifications, while remaining compatible with standard autonomous driving metrics. To enable training of speed-conditioned policies, one approach is to collect expert demonstrations that strictly follow speed requirements, an expensive and unscalable process in the real world. An alternative is to adapt existing regular driving data by treating the speed observed in future frames as the target speed for training. To investigate this, we construct CustomizedSpeedDataset, composed of 2,100 clips annotated with experts demonstrations, enabling systematic investigation of supervision strategies. Our experiments show that, under proper re-annotation, models trained on regular driving data perform comparably to on expert demonstrations, suggesting that speed supervision can be introduced without additional complex real-world data collection. Furthermore, we find that while target-speed following can be achieved without degrading regular driving performance, executing overtaking commands remains challenging due to the inherent difficulty of interactive behaviors. All code, datasets and baselines are available at https://github.com/Thinklab-SJTU/Bench2Drive-Speed

Figures (14)

• Figure 1: Bench2Drive-Speed introduces target-speed commands and overtake/follow instructions, establishing the first benchmark for desired-speed conditioned autonomous driving. We quantitatively evaluate model controllability across multiple dimensions: speed tracking, interaction strategy, comfort, traffic compliance, .
• Figure 2: We present Bench2Drive-Speed, including desired-speed conditioned task - with target speed and overtake/follow commands for speed control; dataset — 2,100 scenarios with extra commands annotated by expert demonstration and virtual target speed strategies; benchmark — controllability metrics (speed adherence, overtake/follow) jointly evaluated with safety, comfort, traffic compliance, ; and baseline — the model takes visual and speed command inputs, capable of following target speed commands while attempting to execute overtake/follow behaviors.
• Figure 3: Three Difficulty Levels in Bench2Drive-Speed. The difficulty of adhering to the desired speed increases from easy to hard. Overtaking and following adherence are evaluated only in the medium and hard scenarios.
• Figure 4: Illustration of Customized Speed Dataset. The dataset includes visual sensor inputs, ego-state information, bounding box annotations, overtaking commands, and target-speed commands from different sources.
• Figure 5: Illustration of different target-speed annotation methods. Expert demonstrations are precise but rely on the data collection model's internal hyperparameters—which are unavailable in practice—whereas re-annotation is more feasible.