TakeAD: Preference-based Post-optimization for End-to-end Autonomous Driving with Expert Takeover Data

TL;DR

TakeAD introduces a takeover-data–driven, two-stage post-optimization framework that combines DAgger imitation and Direct Preference Optimization (DPO) to bridge the open-loop–closed-loop gap in end-to-end autonomous driving. By collecting expert takeover data in shadow mode and training with a hybrid architecture that fuses long-horizon trajectory planning with reactive control, TakeAD achieves state-of-the-art performance on Bench2Drive while promoting recovery capabilities in disengagement scenarios. The approach demonstrates substantial improvements over pure imitation learning and base-end-to-end models, and ablations confirm the value of the multi-modal control branch and iterative post-optimization. Limitations arise from the base model’s perception capabilities (e.g., traffic-light state handling), pointing to future work on stronger bases and continuous learning for further gains.

Abstract

Existing end-to-end autonomous driving methods typically rely on imitation learning (IL) but face a key challenge: the misalignment between open-loop training and closed-loop deployment. This misalignment often triggers driver-initiated takeovers and system disengagements during closed-loop execution. How to leverage those expert takeover data from disengagement scenarios and effectively expand the IL policy's capability presents a valuable yet unexplored challenge. In this paper, we propose TakeAD, a novel preference-based post-optimization framework that fine-tunes the pre-trained IL policy with this disengagement data to enhance the closed-loop driving performance. First, we design an efficient expert takeover data collection pipeline inspired by human takeover mechanisms in real-world autonomous driving systems. Then, this post optimization framework integrates iterative Dataset Aggregation (DAgger) for imitation learning with Direct Preference Optimization (DPO) for preference alignment. The DAgger stage equips the policy with fundamental capabilities to handle disengagement states through direct imitation of expert interventions. Subsequently, the DPO stage refines the policy's behavior to better align with expert preferences in disengagement scenarios. Through multiple iterations, the policy progressively learns recovery strategies for disengagement states, thereby mitigating the open-loop gap. Experiments on the closed-loop Bench2Drive benchmark demonstrate our method's effectiveness compared with pure IL methods, with comprehensive ablations confirming the contribution of each component.

Figures (5)

• Figure 1: Different training paradigms of end-to-end autonomous driving. (a) Imitation learning matches human demonstrations but suffers from the open-loop gap and out-of-distribution problems. (b) The proposed TakeAD method, a novel preference-based post-optimization framework, leverages high-quality expert takeover data for iterative imitation and preference optimization.
• Figure 2: Overall framework of TakeAD. (a) Model architecture of TakeAD. Multi-view images are encoded into environmental tokens and decoded into perception outputs, with parallel branches generating both long-term planning trajectories and instant control signals. (b) Post-optimization pipeline of TakeAD. In the post-optimization phase, we adopt a multi-round iterative paradigm. In each iteration, we sequentially perform expert takeover data collection, DAgger-style imitation, and preference optimization to expand the policy’s capability boundary based on high-quality takeover data. Specifically, the expert takeover data collection process consists of: (1) running the expert policy in “shadow mode” alongside the e2e policy; (2) triggering a takeover when the e2e policy fails; (3) letting the expert take control and record the intervention data; (4) storing expert takeover data.
• Figure 3: Expert takeover data collection pipeline.
• Figure 4: Multi-round Iterative Post-optimization Results.
• Figure 5: Visualization of our E2E AD policy before and after post-optimization in typical scenarios. Note: In CARLA, both throttle and brake can be greater than zero at the same time, but only the brake will take effect in this case.