Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LEAD: Minimizing Learner-Expert Asymmetry in End-to-End Driving

Long Nguyen, Micha Fauth, Bernhard Jaeger, Daniel Dauner, Maximilian Igl, Andreas Geiger, Kashyap Chitta

TL;DR

The work identifies and mitigates learner–expert asymmetries in end-to-end driving by aligning expert demonstrations to sensor-based observations and strengthening navigation intent. It introduces LEAD, a large-scale CARLA dataset, and TFv6, a TransFuser-based policy that delivers state-of-the-art closed-loop performance on CARLA benchmarks and shows sim-to-real gains when paired with real-world data. Key contributions include state and intent alignment techniques, removal of late-stage goal conditioning bottlenecks, and a multi-point navigation conditioning strategy that reduces target-point bias. The findings highlight the practical impact of expert design and intent specification as drivers of real-world robustness in simulation-based imitation learning.

Abstract

Simulators can generate virtually unlimited driving data, yet imitation learning policies in simulation still struggle to achieve robust closed-loop performance. Motivated by this gap, we empirically study how misalignment between privileged expert demonstrations and sensor-based student observations can limit the effectiveness of imitation learning. More precisely, experts have significantly higher visibility (e.g., ignoring occlusions) and far lower uncertainty (e.g., knowing other vehicles' actions), making them difficult to imitate reliably. Furthermore, navigational intent (i.e., the route to follow) is under-specified in student models at test time via only a single target point. We demonstrate that these asymmetries can measurably limit driving performance in CARLA and offer practical interventions to address them. After careful modifications to narrow the gaps between expert and student, our TransFuser v6 (TFv6) student policy achieves a new state of the art on all major publicly available CARLA closed-loop benchmarks, reaching 95 DS on Bench2Drive and more than doubling prior performances on Longest6~v2 and Town13. Additionally, by integrating perception supervision from our dataset into a shared sim-to-real pipeline, we show consistent gains on the NAVSIM and Waymo Vision-Based End-to-End driving benchmarks. Our code, data, and models are publicly available at https://github.com/autonomousvision/lead.

LEAD: Minimizing Learner-Expert Asymmetry in End-to-End Driving

TL;DR

The work identifies and mitigates learner–expert asymmetries in end-to-end driving by aligning expert demonstrations to sensor-based observations and strengthening navigation intent. It introduces LEAD, a large-scale CARLA dataset, and TFv6, a TransFuser-based policy that delivers state-of-the-art closed-loop performance on CARLA benchmarks and shows sim-to-real gains when paired with real-world data. Key contributions include state and intent alignment techniques, removal of late-stage goal conditioning bottlenecks, and a multi-point navigation conditioning strategy that reduces target-point bias. The findings highlight the practical impact of expert design and intent specification as drivers of real-world robustness in simulation-based imitation learning.

Abstract

Simulators can generate virtually unlimited driving data, yet imitation learning policies in simulation still struggle to achieve robust closed-loop performance. Motivated by this gap, we empirically study how misalignment between privileged expert demonstrations and sensor-based student observations can limit the effectiveness of imitation learning. More precisely, experts have significantly higher visibility (e.g., ignoring occlusions) and far lower uncertainty (e.g., knowing other vehicles' actions), making them difficult to imitate reliably. Furthermore, navigational intent (i.e., the route to follow) is under-specified in student models at test time via only a single target point. We demonstrate that these asymmetries can measurably limit driving performance in CARLA and offer practical interventions to address them. After careful modifications to narrow the gaps between expert and student, our TransFuser v6 (TFv6) student policy achieves a new state of the art on all major publicly available CARLA closed-loop benchmarks, reaching 95 DS on Bench2Drive and more than doubling prior performances on Longest6~v2 and Town13. Additionally, by integrating perception supervision from our dataset into a shared sim-to-real pipeline, we show consistent gains on the NAVSIM and Waymo Vision-Based End-to-End driving benchmarks. Our code, data, and models are publicly available at https://github.com/autonomousvision/lead.
Paper Structure (12 sections, 2 figures, 7 tables)

This paper contains 12 sections, 2 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Minimizing Learner-Expert Asymmetry
  4. Preliminaries
  5. State Alignment
  6. Intent Alignment
  7. Discussion
  8. Experiments
  9. Closed-Loop Benchmarks
  10. Real-World Data Benchmarks
  11. Conclusion
  12. Limitations

Figures (2)

  • Figure 1: Performing a task well and teaching it well are not the same. An expert driver (blue bounding box) is most useful when its behavior can be transferred to a student policy (green bounding box) effectively. Current expert drivers for CARLA do not fulfill this requirement. We focus on three common asymmetries that hinder effective transfer. Visibility asymmetry: the expert reacts to occluded actors, leading to non-causal and less useful demonstrations. Uncertainty asymmetry: the expert's noiseless state inputs (e.g., accelerations and velocities of other vehicles) lead to successful but dangerous demonstrations. Intent asymmetry: the student's intent is under-specified (as a single target point) making it unaware of complex multi-lane maneuvers. Our approach reduces expert privileges, enforces sensor-aware demonstrations and redesigns the policy's navigation conditioning, resulting in state-of-the-art closed-loop driving in CARLA.
  • Figure 2: Effect of Aligned Supervision and Conditioning. Infractions counted per 100km, lower is better. Stat: Collisions with Layout; Ped: Collisions with Pedestrian; Veh: Collision Vehicle; OL: Outside Lane; Red: Red Light; Dev: Route Deviation; SI: Stop Infraction; Block: Vehicle Blocked.