ZTRS: Zero-Imitation End-to-end Autonomous Driving with Trajectory Scoring

TL;DR

ZTRS addresses the challenge of end-to-end autonomous driving without relying on expert demonstrations by reframing planning as offline reinforcement learning over high-dimensional sensor inputs. It introduces Exhaustive Policy Optimization (EPO), which densely optimizes over an enumerable trajectory set $\mathcal{A}$ using a reward signal $\Psi(s,a)=\mathcal{E}(s,a)-b(s_t,a_t,a_{t-1})$ derived from the Extended Predictive Driver Model Score $\mathcal{E}$ and a temporal-consistency term $b$. The framework employs a Transformer-based trajectory scorer that jointly uses a discrete action space and a set of trajectory scores $\{\mathcal{S}_i(\cdot|s)\}$, enabling offline training on real-world data and robust performance across Navtest, Navhard, and HUGSIM benchmarks, with state-of-the-art results on Navhard and strong out-of-imitation performance on HUGSIM. The work demonstrates that reward-driven learning over high-dimensional sensory inputs can rival imitation-based methods, offering a data-efficient and safer path to reliable end-to-end autonomous driving. The findings suggest broad implications for offline RL in perception-rich control tasks and pave the way for further exploration of dense, reward-based supervision in real-world robotics.

Abstract

End-to-end autonomous driving maps raw sensor inputs directly into ego-vehicle trajectories to avoid cascading errors from perception modules and to leverage rich semantic cues. Existing frameworks largely rely on Imitation Learning (IL), which can be limited by sub-optimal expert demonstrations and covariate shift during deployment. On the other hand, Reinforcement Learning (RL) has recently shown potential in scaling up with simulations, but is typically confined to low-dimensional symbolic inputs (e.g. 3D objects and maps), falling short of full end-to-end learning from raw sensor data. We introduce ZTRS (Zero-Imitation End-to-End Autonomous Driving with Trajectory Scoring), a framework that combines the strengths of both worlds: sensor inputs without losing information and RL training for robust planning. To the best of our knowledge, ZTRS is the first framework that eliminates IL entirely by only learning from rewards while operating directly on high-dimensional sensor data. ZTRS utilizes offline reinforcement learning with our proposed Exhaustive Policy Optimization (EPO), a variant of policy gradient tailored for enumerable actions and rewards. ZTRS demonstrates strong performance across three benchmarks: Navtest (generic real-world open-loop planning), Navhard (open-loop planning in challenging real-world and synthetic scenarios), and HUGSIM (simulated closed-loop driving). Specifically, ZTRS achieves the state-of-the-art result on Navhard and outperforms IL-based baselines on HUGSIM. Code will be available at https://github.com/woxihuanjiangguo/ZTRS.

Figures (4)

• Figure 1: Comparisons between three paradigms for end-to-end autonomous driving.
• Figure 2: The Overall Framework of ZTRS. Given offline sensor data and a fixed set of trajectories, ZTRS first tokenizes these two modalities. In a Transformer Decoder, the trajectory tokens attend to image tokens to acquire the context. Finally, scoring heads and a policy head map the trajectory tokens to rule-based scores and action likelihoods.
• Figure 3: Visualizations of planned trajectories (blue curves) and the human trajectory (green curves) on the open-loop planning benchmark Navtest.
• Figure 4: Visualizations of planned trajectories (orange dots) on the challenging closed-loop driving benchmark HUGSIM.