Driving on Registers

TL;DR

DrivoR presents a compact, transformer-based end-to-end driving architecture that uses camera-aware register tokens to compress multi-camera perception into a lightweight scene representation. Two disentangled decoders generate and score trajectory proposals, with inference guided by a learned scoring function that can be conditioned to prioritize safety, comfort, or efficiency. The approach achieves state-of-the-art performance on NAVSIM-v1, NAVSIM-v2, and competitive results on HUGSIM, while delivering major efficiency gains over larger ViT baselines. This work demonstrates that a pure-transformer pipeline with targeted token compression can provide accurate, efficient, and adaptable autonomous driving without intermediate BEV representations or large trajectory dictionaries.

Abstract

We present DrivoR, a simple and efficient transformer-based architecture for end-to-end autonomous driving. Our approach builds on pretrained Vision Transformers (ViTs) and introduces camera-aware register tokens that compress multi-camera features into a compact scene representation, significantly reducing downstream computation without sacrificing accuracy. These tokens drive two lightweight transformer decoders that generate and then score candidate trajectories. The scoring decoder learns to mimic an oracle and predicts interpretable sub-scores representing aspects such as safety, comfort, and efficiency, enabling behavior-conditioned driving at inference. Despite its minimal design, DrivoR outperforms or matches strong contemporary baselines across NAVSIM-v1, NAVSIM-v2, and the photorealistic closed-loop HUGSIM benchmark. Our results show that a pure-transformer architecture, combined with targeted token compression, is sufficient for accurate, efficient, and adaptive end-to-end driving. Code and checkpoints will be made available via the project page.

Figures (12)

• Figure 1: DrivoR architecture. The proposed architecture is composed of three transformer blocks: one encoder (perception) and two decoders (trajectory and scoring). The perception encoder compresses perceptual information in camera-aware registers for lightweight subsequent processing in the trajectory and scoring decoders. The decoded trajectories are re-embedded and detached from the gradient computation graph to disentangle scoring and generation. The final trajectory is chosen from the proposal set via the max predicted score.
• Figure 2: Encoder and decoder architectures follow standard transformer architectures, with introduction of sensor registers in the encoder, and using these registers as scene tokens in downstream decoders.
• Figure 3: Cosine similarity between scene tokens. Darker indicates lower cosine similarity. Note the specialized tokens in the front cam, and collapsed tokens in the back cam, showing relative camera compression. Averaged on navval.
• Figure 4: Attention maps of scene tokens. From the final attention layer, front-camera tokens specialize to distinct regions (traffic light, lead vehicle, road edges), while back-camera tokens largely collapse to the same features, aside from a single distinct token.
• Figure 5: Scoring head disentanglement. Dominant cameras are identified via cross-attention between scene tokens and score or trajectory tokens. Trajectory prediction consistently relies on the front camera, while scoring shifts attention based on trajectory behavior—underscoring the need to separate the two pipelines.