Delta-Triplane Transformers as Occupancy World Models

TL;DR

Delta-Triplane Transformers (DTT) tackle the inefficiency of full-state occupancy forecasting by employing a compact triplane latent representation and predicting occupancy changes delta-wise in an autoregressive, plane-specific manner. The approach couples pre-trained triplane encoders/decoders with three plane-specific, multi-scale Transformers to forecast future deltas, which are decoded into occupancy and used as sparse queries for motion planning. This delta-based forecasting reduces variance, enables lighter architectures, and mitigates error accumulation over time, achieving state-of-the-art 4D occupancy forecasting and planning performance with real-time speed (26 FPS). Extensive experiments on Occ3D and nuScenes demonstrate superior mIoU and planning accuracy, validating the practicality of delta-triplane OWMs for autonomous driving.

Abstract

Occupancy World Models (OWMs) aim to predict future scenes via 3D voxelized representations of the environment to support intelligent motion planning. Existing approaches typically generate full future occupancy states from VAE-style latent encodings, which can be computationally expensive and redundant. We propose Delta-Triplane Transformers (DTT), a novel 4D OWM for autonomous driving, that introduces two key innovations: (1) a triplane based representation that encodes 3D occupancy more compactly than previous approaches, and (2) an incremental prediction strategy for OWM that models {\em changes} in occupancy rather than dealing with full states. The core insight is that changes in the compact 3D latent space are naturally sparser and easier to model, enabling higher accuracy with a lighter-weight architecture. Building on this representation, DTT extracts multi-scale motion features from historical data and iteratively predict future triplane deltas. These deltas are combined with past states to decode future occupancy and ego-motion trajectories. Extensive experiments demonstrate that DTT delivers a 1.44$\times$ speedup (26 FPS) over the state of the art, improves mean IoU to 30.85, and reduces the mean absolute planning error to 1.0 meters. Demo videos are provided in the supplementary material.

Figures (8)

• Figure 1: Comparison of occupancy world model architectures: (a) Conventional approaches use a pre-trained VAE to compress occupancy state $S$ into a compact BEV representation, which is then combined with previous actions $a$ to predict future BEVs and actions via a large Transformer. (b) Our method, DTT, adopts more compact and precise triplane representations and uses lightweight Transformers to separately predict the triplane changes of each plane. These deltas are then used to for occupancy forecasting and motion planning.
• Figure 2: (a-b) Comparison between the full occupancy and occupancy changes ($\Delta$) of a scene. In the change map, red indicates newly appeared voxels, while blue denotes disappeared voxels. (c) Distribution of latent-space feature values for full occupancy versus occupancy changes.
• Figure 3: Workflow of DTT. DTT first pre-trains compact triplane representations of occupancy data. It then applies multi-scale Transformers to model temporal dynamics within each plane and predict future triplane changes ($\Delta$). Finally, these changes, combined with the previous triplane, are used to generate future occupancy results and motion proposals.
• Figure 4: Pretrained triplane for OWM.
• Figure 5: Motion planning with DTT.