SparseWorld: A Flexible, Adaptive, and Efficient 4D Occupancy World Model Powered by Sparse and Dynamic Queries

TL;DR

SparseWorld tackles the rigidity and computational burden of grid-based 4D occupancy models by introducing fully sparse, dynamic queries guided by the ego vehicle. It combines Range-Adaptive Perception with State-Conditioned Forecasting and a Temporal-Aware Self-Scheduling training strategy to achieve extended-range, continuous 4D scene understanding and autoregressive planning. Empirical results on Occ3d-nuScenes show state-of-the-art 4D forecasting and planning performance with substantial inference speed-ups over dense baselines. By reframing forecasting as regression over a sparse, continuous 4D space and avoiding in-place voxel classification, SparseWorld reduces feature drift and improves robustness in dynamic driving scenarios.

Abstract

Semantic occupancy has emerged as a powerful representation in world models for its ability to capture rich spatial semantics. However, most existing occupancy world models rely on static and fixed embeddings or grids, which inherently limit the flexibility of perception. Moreover, their ``in-place classification" over grids exhibits a potential misalignment with the dynamic and continuous nature of real scenarios. In this paper, we propose SparseWorld, a novel 4D occupancy world model that is flexible, adaptive, and efficient, powered by sparse and dynamic queries. We propose a Range-Adaptive Perception module, in which learnable queries are modulated by the ego vehicle states and enriched with temporal-spatial associations to enable extended-range perception. To effectively capture the dynamics of the scene, we design a State-Conditioned Forecasting module, which replaces classification-based forecasting with regression-guided formulation, precisely aligning the dynamic queries with the continuity of the 4D environment. In addition, We specifically devise a Temporal-Aware Self-Scheduling training strategy to enable smooth and efficient training. Extensive experiments demonstrate that SparseWorld achieves state-of-the-art performance across perception, forecasting, and planning tasks. Comprehensive visualizations and ablation studies further validate the advantages of SparseWorld in terms of flexibility, adaptability, and efficiency.

Figures (4)

• Figure 1: (a) Perception-forecasting-decoupled methods. (b) Grid feature-based methods. (c) We adopt dynamic query representations, which facilitate continuous and coherent 4D scene forecasting and planning.
• Figure 2: The overall architecture of SparseWorld. A set of learnable queries encoded with the historical trajectory through an Adaptive Scaling are then fed into stacked spatio-temporal decoders, which interacts with multi-frame, multi-view image features via a Temporal-Spatial MHSA. Subsequently, the extended-range queries are dynamically refined through a State-Conditional Forecasting module, which refines their positions while guiding the ego-vehicle’s motion state in an autoregressive manner. Finally, two parallel decoders are employed for forecasting and planning, respectively.
• Figure 3: Visualization of the current and future 3-second ground truth and forecasts. As highlighted in red boxes, PreWorld preworld, which relies on grid-based modeling and voxel-level classification, exhibits severe distortions in long horizon, whereas SparseWorld effectively mitigates such issues.
• Figure 4: Visualizations of intermediate reasoning representations.