DDP-WM: Disentangled Dynamics Prediction for Efficient World Models

TL;DR

DDP-WM presents a disentangled dynamics framework for world modeling that decouples sparse primary dynamics from context-driven background updates to enable efficient, real-time planning. The four-stage pipeline—Historical information fusion, Dynamic localization, Sparse primary dynamics prediction, and Low-Rank correction—paired with a cross-attention-based background update, yields substantial planning accuracy with dramatically reduced compute. Key contributions include the Low-Rank Correction Module, a highly efficient sparse MPC cost strategy, and empirical validation showing up to 9x inference speedups and improved success rates on challenging tasks like Push-T, while preserving or enhancing planning performance. This approach offers a practical path toward high-fidelity, real-time world models suitable for complex robotic manipulation and navigation in dynamic environments.

Abstract

World models are essential for autonomous robotic planning. However, the substantial computational overhead of existing dense Transformerbased models significantly hinders real-time deployment. To address this efficiency-performance bottleneck, we introduce DDP-WM, a novel world model centered on the principle of Disentangled Dynamics Prediction (DDP). We hypothesize that latent state evolution in observed scenes is heterogeneous and can be decomposed into sparse primary dynamics driven by physical interactions and secondary context-driven background updates. DDP-WM realizes this decomposition through an architecture that integrates efficient historical processing with dynamic localization to isolate primary dynamics. By employing a crossattention mechanism for background updates, the framework optimizes resource allocation and provides a smooth optimization landscape for planners. Extensive experiments demonstrate that DDP-WM achieves significant efficiency and performance across diverse tasks, including navigation, precise tabletop manipulation, and complex deformable or multi-body interactions. Specifically, on the challenging Push-T task, DDP-WM achieves an approximately 9 times inference speedup and improves the MPC success rate from 90% to98% compared to state-of-the-art dense models. The results establish a promising path for developing efficient, high-fidelity world models. Codes will be available at https://github.com/HCPLabSYSU/DDP-WM.

Figures (9)

• Figure 1: (a) PCA visualization of internal feature evolution in a dense model. (b) PCA of the difference between consecutive ground-truth features. In (a), PCA projection of features from each predictor layer shows background regions remain largely static, revealing the computational redundancy of dense models. In (b), the PCA of the feature difference shows most regions (green) have near-zero change, demonstrating the inherent sparsity of physical dynamics.
• Figure 2: Overall Performance on Key Benchmarks. For comparability, Success Rates are plotted directly, while Chamfer Distance (CD) values are normalized via the formula $(\text{max}_{\text{CD}} - \text{CD}_{\text{i}}) / (\text{max}_{\text{CD}} - \text{min}_{\text{CD}}) \times 100$.
• Figure 3: Overview of the DDP-WM Framework. Our framework performs prediction through a four-stage decoupled process. (1) Historical Information Fusion: The features of the current frame, $\mathbf{z}_t$, query historical frame features $\mathbf{Z}_{\text{hist}}$ via cross-attention to obtain temporally-aware augmented features $\mathbf{z}'_t$. (2) Dynamic Localization: A lightweight network receives $\mathbf{z}'_t$ and the action $\mathbf{a}_t$ to predict a sparse mask $\mathbf{M}$ that contains only the primary dynamic regions. (3) Sparse Primary Dynamics Prediction: A powerful primary predictor focuses all computation on the sparse foreground features identified by $\mathbf{M}$ to predict the next frame's foreground features, $\mathbf{z}'_{t+1, \text{fg}}$, with high precision. (4) Contextual Background Update: The background features of the current frame, $\mathbf{z}_{t, \text{bg}}$, are updated at a very low cost by querying the newly predicted foreground features $\mathbf{z}'_{t+1, \text{fg}}$ via cross-attention. Finally, the updated foreground and background are combined to constitute the complete latent state of the next frame.
• Figure 4: Overview of Evaluation Environments. Sample frames from the five simulated task domains used in our experiments.
• Figure 5: Comparison of MPC cost function landscapes created by different models on the Push-T task. Both plots show the cost surfaces after 2D perturbation of the action space. (Left) Naive Sparse model (w/o LRM): The cost landscape is rugged and noisy, trapping the optimizer in local minima. (Right) Our DDP-WM model (w/ LRM): In contrast, the landscape is smooth with a clear, funnel-shaped global minimum, enabling efficient optimization.