Mixed-Density Diffuser: Efficient Planning with Non-Uniform Temporal Resolution

TL;DR

The paper tackles the problem of long-horizon offline RL planning with diffusion models by introducing Mixed-Density Diffuser (MDD), a single flat diffusion model that allocates non-uniform temporal density along a trajectory via independent jump sizes $K_i$ across horizon $H$. Built on the Diffusion Veteran framework with a Diffusion Transformer backbone and Monte Carlo guidance, MDD outputs state-only trajectories using a diffusion-based inverse dynamics model, without increasing parameters or inference costs. Empirically, MDD achieves state-of-the-art performance on Maze2D, AntMaze, and Franka Kitchen in D4RL, supporting the claim that non-uniform temporal horizons significantly improve planning efficiency. The work demonstrates that precise temporal density allocation can outperform both uniform-density planners and hierarchical ensembles, and suggests future work on learning optimal density schedules for further gains.

Abstract

Recent studies demonstrate that diffusion planners benefit from sparse-step planning over single-step planning. Training models to skip steps in their trajectories helps capture long-term dependencies without additional or memory computational cost. However, predicting excessively sparse plans degrades performance. We hypothesize this temporal density threshold is non-uniform across a temporal horizon and that certain parts of a planned trajectory should be more densely planned. We propose Mixed-Density Diffuser (MDD), a diffusion planner where the densities throughout the horizon are tunable hyperparameters. We show that MDD achieves a new SOTA across the Maze2D, Franka Kitchen, and Antmaze D4RL task domains.

Figures (1)

• Figure 1: Comparison of trajectory denoising in different diffusion planning approaches. Sparse Density Diffusers in row $1)$ extend temporal planning horizons with comparatively little computational cost at the price of low temporal resolution. High Density Diffusers in row $2)$ compute many steps for shorter temporal horizons creating more continuous planned trajectories. Hierarchical Planners in row $3)$HDliteHDMI generate way-points with a Sparse Density Diffuser and interpolate between with a High-Density Diffuser. MDD (bottom row) utilizes the benefits of sparse and dense planning with a simple, flat framework by treating the sparse and dense observations as a single trajectory for one planner to generate. Row $5)$ shows fully denoised example trajectories that would be non-trivial to generate with Hierarchical Planners, but simple to implement with MDD.