Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Compositional Monte Carlo Tree Diffusion for Extendable Planning

Jaesik Yoon, Hyeonseo Cho, Sungjin Ahn

TL;DR

The paper tackles the inherent horizon limitation of diffusion-based planning by introducing Compositional Monte Carlo Tree Diffusion (C-MCTD), which reasons over plan compositions rather than single trajectories. It presents three variants—Online Composer for globally coherent plan stitching, Distributed Composer for parallel exploration, and Preplan Composer for offline plan graphs—that collectively enable long-horizon planning without retraining. Empirical results across long mazes, multi-object manipulation, and visual navigation show substantial improvements over existing replanning and stitching methods, with Preplan Composer achieving perfect success on particularly challenging tasks and extending plans roughly tenfold beyond training. While noting computational trade-offs and environment-dependence, the work demonstrates that inference-time compositional planning can dramatically expand the capabilities of diffusion planners for complex sequential decision-making.

Abstract

Monte Carlo Tree Diffusion (MCTD) integrates diffusion models with structured tree search to enable effective trajectory exploration through stepwise reasoning. However, MCTD remains fundamentally limited by training trajectory lengths. While periodic replanning allows plan concatenation for longer plan generation, the planning process remains locally confined, as MCTD searches within individual trajectories without access to global context. We propose Compositional Monte Carlo Tree Diffusion (C-MCTD), a framework that elevates planning from individual trajectory optimization to reasoning over complete plan compositions. C-MCTD introduces three complementary components: (1) Online Composer, which performs globally-aware planning by searching across entire plan compositions; (2) Distributed Composer, which reduces search complexity through parallel exploration from multiple starting points; and (3) Preplan Composer, which accelerates inference by leveraging cached plan graphs.

Compositional Monte Carlo Tree Diffusion for Extendable Planning

TL;DR

The paper tackles the inherent horizon limitation of diffusion-based planning by introducing Compositional Monte Carlo Tree Diffusion (C-MCTD), which reasons over plan compositions rather than single trajectories. It presents three variants—Online Composer for globally coherent plan stitching, Distributed Composer for parallel exploration, and Preplan Composer for offline plan graphs—that collectively enable long-horizon planning without retraining. Empirical results across long mazes, multi-object manipulation, and visual navigation show substantial improvements over existing replanning and stitching methods, with Preplan Composer achieving perfect success on particularly challenging tasks and extending plans roughly tenfold beyond training. While noting computational trade-offs and environment-dependence, the work demonstrates that inference-time compositional planning can dramatically expand the capabilities of diffusion planners for complex sequential decision-making.

Abstract

Monte Carlo Tree Diffusion (MCTD) integrates diffusion models with structured tree search to enable effective trajectory exploration through stepwise reasoning. However, MCTD remains fundamentally limited by training trajectory lengths. While periodic replanning allows plan concatenation for longer plan generation, the planning process remains locally confined, as MCTD searches within individual trajectories without access to global context. We propose Compositional Monte Carlo Tree Diffusion (C-MCTD), a framework that elevates planning from individual trajectory optimization to reasoning over complete plan compositions. C-MCTD introduces three complementary components: (1) Online Composer, which performs globally-aware planning by searching across entire plan compositions; (2) Distributed Composer, which reduces search complexity through parallel exploration from multiple starting points; and (3) Preplan Composer, which accelerates inference by leveraging cached plan graphs.
Paper Structure (69 sections, 7 equations, 5 figures, 13 tables, 7 algorithms)

This paper contains 69 sections, 7 equations, 5 figures, 13 tables, 7 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Terminology.
  4. Diffusion models for planning
  5. Monte Carlo Tree Diffusion
  6. Compositional Monte Carlo Tree Diffusion
  7. Online Composer
  8. Stitching-based tree expansion.
  9. Guidance sets as meta-actions.
  10. Fast replanning for simulation.
  11. Scalability considerations.
  12. Distributed Composer
  13. Guidance-oriented parallel tree search.
  14. Strategic tree connection.
  15. Efficient path synthesis.
  16. ...and 54 more sections

Figures (5)

  • Figure 1: Comparison of stitched planning approaches. (a) Replanning: Sequential plan generation without exploring alternatives, leading to myopic decisions. (b) Online Composer: Systematic tree search over plan combinations to avoid local optima. (c) Distributed Composer: Parallel tree growth from multiple origins with strategic connections (red circles) for efficient exploration. (d) Preplan Composer: Leverages prebuilt plan graphs for rapid solution composition with minimal online search.
  • Figure 2: Architecture of Online Composer
  • Figure 3: Illustrations of the environments. Our method is evaluated on three distinct tasks: (a) A long-horizon maze where the agent must navigate from a start to a goal state. (b) A robotic manipulation task requiring precise control of a robotic arm. (c) A visual maze task where observations are provided as first-person view images.
  • Figure 4: Qualitative comparison of stitched planning approaches.(a) MCTD-Replan: Sequential replanning (trajectories shown from light to dark red) enables plan concatenation but results in myopic, short-sighted paths. (b) Online Composer: Plan-level search explores beyond the training data horizon. Gray lines denote discarded candidate plans during the search. (c) Distributed Composer: Search efficiency is improved by initiating searches from multiple starting positions (blue dots). Gray lines are unselected plans originating from the starting positions. (d) Preplan Composer: A pre-built graph is leveraged to generate higher-quality plans with more optimized efficiency. Gray lines represent pre-computed but unused plans.
  • Figure : Distributed Composer