Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reasoning with Autoregressive-Diffusion Collaborative Thoughts

Mu Yuan, Liekang Zeng, Guoliang Xing, Lan Zhang, Yunhao Liu

TL;DR

Reasoning with Autoregressive-Diffusion Collaborative Thoughts tackles the mismatch between symbolic planning and spatial grounding by proposing a closed-loop AR-Diffusion framework. It comprises three components—Planner (autoregressive), Simulator (diffusion), and Critic (vision-language)—to perform a Simulate-Critic-Refine cycle that grounds reasoning in pixel-space priors. The key contributions include formalizing this loop, enabling modality-agnostic task execution, and demonstrating improved spatial reasoning and generation controllability over AR-Only and Diffusion-Only baselines, with notable gains in inference efficiency and robustness. This work suggests a scalable path toward robust spatial intelligence by leveraging mutual critique between symbolic planning and perceptual generation, with future extensions to 3D assets, video dynamics, and efficiency improvements.

Abstract

Autoregressive and diffusion models represent two complementary generative paradigms. Autoregressive models excel at sequential planning and constraint composition, yet struggle with tasks that require explicit spatial or physical grounding. Diffusion models, in contrast, capture rich spatial structure through high-dimensional generation, but lack the stepwise logical control needed to satisfy complex, multi-stage constraints or to reliably identify and correct errors. We introduce Collaborative Thoughts, a unified collaborative framework that enables autoregressive and diffusion models to reason and generate jointly through a closed-loop interaction. In Collaborative Thoughts, autoregressive models perform structured planning and constraint management, diffusion models instantiate these constraints as intermediate visual thoughts, and a vision-based critic module evaluates whether the visual thoughts satisfy the intended structural and physical requirements. This feedback is then used to iteratively refine subsequent planning and generation steps, mitigating error propagation across modalities. Importantly, Collaborative Thoughts uses the same collaborative loop regardless of whether the task is autoregressive question answering or diffusion-based visual generation. Through representative examples, we illustrate how Collaborative Thoughts can improve the reliability of spatial reasoning and the controllability of generation.

Reasoning with Autoregressive-Diffusion Collaborative Thoughts

TL;DR

Reasoning with Autoregressive-Diffusion Collaborative Thoughts tackles the mismatch between symbolic planning and spatial grounding by proposing a closed-loop AR-Diffusion framework. It comprises three components—Planner (autoregressive), Simulator (diffusion), and Critic (vision-language)—to perform a Simulate-Critic-Refine cycle that grounds reasoning in pixel-space priors. The key contributions include formalizing this loop, enabling modality-agnostic task execution, and demonstrating improved spatial reasoning and generation controllability over AR-Only and Diffusion-Only baselines, with notable gains in inference efficiency and robustness. This work suggests a scalable path toward robust spatial intelligence by leveraging mutual critique between symbolic planning and perceptual generation, with future extensions to 3D assets, video dynamics, and efficiency improvements.

Abstract

Autoregressive and diffusion models represent two complementary generative paradigms. Autoregressive models excel at sequential planning and constraint composition, yet struggle with tasks that require explicit spatial or physical grounding. Diffusion models, in contrast, capture rich spatial structure through high-dimensional generation, but lack the stepwise logical control needed to satisfy complex, multi-stage constraints or to reliably identify and correct errors. We introduce Collaborative Thoughts, a unified collaborative framework that enables autoregressive and diffusion models to reason and generate jointly through a closed-loop interaction. In Collaborative Thoughts, autoregressive models perform structured planning and constraint management, diffusion models instantiate these constraints as intermediate visual thoughts, and a vision-based critic module evaluates whether the visual thoughts satisfy the intended structural and physical requirements. This feedback is then used to iteratively refine subsequent planning and generation steps, mitigating error propagation across modalities. Importantly, Collaborative Thoughts uses the same collaborative loop regardless of whether the task is autoregressive question answering or diffusion-based visual generation. Through representative examples, we illustrate how Collaborative Thoughts can improve the reliability of spatial reasoning and the controllability of generation.
Paper Structure (11 sections, 4 equations, 3 figures)

This paper contains 11 sections, 4 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Problem Formulation
  5. The Planner: Semantic-to-Visual Translation
  6. The Simulator: Physical Instantiation
  7. The Critic: Visual-Logic Alignment
  8. Termination and Inference
  9. Demonstrations
  10. Discussion
  11. Conclusion

Figures (3)

  • Figure 1: Traditional chain-of-thought (CoT) wei2022chain ponders queries via only text, and Visualization of Thoughts (VoT) wu2024mindli2025imagine relies on visual input to initiate the visualization of thinking traces. Collaborative Thoughts orchestrates autoregressive and diffusion models to collaboratively think via multimodal reasoning traces.
  • Figure 2: While text-based reasoning (AR-Only) struggles with spatial abstraction and direct generation (Diffusion-Only) lacks structural logic, our framework leverages the complementary strengths of both. The autoregressive (AR) model acts as a planner to break down the diffusion process into four discrete steps. This step-by-step visual verification prevents the error propagation seen in the baselines, allowing for accurate identification of the final geometric shapes.
  • Figure 3: Illustration of reasoning paradigms for geometric problem solving. The proposed AR-Diffusion Collaborative Thoughts (center) creates intermediate visual blueprints to bridge the gap between text and vision. This approach corrects the geometric hallucinations observed in Diffusion-Only methods (right) and significantly improves inference efficiency compared to the AR-Only textual chain-of-thought (left), reducing computational costs by orders of magnitude.