Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Latent-DARM: Bridging Discrete Diffusion And Autoregressive Models For Reasoning

Lina Berrayana, Ahmed Heakl, Abdullah Sohail, Thomas Hofmann, Salman Khan, Wei Chen

TL;DR

This work introduces Latent-DARM, a latent-space communication framework bridging DDLM (planners) and ARM (executors), maximizing collaborative benefits, and approaches the results of state-of-the-art reasoning models while using less than 2.2% of its token budget.

Abstract

Most multi-agent systems rely exclusively on autoregressive language models (ARMs) that are based on sequential generation. Although effective for fluent text, ARMs limit global reasoning and plan revision. On the other hand, Discrete Diffusion Language Models (DDLMs) enable non-sequential, globally revisable generation and have shown strong planning capabilities, but their limited text fluency hinders direct collaboration with ARMs. We introduce Latent-DARM, a latent-space communication framework bridging DDLM (planners) and ARM (executors), maximizing collaborative benefits. Across mathematical, scientific, and commonsense reasoning benchmarks, Latent-DARM outperforms text-based interfaces on average, improving accuracy from 27.0% to 36.0% on DART-5 and from 0.0% to 14.0% on AIME2024. Latent-DARM approaches the results of state-of-the-art reasoning models while using less than 2.2% of its token budget. This work advances multi-agent collaboration among agents with heterogeneous models.

Latent-DARM: Bridging Discrete Diffusion And Autoregressive Models For Reasoning

TL;DR

This work introduces Latent-DARM, a latent-space communication framework bridging DDLM (planners) and ARM (executors), maximizing collaborative benefits, and approaches the results of state-of-the-art reasoning models while using less than 2.2% of its token budget.

Abstract

Most multi-agent systems rely exclusively on autoregressive language models (ARMs) that are based on sequential generation. Although effective for fluent text, ARMs limit global reasoning and plan revision. On the other hand, Discrete Diffusion Language Models (DDLMs) enable non-sequential, globally revisable generation and have shown strong planning capabilities, but their limited text fluency hinders direct collaboration with ARMs. We introduce Latent-DARM, a latent-space communication framework bridging DDLM (planners) and ARM (executors), maximizing collaborative benefits. Across mathematical, scientific, and commonsense reasoning benchmarks, Latent-DARM outperforms text-based interfaces on average, improving accuracy from 27.0% to 36.0% on DART-5 and from 0.0% to 14.0% on AIME2024. Latent-DARM approaches the results of state-of-the-art reasoning models while using less than 2.2% of its token budget. This work advances multi-agent collaboration among agents with heterogeneous models.
Paper Structure (38 sections, 11 equations, 5 figures, 4 tables, 2 algorithms)

This paper contains 38 sections, 11 equations, 5 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Why consider a planner–executor framework?
  3. Contributions.
  4. Organization.
  5. Preliminaries
  6. Definitions
  7. Motivation
  8. DDLMs for planning.
  9. Fluency Problem.
  10. Remark.
  11. Problem Formulation
  12. Latent-Space Communication.
  13. Challenge : Embedding Space Mismatch.
  14. Latent-DARM
  15. System Architecture
  16. ...and 23 more sections

Figures (5)

  • Figure 1: Accuracy–cost trade-offs across planner–executor configurations. Here DDLM refers to a LLada-8B-Instruct and ARM to a Llama-3.2-3B-Instruct model. DDLM$\to$ARM, particularly with latent-space exchange, achieves higher reasoning accuracy at lower token budgets compared to ARM-only models.
  • Figure 2: Overview of the latent-space collaboration pipeline. A discrete diffusion language model (DDLM) generates a latent plan. The plan is projected directly into the autoregressive model (ARM) embedding space through a learned projector (in blue). The ARM then conditions on the plan and the question to produce the final answer.
  • Figure 3: Accuracy comparison of text-space vs. latent-space collaboration.
  • Figure 4: Diagnostic configurations for attributing errors to planner or executor. Setup X tests whether failures stem from the planner: if replacing the diffusion planner (DDLM) with an autoregressive planner (ARM) fixes the output, the error is attributed to the DDLM. Setup Y tests executor reliability: if a diffusion executor succeeds where an ARM executor fails, the limitation lies in the executor.
  • Figure 5: Planner vs. executor failures in text- vs. latent-space collaboration. Results for LLaDA-8B-Instruct and Llama-3.2-3B-Instruct. Latent-space collaboration substantially reduces planning errors compared to text-space.