Planned Diffusion

TL;DR

Planned diffusion tackles the speed-accuracy tension in LLM inference by introducing a two-stage hybrid: an autoregressive planning phase that structures output into independent spans, followed by a parallel diffusion phase that denoises all spans concurrently. The approach is formalized as a factorized model $p_{\text{PD}}(z,x|c)=p_{\text{AR}}(z|c)p_{\text{D}}(x|z,c)$ and supported by a specialized data pipeline, training objective with mixed attention masks, and a variable-step inference scheme using KV caching. Empirical results on AlpacaEval show notable speedups (1.27×–1.81×) with modest drops in LCWR (0.87%–5.4%), and the PD-DA variant improves quality at similar speeds; diffusion-based variants achieve the highest quality but with larger latency. The work demonstrates that a single model can effectively blend autoregressive planning with diffusion-driven parallelism, providing a scalable and tunable path toward faster, higher-quality text generation with broad practical impact.

Abstract

A central challenge in large language model inference is the trade-off between generation speed and output quality. Autoregressive models produce high-quality text but generate tokens sequentially. Diffusion models can generate tokens in parallel but often need many iterations to match the same quality. We propose planned diffusion, a hybrid method that combines the strengths of both paradigms. Planned diffusion works in two stages: first, the model creates a short autoregressive plan that breaks the output into smaller, independent spans. Second, the model generates these spans simultaneously using diffusion. This approach expands the speed-quality Pareto frontier and provides a practical path to faster, high-quality text generation. On AlpacaEval, a suite of 805 instruction-following prompts, planned diffusion achieves Pareto-optimal trade-off between quality and latency, achieving 1.27x to 1.81x speedup over autoregressive generation with only 0.87\% to 5.4\% drop in win rate, respectively. Our sensitivity analysis shows that the planning mechanism of planned diffusion is minimal and reliable, and simple runtime knobs exist to provide flexible control of the quality-latency trade-off.

Figures (5)

• Figure 1: A real example of planned diffusion. (1) Autoregressive Plan: The model first generates a sequential plan using control tags to define the structure and length of independent text spans. (2) Programmatic Scaffold: This plan is then translated into a scaffold where each span is initialized with a corresponding number of mask tokens. (3) Diffusion Denoise: Finally, the model denoises all spans in parallel with diffusion, generating the text for each section simultaneously to produce the complete response.
• Figure 2: The attention mask for planned diffusion combines causal and bidirectional attention. Causal attention is used for sequential planning stages. Bidirectional attention is used within $\texttt{<async>}$ spans for parallel denoising, with concurrent spans isolated from each other. After a $\texttt{<sync/>}$ token, subsequent tokens can to attend to all prior tokens. For illustrative purposes, this example shortens the diffusion spans.
• Figure 3: Evaluation of planned diffusion on the AlpacaEval benchmark. Left: A comparison of latency versus length-controlled win rate shows planned diffusion establishing a new Pareto frontier, offering a better trade-off between speed and quality. Middle: An analysis of the average critical path length reveals that planned diffusion requires substantially fewer sequential forward passes than autoregressive generation. Right: A scaling analysis shows that planned diffusion's win rate continues to improve with more training, while the autoregressive baseline's performance flatlines. Within each method, color brightness encodes training epochs: lightest to darkest corresponds to 2, 4, 8, and 16 epochs. AR: Autoregressive; PD: Planned Diffusion; PD-DA Planned Diffusion with Dense Attention; F-dLLM: Fast-dLLM.
• Figure 4: Additional analysis. Left (Plan ablation): Removing topic from autoregressive plan harms quality yet removing <sync/> tokens boosts speedup without significantly hurting quality. Middle (Span lengths): LCWR peaks at a length-scaling factor of 1.0; length prediction in autoregressive plan does not exhibit systemic error. Right (Quality versus Latency Sweep): Varying the step ratio ($r=\{0.25, 0.5, 0.75, 1\}$) and the confidence threshold ($\tau=\{0.4, 0.5, 0.6, 0.7, 0.8, 0.9\}$) hyperparameters produces a smooth quality–latency trade-off.
• Figure : Planned Diffusion