Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

WavefrontDiffusion: Dynamic Decoding Schedule for Improved Reasoning

Haojin Yang, Rui Hu, Zequn Sun, Rui Zhou, Yujun Cai, Yiwei Wang

TL;DR

WavefrontDiffusion introduces an adaptive, wavefront-based decoding schedule for diffusion-based text generation, expanding a frontier of active tokens from finalized regions to balance semantic coherence with compute efficiency. Across four reasoning and code-generation benchmarks, it achieves state-of-the-art accuracy and higher semantic fidelity without increasing the token-update budget compared to block-based methods. The work also introduces MHCO as a diagnostic of alignment with semantic order and demonstrates robust, practical hyperparameter settings. Together, these results advocate for adaptive decoding schedules to improve long-form, structured generation in diffusion-based models.

Abstract

Diffusion Language Models (DLMs) have shown strong potential for text generation and are becoming a competitive alternative to autoregressive models. The denoising strategy plays an important role in determining the quality of their outputs. Mainstream denoising strategies include Standard Diffusion and BlockDiffusion. Standard Diffusion performs global denoising without restricting the update range, often finalizing incomplete context and causing premature end-of-sequence predictions. BlockDiffusion updates fixed-size blocks in a preset order, but its rigid structure can break apart coherent semantic units and disrupt reasoning. We present WavefrontDiffusion, a dynamic decoding approach that expands a wavefront of active tokens outward from finalized positions. This adaptive process follows the natural flow of semantic structure while keeping computational cost equal to block-based methods. Across four benchmarks in reasoning and code generation, WavefrontDiffusion achieves state-of-the-art performance while producing outputs with higher semantic fidelity, showing the value of adaptive scheduling for more coherent and efficient generation.

WavefrontDiffusion: Dynamic Decoding Schedule for Improved Reasoning

TL;DR

WavefrontDiffusion introduces an adaptive, wavefront-based decoding schedule for diffusion-based text generation, expanding a frontier of active tokens from finalized regions to balance semantic coherence with compute efficiency. Across four reasoning and code-generation benchmarks, it achieves state-of-the-art accuracy and higher semantic fidelity without increasing the token-update budget compared to block-based methods. The work also introduces MHCO as a diagnostic of alignment with semantic order and demonstrates robust, practical hyperparameter settings. Together, these results advocate for adaptive decoding schedules to improve long-form, structured generation in diffusion-based models.

Abstract

Diffusion Language Models (DLMs) have shown strong potential for text generation and are becoming a competitive alternative to autoregressive models. The denoising strategy plays an important role in determining the quality of their outputs. Mainstream denoising strategies include Standard Diffusion and BlockDiffusion. Standard Diffusion performs global denoising without restricting the update range, often finalizing incomplete context and causing premature end-of-sequence predictions. BlockDiffusion updates fixed-size blocks in a preset order, but its rigid structure can break apart coherent semantic units and disrupt reasoning. We present WavefrontDiffusion, a dynamic decoding approach that expands a wavefront of active tokens outward from finalized positions. This adaptive process follows the natural flow of semantic structure while keeping computational cost equal to block-based methods. Across four benchmarks in reasoning and code generation, WavefrontDiffusion achieves state-of-the-art performance while producing outputs with higher semantic fidelity, showing the value of adaptive scheduling for more coherent and efficient generation.

Paper Structure

This paper contains 30 sections, 6 equations, 2 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Preliminaries: Diffusion Language Models
  4. Standard Diffusion Decoding
  5. Block-wise Diffusion Scheduling
  6. WavefrontDiffusion Method
  7. Method Theory
  8. Method: Algorithmic Details
  9. Experiments
  10. Experimental Setup
  11. Main Results
  12. Semantic Quality Evaluation
  13. Measuring Priority Violations via MHCO
  14. Hyperparameter Analysis
  15. Related Work
  16. ...and 15 more sections

Figures (2)

  • Figure 1: Illustration of scheduling strategies in diffusion language models. A Diffusion Language Model predicts all masked tokens. At each step, a scheduling strategy decides which tokens to denoise. There are three strategies: Standard Diffusion uses global scheduling. It always selects tokens from all masked positions. BlockDiffusion limits the choice to a candidate scope. The scope is fixed blocks, similar to semi-autoregression. WavefrontDiffusion also uses a candidate scope. Unlike BlockDiffusion, the scope grows dynamically during decoding.
  • Figure 2: Comparison of MHCO values across strategies. Lower values mean fewer priority violations and closer alignment with semantic order.