Deferred Commitment Decoding for Diffusion Language Models with Confidence-Aware Sliding Windows

TL;DR

The paper addresses Boundary-Induced Context Truncation (BICT), a limitation of block-based diffusion decoding where tokens near block boundaries receive insufficient future context. It introduces Deferred Commitment Decoding (DCD), a training-free strategy that uses a confidence-aware sliding window to defer low-confidence tokens and expand contextual evidence, preserving KV-cache compatibility. Empirical results across full-attention and semi-causal diffusion architectures show average gains of $+1.39\%$ in accuracy with up to $+9.0\%$ in certain setups, along with comparable or improved decoding times, and clear evidence that DCD mitigates BICT by reducing low-confidence events. Overall, DCD offers a simple yet effective approach to enhance both the quality and efficiency of diffusion language model decoding, with broad applicability across benchmarks and caching configurations.

Abstract

Diffusion language models (DLMs) have recently emerged as a strong alternative to autoregressive models by enabling parallel text generation. To improve inference efficiency and KV-cache compatibility, prior work commonly adopts block-based diffusion, decoding tokens block by block. However, this paradigm suffers from a structural limitation that we term Boundary-Induced Context Truncation (BICT): undecoded tokens near block boundaries are forced to commit without access to nearby future context, even when such context could substantially reduce uncertainty. This limitation degrades decoding confidence and generation quality, especially for tasks requiring precise reasoning, such as mathematical problem solving and code generation. We propose Deferred Commitment Decoding (DCD), a novel, training-free decoding strategy that mitigates this issue. DCD maintains a confidence-aware sliding window over masked tokens, resolving low-uncertainty tokens early while deferring high-uncertainty tokens until sufficient contextual evidence becomes available. This design enables effective bidirectional information flow within the decoding window without sacrificing efficiency. Extensive experiments across multiple diffusion language models, benchmarks, and caching configurations show that DCD improves generation accuracy by 1.39% with comparable time on average compared to fixed block-based diffusion methods, with the most significant improvement reaching 9.0%. These results demonstrate that deferring token commitment based on uncertainty is a simple yet effective principle for improving both the quality and efficiency of diffusion language model decoding.

Figures (4)

• Figure 1: An overview of the proposed DCD algorithm.
• Figure 2: Confidence distributions of LLaDA-8B-Instruct, Dream-v0-Base-7B, and Fast-dLLM-v2-7B on GSM8K with DCD and (sub-)block-based decoding using dual cache. We use a log$_{10}$ scale for clarity. The DCD algorithm yields fewer low-confidence decoding steps across these models.
• Figure 3: Ablation studies on LLaDA-8B-Instruct on the MATH500 task with dual-cache DCD. We vary $s_{\text{max}}$, $s_{\text{init}}$, and $\tau_{\text{conf}}$, and evaluate task accuracy and decoding time.
• Figure 4: MBPP Task #129 solved by block-based decoding and the DCD algorithm using Dream-v0-Instruct-7B.