CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Credits

TL;DR

This work tackles inefficiencies in diffusion large language model (dLLM) inference under parallel decoding by exposing two bottlenecks: redundant remasking of tokens that stabilize early and history-agnostic decisions that ignore past convergence signals. It introduces CreditDecoding, a training-free approach that builds a token-level Trace Credit from historical logits and fuses it with current logits via logit augmentation to accelerate convergence for underconfident but correct tokens. Empirical results across eight benchmarks show substantial speedups (up to 5.48×) with minimal accuracy loss and improved robustness, while remaining orthogonal to existing optimizations such as KV caching and EOS stopping, and scaling to long contexts. The method is readily integrable and offers a practical path to faster, more reliable dLLM inference in real-world deployments as base models become stronger.

Abstract

Diffusion large language models (dLLMs) generate text through iterative denoising steps, achieving parallel decoding by denoising only high-confidence positions at each step. However, existing approaches often repetitively remask tokens due to initially low confidence scores, leading to redundant iterations and limiting overall acceleration. Through the analysis of dLLM decoding traces, we observe that the model often determines the final prediction for a token several steps before the decoding step. To leverage this historical information and avoid redundant steps, we introduce the concept of Trace Credit, which quantifies each token's convergence potential by accumulating historical logits. Furthermore, we propose CreditDecoding, a training-free parallel decoding algorithm that accelerates the confidence convergence of correct but underconfident tokens by fusing current logits with Trace Credit. This process significantly reduces redundant iterations and enhances decoding robustness. On eight benchmarks, CreditDecoding achieves a 5.48 times speedup and a 0.48 performance improvement over LLaDA-8B-Instruct, and a 4.11 times speedup with a 0.15 performance improvement over LLaDA-MoE-Instruct. Importantly, CreditDecoding scales effectively to long sequences and is orthogonal to mainstream inference optimizations, making it a readily integrable and versatile solution.

Figures (15)

• Figure 1: Decoding Boundaries on GSM8K
• Figure 2: Decoding Boundaries on HumanEval
• Figure 4: Orthogonality and scalability of CreditDecoding on LLaDA-8B-Instruct. In the left figure(w/o Early Stop), each colored dot denotes an acceleration method, and the corresponding star shows its performance with CreditDecoding. In the right figure(w/ Early Stop), the line indicates the TPF, while the bars represent the average score.
• Figure 5: Confidence rank of the final predicted token at each position during the generation steps(Left: LLaDA-8B-Inst, Middle: Fast-dLLM, Right: CreditDecoding). The correct token refers to the model’s final prediction. Each data point represents the softmax rank of the final output token on a log-scale, color-coded from yellow (top-1) to blue (lower ranks). The red dots denote that the model actually decoded this token at the corresponding step, which is also the Decoding Boundary in Figure\ref{['fig:temperal_gap']}.
• Figure 6: Comparison between standard dLLM parallel decoding (left) and the proposed CreditDecoding (right). The left diagram illustrates how existing methods decode solely on instantaneous predictions at each step, causing the repetitive remasking of correct tokens. In contrast, CreditDecoding maintains a token-level credit value across steps, using Trace Credit as a prior to enhance and calibrate current predictions.