Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models

Shutong Wu, Jiawei Zhang

TL;DR

The paper tackles the bottleneck of DLLM inference speed while preserving generation quality. It introduces Free Dave, a training-free, model-free algorithm that uses the DLLM itself to draft and verify multiple candidate unmasking steps in parallel, resulting in lossless decoding. The authors prove that, with sufficient compute and memory, the method achieves the fewest forward calls under the lossless constraint and demonstrates up to $3.78\times$ throughput gains across math reasoning and code-generation benchmarks. Empirically, FreeDave outperforms threshold-based parallel decoding, maintains accuracy across tasks, and integrates with existing KV-cache frameworks to boost practical performance.

Abstract

Diffusion Large Language Models (DLLMs) have emerged as a new paradigm of language modeling beyond autoregressive next-token prediction. Thanks to their bidirectional attention mechanism, DLLMs are more capable of capturing the connection of context, and thus show unique advantages in challenges like the famous "reversal curse" or learning under data-constrained scenarios. In addition, taking advantage of their inherent modeling foundations, DLLMs have the great potential of efficient inference with parallel decoding algorithms, which enable multi-token prediction per step. However, the high generation quality often requires the number of decoding steps equal to the sequence length, which performs a one-token-per-step decoding, and existing parallel decoding algorithms, which yield suboptimal decoding paths, bring inference speedup at the cost of non-negligible performance degradation. To overcome this challenge, we introduce Free Draft-and-Verification (FreeDave), a novel fast decoding algorithm tailored for DLLMs that achieves lossless parallel decoding without any model modification or extra modules. Specifically, we propose an algorithm of parallel-decoded candidate generation and verification, which is theoretically guaranteed to use the fewest model forward calls to reproduce the same sequence generated by static decoding when enough computation and memory budget is provided. By extensive evaluations on math reasoning and code generation benchmarks across different DLLMs, FreeDave is proven to boost the inference throughput up to $3.78\times$ without performance degradation.

Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models

TL;DR

The paper tackles the bottleneck of DLLM inference speed while preserving generation quality. It introduces Free Dave, a training-free, model-free algorithm that uses the DLLM itself to draft and verify multiple candidate unmasking steps in parallel, resulting in lossless decoding. The authors prove that, with sufficient compute and memory, the method achieves the fewest forward calls under the lossless constraint and demonstrates up to throughput gains across math reasoning and code-generation benchmarks. Empirically, FreeDave outperforms threshold-based parallel decoding, maintains accuracy across tasks, and integrates with existing KV-cache frameworks to boost practical performance.

Abstract

Diffusion Large Language Models (DLLMs) have emerged as a new paradigm of language modeling beyond autoregressive next-token prediction. Thanks to their bidirectional attention mechanism, DLLMs are more capable of capturing the connection of context, and thus show unique advantages in challenges like the famous "reversal curse" or learning under data-constrained scenarios. In addition, taking advantage of their inherent modeling foundations, DLLMs have the great potential of efficient inference with parallel decoding algorithms, which enable multi-token prediction per step. However, the high generation quality often requires the number of decoding steps equal to the sequence length, which performs a one-token-per-step decoding, and existing parallel decoding algorithms, which yield suboptimal decoding paths, bring inference speedup at the cost of non-negligible performance degradation. To overcome this challenge, we introduce Free Draft-and-Verification (FreeDave), a novel fast decoding algorithm tailored for DLLMs that achieves lossless parallel decoding without any model modification or extra modules. Specifically, we propose an algorithm of parallel-decoded candidate generation and verification, which is theoretically guaranteed to use the fewest model forward calls to reproduce the same sequence generated by static decoding when enough computation and memory budget is provided. By extensive evaluations on math reasoning and code generation benchmarks across different DLLMs, FreeDave is proven to boost the inference throughput up to without performance degradation.

Paper Structure

This paper contains 17 sections, 2 theorems, 6 equations, 3 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Diffusion Large Language Models
  4. Fast Decoding for Large Language Models
  5. Methodology
  6. Preliminaries
  7. Autoregressive language models
  8. Diffusion language models
  9. Free Draft-and-Verification Decoding
  10. Remark
  11. Proof Sketch
  12. Experiments
  13. Experimental Settings
  14. Evaluation on Math Reasoning Benchmarks
  15. Evaluation on Code Generation Benchmarks
  16. ...and 2 more sections

Key Result

Theorem 1

In the feasible space $\mathcal{P}(f_\theta, g, \mathbf{T})$, suppose we have a verifier function $h$ defined as where $t_n$ is a timestep and $d$ is a positive integer. Then we can find a feasible path $\mathbf{p} = [\mathcal{I}_{t_{a_0} \rightarrow t_{a_1}}, \dots, \mathcal{I}_{t_{a_{M-1}} \rightarrow t_{a_M}}]$, where $a_0 = 0$, $a_M = N$, and $a_i = h(t_{a_{i-1}}, d), i\in\{1,\dots,M-1\}$.

Figures (3)

  • Figure 1: The overview of FreeDave decoding for DLLMs. Based on the estimated distribution predicted by the DLLM at the current step, the remasking scheduler looks multiple steps ahead and returns multiple draft candidates at those timesteps. Then, at the next step, the DLLM takes those candidates as a batch of inputs in parallel and gets the estimated distribution for each candidate, which is further processed by the remasking scheduler for one more step to get a target sequence. The candidates, as well as their estimated distributions, are then accepted or rejected by matching their targets. The generation and verification of the draft candidates can be understood as byproducts during the normal static decoding without introducing extra cost, except for the memory overhead introduced by the batch forward. Empirically, with a high potential, the inference will get an appreciable speedup.
  • Figure 2: An example of DLLM static decoding and FreeDave decoding. In this example, at each step, static decoding is set to unmask one masked position where the predicted token has the highest confidence. For FreeDave decoding, V-Step takes as input all the candidate sequences in parallel and outputs target sequences (one more decoding step over each candidate sequence) and their estimated distributions, while D-Step samples from the estimated distribution of the maximum matched target sequence.
  • Figure 3: Evaluation of TraDo-4B-Instruct on MATH500 using different draft steps $d$. Specifically, when $d=1$, the decoding strategy is static decoding. We compare accuracy, throughput over time, throughput over NFEs, and GPU memory peak usage (percentage) in the inference stage, which are illustrated from left to right, respectively.

Theorems & Definitions (4)

  • Definition 1: Decoding Path and Oracle Path
  • Definition 2: Feasible Path and Optimal Path
  • Theorem 1: Verification-based Feasible Path Search
  • Lemma 1