Spiffy: Multiplying Diffusion LLM Acceleration via Lossless Speculative Decoding

TL;DR

Spiffy introduces a lossless speculative decoding framework for diffusion LLMs that uses auto-speculation with directed draft graphs and offline calibration to accelerate block-wise denoising. By drafting whole block states and verifying them in parallel, Spiffy increases throughput by about 2.8–3.1× while preserving output distributions, and can multiply the gains of other parallel decoding strategies to reach up to 7.9×. The approach avoids training a separate draft model, remains compatible with KV caching and multi-token unmasking, and demonstrates data-efficient calibration (20–50 samples) with robust cross-task transfer. This work significantly advances practical deployment of dLLMs by closing the gap between diffusion-based and autoregressive decoders in speed, while maintaining reliability and flexibility for integration with existing acceleration techniques.

Abstract

Diffusion LLMs (dLLMs) have recently emerged as a powerful alternative to autoregressive LLMs (AR-LLMs) with the potential to operate at significantly higher token generation rates. However, currently available open-source dLLMs often generate at much lower rates, typically decoding only a single token at every denoising timestep in order to maximize output quality. We present Spiffy, a speculative decoding algorithm that accelerates dLLM inference by $\mathbf{2.8{-}3.1\times}$ while provably preserving the model's output distribution. This work addresses the unique challenges involved in applying ideas from speculative decoding of AR-LLMs to the dLLM setting. Spiffy proposes draft states by leveraging the dLLM's distribution itself in an auto-speculative manner. This approach is efficient and effective, and eliminates the overheads of training and running an independent draft model. To structure the candidate draft states, we propose a novel directed draft graph which is uniquely designed to take advantage of the bidirectional, block-wise nature of dLLM generation and can be verified in parallel by the dLLM. To further optimize the structure of these draft graphs, we introduce an efficient, offline calibration algorithm that procedurally determines high-quality graph configurations. These optimized draft graphs, enabling increased acceptance rates, lead to a significant boost in the overall speedup achieved by the system. Crucially, Spiffy is also complementary to other recent innovations in improving dLLM generation speeds such as KV-caching and multi-token unmasking. We demonstrate that when combined with such parallel decoding algorithms, Spiffy is able to effectively multiply the benefits of these methods leading to total speedups of up to $\mathbf{7.9\times}$.

Figures (9)

• Figure 1: Spiffy's lossless verification allows $X_1(t)$ to advance to state $X_1(t-2)$ with a single model inference. Draft blocks $X_1^1, X_1^2, X_1^3$ are appended to the end of the sequence and model inference is performed in parallel. Each draft is compared against $X_1(t-1)$ to check for acceptance. Since we have access to any accepted draft's distribution at time $t-2$, we skip ahead and predict $X_1(t-2)$. We then repeat this process with the remaining drafts for further acceleration.
• Figure 2: Directed draft graph where each node represents a draft block. Each draft block may have multiple parent states. In the first level, only a single token has been unmasked, $c_{11}$ or $c_{21}$. In the second level, representing the second speculation timestep, $c_{11}$ and $c_{21}$ have been unmasked. This state can be reached if either $c_{11}$ or $c_{21}$ had been unmasked in the previous step, allowing multiple chances to reach the second draft level. The second level additionally contains $c_{11}, c_{31}$, which can be accessed through $c_{11}$ even though $c_{31}$ had not been drafted. In total, there are $3$ routes through the graph that reach the third level $c_{11}, c_{21}, c_{31}$. These drafts may all be verified in parallel using a block-attention mask as shown to the right.
• Figure 3: By leveraging the marginals of $p_\text{TM}(X(t))$ for each position in the block, we sort token candidates by $i=$ token position rank, $j=$ token choice rank.
• Figure 4: Directed draft graph calibrated using Alg. \ref{['alg:calibration']} for LLaDA-Base-8B with 50 samples from HumanEval.
• Figure 5: The model inference time sees a gradual increase from $D=0$ to $D=10$ due to inefficient attention computation. This curve is likely to be flatter in memory-bound systems where latency is less dependent on the input complexity.