DFlash: Block Diffusion for Flash Speculative Decoding

TL;DR

Autoregressive LLM inference is inherently sequential, causing high latency and poor GPU utilization. DFlash introduces a lightweight block diffusion drafter that is conditioned on rich target-model hidden features via KV injection, enabling parallel drafting of token blocks and high acceptance rates. Empirical results show lossless speedups up to around 6× on Qwen3-8B and consistent gains over state-of-the-art EAGLE-3 across math, code, and chat tasks, with efficient long-context training. By reframing diffusion LLMs as fast, specialized drafters for speculative decoding, DFlash demonstrates a practical, scalable approach to accelerating real-world LLM serving.

Abstract

Autoregressive large language models (LLMs) deliver strong performance but require inherently sequential decoding, leading to high inference latency and poor GPU utilization. Speculative decoding mitigates this bottleneck by using a fast draft model whose outputs are verified in parallel by the target LLM; however, existing methods still rely on autoregressive drafting, which remains sequential and limits practical speedups. Diffusion LLMs offer a promising alternative by enabling parallel generation, but current diffusion models typically underperform compared with autoregressive models. In this paper, we introduce DFlash, a speculative decoding framework that employs a lightweight block diffusion model for parallel drafting. By generating draft tokens in a single forward pass and conditioning the draft model on context features extracted from the target model, DFlash enables efficient drafting with high-quality outputs and higher acceptance rates. Experiments show that DFlash achieves over 6x lossless acceleration across a range of models and tasks, delivering up to 2.5x higher speedup than the state-of-the-art speculative decoding method EAGLE-3.

Figures (5)

• Figure 1: Speedup comparison between DFlash, EAGLE-3 against Autoregressive Decoding on Qwen3-8B yang2025qwen3technicalreport with the Transformers backend. Overall, DFlash achieves more than 2.5× higher speedup than EAGLE-3.
• Figure 2: DFlash Inference Design. Hidden context features extracted from the target model are fused and injected into each draft layer's Key-Value cache to enable conditional speculation.
• Figure 3: Draft cost of 1, 3, 5-layer DFlash and 1-layer EAGLE-3.
• Figure 4: DFlash training attention. The target model provides context features (blue) that condition the draft model. The input consists of clean prompt tokens $p$ and clean response tokens $r$. Within each masked block, a subset of clean response tokens (yellow) is randomly sampled as anchors, while mask tokens $m$ (green) mark positions for parallel prediction. Invisible tokens (white) denote the attention mask, which enforces causal consistency and prevents inter-block information leakage during training.
• Figure 5: The loss decay makes training converge faster and better.