LoPA: Scaling dLLM Inference via Lookahead Parallel Decoding

TL;DR

This work tackles the bottleneck of limited parallelism in diffusion LLMs by revealing the critical role of Token Filling Order (TFO) in enabling parallel decoding. It introduces LoPA, a training-free Lookahead Parallel Decoding algorithm that concurrently explores multiple TFOs (anchor plus lookahead branches) and selects the path with the highest future parallelism based on branch confidence, all evaluated in a single forward pass. When integrated with D2F models, LoPA substantially boosts tokens-per-forward, achieving 10.1 TPF on GSM8K for D2F-Dream and 8.3 TPF on HumanEval+ for D2F-DiffuCoder, while preserving or improving quality. Complemented by a specialized Branch Parallel inference system (LoPA-Dist) on CUDA and Ascend, the approach yields near-linear throughput gains (e.g., 1073.86 tokens/s) and demonstrates strong cross-platform generalizability, offering a practical path to scalable, non-autoregressive generation for large diffusion models.

Abstract

Diffusion Large Language Models (dLLMs) have demonstrated significant potential for high-speed inference. However, current confidence-driven decoding strategies are constrained by limited parallelism, typically achieving only 1--3 tokens per forward pass (TPF). In this work, we identify that the degree of parallelism during dLLM inference is highly sensitive to the Token Filling Order (TFO). Then, we introduce Lookahead PArallel Decoding LoPA, a training-free, plug-and-play algorithm, to identify a superior TFO and hence accelerate inference. LoPA concurrently explores distinct candidate TFOs via parallel branches, and selects the one with the highest potential for future parallelism based on branch confidence. We apply LoPA to the state-of-the-art D2F model and observe a substantial enhancement in decoding efficiency. Notably, LoPA increases the TPF of D2F-Dream to 10.1 on the GSM8K while maintaining performance superior to the Dream baseline. Furthermore, to facilitate this unprecedented degree of parallelism, we develop a specialized multi-device inference system featuring Branch Parallelism (BP), which achieves a single-sample throughput of 1073.9 tokens per second under multi-GPU deployment. The code is available at https://github.com/zhijie-group/LoPA.

Figures (5)

• Figure 1: Throughput performance of LoPA. LoPA accelerates the single-sample throughput for D2F-Dream to up to 1073.9 and 856.5 tokens/s on MBPP and GSM8K respectively, significantly outperforming baselines. More details are provided in Table \ref{['tab:baseline_results']}.
• Figure 2: Overview of Lookahead Parallel Decoding (LoPA). In each iteration, LoPA generates a anchor branch alongside multiple lookahead branches (e.g., $B_1, \dots, B_k$) by independently sampling high-confidence positions from the baseline's unfilled set. A branch confidence verification mechanism then evaluates all branches in parallel within a single forward pass, selecting the optimal path to maximize future parallelism.
• Figure 3: Overview of LoPA Branch Parallel Distributed Inference System Design. A key distinction lies in the KV cache management protocol tailored for different backends: LoPA-Dist-NV utilizes a robust two-phase update mechanism to ensure consistency, whereas LoPA-Dist-Ascend adopts a streamlined single-phase update strategy for optimized serving efficiency.
• Figure 4: Scaling Curves of LoPA. LoPA scales the tokens per forward pass (TPS) for D2F-Dream and D2F-DiffuCoder to up to 10.1 and 8.3 on GSM8k and HumanEval+ respectively, with comparable performance.
• Figure 5: Scaling analysis of LoPA on D2F-Dream with varying branch counts. The results illustrate that LoPA effectively scales the TPF of D2F to a peak exceeding 10, thereby significantly reducing the total number of decoding steps.