$ρ$-$\texttt{EOS}$: Training-free Bidirectional Variable-Length Control for Masked Diffusion LLMs

TL;DR

Fixed-length generation in masked diffusion LLMs constrains efficiency and adaptability. ρ-EOS introduces a training-free, single-stage bidirectional length control by monitoring the implicit EOS density $\rho_{\text{EOS}}$ during denoising to decide when to insert or remove MASK tokens. The approach achieves competitive accuracy on math and code benchmarks while substantially improving token utilization and reducing inference latency compared to DAEDAL and fixed-length baselines. By revealing and exploiting a latent length sufficiency signal learned during training, this method enables adaptive generation without extra supervision or multi-stage decoding, with strong practical impact for high-throughput and RL fine-tuning scenarios.

Abstract

Beyond parallel generation and global context modeling, current masked diffusion large language models (dLLMs) suffer from a fundamental limitation: they require a predefined, fixed generation length, which lacks flexibility and forces an inevitable trade-off between output quality and computational efficiency. To address this, we study the denoising dynamics and find that the implicit density ($ρ$) of end-of-sequence ($\texttt{EOS}$) tokens serves as a reliable signal of generation sufficiency. In particular, the evolving implicit $\texttt{EOS}$ density during denoising reveals whether the current masked space is excessive or insufficient, thereby guiding the adjustment direction for generation length. Building on this insight, we propose $\textbf{$ρ$-$\texttt{EOS}$}$, a training-free, single-stage strategy that enables bidirectional variable-length generation for masked dLLMs. Unlike prior two-stage approaches--which require separate length adjustment and iterative mask insertion phases while supporting only unidirectional expansion--$\textbf{$ρ$-$\texttt{EOS}$}$ achieves bidirectional length adjustment within a unified denoising process by continuously estimating the implicit $\texttt{EOS}$ density: excessively high density triggers $\texttt{MASK}$ token contraction, while insufficient density induces expansion. Extensive experiments on mathematics and code benchmarks demonstrate that $\textbf{$ρ$-$\texttt{EOS}$}$ achieves comparable performance while substantially improving inference efficiency and token utilization.

Figures (6)

• Figure 1: The Evolution Trend of Implicit EOS Density During the Denoising Process. (a) Illustration of the calculation for implicit EOS density. (b.1) Trends of implicit EOS density on GSM8k. (b.2) Trends of implicit EOS density on MBPP.
• Figure 2: The Formulation of Implicit and Explicit Concept for Token and Density.(Upper Left) The standard denoising process of masked diffusion large language models (e.g., LLaDA). (Lower Left) The two stage of DAEDAL. (Right) The bidirectional variable-length denoising process of $\rho$-EOS.
• Figure 3: Example of piecewise expansion factor function for constants, linearity, and exponentiation.
• Figure 4: Main Results on LLaDA-Instruct-8B across Four Benchmarks. We compare the $\rho$-EOS performance against DAEDAL and various baseline configurations. $\boldsymbol{Acc}$ denotes accuracy, $\boldsymbol{E_\text{token}}$ is the average effective tokens (the response length excluding trailing padding), $\boldsymbol{N_\text{token}}$ is the average total tokens, and $\boldsymbol{E_\text{ratio}}$ is the effective token ratio. $\boldsymbol{T_\text{runtime}}$ represents the runtime spent on the evaluation (in seconds). The best configuration for the baseline is highlighted in orange. DAEDAL is highlighted in blue, and $\rho$-EOS is highlighted in red. Under the $\rho$-EOS setting, Sym and Asym denote the symmetric and asymmetric lower-upper threshold centered around 0.5. The best results are bold, and the second-best results are underlined.
• Figure 5: Main Results on LLaDA-1.5-8B across Four Benchmarks. We compare the $\rho$-EOS performance against DAEDAL and various baseline configurations. $\boldsymbol{Acc}$ denotes accuracy, $\boldsymbol{E_\text{token}}$ is the average effective tokens (the response length excluding trailing padding), $\boldsymbol{N_\text{token}}$ is the average total tokens, and $\boldsymbol{E_\text{ratio}}$ is the effective token ratio. $\boldsymbol{T_\text{runtime}}$ represents the runtime spent on the evaluation (in seconds). The best configuration for the baseline is highlighted in orange. DAEDAL is highlighted in blue, and $\rho$-EOS is highlighted in red. Under the $\rho$-EOS setting, Sym and Asym denote the symmetric and asymmetric lower-upper threshold centered around 0.5. The best results are bold, and the second-best results are underlined.