Time-Annealed Perturbation Sampling: Diverse Generation for Diffusion Language Models

TL;DR

This paper presents Time-Annealed Perturbation Sampling (TAPS), a training-free inference method for diffusion-based language models that perturbs the conditioning signal with a decaying noise schedule to foster early semantic branching while preserving later fluency. It identifies a temporal division of labor in diffusion generation, where early steps shape global semantics and later steps refine lexical details, and demonstrates that time-annealed perturbations can expand the space of valid outputs without sacrificing quality. Across two backbones (LLaDA-8B-Instruct and TraDo-8B-Instruct) and tasks spanning open-ended writing and mathematical reasoning, TAPS yields consistent improvements in semantic and embedding-level diversity, with competitive or superior quality judged by GPT-4o and human-aligned preference benchmarks. The approach is validated through extensive experiments, ablations on noise scales, perturbation windows, and perturbation strategies, and points to diffusion models as a promising platform for controlled, diverse text generation.

Abstract

Diffusion language models (Diffusion-LMs) introduce an explicit temporal dimension into text generation, yet how this structure can be leveraged to control generation diversity for exploring multiple valid semantic or reasoning paths remains underexplored. In this paper, we show that Diffusion-LMs, like diffusion models in image generation, exhibit a temporal division of labor: early denoising steps largely determine the global semantic structure, while later steps focus on local lexical refinement. Building on this insight, we propose Time-Annealed Perturbation Sampling (TAPS), a training-free inference strategy that encourages semantic branching early in the diffusion process while progressively reducing perturbations to preserve fluency and instruction adherence. TAPS is compatible with both non-autoregressive and semi-autoregressive Diffusion backbones, demonstrated on LLaDA and TraDo in our paper, and consistently improves output diversity across creative writing and reasoning benchmarks without compromising generation quality.

Figures (6)

• Figure 1: Comparison between TAPS and the base models across diversity, generation quality, and reasoning performance. Quality is assessed by GPT, and reasoning is measured via majority voting accuracy on GSM8K (Sec. \ref{['sec:exp']}).
• Figure 2: A conceptual comparison of the inference process between the base Diffusion-LM and our proposed method, TAPS, illustrating different context conditioning behaviors.
• Figure 3: Quality comparison on Novelty-Bench across multiple human preference dimensions. The radar plots compare our method with decoding baselines, showing consistent improvements on creativity-related evaluation dimensions while maintaining comparable overall quality on other dimensions. Results for LLaDA are shown on the left and for TraDo on the right.
• Figure 4: GSM8K accuracy on 300 questions with 10 samples per question. We report single-sample and majority-vote accuracy under three temperatures.
• Figure 5: Toy experiment on semantic branching with TraDo-8B. We compare Standard DLM and TAPS by projecting SBERT embeddings of generated samples into a shared 2D space via t-SNE at three denoising stages (early, mid, final). Convex hulls illustrate semantic coverage. TAPS maintains broader and more multimodal coverage across stages, while Standard DLM exhibits progressive semantic contraction. For the mid-stage snapshot, we retain all predicted tokens before re-masking when computing semantic representations.