Entropy-Guided k-Guard Sampling for Long-Horizon Autoregressive Video Generation

TL;DR

This work tackles error accumulation in long-horizon autoregressive video generation by introducing Entropy-Guided k-Guard (ENkG) sampling. ENkG leverages per-token predictive entropy to dynamically adapt the sampling nucleus and augments it with a $k$-guard to balance stability and diversity, all at inference time. Across multiple AR video models and driving-domain datasets, ENkG yields substantial improvements in video-level and frame-level fidelity (e.g., reduced FVD/FID) and maintains temporal coherence over long sequences, addressing entropy collapse observed in standard decoding. The approach is model-agnostic and training-free, offering a practical, generalizable path to higher-quality long-horizon video synthesis.

Abstract

Autoregressive (AR) architectures have achieved significant successes in LLMs, inspiring explorations for video generation. In LLMs, top-p/top-k sampling strategies work exceptionally well: language tokens have high semantic density and low redundancy, so a fixed size of token candidates already strikes a balance between semantic accuracy and generation diversity. In contrast, video tokens have low semantic density and high spatio-temporal redundancy. This mismatch makes static top-k/top-p strategies ineffective for video decoders: they either introduce unnecessary randomness for low-uncertainty regions (static backgrounds) or get stuck in early errors for high-uncertainty regions (foreground objects). Prediction errors will accumulate as more frames are generated and eventually severely degrade long-horizon quality. To address this, we propose Entropy-Guided k-Guard (ENkG) sampling, a simple yet effective strategy that adapts sampling to token-wise dispersion, quantified by the entropy of each token's predicted distribution. ENkG uses adaptive token candidate sizes: for low-entropy regions, it employs fewer candidates to suppress redundant noise and preserve structural integrity; for high-entropy regions, it uses more candidates to mitigate error compounding. ENkG is model-agnostic, training-free, and adds negligible overhead. Experiments demonstrate consistent improvements in perceptual quality and structural stability compared to static top-k/top-p strategies.

Figures (14)

• Figure 1: The results illustrate the phenomenon of entropy collapse in standard AR decoding, where blue regions indicate low entropy and red regions indicate high entropy. Our method effectively alleviates this issue.
• Figure 2: The visualization of entropy heatmaps. High-entropy regions form repeating textures (e.g., sky, foliage, and road), while low-entropy regions cluster in structured content and distinguishable textures (e.g., boundaries, edges between sky and trees, road markers and lines).
• Figure 3: The overall illustration of our sampling strategy.
• Figure 4: Visual results of DrivingWorld and VaVIM models with our strategy..
• Figure 5: Visual results of Cosmos model with our strategy.