Mind the Generative Details: Direct Localized Detail Preference Optimization for Video Diffusion Models

TL;DR

This work tackles the inefficiency and weak local guidance of existing Direct Preference Optimization (DPO) for text-to-video diffusion. LocalDPO constructs high-quality real-video positives and generates localized negatives via spatio-temporal masking and region-aware restoration, enabling a single-inference, high-confidence preference dataset and fast convergence. A region-aware DPO loss, combined with a hybrid training objective that includes SFT and global DPO terms, enables fine-grained alignment of local details while preserving global coherence. Across large-scale real-video datasets and multiple VDMs, LocalDPO yields consistent gains in visual fidelity, temporal coherence, and human-aligned semantics, demonstrating a scalable, efficient path for improving video generator alignment.

Abstract

Aligning text-to-video diffusion models with human preferences is crucial for generating high-quality videos. Existing Direct Preference Otimization (DPO) methods rely on multi-sample ranking and task-specific critic models, which is inefficient and often yields ambiguous global supervision. To address these limitations, we propose LocalDPO, a novel post-training framework that constructs localized preference pairs from real videos and optimizes alignment at the spatio-temporal region level. We design an automated pipeline to efficiently collect preference pair data that generates preference pairs with a single inference per prompt, eliminating the need for external critic models or manual annotation. Specifically, we treat high-quality real videos as positive samples and generate corresponding negatives by locally corrupting them with random spatio-temporal masks and restoring only the masked regions using the frozen base model. During training, we introduce a region-aware DPO loss that restricts preference learning to corrupted areas for rapid convergence. Experiments on Wan2.1 and CogVideoX demonstrate that LocalDPO consistently improves video fidelity, temporal coherence and human preference scores over other post-training approaches, establishing a more efficient and fine-grained paradigm for video generator alignment.

Figures (12)

• Figure 1: Comparison between (a) vanilla DPO and (b) LocalDPO for video diffusion model (VDM). LocalDPO efficiently constructs positive-negative pairs by locally corrupting real videos, avoiding multi-round sampling, extra critic models, and annotation ambiguities. (c) Quantifies comprison of GPU time in constructing preference pairs.
• Figure 2: Comparison of video pairs generated by CogVideoX-5B from the same prompt but different seeds reveals significant discrepancies in the visual quality of localized regions, with their relative quality varying across frames. These fine-grained, localized preference patterns are overlooked by the vanilla DPO annotation paradigm, motivating our LocalDPO approach.
• Figure 3: Pipeline of locally corrupted videos generation. We first randomly sample several Bézier curves on the original video and ensure that these curves form closed shapes. The interior of each closed shape defines the region to be corrupted in subsequent steps. Then, the masked area of real video is inpainted by the pretrained VDM. Specifically, given the latent of input real video, the model first adds a controlled amount of noise to its latent representation and then denoises it step by step. During each denoising step, the original video latent is re-noised at the noise level corresponding to the next timestep and then fused with the denoised latent via a latent fusion mechanism by $\mathbf{z}_{t-1} = \mathbf{M} \odot \hat{\mathbf{z}}_{t-1} + (1 - \mathbf{M}) \odot \mathbf{z}_{t-1}^{\text{orig}}$.
• Figure 4: Human evaluation of LocalDPO vs. SFT and VanillaDPO. LocalDPO achieves the best results on all dimensions of human evaluation.
• Figure 5: Qualitative Comparison between SFT, Vanilla DPO and LocalDPO for CogVideoX models. Our LocalDPO generates rich textural details, plausible motion, higher aesthetic and fewer artifacts.