PISCES: Annotation-free Text-to-Video Post-Training via Optimal Transport-Aligned Rewards

TL;DR

PISCES addresses the challenge of annotation-free post-training for text-to-video generation by introducing a Dual OT-aligned Rewards framework that aligns text and real-video distributions and enforces token-level grounding. The Distributional OT-aligned Quality Reward maps text embeddings into the real-video manifold to improve global visual quality and temporal coherence, while the Discrete Token-level OT-aligned Semantic Reward uses a partial OT plan with spatio-temporal constraints to align specific text tokens with video regions. Together, these rewards provide strong, scalable supervision that outperforms both annotation-based and annotation-free baselines on VBench and human evaluations for both short- and long-video generation, and are compatible with direct backpropagation and RL fine-tuning. This OT-driven approach offers a principled pathway to scalable, annotation-free reward design in multimodal generation with potential broad impact on practical T2V deployment and future research in OT-based supervision.

Abstract

Text-to-video (T2V) generation aims to synthesize videos with high visual quality and temporal consistency that are semantically aligned with input text. Reward-based post-training has emerged as a promising direction to improve the quality and semantic alignment of generated videos. However, recent methods either rely on large-scale human preference annotations or operate on misaligned embeddings from pre-trained vision-language models, leading to limited scalability or suboptimal supervision. We present $\texttt{PISCES}$, an annotation-free post-training algorithm that addresses these limitations via a novel Dual Optimal Transport (OT)-aligned Rewards module. To align reward signals with human judgment, $\texttt{PISCES}$ uses OT to bridge text and video embeddings at both distributional and discrete token levels, enabling reward supervision to fulfill two objectives: (i) a Distributional OT-aligned Quality Reward that captures overall visual quality and temporal coherence; and (ii) a Discrete Token-level OT-aligned Semantic Reward that enforces semantic, spatio-temporal correspondence between text and video tokens. To our knowledge, $\texttt{PISCES}$ is the first to improve annotation-free reward supervision in generative post-training through the lens of OT. Experiments on both short- and long-video generation show that $\texttt{PISCES}$ outperforms both annotation-based and annotation-free methods on VBench across Quality and Semantic scores, with human preference studies further validating its effectiveness. We show that the Dual OT-aligned Rewards module is compatible with multiple optimization paradigms, including direct backpropagation and reinforcement learning fine-tuning.

Figures (10)

• Figure 1: (a) Baseline (T2V-Turbo-v2) defines rewards over pre-trained VLM text-video embeddings, which suffer from distributional misalignment. (b) PISCES T2V post-training addresses this by formulating reward supervision over an OT-aligned embedding space. We propose a novel Dual OT-aligned Rewards module that aligns text embeddings to the video space, enabling effective visual and semantic alignment. (c) Compared to the baseline, PISCES improves visual quality (temporal coherence, photorealism) and semantic fidelity (object count, attributes) on both short-video (VideoCrafter2) and long-video (HunyuanVideo) generation.
• Figure 2: PISCEST2V Post-Training. We introduce a Dual OT-aligned Rewards module: (i) a distributional OT map $\mathbf{T}^\star$ for Quality Reward via [CLS] representation similarity, and (ii) a discrete OT plan $\mathbf{P}^\star$ with spatio-temporal constraints for Semantic Reward via a Video-Text Matching (VTM) classifier. The rewards module provides supervision for fine-tuning the T2V denoiser and is applicable with direct backpropagation and RL fine-tuning (GRPO).
• Figure 3: Human preference study.PISCES outperforms HunyuanVideo, T2V-Turbo-v2, VideoReward-DPO in visual quality, motion and semantic alignment, validating its effectiveness in T2V.
• Figure 4: Qualitative comparison of T2V models. PISCES produces videos with better semantic fidelity and visual quality, accurately capturing key details such as the reflective wet pavement and vibrant neon lighting.
• Figure 5: Cross-attention maps (left) are diffuse, OT plan without spatio-temporal constraints (middle) misaligns tokens, while our constrained OT plan (right) produces accurate correspondences.