Bridging Text and Video Generation: A Survey
Nilay Kumar, Priyansh Bhandari, G. Maragatham
TL;DR
The paper systematically surveys text-to-video generation, tracing the field from early GAN and VAE approaches to modern diffusion-based architectures, and analyzes how these models address temporal coherence, alignment, and efficiency. It provides a structured review of representative models, datasets, training configurations, and evaluation metrics, underscoring limitations of current benchmarks and the need for perception-aligned evaluation (e.g., VBench). The authors synthesize current open challenges—such as long-range coherence, data availability, and computational costs—and outline promising directions, including dataset enrichment, architecture optimization, and broadened application scopes. Collectively, the work offers a comprehensive foundation for reproducibility and future progress in T2V research and deployment across education, accessibility, and media industries.
Abstract
Text-to-video (T2V) generation technology holds potential to transform multiple domains such as education, marketing, entertainment, and assistive technologies for individuals with visual or reading comprehension challenges, by creating coherent visual content from natural language prompts. From its inception, the field has advanced from adversarial models to diffusion-based models, yielding higher-fidelity, temporally consistent outputs. Yet challenges persist, such as alignment, long-range coherence, and computational efficiency. Addressing this evolving landscape, we present a comprehensive survey of text-to-video generative models, tracing their development from early GANs and VAEs to hybrid Diffusion-Transformer (DiT) architectures, detailing how these models work, what limitations they addressed in their predecessors, and why shifts toward new architectural paradigms were necessary to overcome challenges in quality, coherence, and control. We provide a systematic account of the datasets, which the surveyed text-to-video models were trained and evaluated on, and, to support reproducibility and assess the accessibility of training such models, we detail their training configurations, including their hardware specifications, GPU counts, batch sizes, learning rates, optimizers, epochs, and other key hyperparameters. Further, we outline the evaluation metrics commonly used for evaluating such models and present their performance across standard benchmarks, while also discussing the limitations of these metrics and the emerging shift toward more holistic, perception-aligned evaluation strategies. Finally, drawing from our analysis, we outline the current open challenges and propose a few promising future directions, laying out a perspective for future researchers to explore and build upon in advancing T2V research and applications.