Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AI Powered High Quality Text to Video Generation with Enhanced Temporal Consistency

Piyushkumar Patel

TL;DR

This work tackles text-to-video generation by addressing temporal consistency and fine-grained controllability. It introduces MOVAI, a hierarchical framework built from a Compositional Scene Parser (CSP), a Temporal-Spatial Attention Mechanism (TSAM), and a Progressive Video Refinement (PVR) module, all trained in a staged strategy. Through extensive experiments on standard benchmarks, MOVAI delivers state-of-the-art metrics including lower FVD and LPIPS, higher IS and CLIP alignment, and stronger user preference scores, while enabling detailed scene-level control. The approach holds promise for generating complex, multi-object videos with realistic dynamics and semantic fidelity, with potential impact across media creation and education.

Abstract

Text to video generation has emerged as a critical frontier in generative artificial intelligence, yet existing approaches struggle with maintaining temporal consistency, compositional understanding, and fine grained control over visual narratives. We present MOVAI (Multimodal Original Video AI), a novel hierarchical framework that integrates compositional scene understanding with temporal aware diffusion models for high fidelity text to video synthesis. Our approach introduces three key innovations: (1) a Compositional Scene Parser (CSP) that decomposes textual descriptions into hierarchical scene graphs with temporal annotations, (2) a Temporal-Spatial Attention Mechanism (TSAM) that ensures coherent motion dynamics across frames while preserving spatial details, and (3) a Progressive Video Refinement (PVR) module that iteratively enhances video quality through multi-scale temporal reasoning. Extensive experiments on standard benchmarks demonstrate that MOVAI achieves state-of-the-art performance, improving video quality metrics by 15.3% in LPIPS, 12.7% in FVD, and 18.9% in user preference studies compared to existing methods. Our framework shows particular strength in generating complex multi-object scenes with realistic temporal dynamics and fine-grained semantic control.

AI Powered High Quality Text to Video Generation with Enhanced Temporal Consistency

TL;DR

This work tackles text-to-video generation by addressing temporal consistency and fine-grained controllability. It introduces MOVAI, a hierarchical framework built from a Compositional Scene Parser (CSP), a Temporal-Spatial Attention Mechanism (TSAM), and a Progressive Video Refinement (PVR) module, all trained in a staged strategy. Through extensive experiments on standard benchmarks, MOVAI delivers state-of-the-art metrics including lower FVD and LPIPS, higher IS and CLIP alignment, and stronger user preference scores, while enabling detailed scene-level control. The approach holds promise for generating complex, multi-object videos with realistic dynamics and semantic fidelity, with potential impact across media creation and education.

Abstract

Text to video generation has emerged as a critical frontier in generative artificial intelligence, yet existing approaches struggle with maintaining temporal consistency, compositional understanding, and fine grained control over visual narratives. We present MOVAI (Multimodal Original Video AI), a novel hierarchical framework that integrates compositional scene understanding with temporal aware diffusion models for high fidelity text to video synthesis. Our approach introduces three key innovations: (1) a Compositional Scene Parser (CSP) that decomposes textual descriptions into hierarchical scene graphs with temporal annotations, (2) a Temporal-Spatial Attention Mechanism (TSAM) that ensures coherent motion dynamics across frames while preserving spatial details, and (3) a Progressive Video Refinement (PVR) module that iteratively enhances video quality through multi-scale temporal reasoning. Extensive experiments on standard benchmarks demonstrate that MOVAI achieves state-of-the-art performance, improving video quality metrics by 15.3% in LPIPS, 12.7% in FVD, and 18.9% in user preference studies compared to existing methods. Our framework shows particular strength in generating complex multi-object scenes with realistic temporal dynamics and fine-grained semantic control.

Paper Structure

This paper contains 25 sections, 10 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related Work
  4. Text-to-Image Generation
  5. Video Generation Models
  6. Compositional Understanding
  7. Methodology
  8. Problem Formulation
  9. MOVAI Architecture Overview
  10. Compositional Scene Parser (CSP)
  11. Temporal-Spatial Attention Mechanism (TSAM)
  12. Progressive Video Refinement (PVR)
  13. Training Strategy
  14. Experimental Setup
  15. Datasets
  16. ...and 10 more sections

Figures (3)

  • Figure 1: Overall architecture of MOVAI framework showing the three main components: Compositional Scene Parser (CSP), Temporal-Spatial Attention Mechanism (TSAM), and Progressive Video Refinement (PVR).
  • Figure 2: Detailed system diagram of MOVAI showing data flow, technical specifications, and processing stages. The diagram illustrates how textual input is processed through multiple stages to generate high-quality video output with dimensions and processing times indicated.
  • Figure 3: Qualitative comparison showing generated video frames for complex textual descriptions. MOVAI produces more coherent and detailed results compared to baseline methods.