Generating Animated Layouts as Structured Text Representations

TL;DR

This work tackles the challenge of generating video advertisements with readable text and precisely controlled animated layouts by introducing Animated Layout Generation and a Structured Text (ST) Representation. The authors present VAKER, a text-to-video pipeline that decomposes ad creation into three stages (Banner, Mainground, Animation) and leverages Unstructured Text (UT) Reasoning to translate prompts into ST-Representations, with LoRA-fine-tuned experts guiding each stage. They also build an automated dataset pipeline of 2,224 real ads and introduce evaluation metrics, notably Fréchet Motion Distance ($FMD$) and layout metrics (Overlap, mIoU), demonstrating that VAKER outperforms diffusion-based and static-layout baselines in motion realism and typography fidelity. The work highlights a significant step toward automated, controllable video ad generation, enabling high-quality, text-rich animations with spatial-temporal precision and opening avenues for richer, scalable ad production pipelines.

Abstract

Despite the remarkable progress in text-to-video models, achieving precise control over text elements and animated graphics remains a significant challenge, especially in applications such as video advertisements. To address this limitation, we introduce Animated Layout Generation, a novel approach to extend static graphic layouts with temporal dynamics. We propose a Structured Text Representation for fine-grained video control through hierarchical visual elements. To demonstrate the effectiveness of our approach, we present VAKER (Video Ad maKER), a text-to-video advertisement generation pipeline that combines a three-stage generation process with Unstructured Text Reasoning for seamless integration with LLMs. VAKER fully automates video advertisement generation by incorporating dynamic layout trajectories for objects and graphics across specific video frames. Through extensive evaluations, we demonstrate that VAKER significantly outperforms existing methods in generating video advertisements. Project Page: https://yeonsangshin.github.io/projects/Vaker

Figures (20)

• Figure 1: User study results. We compare VAKER with LayoutPrompter for Layout Quality and Advertising Effectiveness. VAKER outperforms the baseline in both criteria.
• Figure 2: Overview of VAKER. VAKER takes text input from users to generate video advertisements. It does so by producing Structured Text (ST) Representations, a format we propose for encoding video layouts as structured text. The pipeline breaks down video advertisement creation into three components: Banner, Mainground, and Animation. Each component uses a LoRA-adapted language model that first interprets the design in natural language before generating structured specifications for the final video advertisement.
• Figure 2: Videos generated by VAKER. The number on the top-left corner of each frame indicates the frame index.
• Figure 3: Video Advertisement Dataset Construction Pipeline. Our automated pipeline converts video advertisements into three types of data for training VAKER: Structured Text (ST), Unstructured Text (UT) Reasoning, and natural language prompts. Given a video advertisement, the pipeline performs the following steps: (1) extracts the last frame, (2) detects and classifies objects using fine-tuned detection models vargheseyolov8, and (3) generates Banner ST and Mainground ST from the spatial layout, (4) tracks object movements using tracking models yang2021associatingobjectstransformersvideo to generate Animation ST, and finally, (5) uses template-based prompting with LLMs to convert these ST-Representations into UT Reasonings and natural language prompts.
• Figure 3: Videos generated by VAKER. The number on the top-left corner of each frame indicates the frame index.