COMIC: Agentic Sketch Comedy Generation

TL;DR

A fully automated AI system that produces short comedic videos similar to sketch shows such as Saturday Night Live is proposed, with the introduction of LLM critics aligned with real viewer preferences through the analysis of a corpus of comedy videos on YouTube to automatically evaluate humor.

Abstract

We propose a fully automated AI system that produces short comedic videos similar to sketch shows such as Saturday Night Live. Starting with character references, the system employs a population of agents loosely based on real production studio roles, structured to optimize the quality and diversity of ideas and outputs through iterative competition, evaluation, and improvement. A key contribution is the introduction of LLM critics aligned with real viewer preferences through the analysis of a corpus of comedy videos on YouTube to automatically evaluate humor. Our experiments show that our framework produces results approaching the quality of professionally produced sketches while demonstrating state-of-the-art performance in video generation.

Figures (13)

• Figure 1: COMIC is an agentic sketch comedy video generator. It takes images, voices, and brief descriptions as input, and automatically generates funny comedy scripts along with corresponding video and audio. Our method flexibly builds stories around multiple characters and custom backgrounds. Each generated comedy is 1--2 minutes long; please watch them at https://susunghong.github.io/COMIC.
• Figure 2: Overall agentic flow. Our method is loosely modeled on human production studios, with agentic counterparts for each role, such as writer, critic, and director. The writing and rendering loops allow us to generate scripts and videos with sufficient breadth and depth through island-based competition and iteration, as illustrated in Fig. \ref{['fig:writing_archipelago']} and Fig. \ref{['fig:rendering_archipelago']}, respectively.
• Figure 3: Sketch comedy videos featuring various generated situations. See our project page for videos of these results.
• Figure 4: Script writing stage. Isolated script populations evolve on separate islands under distinct critic committees sampled from the aligned critic pool. Losing scripts are refined through round-robin pairwise tournaments by each island's critic committee, driving improvement while supporting aesthetic diversity across islands.
• Figure 5: Video rendering stage. Scene directions are generated and critic-refined for each script. Single-elimination tournaments operate at both shot and video levels, selecting the best revision across history and the best video across diverse realizations.