An Automatic Deep Learning Approach for Trailer Generation through Large Language Models

TL;DR

The paper tackles automating movie trailer generation by introducing an end-to-end multimodal framework in which a Large Language Model coordinates preparation, visual selection, voice-over, and soundtrack generation. It leverages frame extraction, scene segmentation, semantic frame embedding, and AI-generated dialogues and music to produce a cohesive trailer. Experimental evaluation against two prior methods with human raters indicates improvements in narrative coherence and overall trailer appeal, though AI-generated soundtracks introduce some variability. This work demonstrates the feasibility of fully automated, narrative-driven trailers and highlights areas for further enhancement, such as action-aware coherence and more sophisticated audio synthesis.

Abstract

Trailers are short promotional videos designed to provide audiences with a glimpse of a movie. The process of creating a trailer typically involves selecting key scenes, dialogues and action sequences from the main content and editing them together in a way that effectively conveys the tone, theme and overall appeal of the movie. This often includes adding music, sound effects, visual effects and text overlays to enhance the impact of the trailer. In this paper, we present a framework exploiting a comprehensive multimodal strategy for automated trailer production. Also, a Large Language Model (LLM) is adopted across various stages of the trailer creation. First, it selects main key visual sequences that are relevant to the movie's core narrative. Then, it extracts the most appealing quotes from the movie, aligning them with the trailer's narrative. Additionally, the LLM assists in creating music backgrounds and voiceovers to enrich the audience's engagement, thus contributing to make a trailer not just a summary of the movie's content but a narrative experience in itself. Results show that our framework generates trailers that are more visually appealing to viewers compared to those produced by previous state-of-the-art competitors.

Figures (2)

• Figure 1: The framework's structure. We report all the four principal phases and the inner subphases. The orange boxes highlight the specific subphases where the Large Language Model plays an active role.
• Figure 3: Comparison of the three automatic trailer generation methods based on the total scores achieved across three movies.