Vidmento: Creating Video Stories Through Context-Aware Expansion With Generative Video

TL;DR

Vidmento addresses narrative gaps in capture-first video by introducing Generative Expansion, a design framework for context-aware blending of captured and generated media. The authors instantiate Vidmento as a unified authoring tool with a canvas and script editor, guided by four design goals (identify ideas, highlight expansion opportunities, generate contextually connected clips, and preserve authorial control). Through formative interviews and a user study with 12 creators, the work reveals opportunities for narrative construction, exploration, and expressive reinterpretation, while highlighting challenges in integration, authenticity, and output refinement. The findings suggest that context-aware blending can expand expressive possibilities in video storytelling without eroding creator ownership, pointing to a promising direction for AI-augmented, narrative-driven video authoring.

Abstract

Video storytelling is often constrained by available material, limiting creative expression and leaving undesired narrative gaps. Generative video offers a new way to address these limitations by augmenting captured media with tailored visuals. To explore this potential, we interviewed eight video creators to identify opportunities and challenges in integrating generative video into their workflows. Building on these insights and established filmmaking principles, we developed Vidmento, a tool for authoring hybrid video stories that combine captured and generated media through context-aware expansion. Vidmento surfaces opportunities for story development, generates clips that blend stylistically and narratively with surrounding media, and provides controls for refinement. In a study with 12 creators, Vidmento supported narrative development and exploration by systematically expanding initial materials with generative media, enabling expressive video storytelling aligned with creative intent. We highlight how creators bridge story gaps with generative content and where they find this blending capability most valuable.

Figures (17)

• Figure 1: Overview of generative expansion, a design framework for contextually expanding existing footage with the affordances of generative video. This approach is designed to provide structured support for video storytelling, while offering flexibility for creative exploration and iteration.
• Figure 2: Vidmento's canvas view. (a) To the left, there are different options for exploring and organizing visual materials. (b) Users can create and compare different story versions by duplicating the current version or prompting for a custom variation. (c) Clicking on a file node (captured or generated) reveals its description, which can be edited or regenerated.
• Figure 3: Vidmento's script editor. (a) Suggestions for script expansion appear in the sidebar. (b) Each includes an explanation and tips for addressing it. (c) Clicking "address" highlights relevant visuals in the canvas and (d) suggests modifications via the inline refinement menu. (e) Users can add ideas or story context in the editor notes to steer system generations.
• Figure 4: Shaping a video story with Vidmento from (a) an initial set of materials by (b) grouping shots into scenes and (c) connecting scenes into a sequence.
• Figure 5: Expanding a scene by (a) sequencing visuals and suggesting new content or (b) adding a transition shot between existing footage. (c) Each generated shot comes with three options, and an explanation for how it strengthens the narrative.