Generative Outpainting To Enhance the Memorability of Short-Form Videos

TL;DR

The results of using generative outpainting to expand the screen size of a short-form video with a view to improving its memorability are presented.

Abstract

With the expanding use of the short-form video format in advertising, social media, entertainment, education and more, there is a need for such media to both captivate and be remembered. Video memorability indicates to us how likely a video is to be remembered by a viewer who has no emotional or personal connection with its content. This paper presents the results of using generative outpainting to expand the screen size of a short-form video with a view to improving its memorability. Advances in machine learning and deep learning are compared and leveraged to understand how extending the borders of video screensizes can affect their memorability to viewers. Using quantitative evaluation we determine the best-performing model for outpainting and the impact of outpainting based on image saliency on video memorability scores

Figures (8)

• Figure 1: Memorability scores from prediction model sweeney2021predicting vs. groundtruth scores for 100 X Memento10k videos. Average score from model = 0.788, average groundtruth score = 0.768
• Figure 2: Sample outpainting of a video frame using M3DDM, which allows for only 256x256 (1:1) resolution for outpainted videos.
• Figure 3: A breakdown of the MOTIA approach to video outpainting.
• Figure 4: Example saliency map and four quadrants.
• Figure 5: Original video frames with outpainted results for MOTIA and M3DDM models.