MiVID: Multi-Strategic Self-Supervision for Video Frame Interpolation using Diffusion Model

TL;DR

MiVID addresses video frame interpolation under occlusion and motion uncertainty by leveraging a self-supervised diffusion prior. It combines a lightweight 3D U‑Net with temporal attention and a multi-strategy masking regime to synthesize temporally coherent intermediate frames without high-frame-rate supervision. Trained entirely on CPU, MiVID achieves competitive PSNR/SSIM and improved perceptual quality on UCF101-7 and DAVIS-7 within 50 epochs, highlighting diffusion priors as scalable priors for VFI. This work demonstrates a practical path toward accessible, generalizable VFI systems and suggests future extensions to longer sequences and semantic conditioning.

Abstract

Video Frame Interpolation (VFI) remains a cornerstone in video enhancement, enabling temporal upscaling for tasks like slow-motion rendering, frame rate conversion, and video restoration. While classical methods rely on optical flow and learning-based models assume access to dense ground-truth, both struggle with occlusions, domain shifts, and ambiguous motion. This article introduces MiVID, a lightweight, self-supervised, diffusion-based framework for video interpolation. Our model eliminates the need for explicit motion estimation by combining a 3D U-Net backbone with transformer-style temporal attention, trained under a hybrid masking regime that simulates occlusions and motion uncertainty. The use of cosine-based progressive masking and adaptive loss scheduling allows our network to learn robust spatiotemporal representations without any high-frame-rate supervision. Our framework is evaluated on UCF101-7 and DAVIS-7 datasets. MiVID is trained entirely on CPU using the datasets and 9-frame video segments, making it a low-resource yet highly effective pipeline. Despite these constraints, our model achieves optimal results at just 50 epochs, competitive with several supervised baselines.This work demonstrates the power of self-supervised diffusion priors for temporally coherent frame synthesis and provides a scalable path toward accessible and generalizable VFI systems.

Figures (4)

• Figure 1: Architecture of our proposed MiVID workflow.
• Figure 2: Reconstruction output of a sample from the UCF101-7 dataset. The order of images in the figure denotes Original, Masked, and Reconstructed frames respectively.
• Figure 3: Reconstruction output of a sample from the DAVIS-7 dataset. The order of images in the figure denotes Original, Masked, and Reconstructed frames respectively.
• Figure 4: The figure shows the relative improvements of MiVID over the best supervised baselines on UCF101-7 and DAVIS-7 datasets. On UCF101-7, MiVID achieves an advantage of 0.7$\%$ in PSNR, 2.6$\%$ in SSIM, and 10.9$\%$ in LPIPS. On DAVIS-7, the improvements are more pronounced, with 3.0$\%$ in PSNR, 57.0$\%$ in SSIM, and 5.0$\%$ in LPIPS.