BVI-VFI: A Video Quality Database for Video Frame Interpolation

TL;DR

BVI-VFI introduces the first bespoke video quality database for video frame interpolation, containing 108 reference sequences and 540 distorted interpolations derived from 36 source videos across three resolutions (540p, 1080p, 4K) and three frame rates (30, 60, 120fps). A large-scale subjective study with 189 participants yields ground-truth perceptual scores, enabling a comprehensive analysis of how VFI algorithms, frame rate, and content affect quality. The work benchmarks 33 objective quality metrics, revealing that none consistently align with human judgments (FAST achieves the best overall SRCC ~0.70), and cross-validation shows learning-based metrics have limited generalization without careful content splitting. The database serves as a crucial resource for developing VFI-specific perceptual metrics and for benchmarking VFI algorithms, with public release to accelerate research in perceptual VFI quality assessment.

Abstract

Video frame interpolation (VFI) is a fundamental research topic in video processing, which is currently attracting increased attention across the research community. While the development of more advanced VFI algorithms has been extensively researched, there remains little understanding of how humans perceive the quality of interpolated content and how well existing objective quality assessment methods perform when measuring the perceived quality. In order to narrow this research gap, we have developed a new video quality database named BVI-VFI, which contains 540 distorted sequences generated by applying five commonly used VFI algorithms to 36 diverse source videos with various spatial resolutions and frame rates. We collected more than 10,800 quality ratings for these videos through a large scale subjective study involving 189 human subjects. Based on the collected subjective scores, we further analysed the influence of VFI algorithms and frame rates on the perceptual quality of interpolated videos. Moreover, we benchmarked the performance of 33 classic and state-of-the-art objective image/video quality metrics on the new database, and demonstrated the urgent requirement for more accurate bespoke quality assessment methods for VFI. To facilitate further research in this area, we have made BVI-VFI publicly available at https://github.com/danier97/BVI-VFI-database.

Figures (11)

• Figure 1: Sample frames from the 36 source sequences of the BVI-VFI database. (1)-(12): sequences at 960$\times$540. (13)-(24): sequences at 1920$\times$1080. (25)-(36): sequences at 3840$\times$2160.
• Figure 2: Example frame blocks generated by five VFI algorithms. It is noted that for frame repeating, although the result appears less distorted, the video exhibits motion juddering.
• Figure 3: Scatter plot of DMOS values given by two randomly separated equal-size groups of subjects. Note that lower DMOS indicates higher perceived quality.
• Figure 4: (a) Histogram of median SRCC achieved by each user in self-consistency examination based MOS on reference videos. (b) Histogram of the average standard deviation of multiple scores given to the same reference video by each subject.
• Figure 5: (a) Final DMOS distribution. (b) Standard deviation of DMOS over subjects. (c) Subject bias distribution. (d) Subject inconsistency distribution.