Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Subjective Quality Study for Video Frame Interpolation

Duolikun Danier, Fan Zhang, David Bull

TL;DR

A subjective quality study for VFI based on a newly developed video database, BVI-VFI, shows that there is an urgent need to develop a bespoke perceptual quality metric for V FI.

Abstract

Video frame interpolation (VFI) is one of the fundamental research areas in video processing and there has been extensive research on novel and enhanced interpolation algorithms. The same is not true for quality assessment of the interpolated content. In this paper, we describe a subjective quality study for VFI based on a newly developed video database, BVI-VFI. BVI-VFI contains 36 reference sequences at three different frame rates and 180 distorted videos generated using five conventional and learning based VFI algorithms. Subjective opinion scores have been collected from 60 human participants, and then employed to evaluate eight popular quality metrics, including PSNR, SSIM and LPIPS which are all commonly used for assessing VFI methods. The results indicate that none of these metrics provide acceptable correlation with the perceived quality on interpolated content, with the best-performing metric, LPIPS, offering a SROCC value below 0.6. Our findings show that there is an urgent need to develop a bespoke perceptual quality metric for VFI. The BVI-VFI dataset is publicly available and can be accessed at https://danier97.github.io/BVI-VFI/.

A Subjective Quality Study for Video Frame Interpolation

TL;DR

A subjective quality study for VFI based on a newly developed video database, BVI-VFI, shows that there is an urgent need to develop a bespoke perceptual quality metric for V FI.

Abstract

Video frame interpolation (VFI) is one of the fundamental research areas in video processing and there has been extensive research on novel and enhanced interpolation algorithms. The same is not true for quality assessment of the interpolated content. In this paper, we describe a subjective quality study for VFI based on a newly developed video database, BVI-VFI. BVI-VFI contains 36 reference sequences at three different frame rates and 180 distorted videos generated using five conventional and learning based VFI algorithms. Subjective opinion scores have been collected from 60 human participants, and then employed to evaluate eight popular quality metrics, including PSNR, SSIM and LPIPS which are all commonly used for assessing VFI methods. The results indicate that none of these metrics provide acceptable correlation with the perceived quality on interpolated content, with the best-performing metric, LPIPS, offering a SROCC value below 0.6. Our findings show that there is an urgent need to develop a bespoke perceptual quality metric for VFI. The BVI-VFI dataset is publicly available and can be accessed at https://danier97.github.io/BVI-VFI/.
Paper Structure (10 sections, 4 figures, 2 tables)

This paper contains 10 sections, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. The BVI-VFI Database
  3. Reference Sequence Selection
  4. Test Sequence Generation
  5. Subjective Assessment Experiment
  6. Experiment Setup
  7. Experimental Procedure
  8. Data Processing
  9. Results and Discussion
  10. Conclusion

Figures (4)

  • Figure 1: Sample frames from the reference sequences in BVI-VFI database.
  • Figure 2: Example blocks generated by various VFI algorithms. It should be noted that for frame repeating, although the result seems less distorted, the video exhibits motion juddering.
  • Figure 3: The DMOS values for 5 VFI methods at 3 frame rates. The error bar denotes the standard error over sequences. Note that lower DMOS values correspond to better visual quality.
  • Figure 4: Plots of DMOS values against the scores of selected quality metrics. PSNR, SSIM and LPIPS are commonly used in VFI, and VMAF is the second best performing metric. The blue lines are the logistic function fitted on the BVI-VFI database.