Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hierarchical Frequency-based Upsampling and Refining for Compressed Video Quality Enhancement

Qianyu Zhang, Bolun Zheng, Xinying Chen, Quan Chen, Zhunjie Zhu, Canjin Wang, Zongpeng Li, Chengang Yan

TL;DR

This work proposes a hierarchical frequency-based upsampling and refining neural network (HFUR) for compressed video quality enhancement that achieves state-of-the-art performance for both constant bit rate and constant QP modes.

Abstract

Video compression artifacts arise due to the quantization operation in the frequency domain. The goal of video quality enhancement is to reduce compression artifacts and reconstruct a visually-pleasant result. In this work, we propose a hierarchical frequency-based upsampling and refining neural network (HFUR) for compressed video quality enhancement. HFUR consists of two modules: implicit frequency upsampling module (ImpFreqUp) and hierarchical and iterative refinement module (HIR). ImpFreqUp exploits DCT-domain prior derived through implicit DCT transform, and accurately reconstructs the DCT-domain loss via a coarse-to-fine transfer. Consequently, HIR is introduced to facilitate cross-collaboration and information compensation between the scales, thus further refine the feature maps and promote the visual quality of the final output. We demonstrate the effectiveness of the proposed modules via ablation experiments and visualized results. Extensive experiments on public benchmarks show that HFUR achieves state-of-the-art performance for both constant bit rate and constant QP modes.

Hierarchical Frequency-based Upsampling and Refining for Compressed Video Quality Enhancement

TL;DR

This work proposes a hierarchical frequency-based upsampling and refining neural network (HFUR) for compressed video quality enhancement that achieves state-of-the-art performance for both constant bit rate and constant QP modes.

Abstract

Video compression artifacts arise due to the quantization operation in the frequency domain. The goal of video quality enhancement is to reduce compression artifacts and reconstruct a visually-pleasant result. In this work, we propose a hierarchical frequency-based upsampling and refining neural network (HFUR) for compressed video quality enhancement. HFUR consists of two modules: implicit frequency upsampling module (ImpFreqUp) and hierarchical and iterative refinement module (HIR). ImpFreqUp exploits DCT-domain prior derived through implicit DCT transform, and accurately reconstructs the DCT-domain loss via a coarse-to-fine transfer. Consequently, HIR is introduced to facilitate cross-collaboration and information compensation between the scales, thus further refine the feature maps and promote the visual quality of the final output. We demonstrate the effectiveness of the proposed modules via ablation experiments and visualized results. Extensive experiments on public benchmarks show that HFUR achieves state-of-the-art performance for both constant bit rate and constant QP modes.
Paper Structure (12 sections, 17 equations, 7 figures, 5 tables)

This paper contains 12 sections, 17 equations, 7 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods
  4. Preliminary
  5. Implicit Frequency Upsampling
  6. Hierarchical and Iterative Refinement
  7. Experiments
  8. Dataset
  9. Implementation Details
  10. Comparison with State-of-the-Art Approaches
  11. Ablation Study
  12. Conclusion

Figures (7)

  • Figure 1: Blurring (blue patch) and artifacts (red patch) in compressed video. Existing methods fall short in reconstructing visually-pleasant outcomes, yielding results that are excessively smooth or still exhibit some artifacts.
  • Figure 2: Overview of HFUR for compressed video quality enhancement. HFUR consists of two modules: implicit frequency upsampling module and hierarchical and iterative refinement module. Note that ImpFreqUp$^*$ is a special upsampling module that achieves $\times 1$ upsampling.
  • Figure 3: Architecture of ImpFreqUp.
  • Figure 4: Architecture of HIR
  • Figure 5: Qualitative results on the state-of-the-art methods and our method on CBR. The test video name (from top to bottom): BasketballPass, PartyScene, and RaceHorses.
  • ...and 2 more figures