Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Compressed Domain Prior-Guided Video Super-Resolution for Cloud Gaming Content

Qizhe Wang, Qian Yin, Zhimeng Huang, Weijia Jiang, Yi Su, Siwei Ma, Jiaqi Zhang

TL;DR

A novel lightweight network called Coding Prior-Guided Super-Resolution (CPGSR) is proposed to address the SR challenges in compressed game video content and proposes a partitioned focal frequency loss to effectively guide the model's focus on preserving high-frequency information.

Abstract

Cloud gaming is an advanced form of Internet service that necessitates local terminals to decode within limited resources and time latency. Super-Resolution (SR) techniques are often employed on these terminals as an efficient way to reduce the required bit-rate bandwidth for cloud gaming. However, insufficient attention has been paid to SR of compressed game video content. Most SR networks amplify block artifacts and ringing effects in decoded frames while ignoring edge details of game content, leading to unsatisfactory reconstruction results. In this paper, we propose a novel lightweight network called Coding Prior-Guided Super-Resolution (CPGSR) to address the SR challenges in compressed game video content. First, we design a Compressed Domain Guided Block (CDGB) to extract features of different depths from coding priors, which are subsequently integrated with features from the U-net backbone. Then, a series of re-parameterization blocks are utilized for reconstruction. Ultimately, inspired by the quantization in video coding, we propose a partitioned focal frequency loss to effectively guide the model's focus on preserving high-frequency information. Extensive experiments demonstrate the advancement of our approach.

Compressed Domain Prior-Guided Video Super-Resolution for Cloud Gaming Content

TL;DR

A novel lightweight network called Coding Prior-Guided Super-Resolution (CPGSR) is proposed to address the SR challenges in compressed game video content and proposes a partitioned focal frequency loss to effectively guide the model's focus on preserving high-frequency information.

Abstract

Cloud gaming is an advanced form of Internet service that necessitates local terminals to decode within limited resources and time latency. Super-Resolution (SR) techniques are often employed on these terminals as an efficient way to reduce the required bit-rate bandwidth for cloud gaming. However, insufficient attention has been paid to SR of compressed game video content. Most SR networks amplify block artifacts and ringing effects in decoded frames while ignoring edge details of game content, leading to unsatisfactory reconstruction results. In this paper, we propose a novel lightweight network called Coding Prior-Guided Super-Resolution (CPGSR) to address the SR challenges in compressed game video content. First, we design a Compressed Domain Guided Block (CDGB) to extract features of different depths from coding priors, which are subsequently integrated with features from the U-net backbone. Then, a series of re-parameterization blocks are utilized for reconstruction. Ultimately, inspired by the quantization in video coding, we propose a partitioned focal frequency loss to effectively guide the model's focus on preserving high-frequency information. Extensive experiments demonstrate the advancement of our approach.
Paper Structure (16 sections, 7 equations, 4 figures, 3 tables)

This paper contains 16 sections, 7 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Method
  4. Experiments
  5. Conclusion
  6. Acknowledgement
  7. References

Figures (4)

  • Figure 1: Performance comparison of our CPGSR with nine other lightweight models on the proposed dataset, which is evaluated in terms of PSNRY and model parameter size.
  • Figure 2: The architecture of Coding Prior-Guided Super-Resolution (CPGSR). The proposed network is composed of three parts: Compressed Domain Guided Block (CDGB), mutual information processing and reconstruction. The CDGB part extracts compression features from coding priors. Then the compression features of different depths are fed into the U-net backbone and fused with the input features. Finally, m Repconv blocks are used for the reconstruction of the frame.
  • Figure 3: The architecture of CDGB. A pair of affine transformation parameters is derived based on both the LR features and the coding priors, then compression features of different depths are generated by Repconvs and two PAC blocks. In the PAC block, the partition map is transformed into Pixel-Adaptive weights and participates in the convolution process.
  • Figure 4: (a) The original decoded frame in grayscale (b) The partition map of the frame