Hybrid Local-Global Context Learning for Neural Video Compression

TL;DR

This paper proposes a hybrid context generation module, which combines the advantages of the above methods in an optimal way and achieves accurate compensation at a low bit cost and can significantly enhance the state-of-the-art methods on standard test datasets.

Abstract

In neural video codecs, current state-of-the-art methods typically adopt multi-scale motion compensation to handle diverse motions. These methods estimate and compress either optical flow or deformable offsets to reduce inter-frame redundancy. However, flow-based methods often suffer from inaccurate motion estimation in complicated scenes. Deformable convolution-based methods are more robust but have a higher bit cost for motion coding. In this paper, we propose a hybrid context generation module, which combines the advantages of the above methods in an optimal way and achieves accurate compensation at a low bit cost. Specifically, considering the characteristics of features at different scales, we adopt flow-guided deformable compensation at largest-scale to produce accurate alignment in detailed regions. For smaller-scale features, we perform flow-based warping to save the bit cost for motion coding. Furthermore, we design a local-global context enhancement module to fully explore the local-global information of previous reconstructed signals. Experimental results demonstrate that our proposed Hybrid Local-Global Context learning (HLGC) method can significantly enhance the state-of-the-art methods on standard test datasets.

Figures (5)

• Figure 1: The overview of the proposed Hybrid Local-Global Context learning method.
• Figure 2: Illustration for the hybrid context generation module.
• Figure 3: Our proposed local-global context enhancement module.
• Figure 4: The network structure of multi-scale context fusion module.
• Figure 5: RD-curves on the HEVC B, C and D datasets.