Group-aware Parameter-efficient Updating for Content-Adaptive Neural Video Compression

TL;DR

This work tackles the challenge of generalization and error accumulation in content-adaptive neural video compression (NVC) by introducing Group-aware Parameter-efficient Updating (GPU). GPU combines patch-based GoP updating to minimize error propagation with encoder-side adapters (serial and parallel) to enable efficient, parameter-light updates. The method integrates into a contemporary NVC framework and yields substantial rate-distortion gains across six video benchmarks and a cardiac MRI dataset, often outperforming traditional codecs like VVC/H.266 and prior content-adaptive NVC methods while updating less than 8% of encoder parameters. The approach enhances cross-domain adaptability, including medical imaging, and offers a practical, architecture-agnostic path toward efficient content-adaptive NVC.

Abstract

Content-adaptive compression is crucial for enhancing the adaptability of the pre-trained neural codec for various contents. Although these methods have been very practical in neural image compression (NIC), their application in neural video compression (NVC) is still limited due to two main aspects: 1), video compression relies heavily on temporal redundancy, therefore updating just one or a few frames can lead to significant errors accumulating over time; 2), NVC frameworks are generally more complex, with many large components that are not easy to update quickly during encoding. To address the previously mentioned challenges, we have developed a content-adaptive NVC technique called Group-aware Parameter-Efficient Updating (GPU). Initially, to minimize error accumulation, we adopt a group-aware approach for updating encoder parameters. This involves adopting a patch-based Group of Pictures (GoP) training strategy to segment a video into patch-based GoPs, which will be updated to facilitate a globally optimized domain-transferable solution. Subsequently, we introduce a parameter-efficient delta-tuning strategy, which is achieved by integrating several light-weight adapters into each coding component of the encoding process by both serial and parallel configuration. Such architecture-agnostic modules stimulate the components with large parameters, thereby reducing both the update cost and the encoding time. We incorporate our GPU into the latest NVC framework and conduct comprehensive experiments, whose results showcase outstanding video compression efficiency across four video benchmarks and adaptability of one medical image benchmark.

Figures (8)

• Figure 1: The overview of our Patch-based Group of Pictures (GoP) updating strategy. It begins by segmenting each video into multiple GoPs as in conventional video codecs. We then spatially divide each GoP into several patch-based GoPs. Following this, we apply our Group-aware Parameter-efficient Updating (GPU) technique to update these patch-based GoPs sequentially. Once all the patch-based GoPs have been optimized, we feed the integrated GoP into the Neural Video Compression (NVC) framework to generate the compressed bit-stream. This process is repeated iteratively until the entire video has been compressed.
• Figure 2: The configurations of how we place our adaptor in (a) parallel and (b) serial ways integrated with our different coding components, and (c) the architecture of our adaptor.
• Figure 3: Overview of our content-adaptive NVC framework with proposed adapters. During online updating, our approach aligns with previous content-adaptive techniques, which only update the modules utilized during the encoding phase (blue). Modules that are also relevant to the decoding phase (white) remain unaltered. The adaptors are integrated within these encoding-specific modules, allowing for the freezing of existing module parameters while updating those within themselves.
• Figure 4: Rate-distortion (i.e., bit-rates vs PSNR) performance comparison between our and other state-of-the-art video compression methods on the standard video compression benchmarks, UVG, MCL-JCV, HEVC Class B, C, D, and E.
• Figure 5: Rate-distortion (i.e., bit-rates vs PSNR) performance comparison of our and other state-of-the-art video codecs on medical volumetric image compression benchmark, ACDC.