Frequency-aware Neural Representation for Videos
Jun Zhu, Xinfeng Zhang, Lv Tang, Junhao Jiang, Gai Zhang, Jia Wang
TL;DR
FaNeRV addresses spectral bias in implicit neural representations for video compression by decoupling low- and high-frequency content through multi-resolution supervision, dynamic high-frequency injection, and a frequency-decomposed network. The method updates targets via $V_{target}=V_S+\beta\cdot R_{HF}$ with $R_{HF}=\mathcal{H}(V_S-\hat{V}_S)$. It demonstrates strong rate-distortion gains over INR baselines and competitive performance with traditional codecs on standard benchmarks, and supports scalable coding within a single model. This frequency-aware design provides a practical route to high-quality neural video codecs and suggests broader utility for spectral-aware representations.
Abstract
Implicit Neural Representations (INRs) have emerged as a promising paradigm for video compression. However, existing INR-based frameworks typically suffer from inherent spectral bias, which favors low-frequency components and leads to over-smoothed reconstructions and suboptimal rate-distortion performance. In this paper, we propose FaNeRV, a Frequency-aware Neural Representation for videos, which explicitly decouples low- and high-frequency components to enable efficient and faithful video reconstruction. FaNeRV introduces a multi-resolution supervision strategy that guides the network to progressively capture global structures and fine-grained textures through staged supervision . To further enhance high-frequency reconstruction, we propose a dynamic high-frequency injection mechanism that adaptively emphasizes challenging regions. In addition, we design a frequency-decomposed network module to improve feature modeling across different spectral bands. Extensive experiments on standard benchmarks demonstrate that FaNeRV significantly outperforms state-of-the-art INR methods and achieves competitive rate-distortion performance against traditional codecs.