Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Content-Aware Frequency Encoding for Implicit Neural Representations with Fourier-Chebyshev Features

Junbo Ke, Yangyang Xu, You-Wei Wen, Chao Wang

TL;DR

Content-Aware Frequency Encoding (CAFE), which builds upon Fourier features through multiple parallel linear layers combined via a Hadamard product, and extends this framework to CAFE+, which incorporates Chebyshev features as a complementary component to Fourier bases, providing a stronger and more stable frequency representation.

Abstract

Implicit Neural Representations (INRs) have emerged as a powerful paradigm for various signal processing tasks, but their inherent spectral bias limits the ability to capture high-frequency details. Existing methods partially mitigate this issue by using Fourier-based features, which usually rely on fixed frequency bases. This forces multi-layer perceptrons (MLPs) to inefficiently compose the required frequencies, thereby constraining their representational capacity. To address this limitation, we propose Content-Aware Frequency Encoding (CAFE), which builds upon Fourier features through multiple parallel linear layers combined via a Hadamard product. CAFE can explicitly and efficiently synthesize a broader range of frequency bases, while the learned weights enable the selection of task-relevant frequencies. Furthermore, we extend this framework to CAFE+, which incorporates Chebyshev features as a complementary component to Fourier bases. This combination provides a stronger and more stable frequency representation. Extensive experiments across multiple benchmarks validate the effectiveness and efficiency of our approach, consistently achieving superior performance over existing methods. Our code is available at https://github.com/JunboKe0619/CAFE.

Content-Aware Frequency Encoding for Implicit Neural Representations with Fourier-Chebyshev Features

TL;DR

Content-Aware Frequency Encoding (CAFE), which builds upon Fourier features through multiple parallel linear layers combined via a Hadamard product, and extends this framework to CAFE+, which incorporates Chebyshev features as a complementary component to Fourier bases, providing a stronger and more stable frequency representation.

Abstract

Implicit Neural Representations (INRs) have emerged as a powerful paradigm for various signal processing tasks, but their inherent spectral bias limits the ability to capture high-frequency details. Existing methods partially mitigate this issue by using Fourier-based features, which usually rely on fixed frequency bases. This forces multi-layer perceptrons (MLPs) to inefficiently compose the required frequencies, thereby constraining their representational capacity. To address this limitation, we propose Content-Aware Frequency Encoding (CAFE), which builds upon Fourier features through multiple parallel linear layers combined via a Hadamard product. CAFE can explicitly and efficiently synthesize a broader range of frequency bases, while the learned weights enable the selection of task-relevant frequencies. Furthermore, we extend this framework to CAFE+, which incorporates Chebyshev features as a complementary component to Fourier bases. This combination provides a stronger and more stable frequency representation. Extensive experiments across multiple benchmarks validate the effectiveness and efficiency of our approach, consistently achieving superior performance over existing methods. Our code is available at https://github.com/JunboKe0619/CAFE.
Paper Structure (25 sections, 6 theorems, 38 equations, 20 figures, 8 tables)

This paper contains 25 sections, 6 theorems, 38 equations, 20 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Preliminaries
  5. Content-Aware Frequency Encoding
  6. Analysis of CAFE
  7. CAFE+ with Fourier-Chebyshev Features
  8. Experiments
  9. Main results
  10. Ablation study
  11. Conclusion
  12. Proofs of Theorems
  13. Experimental Settings for Main Tasks
  14. Implementation Note.
  15. Additional Experiments
  16. ...and 10 more sections

Key Result

Theorem 1

Let $f_{\boldsymbol{\theta}}: \mathbb{R}^{D} \to \mathbb{R}$ be an INR of the form of Eq. eq:INR with $\rho(z) = \sum_{k=0}^{K} \alpha_k z^k$. The network takes an input encoding $\gamma(\mathbf{x})= \sin(\mathbf{\Omega} \mathbf{x} + \mathbf{\bm{\phi}})$, where $\mathbf{\bm{\Omega}} \in \mathbb{R}^{ where $c_{\bm{\omega}'} \in \mathbb{R}$ are coefficients determined by the network parameters. The

Figures (20)

  • Figure 1: Effect of the total number of linear layers on the image fitting task in terms of PSNR. For the RFF baselines, linear layers exist only in the MLP. For our methods (CAFE and CAFE+), the total number of linear layers includes those in both the encoding stage and the MLP. Any increase in this total arises from additional layers in the encoding stage, while the MLP depth remains fixed. The legend indicates the number of hidden neurons per layer.
  • Figure 2: Qualitative comparison of RFF, CAFE, and CAFE+ under the same number of parameters for the image fitting task. High- and low-frequency regions are zoomed in, and corresponding error maps are provided for a detailed evaluation.
  • Figure 3: NTK matrix for the CAFE and RFF. The network width and the number of sampled points were both set to 1024.
  • Figure 4: Overall framework of CAFE+. CAFE+ maps the input coordinates separately to Chebyshev and Fourier features, which are then concatenated to form the inputs of multiple parallel linear layers. The outputs of these layers are combined via the Hadamard product to obtain the CAFE+ encoded features, which are subsequently fed into the backbone MLP to produce the corresponding values.
  • Figure 5: Illustration of the complementary roles of Chebyshev and Fourier features. The network is first trained with the CAFE+, and during inference, we deactivate the neurons corresponding to either $\Phi_{\text{CF}}(\mathbf{x})$ or $\Phi_{\text{FF}}(\mathbf{x})$ by setting them to zero to isolate their individual effects. CF-only corresponds to using $[\,\mathbf{0},\, \Phi_{\text{CF}}(\mathbf{x})\,]$ while FF-only corresponds to $[\,\Phi_{\text{FF}}(\mathbf{x}),\, \mathbf{0}\,]$.
  • ...and 15 more figures

Theorems & Definitions (11)

  • Definition 1: Fourier Features tancik2020fourier
  • Theorem 1: yuce2022structured
  • Theorem 2
  • Definition 2: Chebyshev Features
  • Theorem 3
  • Theorem
  • Lemma 1: Chebyshev Product
  • proof
  • Lemma 2: Chebyshev Power
  • proof
  • ...and 1 more