Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cross-Frequency Implicit Neural Representation with Self-Evolving Parameters

Chang Yu, Yisi Luo, Kai Ye, Xile Zhao, Deyu Meng

TL;DR

This work introduces CF-INR, a cross-frequency implicit neural representation that leverages the Haar wavelet transform to decouple data into four frequency components, each modeled by dedicated INRs. It develops a tensor-decomposition framework that shares a spectral core while decoupling spatial factors, enabling efficient modeling of inter- and intra-frequency relationships. The authors derive theoretical notions of cross-frequency rank and Laplacian smoothness to drive self-evolving updates of per-frequency ranks and frequency parameters, reducing manual tuning. Extensive experiments across image regression, inpainting, hyperspectral denoising, and cloud removal demonstrate superior accuracy and robustness compared with state-of-the-art INR and model-based methods, highlighting CF-INR’s potential for versatile continuous data representation and recovery.

Abstract

Implicit neural representation (INR) has emerged as a powerful paradigm for visual data representation. However, classical INR methods represent data in the original space mixed with different frequency components, and several feature encoding parameters (e.g., the frequency parameter $ω$ or the rank $R$) need manual configurations. In this work, we propose a self-evolving cross-frequency INR using the Haar wavelet transform (termed CF-INR), which decouples data into four frequency components and employs INRs in the wavelet space. CF-INR allows the characterization of different frequency components separately, thus enabling higher accuracy for data representation. To more precisely characterize cross-frequency components, we propose a cross-frequency tensor decomposition paradigm for CF-INR with self-evolving parameters, which automatically updates the rank parameter $R$ and the frequency parameter $ω$ for each frequency component through self-evolving optimization. This self-evolution paradigm eliminates the laborious manual tuning of these parameters, and learns a customized cross-frequency feature encoding configuration for each dataset. We evaluate CF-INR on a variety of visual data representation and recovery tasks, including image regression, inpainting, denoising, and cloud removal. Extensive experiments demonstrate that CF-INR outperforms state-of-the-art methods in each case.

Cross-Frequency Implicit Neural Representation with Self-Evolving Parameters

TL;DR

This work introduces CF-INR, a cross-frequency implicit neural representation that leverages the Haar wavelet transform to decouple data into four frequency components, each modeled by dedicated INRs. It develops a tensor-decomposition framework that shares a spectral core while decoupling spatial factors, enabling efficient modeling of inter- and intra-frequency relationships. The authors derive theoretical notions of cross-frequency rank and Laplacian smoothness to drive self-evolving updates of per-frequency ranks and frequency parameters, reducing manual tuning. Extensive experiments across image regression, inpainting, hyperspectral denoising, and cloud removal demonstrate superior accuracy and robustness compared with state-of-the-art INR and model-based methods, highlighting CF-INR’s potential for versatile continuous data representation and recovery.

Abstract

Implicit neural representation (INR) has emerged as a powerful paradigm for visual data representation. However, classical INR methods represent data in the original space mixed with different frequency components, and several feature encoding parameters (e.g., the frequency parameter or the rank ) need manual configurations. In this work, we propose a self-evolving cross-frequency INR using the Haar wavelet transform (termed CF-INR), which decouples data into four frequency components and employs INRs in the wavelet space. CF-INR allows the characterization of different frequency components separately, thus enabling higher accuracy for data representation. To more precisely characterize cross-frequency components, we propose a cross-frequency tensor decomposition paradigm for CF-INR with self-evolving parameters, which automatically updates the rank parameter and the frequency parameter for each frequency component through self-evolving optimization. This self-evolution paradigm eliminates the laborious manual tuning of these parameters, and learns a customized cross-frequency feature encoding configuration for each dataset. We evaluate CF-INR on a variety of visual data representation and recovery tasks, including image regression, inpainting, denoising, and cloud removal. Extensive experiments demonstrate that CF-INR outperforms state-of-the-art methods in each case.

Paper Structure

This paper contains 35 sections, 7 theorems, 28 equations, 14 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. High Frequency-Enhanced INR
  4. Wavelet Transform in Deep Learning
  5. The Proposed CF-INR
  6. Notations and Preliminaries
  7. Tensor Notations
  8. Haar Wavelet Transform
  9. Haar Wavelet Transform-Induced CF-INR
  10. Efficient Tensor Decomposition Paradigm for CF-INR
  11. Cross-Frequency Tensor Decomposition for CF-INR
  12. Spectrally Coupled and Spatially Decoupled Paradigm
  13. Efficiency Analysis for CF-INR
  14. Self-Evolving Cross-Frequency Low-Rankness
  15. Cross-Frequency Tensor Rank
  16. ...and 20 more sections

Key Result

Lemma 1

For a tensor $\mathcal{X} \in \mathbb{R}^{n_1 \times n_2 \times n_3}$ with Tucker rank $\mathrm{rank}_{T}(\mathcal{X}) = (r_1, r_2, r_3)$, there exist a core tensor $\mathcal{C} \in \mathbb{R}^{r_1 \times r_2 \times r_3}$ and three factor matrices $\boldsymbol{U} \in \mathbb{R}^{n_1 \times r_1}$, $\

Figures (14)

  • Figure 1: The frequency decoupling of CF-INR enables more accurate continuous data representation. Top: fitting a multispectral image using different INRs (WIRE Wire, SIREN activation_function1, and the proposed frequency decoupled CF-INR), along with zoom-in figures and the corresponding error maps. Bottom: the wavelet coefficients learned by CF-INR with the corresponding Fourier spectrum.
  • Figure 2: (a) The conventional CF-INR optimization model \ref{['direct']} for continuous data representation. (b) The proposed generative CF-INR optimization model \ref{['indirect']} for continuous data representation, which solely uses the inverse HWT.
  • Figure 3: Illustration of the proposed cross-frequency tensor decomposition with spectral coupling and spatial decoupling paradigms for CF-INR, which enjoys better computational efficiency and exploits both the intra- and inter-relationships among different wavelet coefficients.
  • Figure 4: Cumulative energy curves (defined by $\mathrm{CE}(k)={\sum_{i=1}^k \sigma_i}/{\sum_{i=1}^n \sigma_i}$, where $\sigma_i$ denotes the $i$-th singular value) of the unfolded wavelet coefficient matrices along the $x$ and $y$ spatial dimensions of an image.
  • Figure 5: Histograms of the Laplacian values derived from the wavelet coefficients of different images Face, Cups, and Cloth.
  • ...and 9 more figures

Theorems & Definitions (12)

  • Definition 1: Tensor Tucker rank Tucker
  • Lemma 1: Tensor Tucker decomposition Tucker
  • Definition 2: Haar discrete wavelet transform wavelets
  • Definition 3: Frontal slice-wise HWTwavelets
  • Definition 4
  • Lemma 2: Relationship between CF-Rank and Tucker rank
  • Lemma 3: The convex envelope of CF-Rank
  • Theorem 5: Smoothness bound for wavelet coefficients of a tensor
  • Definition 6: Discrete Laplacian of a tensorLaplacian
  • Corollary 7: Laplacian bounds for wavelet coefficients of a tensor
  • ...and 2 more