Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improved Implicit Neural Representation with Fourier Reparameterized Training

Kexuan Shi, Xingyu Zhou, Shuhang Gu

TL;DR

A Fourier reparameterization method which learns coefficient matrix of fixed Fourier bases to compose the weights of MLP is proposed and theoretically prove that weight reparameterization could provide a chance to alleviate the spectral bias of MLP.

Abstract

Implicit Neural Representation (INR) as a mighty representation paradigm has achieved success in various computer vision tasks recently. Due to the low-frequency bias issue of vanilla multi-layer perceptron (MLP), existing methods have investigated advanced techniques, such as positional encoding and periodic activation function, to improve the accuracy of INR. In this paper, we connect the network training bias with the reparameterization technique and theoretically prove that weight reparameterization could provide us a chance to alleviate the spectral bias of MLP. Based on our theoretical analysis, we propose a Fourier reparameterization method which learns coefficient matrix of fixed Fourier bases to compose the weights of MLP. We evaluate the proposed Fourier reparameterization method on different INR tasks with various MLP architectures, including vanilla MLP, MLP with positional encoding and MLP with advanced activation function, etc. The superiority approximation results on different MLP architectures clearly validate the advantage of our proposed method. Armed with our Fourier reparameterization method, better INR with more textures and less artifacts can be learned from the training data.

Improved Implicit Neural Representation with Fourier Reparameterized Training

TL;DR

A Fourier reparameterization method which learns coefficient matrix of fixed Fourier bases to compose the weights of MLP is proposed and theoretically prove that weight reparameterization could provide a chance to alleviate the spectral bias of MLP.

Abstract

Implicit Neural Representation (INR) as a mighty representation paradigm has achieved success in various computer vision tasks recently. Due to the low-frequency bias issue of vanilla multi-layer perceptron (MLP), existing methods have investigated advanced techniques, such as positional encoding and periodic activation function, to improve the accuracy of INR. In this paper, we connect the network training bias with the reparameterization technique and theoretically prove that weight reparameterization could provide us a chance to alleviate the spectral bias of MLP. Based on our theoretical analysis, we propose a Fourier reparameterization method which learns coefficient matrix of fixed Fourier bases to compose the weights of MLP. We evaluate the proposed Fourier reparameterization method on different INR tasks with various MLP architectures, including vanilla MLP, MLP with positional encoding and MLP with advanced activation function, etc. The superiority approximation results on different MLP architectures clearly validate the advantage of our proposed method. Armed with our Fourier reparameterization method, better INR with more textures and less artifacts can be learned from the training data.
Paper Structure (25 sections, 2 theorems, 32 equations, 16 figures, 6 tables)

This paper contains 25 sections, 2 theorems, 32 equations, 16 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. The formulation of INRs
  5. Fourier reparameterization
  6. Discussion
  7. Implementation details
  8. Experimental Analysis on Simple Function Approximation
  9. Experimental settings
  10. Approximation results and analysis
  11. Experimental Results on Vision Applications
  12. 2D Color image approximation
  13. Representing shapes with signed distance functions
  14. Learning neural radiance fields
  15. Ablation study
  16. ...and 10 more sections

Key Result

Theorem 1

(Theorem 1 in xu_fourier_framework) Consider a MLP with one hidden layer using tanh function $\sigma (x)$ as the activation function. For any frequencies $k_{1}$ and $k_{2}$ such that $k_{1} > k_{2} > 0$ and there exist $c_{1},c_{2},$ such that $A(k_{1})>c_{1}>0,A(k_{1})<c_{2}<\infty$, we have where $B_{\delta }$ is a ball with radius $\delta$ centered at the origin and $\mu(\cdot)$ is the Lebesg

Figures (16)

  • Figure 2: Visualization of simple function. The left side of the first row displays the visualization of the 1D simple function on the $x-y$ coordinate axis. The left side of the second row shows the amplitude of the function in the frequency domain. The right side presents the average loss curve with the shaded area indicating the fluctuation from 100 repetitions.
  • Figure 3: Evolution of frequency-specific approximation error with training iterations of four different methods (x-axis for training step, y-axis for frequency and colormap for relative approximation error).
  • Figure 4: The magnitude of different NTK eigenvalues. 'First' denotes the percenatge of the largest eigenvalue, 'Second' represents the percentage of the second-largest eigenvalue. 'Remain' refers to the percentage of the summation of remaining eigenvalues.
  • Figure 5: Visual examples of the 2D color image approximation results (PSNR) by different methods. Detailed experimental settings can be found in section \ref{['2D_image']}.
  • Figure 6: Visual examples of the shape representation results (IOU) by different methods. More experimental details can be found in section \ref{['SDF']}.
  • ...and 11 more figures

Theorems & Definitions (3)

  • Theorem 1
  • Theorem 2
  • proof