H-SIREN: Improving implicit neural representations with hyperbolic periodic functions

TL;DR

This work provides a simple solution to mitigate the use of sinusoidal activation functions for implicit neural representations by changing the activation function at the first layer from $\sin(x)$ to $\sin(\sinh(2x))$.

Abstract

Implicit neural representations (INR) have been recently adopted in various applications ranging from computer vision tasks to physics simulations by solving partial differential equations. Among existing INR-based works, multi-layer perceptrons with sinusoidal activation functions find widespread applications and are also frequently treated as a baseline for the development of better activation functions for INR applications. Recent investigations claim that the use of sinusoidal activation functions could be sub-optimal due to their limited supported frequency set as well as their tendency to generate over-smoothed solutions. We provide a simple solution to mitigate such an issue by changing the activation function at the first layer from $\sin(x)$ to $\sin(\sinh(2x))$. We demonstrate H-SIREN in various computer vision and fluid flow problems, where it surpasses the performance of several state-of-the-art INRs.

Figures (9)

• Figure 1: Plots comparing different activation functions: SIREN (left), FINER (middle) and H-SIREN (right).
• Figure 2: (a) Activation spectrum after hidden layer 1, 2 and 5, for MLP with different activation functions at initialization. (b) Results for fitting a simple function using MLPs with different activation functions.
• Figure 3: Comparison between INRs with different activation functions for fitting 2D images.
• Figure 4: Comparison among INRs with different activation functions for fitting the "bush" video. From top to bottom row: frame 10, 200, 290.
• Figure 5: Comparison among different activation functions for NeRF. Test image 62 of the "lego" scene is plotted.