Dynamical Implicit Neural Representations

TL;DR

Dynamical Implicit Neural Representations (DINR) tackle spectral bias in implicit neural representations by replacing static depth with continuous-time latent dynamics, enabling richer frequency representations without increasing model size. The authors establish theoretical gains in expressivity and trainability via Rademacher complexity and Neural Tangent Kernel analyses, and introduce a kinetic energy regularizer to balance expressivity with generalization. Empirically, DINR consistently outperforms traditional INRs on data compression and field reconstruction across diverse scientific domains, while also showing improved convergence stability and robustness to noise. The work offers a principled framework for designing parameter-efficient, high-fidelity implicit representations with broad applicability to vision, graphics, and scientific computing.

Abstract

Implicit Neural Representations (INRs) provide a powerful continuous framework for modeling complex visual and geometric signals, but spectral bias remains a fundamental challenge, limiting their ability to capture high-frequency details. Orthogonal to existing remedy strategies, we introduce Dynamical Implicit Neural Representations (DINR), a new INR modeling framework that treats feature evolution as a continuous-time dynamical system rather than a discrete stack of layers. This dynamical formulation mitigates spectral bias by enabling richer, more adaptive frequency representations through continuous feature evolution. Theoretical analysis based on Rademacher complexity and the Neural Tangent Kernel demonstrates that DINR enhances expressivity and improves training dynamics. Moreover, regularizing the complexity of the underlying dynamics provides a principled way to balance expressivity and generalization. Extensive experiments on image representation, field reconstruction, and data compression confirm that DINR delivers more stable convergence, higher signal fidelity, and stronger generalization than conventional static INRs.

Figures (6)

• Figure 1: Left: three example functions from the original function class $\mathcal{F}$. Right: three example functions from the expanded function class $\Phi_{\mathrm{dyn}}\left(\mathcal{F}\right)$ obtained using latent dynamics. Different colors indicate distinct functions within each class.
• Figure 2: Qualitative evaluation of DINR for 2D signal representation. We compare reconstructions from FFNet/SIREN and Dynamical FFNet/SIREN on turbulence and weather data. For each method, we display the reconstructed image, a pixel-wise error map, and the 2D power spectrum of the reconstruction. Error maps are normalized within each model architecture type to highlight the relative distribution of errors; lighter colors indicate lower error.
• Figure 3: Qualitative results of INRs and DINRs on cloud data.
• Figure 4: Qualitative results of INRs and DINRs on Cryo-EM data.
• Figure 5: Task 3 Results Summary: (A) Effective rank of the empirical NTK during training. (B) Condition number of the empirical NTK during training. (C) Decay rate of leading eigenvalues.

Theorems & Definitions (24)

• Theorem 4.1: Expressivity, Informal
• Theorem 4.2: Gradient Diversity, Informal
• Definition B.1: Empirical Rademacher Complexity
• Lemma B.2: Lipschitz Continuity of the Discrete Flow Map
• proof
• Proposition B.3: Rademacher Complexity of DINRs
• proof
• Corollary B.4
• proof
• Proposition B.5: Rademacher Complexity of INRs