Where Do We Stand with Implicit Neural Representations? A Technical and Performance Survey

TL;DR

This survey analyzes implicit neural representations (INRs) as continuous, resolution-independent encodings that address the memory and discretisation challenges of explicit representations. It proposes a four-way taxonomy—activation functions, positional encoding, combined activation+encoding, and network design—and rigorously compares SOTA INR methods across 1D audio, 2D CT/denoising/SR, and 3D occupancy tasks. Key findings show Incode and Fourier Reparameterised Training (Fr) achieving top fidelity, while Fr and Finer provide favorable speed–accuracy trade-offs; frequency-compact representations emerge as crucial for high-frequency details and 3D geometry. The work offers practical guidance for method selection and identifies avenues for future research, including more expressive activations, scalable high-dimensional INRs, and hybrids with explicit representations.

Abstract

Implicit Neural Representations (INRs) have emerged as a paradigm in knowledge representation, offering exceptional flexibility and performance across a diverse range of applications. INRs leverage multilayer perceptrons (MLPs) to model data as continuous implicit functions, providing critical advantages such as resolution independence, memory efficiency, and generalisation beyond discretised data structures. Their ability to solve complex inverse problems makes them particularly effective for tasks including audio reconstruction, image representation, 3D object reconstruction, and high-dimensional data synthesis. This survey provides a comprehensive review of state-of-the-art INR methods, introducing a clear taxonomy that categorises them into four key areas: activation functions, position encoding, combined strategies, and network structure optimisation. We rigorously analyse their critical properties, such as full differentiability, smoothness, compactness, and adaptability to varying resolutions while also examining their strengths and limitations in addressing locality biases and capturing fine details. Our experimental comparison offers new insights into the trade-offs between different approaches, showcasing the capabilities and challenges of the latest INR techniques across various tasks. In addition to identifying areas where current methods excel, we highlight key limitations and potential avenues for improvement, such as developing more expressive activation functions, enhancing positional encoding mechanisms, and improving scalability for complex, high-dimensional data. This survey serves as a roadmap for researchers, offering practical guidance for future exploration in the field of INRs. We aim to foster new methodologies by outlining promising research directions for INRs and applications.

Figures (15)

• Figure 1: The four categories of state-of-the-art (SOTA) implicit neural representation (INR) methods. The yellow blocks highlight the specific components each method enhances. Specifically: (a) focuses on improving activation functions, (b) enhances position encoding, (c) integrates both (a) and (b) to simultaneously improve activation functions and position encoding, and (d) advances the overall network structure.
• Figure 2: A comprehensive comparison of the INR methods, each represented by a method card. The cards outline key properties, including frequency compactness, spatial compactness, adaptability of the methods, and implementation details with the number of hyperparameters. The categories mentioned correspond to those in Figure \ref{['fig:teaser']}.
• Figure 3: Visualisation of the activation functions used in the method, categorised as (a) in Figure \ref{['fig:teaser']}.
• Figure 4: The visualisation with $L_2$ loss metric comparison on audio reconstruction task across the 8 implicit neural representation methods.
• Figure 5: The visual comparison of CT reconstruction results using the 8 implicit neural representation methods across 20, 50, 100, 200, and 300 projections.