A New Perspective To Understanding Multi-resolution Hash Encoding For Neural Fields

TL;DR

This paper addresses why multi-resolution hash grids, as used in Instant-NGP, dramatically boost the expressivity of neural fields. It introduces domain manipulation as a ground-up explanation, demonstrating that grid operations create additional turning points and multiply the network’s effective piecewise linear segments, with flipping playing a key role. Empirical support comes from 1D toy-signal experiments, and the insights are extended to higher dimensions, highlighting the role of feature dimension F and multi-resolution design. The work offers a principled framework to guide hash-grid hyperparameter choices and motivates future exploration of hashing interactions and convergence properties.

Abstract

Instant-NGP has been the state-of-the-art architecture of neural fields in recent years. Its incredible signal-fitting capabilities are generally attributed to its multi-resolution hash grid structure and have been used and improved in numerous following works. However, it is unclear how and why such a hash grid structure improves the capabilities of a neural network by such great margins. A lack of principled understanding of the hash grid also implies that the large set of hyperparameters accompanying Instant-NGP could only be tuned empirically without much heuristics. To provide an intuitive explanation of the working principle of the hash grid, we propose a novel perspective, namely domain manipulation. This perspective provides a ground-up explanation of how the feature grid learns the target signal and increases the expressivity of the neural field by artificially creating multiples of pre-existing linear segments. We conducted numerous experiments on carefully constructed 1-dimensional signals to support our claims empirically and aid our illustrations. While our analysis mainly focuses on 1-dimensional signals, we show that the idea is generalizable to higher dimensions.

Figures (8)

• Figure 1: NGP architecture. Taken from muller2022instant.
• Figure 2: Illustration of domain manipulation at different input and feature dimensions.
• Figure 3: Components of the error between the learnt function and the target function. Taken from raghu2017expressive.
• Figure 4: Number of domain flips, MLP segments, and prediction segments when fitting to different signal frequency bandwidths.
• Figure 5: Visualizing the effects of a 1-dimensional, single-resolution, no-hashing grid when fitting to a 1-dimensional signal.