Lagrangian Hashing for Compressed Neural Field Representations

TL;DR

LagHash tackles the memory-efficiency of neural fields by marrying Eulerian hash-grid representations with a Lagrangian, point-based Gaussian mixture in the high-resolution levels. It extends InstantNGP by storing a small MoG per hash bucket, enabling a mixture-of-Gaussians representation that concentrates capacity where needed, guided by an EM-inspired loss that aligns Gaussians with surface regions. The method achieves high-fidelity reconstructions with substantially fewer parameters across 2D gigapixel fitting and 3D NeRF tasks, outperforming or matching state-of-the-art baselines like InstantNGP and CompactNGP in PSNR at similar or reduced model sizes. This hybrid, adaptive representation offers practical benefits for compact neural field codecs and scalable rendering, with ablations validating the importance of the guidance and distortion terms and the chosen Gaussian configuration.

Abstract

We present Lagrangian Hashing, a representation for neural fields combining the characteristics of fast training NeRF methods that rely on Eulerian grids (i.e.~InstantNGP), with those that employ points equipped with features as a way to represent information (e.g. 3D Gaussian Splatting or PointNeRF). We achieve this by incorporating a point-based representation into the high-resolution layers of the hierarchical hash tables of an InstantNGP representation. As our points are equipped with a field of influence, our representation can be interpreted as a mixture of Gaussians stored within the hash table. We propose a loss that encourages the movement of our Gaussians towards regions that require more representation budget to be sufficiently well represented. Our main finding is that our representation allows the reconstruction of signals using a more compact representation without compromising quality.

Figures (8)

• Figure 1: We introduce a hybrid representation that is simultaneously Eulerian (grids) and Lagrangian (points), which realizes high-quality novel view synthesis as shown above, while at the same time being more memory efficient.
• Figure 2: (1) Hashing of voxel vertices: For any given input coordinate $x_i$, our method identifies surrounding voxels across $L$ Levels of detail (Lods) (Only one Lod is showed for convenience). Indices are then assigned to the vertices of these voxels, through hashing procedure. (2) Lookup to buckets: for all resulting corner indices, we look up the corresponding $B$ buckets, containing $K$ feature vector and their corresponding $\mu_{k}$ position. (3) Gaussian interpolation: We compute Gaussian weights with respect to the input position for every feature vector in the bucket. (4) Feature aggregation: We multiply the Gaussian weights for the feature corresponding to the feature vector and aggregate them from every level of detail. (5) Neural Network: the resulting concatenated features are mapped to the input domain by the Neural Network.
• Figure 3: Qualitative comparisons on the giga-pixel images. On each image, we show the reconstruction quality (PSNR) together with the number of parameters.
• Figure 4: Qualitative comparisons on the Synthetic NeRF Dataset mildenhall2021nerf. The leftmost column (reconstruction) shows the full-image results of our method and the rightmost column shows the Lagrangian Representation which is learned by our model for the particular scene.
• Figure 5: Qualitative comparisons on the Tanks and Temples dataset knapitsch2017tanks. The leftmost column (reconstruction) shows the full-image results of our method and the Rightmost column shows the Lagrangian Representation which is learned by our model.