Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Geometric Neural Process Fields

Wenzhe Yin, Zehao Xiao, Jiayi Shen, Yunlu Chen, Cees G. M. Snoek, Jan-Jakob Sonke, Efstratios Gavves

TL;DR

Geometric Neural Process Fields (G-NPF) tackle neural field generalization under uncertainty by casting NeF adaptation as probabilistic inference conditioned on geometric priors. The method introduces geometric bases and a hierarchical latent structure (global $z_g$ and local $z_{l,m}$) to jointly encode object-level geometry and coordinate-specific details, enabling robust generalization across 1D, 2D, and 3D signals, including NeRF-style radiance fields. Evaluation across 2D image regression, 3D novel-view synthesis, and 1D GP-like functions shows improved reconstruction fidelity and uncertainty quantification over strong probabilistic baselines. The approach integrates differentiable volume rendering with a transformer-based basis encoder and modulated neural fields, offering a versatile, uncertainty-aware framework for fast adaptation to unseen signals and scenes.

Abstract

This paper addresses the challenge of Neural Field (NeF) generalization, where models must efficiently adapt to new signals given only a few observations. To tackle this, we propose Geometric Neural Process Fields (G-NPF), a probabilistic framework for neural radiance fields that explicitly captures uncertainty. We formulate NeF generalization as a probabilistic problem, enabling direct inference of NeF function distributions from limited context observations. To incorporate structural inductive biases, we introduce a set of geometric bases that encode spatial structure and facilitate the inference of NeF function distributions. Building on these bases, we design a hierarchical latent variable model, allowing G-NPF to integrate structural information across multiple spatial levels and effectively parameterize INR functions. This hierarchical approach improves generalization to novel scenes and unseen signals. Experiments on novel-view synthesis for 3D scenes, as well as 2D image and 1D signal regression, demonstrate the effectiveness of our method in capturing uncertainty and leveraging structural information for improved generalization.

Geometric Neural Process Fields

TL;DR

Geometric Neural Process Fields (G-NPF) tackle neural field generalization under uncertainty by casting NeF adaptation as probabilistic inference conditioned on geometric priors. The method introduces geometric bases and a hierarchical latent structure (global and local ) to jointly encode object-level geometry and coordinate-specific details, enabling robust generalization across 1D, 2D, and 3D signals, including NeRF-style radiance fields. Evaluation across 2D image regression, 3D novel-view synthesis, and 1D GP-like functions shows improved reconstruction fidelity and uncertainty quantification over strong probabilistic baselines. The approach integrates differentiable volume rendering with a transformer-based basis encoder and modulated neural fields, offering a versatile, uncertainty-aware framework for fast adaptation to unseen signals and scenes.

Abstract

This paper addresses the challenge of Neural Field (NeF) generalization, where models must efficiently adapt to new signals given only a few observations. To tackle this, we propose Geometric Neural Process Fields (G-NPF), a probabilistic framework for neural radiance fields that explicitly captures uncertainty. We formulate NeF generalization as a probabilistic problem, enabling direct inference of NeF function distributions from limited context observations. To incorporate structural inductive biases, we introduce a set of geometric bases that encode spatial structure and facilitate the inference of NeF function distributions. Building on these bases, we design a hierarchical latent variable model, allowing G-NPF to integrate structural information across multiple spatial levels and effectively parameterize INR functions. This hierarchical approach improves generalization to novel scenes and unseen signals. Experiments on novel-view synthesis for 3D scenes, as well as 2D image and 1D signal regression, demonstrate the effectiveness of our method in capturing uncertainty and leveraging structural information for improved generalization.

Paper Structure

This paper contains 60 sections, 36 equations, 17 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Neural (Radiance) Fields
  4. Neural Processes
  5. Geometric Neural Process Fields
  6. Probabilistic Neural Process Fields
  7. Adding Geometric Priors to Probabilistic Neural Process Fields
  8. Adding Hierarchical Priors to Probabilistic Neural Process Fields
  9. Implementation
  10. Geometric basis functions.
  11. Global latent variables.
  12. Local latent variables.
  13. Predictive distribution.
  14. Training objective
  15. G-NPF in 1D, 2D, 3D
  16. ...and 45 more sections

Figures (17)

  • Figure 1: Illustration of the proposed G-NPF.
  • Figure 2: Graphical model for the proposed geometric neural processes.
  • Figure 3: Visualizations of image regression results on CelebA (left) and Imagenette (right).
  • Figure 4: Qualitative results of the proposed G-NPF on novel view synthesis of ShapeNet objects. Both 1-view (top) and 2-view (bottom) context results are presented.
  • Figure 5: Using transformer encoder to generate ray-specific latent variable $\mathbf{z}_l$.
  • ...and 12 more figures