Object Registration in Neural Fields

TL;DR

The paper tackles robust 6-DoF registration between neural fields, enabling pose estimation between a scene NF and an object NF library. It introduces Reg-NF, an extension of nerf2nerf that uses bidirectional surface-sample optimization and NeuS-based SDFs to estimate the relative transform $\mathbf{T}$ between $a$ and $b^q$, handling scale differences. The authors demonstrate improved registration accuracy and robustness on the Object NF Registration (ONR) dataset, and show practical uses in object completion and scene generation via library substitution. This work enables robust, data-driven scene editing and enhanced object-centric representations for robotics applications.

Abstract

Neural fields provide a continuous scene representation of 3D geometry and appearance in a way which has great promise for robotics applications. One functionality that unlocks unique use-cases for neural fields in robotics is object 6-DoF registration. In this paper, we provide an expanded analysis of the recent Reg-NF neural field registration method and its use-cases within a robotics context. We showcase the scenario of determining the 6-DoF pose of known objects within a scene using scene and object neural field models. We show how this may be used to better represent objects within imperfectly modelled scenes and generate new scenes by substituting object neural field models into the scene.

Figures (10)

• Figure 1: The use-case for object registration in neural fields examined in this paper. After an object neural field (NF) is registered with its counterpart in a scene and 6-DoF pose has been attained, the library NF can be substituted into the scene to fix errors in the scene NF or new scenes can be generated using other object NFs and the known object pose in the scene.
• Figure 2: Overview of the Reg-NF registration process hausler2024reg. Blue and orange denotes surface sample points from the scene and library NFs for a matched object respectively. Green points represent the target alignment during optimisation. After surface extraction and an initial registration estimate, bi-directional optimisation iterates till convergence. Final output is a 6-DoF transformation matrix between models.
• Figure 3: Example of multiple views used for surface point extraction.
• Figure 4: Example images of object model images in ONR dataset.
• Figure 5: Scene models in ONR dataset.