Category-level Neural Field for Reconstruction of Partially Observed Objects in Indoor Environment

TL;DR

The paper introduces category-level neural fields to improve reconstruction of partially observed indoor objects by sharing cross-object 3D information within a category. It employs uncertainty-guided selection of a representative object and robust registration to a Normalized Object Centric Space (NOCS), followed by subcategorization to keep shared information meaningful. A category-level NeRF framework with per-object latent shape and texture codes is trained jointly with a background model, using RGB-D supervision and opacity regularization. Experiments on Replica and ScanNet show improved reconstruction of unobserved regions and compelling ablations, indicating practical gains in scene understanding and editing for cluttered indoor environments.

Abstract

Neural implicit representation has attracted attention in 3D reconstruction through various success cases. For further applications such as scene understanding or editing, several works have shown progress towards object compositional reconstruction. Despite their superior performance in observed regions, their performance is still limited in reconstructing objects that are partially observed. To better treat this problem, we introduce category-level neural fields that learn meaningful common 3D information among objects belonging to the same category present in the scene. Our key idea is to subcategorize objects based on their observed shape for better training of the category-level model. Then we take advantage of the neural field to conduct the challenging task of registering partially observed objects by selecting and aligning against representative objects selected by ray-based uncertainty. Experiments on both simulation and real-world datasets demonstrate that our method improves the reconstruction of unobserved parts for several categories.

Figures (8)

• Figure 1: Our method reconstructs objects using category-level models. Objects belonging to the same category share common shape properties, which help to reconstruct unobserved parts plausibly. On the other hand, unobserved parts of objects that are reconstructed by the object-level model (vMAP) tend to over-smooth or fail to recover complete geometry.
• Figure 2: Overview of the proposed framework
• Figure 3: A schematic diagram of subcategorization module
• Figure 4: Reconstruction of unobserved region
• Figure 5: Visualization of proposed reliability metric. Color value in each spherical surface point indicates $u(r)$ and $g(u)$ value of the point's corresponding ray. Both plots are oriented as same as the object in the scene.