AniSDF: Fused-Granularity Neural Surfaces with Anisotropic Encoding for High-Fidelity 3D Reconstruction

TL;DR

AniSDF tackles the trade-off between geometric fidelity and photorealistic rendering in neural 3D reconstruction by introducing fused-granularity neural surfaces and blended radiance fields with anisotropic encoding. The method jointly optimizes surface geometry and appearance, balancing coarse and fine hash-grid representations while disentangling diffuse from specular components through ASG-encoded radiance. Key contributions include a unified SDF-based architecture, a fused-granularity geometry representation, and a physics-informed blending of view-based and reflection-based radiance fields, all trained with a multi-term loss. Empirically, AniSDF achieves state-of-the-art geometry accuracy and novel-view synthesis on multiple datasets, including challenging reflective and luminous scenes, highlighting its potential for relighting and deformation tasks.

Abstract

Neural radiance fields have recently revolutionized novel-view synthesis and achieved high-fidelity renderings. However, these methods sacrifice the geometry for the rendering quality, limiting their further applications including relighting and deformation. How to synthesize photo-realistic rendering while reconstructing accurate geometry remains an unsolved problem. In this work, we present AniSDF, a novel approach that learns fused-granularity neural surfaces with physics-based encoding for high-fidelity 3D reconstruction. Different from previous neural surfaces, our fused-granularity geometry structure balances the overall structures and fine geometric details, producing accurate geometry reconstruction. To disambiguate geometry from reflective appearance, we introduce blended radiance fields to model diffuse and specularity following the anisotropic spherical Gaussian encoding, a physics-based rendering pipeline. With these designs, AniSDF can reconstruct objects with complex structures and produce high-quality renderings. Furthermore, our method is a unified model that does not require complex hyperparameter tuning for specific objects. Extensive experiments demonstrate that our method boosts the quality of SDF-based methods by a great scale in both geometry reconstruction and novel-view synthesis.

Figures (23)

• Figure 1: The left part demonstrates the ability of AniSDF to produce accurate geometry and high-quality rendering results. The right part presents its capability to handle various scenes including complex object, luminous object, highly reflective object, and fuzzy object.
• Figure 2: Pipeline of our method for 3D reconstruction. We utilize a fused-granularity neural surface structure where we make the most of coarse grids and fine grids for accurate surface reconstruction. We then employ a view-based radiance field and reflection-based radiance field to model diffuse part and specular part accordingly. By learning a 3D weight field, we blend the radiance fields to obtain high-fidelity renderings.
• Figure 3: Comparison on NeRF synthetic dataset with previous surface reconstruction methods. Our model yields the most accurate geometry reconstruction and highest-quality rendering at the same time. Our model can handle the semi-transparent structure and produce accurate renderings for the specular parts.
• Figure 4: Comparison on Shiny Blender dataset with previous surface reconstruction methods. Our model achieves the most accurate surface reconstruction for reflective objects. In addition, our method can reconstruct luminous objects while all the other methods fail to reconstruct surfaces.
• Figure 5: Ablation results on ASG encoding. We demonstrate the ability to synthesize specular details with the use of ASG encoding.