Micro-macro Wavelet-based Gaussian Splatting for 3D Reconstruction from Unconstrained Images

TL;DR

The paper tackles 3D reconstruction from unconstrained image collections by introducing Micro-macro Wavelet-based Gaussian Splatting (MW-GS), which explicitly decouples appearance into global, refined, and intrinsic components. It introduces Micro-macro Projection for diverse, multi-scale refinement and Wavelet-based Sampling to leverage frequency-domain information, all fused through a Hierarchical Residual Fusion Network. The approach yields state-of-the-art rendering quality on challenging datasets and shows robustness to transient objects via a visibility-guided optimization strategy. These innovations collectively improve both local detail and global appearance fidelity, enabling more accurate and efficient 3D reconstructions from diverse photo collections.

Abstract

3D reconstruction from unconstrained image collections presents substantial challenges due to varying appearances and transient occlusions. In this paper, we introduce Micro-macro Wavelet-based Gaussian Splatting (MW-GS), a novel approach designed to enhance 3D reconstruction by disentangling scene representations into global, refined, and intrinsic components. The proposed method features two key innovations: Micro-macro Projection, which allows Gaussian points to capture details from feature maps across multiple scales with enhanced diversity; and Wavelet-based Sampling, which leverages frequency domain information to refine feature representations and significantly improve the modeling of scene appearances. Additionally, we incorporate a Hierarchical Residual Fusion Network to seamlessly integrate these features. Extensive experiments demonstrate that MW-GS delivers state-of-the-art rendering performance, surpassing existing methods.

Figures (6)

• Figure 1: Overview of Micro-macro Wavelet-based Gaussian Splatting (MW-GS). Beginning with a reference image, CNN-based sub-networks extract global appearance features, a visibility map, and a set of feature maps which then undergo wavelet transform to capture multi-resolution information for refined appearance modeling. 3D Gaussian points are projected onto these feature maps using Micro-macro Projection, which employs adaptive sampling within narrow and broad conical frustums to capture richer information. The global, refined, and intrinsic features are subsequently fused through a Hierarchical Residual Fusion Network (HRFN) to generate the final color, which is rendered using a tile rasterizer for high-quality 3D reconstruction.
• Figure 2: Sampling comparison between GS-W and the proposed MW-GS. Our approach integrates both narrow and broad conical frustums with wavelet-based sampling, allowing for a more comprehensive capture of features and resulting in enhanced accuracy.
• Figure 3: Qualitative comparison on three datasets. Red and blue crops emphasize that MW-GS can recover finer details.
• Figure 4: Qualitative results of appearance transfer.
• Figure 5: Ablation studies by visualization.