FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Views

TL;DR

FLARE tackles uncalibrated sparse-view 3D reconstruction by introducing a cascaded, feed-forward framework that uses camera pose estimation as a geometric prior to drive geometry and appearance learning. The method jointly learns a neural pose predictor, camera-centric geometry with a global projection, and 3D Gaussian-based appearance with differentiable rendering, achieving state-of-the-art results while maintaining sub-second inference. Key contributions include the pose-guided two-stage geometry learning, the use of camera-centric point maps and a learnable geometry projector, and the 3D Gaussian splatting pipeline for photorealistic novel-view synthesis. The approach generalizes well across diverse real-world scenes and varying numbers of input views, enabling practical sparse-view 3D reconstruction and rendering without extrinsic camera information.

Abstract

We present FLARE, a feed-forward model designed to infer high-quality camera poses and 3D geometry from uncalibrated sparse-view images (i.e., as few as 2-8 inputs), which is a challenging yet practical setting in real-world applications. Our solution features a cascaded learning paradigm with camera pose serving as the critical bridge, recognizing its essential role in mapping 3D structures onto 2D image planes. Concretely, FLARE starts with camera pose estimation, whose results condition the subsequent learning of geometric structure and appearance, optimized through the objectives of geometry reconstruction and novel-view synthesis. Utilizing large-scale public datasets for training, our method delivers state-of-the-art performance in the tasks of pose estimation, geometry reconstruction, and novel view synthesis, while maintaining the inference efficiency (i.e., less than 0.5 seconds). The project page and code can be found at: https://zhanghe3z.github.io/FLARE/

Figures (6)

• Figure 1: We present flare, a feed-forward approach that simultaneously recovers high-quality poses, geometry and appearance from uncalibrated sparse views within 0.5s. Our model excels in the scenarios with a camera circling around the subject, and also shows robust generalization to real-world casual captures, such as an indoor bedroom. In the central area below, we casually captured six random bedroom images with minimal overlap. Our method demonstrates high-quality geometry reconstruction even in this challenging case.
• Figure 2: Illustration of our pipeline. Given uncalibrated sparse views, our model can infer high-quality camera poses, geometry and appearance in a single feed-forward pass. We use camera poses as proxies to guide subsequent geometry and appearance learning. Given initial pose estimates, we first compute camera-centric geometry, then project it into a global scene representation. Finally, we form 3D Gaussians on top of the scene geometry to enable photo-realistic novel-view synthesis.
• Figure 3: Qualitative Comparison results for sparse-view 3D reconstruction. We visualize the 3D pointmaps of MASt3R leroy2024grounding, DUSt3R wang2024dust3r, Spann3R wang2024spann3r and our flare on ETH3D and TUM dataset.
• Figure 4: Qualitative Comparison results for novel-view synthesis. We visualize the rendering results from the DL3DV dataset, which shows that our method obtains high-quality rendering from sparse-view, uncalibrated input images.
• Figure 5: Relationship between MVSplat Performance and Input Views.