Light3R-SfM: Towards Feed-forward Structure-from-Motion

TL;DR

Light3R-SfM introduces a fully feed-forward SfM pipeline that replaces traditional global optimization with a latent global alignment module based on attention, enabling scalable, offline reconstruction from unordered image collections. By constructing a sparse scene graph via a retrieval-guided shortest path tree and decoding per-edge pointmaps that are globally accumulated with Procrustes alignment, the method achieves competitive accuracy while drastically reducing runtime. Extensive experiments on Tanks&Temples, CO3Dv2, and Waymo demonstrate strong generalization and clear efficiency gains over optimization-based and online memory-based baselines, though some tight-threshold accuracies still lag state-of-the-art. The work highlights the potential of data-driven SfM for large-scale, in-the-wild 3D reconstruction and points to future directions in dynamic scenes and scalability to very large image sets.

Abstract

We present Light3R-SfM, a feed-forward, end-to-end learnable framework for efficient large-scale Structure-from-Motion (SfM) from unconstrained image collections. Unlike existing SfM solutions that rely on costly matching and global optimization to achieve accurate 3D reconstructions, Light3R-SfM addresses this limitation through a novel latent global alignment module. This module replaces traditional global optimization with a learnable attention mechanism, effectively capturing multi-view constraints across images for robust and precise camera pose estimation. Light3R-SfM constructs a sparse scene graph via retrieval-score-guided shortest path tree to dramatically reduce memory usage and computational overhead compared to the naive approach. Extensive experiments demonstrate that Light3R-SfM achieves competitive accuracy while significantly reducing runtime, making it ideal for 3D reconstruction tasks in real-world applications with a runtime constraint. This work pioneers a data-driven, feed-forward SfM approach, paving the way toward scalable, accurate, and efficient 3D reconstruction in the wild.

Figures (11)

• Figure 1: Processing speed vs. accuracy for various SfM methods. Our work significantly decreases the runtime across various sizes of image collections compared to traditional pipelines while obtaining comparable accuracy. Results are measured on the Tanks&Temples dataset.
• Figure 2: Light3R-SfM Pipeline. Given an unordered set of images, we first encode them to obtain image tokens from which we average pool global features for constructing a shortest path tree. We next feed image tokens into our attention-based latent global alignment to enable global context sharing. Afterwards, for each edge in the SPT, we decode pairwise pointmaps using the implicitly aligned feature tokens. Finally, we use global accumulation to obtain globally aligned pointmaps per image.
• Figure 3: Qualitative comparison on a Waymo scene. Note how the MASt3R-SfM reconstruction does not truthfully reconstruct the 90° turn, while Spann3R predictions degrade after tens of frames.
• Figure 4: Reconstructing opposite-oriented cameras. After conditioning Light3R-SfM's decoder with the output from our global latent alignment, it is able to predict pointmaps even for images recorded in opposite directions, suggesting the latent global alignment has learned a representation of the entire scene.
• Figure 5: CDF of pose errors on 100-view Tanks&Temples scenes.