Indoor 3D Reconstruction with an Unknown Camera-Projector Pair

TL;DR

This work tackles indoor 3D reconstruction with an unknown camera-projector pair by introducing a cuboid corner (C2) as a robust, common cue to derive sufficient two-view constraints for CPP self-calibration. The authors derive single- and two-view geometry around C2, showing that the CPP calibration can be reduced to a univariate problem in the camera focal length $f_c$, via a deterministic mapping to the projector intrinsics $K_p=g(K_c)$. A cycle-consistent optimization objective drives accurate intrinsic estimation, enabling dense 3D reconstructions that rival multi-view and learning-based methods while requiring only two views. The approach extends naturally to two-view SfM and highlights the importance of the cycle term in calibration, though it remains limited by degenerate C2 configurations and the need for manual segmentation of faces. Overall, the method offers a practical pathway for reliable indoor SL with unknown CPP and demonstrates potential for sparse-view camera self-calibration.

Abstract

Structured light-based method with a camera-projector pair (CPP) plays a vital role in indoor 3D reconstruction, especially for scenes with weak textures. Previous methods usually assume known intrinsics, which are pre-calibrated from known objects, or self-calibrated from multi-view observations. It is still challenging to reliably recover CPP intrinsics from only two views without any known objects. In this paper, we provide a simple yet reliable solution. We demonstrate that, for the first time, sufficient constraints on CPP intrinsics can be derived from an unknown cuboid corner (C2), e.g. a room's corner, which is a common structure in indoor scenes. In addition, with only known camera principal point, the complex multi-variable estimation of all CPP intrinsics can be simplified to a simple univariable optimization problem, leading to reliable calibration and thus direct 3D reconstruction with unknown CPP. Extensive results have demonstrated the superiority of the proposed method over both traditional and learning-based counterparts. Furthermore, the proposed method also demonstrates impressive potential to solve similar tasks without active lighting, such as sparse-view structure from motion.

Figures (9)

• Figure 1: Reconstructing indoor scenes with an unknown CPP is challenging, since the self-calibration only from two views is ill-posed and that few scene cues are available due to texture-lessness. We propose to leverage an unknown cuboid corner (C2) to extract sufficient constraints for calibration. As shown in \ref{['fig1']}(a), a C2 can be a room corner with partially observed walls and floor as its faces, see \ref{['fig1']}(b), or, more formally, a Tri-rectangular tetrahedron in \ref{['fig1']}(c). Compared with two-view COLMAP r2 and PlaneFormer r12, which struggle to reconstruct the scene with significant distortion, see \ref{['fig1']}(d) and \ref{['fig1']}(e), our method achieves impressive result, see \ref{['fig1']}(f).
• Figure 2: Solutions of a C2 from its image: (a) configurations with only one real solution and (b) that with two real solutions, respectively, which correspond to two C2’s with different convexity.
• Figure 3: Transferring from the camera to projector. (a) camera to C2, (b) C2 to $\boldsymbol{X}_{\mathrm{S}}(i)$, and (c) $\boldsymbol{X}_{\mathrm{S}}(i)$ to projector, where $S$ can be $A$, $B$ or $C$.
• Figure 4: Pseudocode of our algorithm. Note that an exact search is used for illustration.
• Figure 5: The camera view (the first row) and C2's with partially observed faces and their vertices of indoor scenes No. 1 - No. 4. Please note that the images are cropped to the common field of view (FOV) of the camera and projector. Readers are suggested to Supplementary material for a full version.