Refractive COLMAP: Refractive Structure-from-Motion Revisited

TL;DR

The authors address underwater 3D reconstruction by introducing a generalized refractive SfM (RSfM) framework that integrates refraction directly into COLMAP. The method uses Snell’s law to model flat-port and dome-port interfaces, replaces rays with stable virtual cameras, and employs GP3P for absolute pose with a robust BestApprox 3-point relative pose approach, followed by optimization on virtual image planes. Key contributions include automatic refractive parameter refinement during bundle adjustment, scale-aware processing, and extensive validations on synthetic re-renderings and real underwater data, demonstrating accuracy and robustness beyond the UWPinhole approximation. The implementation is released as an open-source COLMAP extension, enabling scalable, open-access underwater 3D reconstruction under varied refractive configurations with improved pose and model fidelity.

Abstract

In this paper, we present a complete refractive Structure-from-Motion (RSfM) framework for underwater 3D reconstruction using refractive camera setups (for both, flat- and dome-port underwater housings). Despite notable achievements in refractive multi-view geometry over the past decade, a robust, complete and publicly available solution for such tasks is not available at present, and often practical applications have to resort to approximating refraction effects by the intrinsic (distortion) parameters of a pinhole camera model. To fill this gap, we have integrated refraction considerations throughout the entire SfM process within the state-of-the-art, open-source SfM framework COLMAP. Numerical simulations and reconstruction results on synthetically generated but photo-realistic images with ground truth validate that enabling refraction does not compromise accuracy or robustness as compared to in-air reconstructions. Finally, we demonstrate the capability of our approach for large-scale refractive scenarios using a dataset consisting of nearly 6000 images. The implementation is released as open-source at: https://cau-git.rz.uni-kiel.de/inf-ag-koeser/colmap_underwater.

Figures (7)

• Figure 1: Results of the reconstruction on a rendered large-scale AUV-based seafloor mapping dataset containing 5740 refractive flat-port images. Top: Using the perspective camera model underwater creates a curved seafloor reconstruction. Bottom: Our proposed RSfM.
• Figure 2: A schematic illustration of the refractive camera models. The scene points $\hbox{\boldmath$X$}$ are observed by the camera at image points $\hbox{\boldmath$x$}$ through the interface. The virtual cameras $V$ are depicted by differently colored dashed triangles situated along the refraction axis $A$. Left: Flat-port. Right: Dome-port.
• Figure 3: A schematic illustration of the feature-dependent virtual epipolar geometry ${{{\bf E}}}^v$, which relates the relative pose $^b{{{\bf R}}}_{a}, ^b\hbox{\boldmath$t$}_a$ of two frames.
• Figure 4: Numerical evaluation results of the absolute pose estimation across various refractive camera configurations.
• Figure 5: Numerical evaluation results of the relative pose estimation under different noise and outlier conditions.