Minimal Perspective Autocalibration

TL;DR

This work introduces a depth-based formulation for minimal autocalibration in two- and three-view settings with constant intrinsics, solving for the calibration matrix $K$ and unknown depths $\lambda_{ip}$ using depth-consistency constraints rather than Kruppa's equations. A complete taxonomy of 80 minimal relaxation problems is developed by enumerating which depth constraints to drop, represented via 4-colorings of the complete point-pair graph and organized into isomorphism classes for offline solver design. The authors build practical solvers using homotopy continuation, enabling offline solution enumeration and online parameter tracking, and demonstrate superior accuracy over Kruppa and modulus-based methods on synthetic and real data, including effective COLMAP integration. The approach avoids common degeneracies of Kruppa-based autocalibration, provides robust initialization options (e.g., partial knowledge of $K$ such as zero skew or square pixels), and delivers a scalable offline/online pipeline suitable for real-world calibration and 3D reconstruction tasks.

Abstract

We introduce a new family of minimal problems for reconstruction from multiple views. Our primary focus is a novel approach to autocalibration, a long-standing problem in computer vision. Traditional approaches to this problem, such as those based on Kruppa's equations or the modulus constraint, rely explicitly on the knowledge of multiple fundamental matrices or a projective reconstruction. In contrast, we consider a novel formulation involving constraints on image points, the unknown depths of 3D points, and a partially specified calibration matrix $K$. For $2$ and $3$ views, we present a comprehensive taxonomy of minimal autocalibration problems obtained by relaxing some of these constraints. These problems are organized into classes according to the number of views and any assumed prior knowledge of $K$. Within each class, we determine problems with the fewest -- or a relatively small number of -- solutions. From this zoo of problems, we devise three practical solvers. Experiments with synthetic and real data and interfacing our solvers with COLMAP demonstrate that we achieve superior accuracy compared to state-of-the-art calibration methods. The code is available at https://github.com/andreadalcin/MinimalPerspectiveAutocalibration

Figures (8)

• Figure 1: Illustrating the setup of equations \ref{['eq:zx=K[R|-RC][X;;]']} and \ref{['eq:depth-equation-2']}.
• Figure 2: Minimal relaxations $\mathbb{V} (\mathbf{g}_1)$, $\mathbb{V} (\mathbf{g}_2)$, w/ $X = \mathbb{V} (\mathbf{g}_1, \mathbf{g}_2)$, $\mathbf{g}_1 (p_1, p_2, x) = x^2 - p_1$, and $\mathbf{g}_2 (p_1,p_2, x) = p_2 x^2 -1.$
• Figure 3: Non-isomorphic 4-colorings when $L=2$ in 3 views. 3D point pairs $(p,q)$ are colored according to the removal of depth equations $d_{i,j,pq}$ in the relaxed subsystem. $W$ (white) indicates removal from both image pairs, $\textcolor{blue}{B}$ no removal. $\textcolor{red}{R}$ and $\textcolor{green}{G}$ indicate removal in image pairs (1,3) and (1,2), respectively.
• Figure 4: Autocalibration Evaluation on Synthetic Images. Solver accuracy is assessed under varying levels of zero-mean Gaussian noise (denoted by $\sigma$ on the x-axis) applied to pixel coordinates. Mean reprojection error and relative errors in focal lengths $\Delta fg$, principal point $\Delta uv$, and skew $\Delta s$ are reported. For error measures, boxes represent the interquartile range of error distribution. The right-most plot illustrates the failure rate as a percentage, with $\mathtt{ffuv0}$, $\mathtt{fguv0}$, and $\mathtt{fguvs}$ excluded due to no failures.
• Figure 5: Visualization of a representative of the equivalence class of minimal relaxations for the $\mathtt{11000}$ problem obtained by dropping a constraint $d_{1,2,pq}$ for view pair $(1,2)$.