Advances in Global Solvers for 3D Vision

TL;DR

This survey analyzes the rise of global solvers for 3D vision, focusing on certifiable optimization to address nonconvex geometric estimation. It introduces a unified taxonomy of Branch-and-Bound, Convex Relaxation, and Graduated Non-Convexity, detailing theoretical foundations, algorithmic designs, and practical enhancements across robustness and scalability. Ten fundamental vision tasks are evaluated to illuminate optimality guarantees, outlier robustness, and scalability trade-offs, and the authors outline future directions including scaling guarantees, data-driven priors, and standardized benchmarks. The work also emphasizes societal implications and provides open resources, tutorials, and a continuously-updated literature repository to bridge theory and real-world, trustworthy perception systems.

Abstract

Global solvers have emerged as a powerful paradigm for 3D vision, offering certifiable solutions to nonconvex geometric optimization problems traditionally addressed by local or heuristic methods. This survey presents the first systematic review of global solvers in geometric vision, unifying the field through a comprehensive taxonomy of three core paradigms: Branch-and-Bound (BnB), Convex Relaxation (CR), and Graduated Non-Convexity (GNC). We present their theoretical foundations, algorithmic designs, and practical enhancements for robustness and scalability, examining how each addresses the fundamental nonconvexity of geometric estimation problems. Our analysis spans ten core vision tasks, from Wahba problem to bundle adjustment, revealing the optimality-robustness-scalability trade-offs that govern solver selection. We identify critical future directions: scaling algorithms while maintaining guarantees, integrating data-driven priors with certifiable optimization, establishing standardized benchmarks, and addressing societal implications for safety-critical deployment. By consolidating theoretical foundations, practical advances, and broader impacts, this survey provides a unified perspective and roadmap toward certifiable, trustworthy perception for real-world applications. A continuously-updated literature summary and companion code tutorials are available at https://github.com/ericzzj1989/Awesome-Global-Solvers-for-3D-Vision.

Figures (7)

• Figure 1: Taxonomy of global solvers for 3D vision problems. The field is organized around three main components: (1) Global Solvers (\ref{['sec:bnb', 'sec:relaxation', 'sec:gnc']}), encompassing Branch-and-Bound, Convex Relaxation (Shor's relaxation, Moment-SOS relaxation, and other relaxation techniques), and Graduated Non-Convexity; (2) Comparative Analysis (\ref{['sec:compare']}), examining optimality guarantees, computational complexity and scalability, robustness to outliers, and method selection and deployment; and (3) Tasks and Applications (\ref{['sec:app']}), covering ten fundamental problems in 3D computer vision.
• Figure 2: Branch-and-Bound (BnB) algorithm on a one-dimensional non-convex minimization problem. (a) The original problem: minimize $f(x)$ over domain $\mathcal{X}_0$. The function has a global minimum and a local minimum. (b) Branching partitions the domain into subregions $\mathcal{X}_1$ and $\mathcal{X}_2$, while bounding computes lower bounds $L_1$ and $L_2$ for the respective regions and an upper bound $U^*$ from a feasible point. (c) Pruning eliminates region $\mathcal{X}_1$ since $L_1 \!>\! U^*$, while region $\mathcal{X}_2$ remains active since $L_2 \!<\! U^*$. (d) After further iterations, the algorithm converges to the global optimum $x^*$ with certified optimal value $f^*$.
• Figure 3: Shor's relaxation for solving nonconvex QCQPs. The original nonconvex problem \ref{['eq:qcqp']} is transformed into a convex SDP via lifting to matrix variable $\mathbf{X} = \mathbf{x}\mathbf{x}^\top$ and semidefinite relaxation. The primal SDP \ref{['eq:sdp']} and its dual \ref{['eq:sdp_dual']} are related through SDP duality. In practice, the global optimum is computed using convex SDP solvers based on IPMs aps2019mosektutuncu2003solvingsturm1999using. The theoretical properties including tightness of the relaxation and certification of global optimality are discussed in \ref{['subsec:shor']}.
• Figure 4: Moment-SOS (Lasserre) hierarchy for polynomial optimization. Left: Lower bounds $\gamma_d$ from successive SDP relaxations converge monotonically to the global optimum$f^*$, with tightness achieved at finite order $d^*$. Right: The moment matrix dimension $\tbinom{n+d}{d}$ grows combinatorially with relaxation order $d$, illustrating the computational cost of tightening the bounds. For many 3D vision problems, low-order relaxations ($d \!=\! 2$--$4$) suffice for tightness.
• Figure 5: Graduated Non-Convexity (GNC) for robust optimization. Left: The surrogate loss $\rho(r,\mu)$ transitions from convex (large $\mu$) to nonconvex (small $\mu$) via homotopy, enabling progressive outlier down-weighting. Right: By solving a sequence of subproblems with decreasing $\mu$ (top to bottom), GNC tracks a solution path that starts from the global minimum of the convex surrogate and converges to a near-global solution of the target nonconvex objective through warm-starting.