Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generalized Orthogonal Procrustes Problem under Arbitrary Adversaries

Shuyang Ling

TL;DR

This work tackles the generalized orthogonal Procrustes problem (GOPP) under arbitrary additive adversaries by analyzing a semidefinite relaxation (SDR) and an iterative generalized power method (GPM). It proves the SDR is tight and recovers the least-squares estimator exactly in the high-SNR regime, and that GPM with spectral initialization converges linearly to the SDR’s global minimizer under similar conditions, all without statistical priors on noise. It further studies the low-rank (BM) factorization landscape, showing no spurious local minima when the factor rank is sufficiently large, thus explaining the practical success of nonconvex approaches. The results apply to Gaussian noise and uniform corruption models, providing explicit SNR thresholds and high-probability convergence guarantees, with implications for robust alignment in computer vision, statistics, and related fields.

Abstract

The generalized orthogonal Procrustes problem (GOPP) plays a fundamental role in several scientific disciplines including statistics, imaging science and computer vision. Despite its tremendous practical importance, it is generally an NP-hard problem to find the least squares estimator. We study the semidefinite relaxation (SDR) and an iterative method named generalized power method (GPM) to find the least squares estimator, and investigate the performance under a signal-plus-noise model. We show that the SDR recovers the least squares estimator exactly and moreover the generalized power method with a proper initialization converges linearly to the global minimizer to the SDR, provided that the signal-to-noise ratio is large. The main technique follows from showing the nonlinear mapping involved in the GPM is essentially a local contraction mapping and then applying the well-known Banach fixed-point theorem finishes the proof. In addition, we analyze the low-rank factorization algorithm and show the corresponding optimization landscape is free of spurious local minimizers under nearly identical conditions that enables the success of SDR approach. The highlight of our work is that the theoretical guarantees are purely algebraic and do not assume any statistical priors of the additive adversaries, and thus it applies to various interesting settings.

Generalized Orthogonal Procrustes Problem under Arbitrary Adversaries

TL;DR

This work tackles the generalized orthogonal Procrustes problem (GOPP) under arbitrary additive adversaries by analyzing a semidefinite relaxation (SDR) and an iterative generalized power method (GPM). It proves the SDR is tight and recovers the least-squares estimator exactly in the high-SNR regime, and that GPM with spectral initialization converges linearly to the SDR’s global minimizer under similar conditions, all without statistical priors on noise. It further studies the low-rank (BM) factorization landscape, showing no spurious local minima when the factor rank is sufficiently large, thus explaining the practical success of nonconvex approaches. The results apply to Gaussian noise and uniform corruption models, providing explicit SNR thresholds and high-probability convergence guarantees, with implications for robust alignment in computer vision, statistics, and related fields.

Abstract

The generalized orthogonal Procrustes problem (GOPP) plays a fundamental role in several scientific disciplines including statistics, imaging science and computer vision. Despite its tremendous practical importance, it is generally an NP-hard problem to find the least squares estimator. We study the semidefinite relaxation (SDR) and an iterative method named generalized power method (GPM) to find the least squares estimator, and investigate the performance under a signal-plus-noise model. We show that the SDR recovers the least squares estimator exactly and moreover the generalized power method with a proper initialization converges linearly to the global minimizer to the SDR, provided that the signal-to-noise ratio is large. The main technique follows from showing the nonlinear mapping involved in the GPM is essentially a local contraction mapping and then applying the well-known Banach fixed-point theorem finishes the proof. In addition, we analyze the low-rank factorization algorithm and show the corresponding optimization landscape is free of spurious local minimizers under nearly identical conditions that enables the success of SDR approach. The highlight of our work is that the theoretical guarantees are purely algebraic and do not assume any statistical priors of the additive adversaries, and thus it applies to various interesting settings.

Paper Structure

This paper contains 22 sections, 15 theorems, 108 equations, 1 figure, 1 algorithm.

Table of Contents

  1. Introduction
  2. Least squares estimation
  3. Convex relaxation and its tightness
  4. Generalized power method (GPM)
  5. Optimization landscape of low-rank factorization
  6. Main results
  7. Main theorems
  8. Applications in two statistical models
  9. Example: Gaussian noise model.
  10. Example: uniform corruption model
  11. Gaussian noise model
  12. Uniform corruption model
  13. Proofs
  14. An overview of the proof
  15. ${\cal T}$ is a contraction mapping on $\mathcal{N}_{\epsilon}$
  16. ...and 7 more sections

Key Result

Theorem 2.1

Suppose the def:sdr is tight and recovers the unique global minimizer to def:od, i.e., the global minimizer to the SDR is unique and exactly rank-$d$.

Figures (1)

  • Figure 1: Phase transition for the tightness/global convergence of the GPM. Left column: black region: the tightness of the \ref{['def:gpm']} fails; white region: the tightness holds; Right column: the frequency of success v.s. $\|\boldsymbol{O}\boldsymbol{A}\|/\|\boldsymbol{\Delta}\|$

Theorems & Definitions (21)

  • Theorem 2.1: Tightness of the SDR
  • Theorem 2.2: Convergence of the GPM
  • Theorem 2.3: Optimization landscape of the Burer-Monteiro factorization
  • Corollary 2.4
  • Corollary 2.5
  • Theorem 3.1: Characterization of global optimality
  • Theorem 3.2
  • Proposition 3.3
  • Proposition 3.4: Lipschitz continuity of ${\cal T}$ on $\mathcal{N}_{\epsilon}$
  • Proposition 3.5: ${\cal T}$ maps $\mathcal{N}_{\epsilon}$ to itself
  • ...and 11 more