Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

QuadSync: Quadrifocal Tensor Synchronization via Tucker Decomposition

Daniel Miao, Gilad Lerman, Joe Kileel

TL;DR

This work provides a new framework to recover cameras from the corresponding collection of quadrifocal tensors, and develops the first synchronization algorithm for quadrifocal tensors, using Tucker decomposition, alternating direction method of multipliers, and iteratively reweighted least squares.

Abstract

In structure from motion, quadrifocal tensors capture more information than their pairwise counterparts (essential matrices), yet they have often been thought of as impractical and only of theoretical interest. In this work, we challenge such beliefs by providing a new framework to recover $n$ cameras from the corresponding collection of quadrifocal tensors. We form the block quadrifocal tensor and show that it admits a Tucker decomposition whose factor matrices are the stacked camera matrices, and which thus has a multilinear rank of (4,~4,~4,~4) independent of $n$. We develop the first synchronization algorithm for quadrifocal tensors, using Tucker decomposition, alternating direction method of multipliers, and iteratively reweighted least squares. We further establish relationships between the block quadrifocal, trifocal, and bifocal tensors, and introduce an algorithm that jointly synchronizes these three entities. Numerical experiments demonstrate the effectiveness of our methods on modern datasets, indicating the potential and importance of using higher-order information in synchronization.

QuadSync: Quadrifocal Tensor Synchronization via Tucker Decomposition

TL;DR

This work provides a new framework to recover cameras from the corresponding collection of quadrifocal tensors, and develops the first synchronization algorithm for quadrifocal tensors, using Tucker decomposition, alternating direction method of multipliers, and iteratively reweighted least squares.

Abstract

In structure from motion, quadrifocal tensors capture more information than their pairwise counterparts (essential matrices), yet they have often been thought of as impractical and only of theoretical interest. In this work, we challenge such beliefs by providing a new framework to recover cameras from the corresponding collection of quadrifocal tensors. We form the block quadrifocal tensor and show that it admits a Tucker decomposition whose factor matrices are the stacked camera matrices, and which thus has a multilinear rank of (4,~4,~4,~4) independent of . We develop the first synchronization algorithm for quadrifocal tensors, using Tucker decomposition, alternating direction method of multipliers, and iteratively reweighted least squares. We further establish relationships between the block quadrifocal, trifocal, and bifocal tensors, and introduce an algorithm that jointly synchronizes these three entities. Numerical experiments demonstrate the effectiveness of our methods on modern datasets, indicating the potential and importance of using higher-order information in synchronization.
Paper Structure (41 sections, 10 theorems, 33 equations, 6 figures, 7 tables, 2 algorithms)

This paper contains 41 sections, 10 theorems, 33 equations, 6 figures, 7 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Relevant Previous Work
  3. Our Contributions
  4. Background
  5. Quadrifocal Tensors
  6. Tucker Decomposition
  7. The Block Quadrifocal Tensor
  8. Properties of the Block Quadrifocal Tensor
  9. Method
  10. QuadSync: An ADMM-IRLS Scheme
  11. IRLS Outer Loop
  12. ADMM Inner Loop
  13. A Joint Optimization Framework
  14. Joint Opt. with ADMM-IRLS
  15. Experiments
  16. ...and 26 more sections

Key Result

Theorem 2.1

A given matrix $\mathcal{F}^n\in \mathbb{R}^{3n \times 3n}$ is consistent as an $n$-view fundamental matrix with some set of $n$ cameras whose centers are not all collinear if and only if:

Figures (6)

  • Figure 1: Mean location error for ETH3D datasets
  • Figure 2: Mean location error for EPFL datasets
  • Figure 3: QuadSync retrieved camera poses on near-collinear views from plant_scene_1 dataset from ETH3D SLAM.
  • Figure 4: Randomized updates in QuadSync tested on ETH3D 'relief' dataset.
  • Figure 5: Ground truth location of 10 cameras. $\times$
  • ...and 1 more figures

Theorems & Definitions (23)

  • Theorem 2.1: kasten2019gpsfm
  • Theorem 2.2: miao2024tensor
  • Theorem 3.1
  • Remark 1
  • Corollary 3.1.1: Projective Reconstruction
  • Remark 2
  • Definition 1
  • Theorem 3.2
  • Proposition 3.3
  • Theorem 3.4
  • ...and 13 more