Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tensor-Based Synchronization and the Low-Rankness of the Block Trifocal Tensor

Daniel Miao, Gilad Lerman, Joe Kileel

TL;DR

This work establishes an explicit Tucker factorization of the block trifocal tensor, revealing a low multilinear rank of $(6,4,4) independent of the number of cameras under appropriate scaling conditions, and proves that this rank constraint provides sufficient information for camera recovery in the noiseless case.

Abstract

The block tensor of trifocal tensors provides crucial geometric information on the three-view geometry of a scene. The underlying synchronization problem seeks to recover camera poses (locations and orientations up to a global transformation) from the block trifocal tensor. We establish an explicit Tucker factorization of this tensor, revealing a low multilinear rank of $(6,4,4)$ independent of the number of cameras under appropriate scaling conditions. We prove that this rank constraint provides sufficient information for camera recovery in the noiseless case. The constraint motivates a synchronization algorithm based on the higher-order singular value decomposition of the block trifocal tensor. Experimental comparisons with state-of-the-art global synchronization methods on real datasets demonstrate the potential of this algorithm for significantly improving location estimation accuracy. Overall this work suggests that higher-order interactions in synchronization problems can be exploited to improve performance, beyond the usual pairwise-based approaches.

Tensor-Based Synchronization and the Low-Rankness of the Block Trifocal Tensor

TL;DR

This work establishes an explicit Tucker factorization of the block trifocal tensor, revealing a low multilinear rank of $(6,4,4) independent of the number of cameras under appropriate scaling conditions, and proves that this rank constraint provides sufficient information for camera recovery in the noiseless case.

Abstract

The block tensor of trifocal tensors provides crucial geometric information on the three-view geometry of a scene. The underlying synchronization problem seeks to recover camera poses (locations and orientations up to a global transformation) from the block trifocal tensor. We establish an explicit Tucker factorization of this tensor, revealing a low multilinear rank of independent of the number of cameras under appropriate scaling conditions. We prove that this rank constraint provides sufficient information for camera recovery in the noiseless case. The constraint motivates a synchronization algorithm based on the higher-order singular value decomposition of the block trifocal tensor. Experimental comparisons with state-of-the-art global synchronization methods on real datasets demonstrate the potential of this algorithm for significantly improving location estimation accuracy. Overall this work suggests that higher-order interactions in synchronization problems can be exploited to improve performance, beyond the usual pairwise-based approaches.
Paper Structure (24 sections, 4 theorems, 40 equations, 2 figures, 4 tables, 2 algorithms)

This paper contains 24 sections, 4 theorems, 40 equations, 2 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Low-rankness of the block trifocal tensor
  3. Background
  4. Cameras and 3D geometry
  5. Trifocal tensors
  6. Tucker decomposition and the multilinear rank of tensors
  7. Low Tucker rank of the block trifocal tensor and one shot camera retrieval
  8. Synchronization of the block trifocal tensor
  9. Higher-order SVD with a hard threshold (HOSVD-HT)
  10. Scale recovery
  11. Synchronization algorithm
  12. Numerical experiments
  13. EPFL dataset
  14. Photo Tourism
  15. Conclusion
  16. ...and 9 more sections

Key Result

Proposition 1

We have the following observations for the block trifocal tensor $T^n$. For all distinct $i, j \in [n]$, we have the following properties:

Figures (2)

  • Figure 1: EPFL translation error comparison between our method, NRFM initialized by LUD, LUD, and NRFM initialized randomly. BATA(MPLS) stands for BATA initialized by MPLS. HZ8 stands for HerzP8, FP11 for FountainP11, HZ25 for Herz P25, EN10 for EntryP10, CS19 for CastleP19, CS30 for CastleP30.
  • Figure 2: Photo Tourism translation error comparison between our method, NRFM initialized by LUD, LUD, NRFM initialized randomly, and BATA initialized with MPLS. Note that we have not been able to acquire results for Piccadilly for BATA + MPLS.

Theorems & Definitions (8)

  • Proposition 1
  • Theorem 1: Tucker factorization and low multilinear rank of block trifocal tensor
  • proof
  • Proposition 2: One shot camera pose retrieval
  • Theorem 2
  • proof : Sketch
  • proof : Proof for (i)
  • proof