Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PRAGO: Differentiable Multi-View Pose Optimization From Objectness Detections

Matteo Taiana, Matteo Toso, Stuart James, Alessio Del Bue

TL;DR

This work shows how the objectness pose-refinement module in PRAGO is able to refine the inherent ambiguities in pairwise relative pose estimation without removing edges and avoiding making early decisions on the viability of graph edges.

Abstract

Robustly estimating camera poses from a set of images is a fundamental task which remains challenging for differentiable methods, especially in the case of small and sparse camera pose graphs. To overcome this challenge, we propose Pose-refined Rotation Averaging Graph Optimization (PRAGO). From a set of objectness detections on unordered images, our method reconstructs the rotational pose, and in turn, the absolute pose, in a differentiable manner benefiting from the optimization of a sequence of geometrical tasks. We show how our objectness pose-refinement module in PRAGO is able to refine the inherent ambiguities in pairwise relative pose estimation without removing edges and avoiding making early decisions on the viability of graph edges. PRAGO then refines the absolute rotations through iterative graph construction, reweighting the graph edges to compute the final rotational pose, which can be converted into absolute poses using translation averaging. We show that PRAGO is able to outperform non-differentiable solvers on small and sparse scenes extracted from 7-Scenes achieving a relative improvement of 21% for rotations while achieving similar translation estimates.

PRAGO: Differentiable Multi-View Pose Optimization From Objectness Detections

TL;DR

This work shows how the objectness pose-refinement module in PRAGO is able to refine the inherent ambiguities in pairwise relative pose estimation without removing edges and avoiding making early decisions on the viability of graph edges.

Abstract

Robustly estimating camera poses from a set of images is a fundamental task which remains challenging for differentiable methods, especially in the case of small and sparse camera pose graphs. To overcome this challenge, we propose Pose-refined Rotation Averaging Graph Optimization (PRAGO). From a set of objectness detections on unordered images, our method reconstructs the rotational pose, and in turn, the absolute pose, in a differentiable manner benefiting from the optimization of a sequence of geometrical tasks. We show how our objectness pose-refinement module in PRAGO is able to refine the inherent ambiguities in pairwise relative pose estimation without removing edges and avoiding making early decisions on the viability of graph edges. PRAGO then refines the absolute rotations through iterative graph construction, reweighting the graph edges to compute the final rotational pose, which can be converted into absolute poses using translation averaging. We show that PRAGO is able to outperform non-differentiable solvers on small and sparse scenes extracted from 7-Scenes achieving a relative improvement of 21% for rotations while achieving similar translation estimates.
Paper Structure (17 sections, 14 equations, 4 figures, 4 tables)

This paper contains 17 sections, 14 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Relative Pose Estimation
  4. Motion Averaging
  5. Rotation Averaging
  6. Translation Averaging
  7. PRAGO for Objectness Refined Rotation Averaging
  8. Preliminary Problem Formulation
  9. Relative Pose Refinement
  10. Rotation Averaging
  11. Translation Averaging
  12. Evaluation
  13. PRAGO Evaluation
  14. Ablations
  15. Analysis of the effect of pose refinement
  16. ...and 2 more sections

Figures (4)

  • Figure 1: From a set of unordered images a), we compute objectness detections and construct an initial relative pose graph from b) the 5-Point pose estimates. PRAGO refines the relative pose using the objectness detections resolving, among others, the chirality issues before inferring the rotational poses using RAGO blocks which iteratively infer the rotational pose by reweighting the graph edges. The absolute rotations and refined translations are then combined in e) translation averaging using BATA to construct the f) absolute camera poses.
  • Figure 2: Histograms with natural (left) and logarithmic frequency axis (right) of absolute orientation errors produced by PRAGO (green) and by applying the pose refinement and the rotation averaging modules independently (pink). Histograms are overlaid with partial transparency.
  • Figure 3: Effect of objectness-based pose refinement on the input relative poses. Left and center, natural and logarithmic histograms of edge-wise orientation error, respectively, for the test set with raw initial poses (violet) and refined poses (green). Right, shows a matrix representing the refinement effect based on relative orientation error. The color represents $log_{10}$ of the number of graphs that belong to one cell of the matrix.
  • Figure 4: Natural (top) and logarithmic (bottom) histograms of the absolute orientation error achieved by different rotation averaging methods (EIG-SE3, NeuRoRA and RAGO) applied either on raw relative poses (blue) or on refined relative poses (yellow).