DynaWeightPnP: Toward global real-time 3D-2D solver in PnP without correspondences

TL;DR

This work proposes DynaWeightPnP, introducing a dynamic weighting sub-problem and an alternative searching algorithm designed to enhance pose estimation and alignment accuracy, and identifies a unique and interesting observability issue in correspondence-free PnP: the numerical ambiguity between rotation and translation.

Abstract

This paper addresses a special Perspective-n-Point (PnP) problem: estimating the optimal pose to align 3D and 2D shapes in real-time without correspondences, termed as correspondence-free PnP. While several studies have focused on 3D and 2D shape registration, achieving both real-time and accurate performance remains challenging. This study specifically targets the 3D-2D geometric shape registration tasks, applying the recently developed Reproducing Kernel Hilbert Space (RKHS) to address the "big-to-small" issue. An iterative reweighted least squares method is employed to solve the RKHS-based formulation efficiently. Moreover, our work identifies a unique and interesting observability issue in correspondence-free PnP: the numerical ambiguity between rotation and translation. To address this, we proposed DynaWeightPnP, introducing a dynamic weighting sub-problem and an alternative searching algorithm designed to enhance pose estimation and alignment accuracy. Experiments were conducted on a typical case, that is, a 3D-2D vascular centerline registration task within Endovascular Image-Guided Interventions (EIGIs). Results demonstrated that the proposed algorithm achieves registration processing rates of 60 Hz (without post-refinement) and 31 Hz (with post-refinement) on modern single-core CPUs, with competitive accuracy comparable to existing methods. These results underscore the suitability of DynaWeightPnP for future robot navigation tasks like EIGIs.

Figures (11)

• Figure 1: The upper two figures show the typical "big-to-small" alignment. The green shape is the projection of the pre-operative 3D vessel, which is in large size. The upper left figure is the proposed correspondence-free PnP on RKHS space, which is robust to "big-to-small" alignment. The upper right figure is our implementation of rivest2012nonrigid, which uses the Huber loss. The bottom figures are three loss functions. Only the "Exp" (stands for exponential) loss manages to lower the impacts of "outlier" points.
• Figure 2: Illustrated is an example of scale ambiguity in the correspondence-free PnP process. The red and green points represent the 2D points and projected 3D points, respectively. As the 3D points move farther away from the sensor plane, their projected 2D points continue to shrink.
• Figure 3: The figure shows the ambiguity of rotation and translation. Rotating and translating the cube object yield similar observations.
• Figure 4: The figure illustrates the concept of alternative direction searching. After the state falls into a local minimum (steps 1 and 3), the auxiliary problem helps the state jump out (steps 2 and 4). This alternative direction searching method thus aids the algorithm in converging to a better local minimum.
• Figure 5: Illustrated is the basic iterative searching structure shown in Algorithm \ref{['Algorithm_ADMM']}.