Multiview Point Cloud Registration via Optimization in an Autoencoder Latent Space

TL;DR

POLAR addresses the challenge of registering a large number of degraded 3D views of the same object by performing optimization in the latent space of a pretrained autoencoder. It learns a global latent template and aligns views by a degradation-aware latent loss that accounts for anisotropic noise, partial visibility, and outliers, coupled with a robust multistart optimization (including FLAMES) to escape local minima. The method achieves state-of-the-art performance on synthetic and real-world data, demonstrating strong robustness to severe degradations and scalability to many views, with linear-time scaling. This latent-space, globally coherent registration is particularly valuable for object-level reconstruction in microscopy and related 3D sensing tasks where degraded observations are common and large transformations are encountered.

Abstract

Point cloud rigid registration is a fundamental problem in 3D computer vision. In the multiview case, we aim to find a set of 6D poses to align a set of objects. Methods based on pairwise registration rely on a subsequent synchronization algorithm, which makes them poorly scalable with the number of views. Generative approaches overcome this limitation, but are based on Gaussian Mixture Models and use an Expectation-Maximization algorithm. Hence, they are not well suited to handle large transformations. Moreover, most existing methods cannot handle high levels of degradations. In this paper, we introduce POLAR (POint cloud LAtent Registration), a multiview registration method able to efficiently deal with a large number of views, while being robust to a high level of degradations and large initial angles. To achieve this, we transpose the registration problem into the latent space of a pretrained autoencoder, design a loss taking degradations into account, and develop an efficient multistart optimization strategy. Our proposed method significantly outperforms state-of-the-art approaches on synthetic and real data. POLAR is available at github.com/pypolar/polar or as a standalone package which can be installed with pip install polaregistration.

Figures (19)

• Figure 1: Overview of the proposed method. 1. Once and for all, before any registration, an autoencoder is trained to reconstruct point clouds (\ref{['eq:bidirectional_chamfer']}). 2. To register a set of views, an optimization problem is iteratively solved within the learnt latent space (\ref{['eq:degraded_latent_optim_i']}).
• Figure 2: Graphical illustration of the loss computation in POLAR. $\mathcal{X} = \left\{ \bm{X}_1 \in \mathbb{R}^{k_1}, \ldots, \bm{X}_N \in \mathbb{R}^{k_N}\right\}$ denotes the views to register. $\bm{Z}^{\text{data}} = \left[e(\bm{X}_1), \ldots, e(\bm{X}_N) \right] \in \mathbb{R}^{N \times l}$ is the matrix of their encodings. $\bm{P} = \mathrm{d}(\bm{z}) \in \mathbb{R}^{k \times 3}$ is the estimated template. $\bm{V} = \left[ \rho , \ldots, \rho \right] \in \mathbb{R}^{N \times k \times 3}$ denotes the views obtained by applying the estimated motions $\bm{\rho}$ to the estimated template $\bm{P}$. Finally, $\bm{Z} = \left[\mathrm{e}( \rho ), \ldots, \mathrm{e}( \rho ) \right] \in \mathbb{R}^{N \times l}$ is the matrix of the latent vectors obtained by encoding these estimated views.
• Figure 3: Schematic representation of nearest neighbor distances in three scenarios. The estimated template in the $i$-th pose $\rho$ is translated away from the corresponding view $\bm{X}_i$ for visualization purpose but should be seen as superimposed, such that blue arrows denote null distances. (a) When the two objects are identical, all distances are null. (b) In case of occlusion, the distance from a point in the template to its nearest neighbor in the view is non-zero if and only if this point is occluded in the view (red arrows). (c) Similarly, the distance from a point in the view to its nearest neighbor in the template is non-zero if and only if this point is not part of the template, i.e. if and only if it is an outlier (red arrows).
• Figure 4: Visual results of the masking operations (\ref{['sec:latent_loss']}.c, \ref{['sec:latent_loss']}.d) in case of occlusion and outliers. Red points in the template (a) and in the view (b) will be discarded for the loss computation.
• Figure 5: Landscape of the loss in \ref{['eq:latent_optim']} with respect to the two first Euler angles (the loss is summed over the third Euler angle). The white crosses are the results of the $\mathrm{SO(3)}$ local minima search (\ref{['sec:optimization']}). In the case of an airplane, two minima coexist, corresponding to a $\ang{180}$ rotation along the fuselage axis.