Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Model 3D Registration: Finding Multiple Moving Objects in Cluttered Point Clouds

David Jin, Sushrut Karmalkar, Harry Zhang, Luca Carlone

TL;DR

A simple approach based on Expectation-Maximization (EM) is proposed and evaluated in simulated and real datasets ranging from table-top scenes to self-driving scenarios and it is demonstrated its effectiveness when combined with state-of-the-art scene flow methods to establish dense correspondences.

Abstract

We investigate a variation of the 3D registration problem, named multi-model 3D registration. In the multi-model registration problem, we are given two point clouds picturing a set of objects at different poses (and possibly including points belonging to the background) and we want to simultaneously reconstruct how all objects moved between the two point clouds. This setup generalizes standard 3D registration where one wants to reconstruct a single pose, e.g., the motion of the sensor picturing a static scene. Moreover, it provides a mathematically grounded formulation for relevant robotics applications, e.g., where a depth sensor onboard a robot perceives a dynamic scene and has the goal of estimating its own motion (from the static portion of the scene) while simultaneously recovering the motion of all dynamic objects. We assume a correspondence-based setup where we have putative matches between the two point clouds and consider the practical case where these correspondences are plagued with outliers. We then propose a simple approach based on Expectation-Maximization (EM) and establish theoretical conditions under which the EM approach converges to the ground truth. We evaluate the approach in simulated and real datasets ranging from table-top scenes to self-driving scenarios and demonstrate its effectiveness when combined with state-of-the-art scene flow methods to establish dense correspondences.

Multi-Model 3D Registration: Finding Multiple Moving Objects in Cluttered Point Clouds

TL;DR

A simple approach based on Expectation-Maximization (EM) is proposed and evaluated in simulated and real datasets ranging from table-top scenes to self-driving scenarios and it is demonstrated its effectiveness when combined with state-of-the-art scene flow methods to establish dense correspondences.

Abstract

We investigate a variation of the 3D registration problem, named multi-model 3D registration. In the multi-model registration problem, we are given two point clouds picturing a set of objects at different poses (and possibly including points belonging to the background) and we want to simultaneously reconstruct how all objects moved between the two point clouds. This setup generalizes standard 3D registration where one wants to reconstruct a single pose, e.g., the motion of the sensor picturing a static scene. Moreover, it provides a mathematically grounded formulation for relevant robotics applications, e.g., where a depth sensor onboard a robot perceives a dynamic scene and has the goal of estimating its own motion (from the static portion of the scene) while simultaneously recovering the motion of all dynamic objects. We assume a correspondence-based setup where we have putative matches between the two point clouds and consider the practical case where these correspondences are plagued with outliers. We then propose a simple approach based on Expectation-Maximization (EM) and establish theoretical conditions under which the EM approach converges to the ground truth. We evaluate the approach in simulated and real datasets ranging from table-top scenes to self-driving scenarios and demonstrate its effectiveness when combined with state-of-the-art scene flow methods to establish dense correspondences.
Paper Structure (11 sections, 1 theorem, 7 equations, 2 figures, 1 table, 1 algorithm)

This paper contains 11 sections, 1 theorem, 7 equations, 2 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. An Expectation-Maximization Approach to Multi-Model Registration
  4. Theoretical Analysis
  5. Experiments
  6. Baselines and Initialization
  7. Metrics
  8. PASCAL3D+
  9. FlyingThings3D
  10. KITTI
  11. Conclusion

Key Result

Theorem 3

In the setting of Definition set:problem, assume that the initial clustering $\mathcal{H}$ is $(\tau,\alpha,m_0)$-good in the sense of Definition def:initial_conditions, for some sufficiently large $m_0$. Then, running alg:EM, with high probability (dependent on $m_0$), returns $\mathcal{H}'$, a par

Figures (2)

  • Figure 1: We propose an Expectation-Maximization approach for multi-model 3D registration, which aims to recover the motion of all objects (and background) in a scene from point cloud observations. The figure reports two results produced by our approach on the KITTI dataset. Note that the two cars on the left of the bottom figure are stationary, hence they are correctly deemed to be part of the background.
  • Figure 2: Results (IoU, per-point error, rotation error, and translation error) on (a) FlyingThings3D and (b) KITTI. We evaluate two variations of our method with two different initializations (SAM and Euclidean) and four baselines: Naive, Sequential RANSAC (SeqR), T-Linkage with SAM initialization (T-L (SAM)), and T-Linkage with Euclidean initialization (T-L (Euc.)).

Theorems & Definitions (4)

  • Definition 1: Ground Truth
  • Definition 2: Good Clustering
  • Theorem 3: Expectation-Maximization Guarantee
  • Remark 4: Novelty