Instance by Instance: An Iterative Framework for Multi-instance 3D Registration

TL;DR

The paper tackles MI-3DReg, where multiple repeated object instances must be registered in a common frame. It introduces an iterative framework, IBI, to register instances sequentially from easy to hard, progressively removing outliers, and proposes IBI-S2DC to mine sparse seeds and expand to dense, robust correspondences. The approach combines game-theoretic seed mining (GTM), voting-based enhancement, guided sample consensus (GSAC), and global overlap-based hypothesis validation, achieving state-of-the-art performance and strong robustness to high outlier ratios on both synthetic and real datasets. The work demonstrates substantial improvements in mean hit F1 (MHF1) over existing methods and provides detailed ablations confirming the contribution of each module, with practical implications for scalable, robust MI-3DReg in robotics and perception systems.

Abstract

Multi-instance registration is a challenging problem in computer vision and robotics, where multiple instances of an object need to be registered in a standard coordinate system. In this work, we propose the first iterative framework called instance-by-instance (IBI) for multi-instance 3D registration (MI-3DReg). It successively registers all instances in a given scenario, starting from the easiest and progressing to more challenging ones. Throughout the iterative process, outliers are eliminated continuously, leading to an increasing inlier rate for the remaining and more challenging instances. Under the IBI framework, we further propose a sparse-to-dense-correspondence-based multi-instance registration method (IBI-S2DC) to achieve robust MI-3DReg. Experiments on the synthetic and real datasets have demonstrated the effectiveness of IBI and suggested the new state-of-the-art performance of IBI-S2DC, e.g., our MHF1 is 12.02%/12.35% higher than the existing state-of-the-art method ECC on the synthetic/real datasets.

Figures (12)

• Figure 1: Comparison of one-shot method and our IBI for MI-3DReg. The inliers and outliers are visualized in green and red, respectively. The green and red bounding boxes represent the ground truth and estimated poses of instances, respectively.
• Figure 2: Schematic illustration of the IBI framework.
• Figure 3: The pipeline of IBI-S2DC. 1. SCS: Mine a sparse yet consistent correspondence set $\mathbf{C}_{s}$ under GTM guidance. 2. CE: Enhance consistency to achieve a dense correspondence set $\mathbf{C}_{d}$ based on a voting scheme. 3. TE: Estimate the transformation for the current instance based on a guided sample consensus estimator (GSAC). 4. HV: Validate the hypothesis globally using the point cloud overlap rate.
• Figure 4: The pipeline of the voting-based correspondence enhancement. After mining a sparse correspondence set, the inlier rate of the instance rises but the number of inliers is very small. The voting-based enhancement helps to retrieve significantly more inliers while maintaining a high inlier rate, resulting in a consistent and dense correspondence set.
• Figure 5: In challenging registration cases, correspondences may still exhibit multiple consistencies and suffer from outliers after the CE step. In (a), the dense correspondences have a high inlier rate, while multiple-consistencies exist. In (b), there are multiple consistencies and many outliers in the dense correspondence set.