Towards Zero-Shot Point Cloud Registration Across Diverse Scales, Scenes, and Sensor Setups

TL;DR

BUFFER-X tackles zero-shot point cloud registration by diagnosing three generalization bottlenecks: fixed hyperparameters, brittle learned keypoints, and absolute-coordinate scale sensitivity. It introduces a training-free pipeline with geometric bootstrapping to estimate scale-aware voxel sizes and radii, a detector-free tri-scale patch embedder with FPS sampling, and a hierarchical inlier search for cross-scale consensus, achieving robust cross-domain alignment without test-domain tuning. BUFFER-X-Lite further accelerates the pipeline via adaptive scale processing and a fast KISS-Matcher solver, enabling practical deployment with substantial speedups. Evaluated on 12 diverse datasets including cross-sensor LiDAR scenarios, BUFFER-X demonstrates strong out-of-the-box zero-shot generalization, while BUFFER-X-Lite offers a favorable speed-accuracy trade-off for real-world robotics applications.

Abstract

Some deep learning-based point cloud registration methods struggle with zero-shot generalization, often requiring dataset-specific hyperparameter tuning or retraining for new environments. We identify three critical limitations: (a) fixed user-defined parameters (e.g., voxel size, search radius) that fail to generalize across varying scales, (b) learned keypoint detectors exhibit poor cross-domain transferability, and (c) absolute coordinates amplify scale mismatches between datasets. To address these three issues, we present BUFFER-X, a training-free registration framework that achieves zero-shot generalization through: (a) geometric bootstrapping for automatic hyperparameter estimation, (b) distribution-aware farthest point sampling to replace learned detectors, and (c) patch-level coordinate normalization to ensure scale consistency. Our approach employs hierarchical multi-scale matching to extract correspondences across local, middle, and global receptive fields, enabling robust registration in diverse environments. For efficiency-critical applications, we introduce BUFFER-X-Lite, which reduces total computation time by 43% (relative to BUFFER-X) through early exit strategies and fast pose solvers while preserving accuracy. We evaluate on a comprehensive benchmark comprising 12 datasets spanning object-scale, indoor, and outdoor scenes, including cross-sensor registration between heterogeneous LiDAR configurations. Results demonstrate that our approach generalizes effectively without manual tuning or prior knowledge of test domains. Code: https://github.com/MIT-SPARK/BUFFER-X.

Figures (15)

• Figure 1: Success rate (unit: %) of zero-shot point cloud registration with state-of-the-art approaches on 12 datasets Wu15cvpr-ModelNetZeng17cvpr-3dmatchHuang21cvpr-PREDATORRegistrationGeiger13ijrr-KITTIYeshwanth23iccv-Scannet++Qingqing22iros-TIERSRamezani20iros-NewerCollegeSun20cvpr-WaymoDatasetTian23iros-KimeraMultiExperimentsPomerleau12ijrr-ETHJung23ijrr-HeLiPR. Without any prior information or manual parameter tuning for the test datasets, our BUFFER-X shows robust generalization capability across diverse scales, scenes, and sensor setups even though the network is only trained on the 3DMatch dataset Zeng17cvpr-3dmatch. In particular, our proposed approach can operate on CAD models (from left to right, the point clouds represent a plant, a laptop, and a sofa, respectively) as well as on heterogeneous LiDAR point clouds.
• Figure 2: (a) Variation in the number of points after voxelization with different voxel sizes $v$ across datasets. Even in indoor scenes, point counts vary significantly depending on the sensor type (i.e.,TIERSQingqing22iros-TIERS vs. 3DMatchZeng17cvpr-3dmatch). Notably, TIERS and KITTIGeiger13ijrr-KITTI, both using omnidirectional LiDARs, yield different point densities due to indoor vs. outdoor environments. (b) Empirical distribution of the datasets’ maximum range.
• Figure 3: Overview of our BUFFER-X, which mainly consists of three steps. (a) Geometric bootstrapping (\ref{['sec:geometric']}) to determine the appropriate voxel size and radii for the given source $\mathcal{P}$ and target $\mathcal{Q}$ clouds. (b) Multi-scale patch embedder (\ref{['sec:tri-scale-patch']}) to generate patch-wise descriptor $\mathcal{S}_\xi$ for multiple scales $\xi \in \{l, m, g\}$, where $l$, $m$, and $g$ represent local, middle, and global scales, respectively. Specifically, Mini-SpinNet Ao23CVPR-BUFFER outputs cylindrical feature maps $\mathcal{C}_\xi$ and vector feature sets $\mathcal{F}_\xi$. (c) Hierarchical inlier search (\ref{['sec:hierarchical']}), which first performs nearest neighbor-based intra-scale matching using $\mathcal{F}^\mathcal{P}_\xi$ and $\mathcal{F}^\mathcal{Q}_\xi$ at each scale, followed by pairwise transformation estimation. Finally, it identifies globally consistent inliers $\mathcal{I}$ across all scales to refine correspondences based on consensus maximization Sun22ral-TriVoCZhang24tpami-AcceleratingGloballyCM.
• Figure 4: (a) Visual description of local ($r_l$), middle ($r_m$), and global ($r_g$) radii for the same point to illustrate scale differences and (b) normalized patches ranging from $[-1, 1]$. Note that their reference frames follow the eigenvectors obtained from principal component analysis (PCA) Lim21ral-PatchworkAlexiou24jivp-PointPCA. The $z$-axis is assigned to the eigenvector ${\bm v}_3$, which corresponds to the smallest eigenvalue.
• Figure 5: Dataset statistics and characteristics analysis, which demonstrates the diversity of point cloud characteristics across different environments, sensor types, and acquisition scenarios in our generalizability benchmark. (a)-(b) Box plots of mean and 99th percentile range per frame across different datasets, respectively. (c) Distribution of the number of point clouds and the number of selected test pairs across indoor and outdoor environments. (d)-(e) Gaussian distribution of ranges for indoor and outdoor scenes, respectively, and (f)-(g) those for different sensor types in the TIERS and KAIST sequences.