Incremental Multimodal Surface Mapping via Self-Organizing Gaussian Mixture Models

TL;DR

This work addresses the challenge of real-time, high-fidelity multimodal surface mapping under bandwidth and computation constraints by modeling the environment with Self-Organizing Gaussian Mixture Models (SOGMMs). It introduces Local p_L and Global p_G GMMs and a Spatial Hash H to enable fast submap extraction and incremental updates, using a marginal 3D likelihood to determine relevance and a hashing scheme to limit computation. Key contributions include a computationally efficient submap extraction method, an incremental update rule for the global GMM, and extensive evaluations showing improved map fidelity at lower memory costs, with an open-source release. The approach achieves significant speedups over prior GMM-based methods while maintaining or improving reconstruction quality, supporting scalable, multi-robot exploration in unstructured environments.

Abstract

This letter describes an incremental multimodal surface mapping methodology, which represents the environment as a continuous probabilistic model. This model enables high-resolution reconstruction while simultaneously compressing spatial and intensity point cloud data. The strategy employed in this work utilizes Gaussian mixture models (GMMs) to represent the environment. While prior GMM-based mapping works have developed methodologies to determine the number of mixture components using information-theoretic techniques, these approaches either operate on individual sensor observations, making them unsuitable for incremental mapping, or are not real-time viable, especially for applications where high-fidelity modeling is required. To bridge this gap, this letter introduces a spatial hash map for rapid GMM submap extraction combined with an approach to determine relevant and redundant data in a point cloud. These contributions increase computational speed by an order of magnitude compared to state-of-the-art incremental GMM-based mapping. In addition, the proposed approach yields a superior tradeoff in map accuracy and size when compared to state-of-the-art mapping methodologies (both GMM- and not GMM-based). Evaluations are conducted using both simulated and real-world data. The software is released open-source to benefit the robotics community.

Figures (7)

• Figure 1: A reconstructed point cloud using spatial and intensity information inferred from the compact multimodal point cloud model created using the proposed approach. The representation leverages a formulation that has been demonstrated to be amenable for higher level robot autonomy objectives like exploration in complex, unstructured 3D environments. A video is available at: https://youtu.be/VgPEEcbUAnY.
• Figure 2: Information flow during surface point cloud modeling via the proposed incremental mapping approach (\ref{['ssec:proposed-mapping']}).
• Figure 3: Illustration of the relevant point cloud calculation using two multimodal point clouds, $\mathcal{Z}_1$ and $\mathcal{Z}_2$ (\ref{['sssec:local-sogmm']}). The objective is to find the relevant point cloud, $\mathcal{Z}^r_2$, from$\mathcal{Z}_2$ using $p_{G}$, which is created from $\mathcal{Z}_1$. \ref{['sfig:given-pclds']} shows the 3D parts of these point clouds in different colors and the associated 3D poses. \ref{['sfig:4d-check']} shows $\mathcal{Z}^r_2$ and $\mathcal{Z}_1$ with intensity values, when \ref{['eq:zr-4d']} is used. \ref{['sfig:3d-check']} shows the same but when \ref{['eq:zr-3d']} is used. Notice that in the former case $\mathcal{Z}^r_2$ contains more misclassified points that overlap with $p_{G}$ than in the latter case. \ref{['sfig:3d-check-fov']} shows the output $\mathcal{Z}^r_2$ when only a subset ($|\mathcal{B}| = 480$) of components in $p_{G}$ ($|\mathcal{K}| = 1165$) derived using the hash table $H$ are used. This output is similar to \ref{['sfig:3d-check']}. The point clouds are sourced from from the real-world Lounge dataset zhou_dense_2013. This figure is best viewed in color.
• Figure 4: Comparison of the relevant subset $\mathcal{Z}^r$ calculation time between the prior work on multimodal GMM mapping srivastava_efficient_2017 and the proposed approach. The per-frame calculation time in seconds is plotted for \ref{['sfig:ll-base']} different values of fixed numbers of components $|\mathcal{J}|$ and \ref{['sfig:ll-isogmm']} different values of the bandwidth parameter $\sigma$ for the proposed method. \ref{['sfig:ll-times']} Notice that the spatial hash (\ref{['sssec:spatial-hash-global']}) enables an order of magnitude improvement and that the performance gains increase monotonically with model size. \ref{['sfig:ll-sp-abl']} shows an ablation of calculation times for different values of the spatial hash resolution parameter $\alpha$.
• Figure 5: Quantitative comparison of \ref{['sfig:lr-mre']} reconstruction error, \ref{['sfig:lr-prec']} precision, \ref{['sfig:lr-rec']} recall, and \ref{['sfig:lr-psnr']} PSNR as a function of the map size in megabytes (M) for each approach. The dataset under consideration is the synthetic D1 dataset shown in \ref{['sfig:lr-gt']}. Note that the proposed approach yields a map that requires less disk space than the competing methods while demonstrating at par or better reconstruction accuracy (i.e., low reconstruction error and high precision).