E2-BKI: Evidential Ellipsoidal Bayesian Kernel Inference for Uncertainty-aware Gaussian Semantic Mapping

TL;DR

This work tackles the challenge of uncertainty-aware 3D semantic mapping in outdoor environments by extending Bayesian Kernel Inference with evidential, geometry-aligned processing. It introduces Evidential Ellipsoidal BKI (E2-BKI), which aggregates sparse, noisy semantic observations into anisotropic Gaussian primitives, fuses them with semantic uncertainty via Evidential Deep Learning, and performs geometry-consistent, uncertainty-aware inference with an ellipsoidal kernel. The approach systematically addresses semantic, spatial, and observation uncertainties through semantic evidence, adaptive kernels, merging/pruning of primitives, and uncertainty decomposition, achieving improved semantic accuracy, calibration, robustness to sparsity and noisy predictions, and versatile map representations including BEV and continuous querying, all with real-time performance. Extensive experiments on off-road and urban outdoor datasets demonstrate consistent gains over prior continuous semantic mapping methods, with strong uncertainty calibration and practical runtime, highlighting the method's potential for robust autonomous mapping. The framework's modular Gaussian-primitive representation and geometry-aligned fusion offer practical benefits for multi-resolution mapping, adaptive sampling, and open-set extensions in real-world robotics.

Abstract

Semantic mapping aims to construct a 3D semantic representation of the environment, providing essential knowledge for robots operating in complex outdoor settings. While Bayesian Kernel Inference (BKI) addresses discontinuities of map inference from sparse sensor data, existing semantic mapping methods suffer from various sources of uncertainties in challenging outdoor environments. To address these issues, we propose an uncertainty-aware semantic mapping framework that handles multiple sources of uncertainties, which significantly degrade mapping performance. Our method estimates uncertainties in semantic predictions using Evidential Deep Learning and incorporates them into BKI for robust semantic inference. It further aggregates noisy observations into coherent Gaussian representations to mitigate the impact of unreliable points, while employing geometry-aligned kernels that adapt to complex scene structures. These Gaussian primitives effectively fuse local geometric and semantic information, enabling robust, uncertainty-aware mapping in complex outdoor scenarios. Comprehensive evaluation across diverse off-road and urban outdoor environments demonstrates consistent improvements in mapping quality, uncertainty calibration, representational flexibility, and robustness, while maintaining real-time efficiency. Our project website: https://e2-bki.github.io

Figures (17)

• Figure 1: Evidential Ellipsoidal BKI (E2-BKI) combines uncertainty-aware processing that prioritizes reliable observations with anisotropic kernels that align with local scene geometry, enabling the construction of accurate and reliable semantic maps from sparse, noisy, and uncertain semantic points.
• Figure 2: Key limitations of S-BKI and our solutions. The static isotropic kernel in S-BKI does not account for (a) semantic uncertainty arising from neural network predictions, (b) spatial uncertainty caused by misalignment with local scene geometry, and (c) observation uncertainty from distant noisy measurements. Our method addresses these by (a) prioritizing reliable information via uncertainty estimates, (b) adapting kernels to local scene geometry, and (c) leveraging local context through Gaussian primitives.
• Figure 3: Observation uncertainty varies with sensor distance. Left: sensor-agnostic schematic showing dense, high-resolution proximal observations and sparse, low-detail distal observations. Right: RGB image with semantic labels and LiDAR measurements overlaid; sparser returns at longer ranges lead to higher observation uncertainty.
• Figure 4: Overview of semantic mapping with Evidential Ellipsoidal BKI (E2-BKI). Given evidential points with semantic probability $\mathbf p_n$ and uncertainty $u_n$ (Section \ref{['Method:EDL']}), our method operates through three key stages: (Section \ref{['Method:Init']}) Gaussian Initialization aggregates evidential points into anisotropic Gaussian primitives encoding local geometry and semantics; (Section \ref{['Method:Refine']}) Gaussian Refinement merges spatially coherent primitives and prunes unreliable primitives; and (Sections \ref{['Method:Kernel']} and \ref{['Method:EEBKI']}) Evidential Ellipsoidal BKI performs uncertainty-aware semantic mapping using Gaussian primitives.
• Figure 5: Qualitative comparison of semantic mapping results on RELLIS-3D, OffRoad, and KITTI-360. Compared to baselines, our method produces more accurate and visually consistent semantic reconstructions across diverse scenes. The color scheme for semantic classes follows Table \ref{['tab:offroad_quant']} and Table \ref{['tab:urban_quant']}.