Active Semantic Mapping of Horticultural Environments Using Gaussian Splatting

TL;DR

This work tackles the challenge of efficient and accurate semantic 3D reconstruction in horticultural environments by fusing a low-resolution semantic Octomap for exploration and safe planning with a dense semantic 3D Gaussian Splatting representation for high-fidelity fruit reconstruction. A target-aware viewpoint sampling strategy and graph-based planning guide data collection around semantic clusters and crop rows, while a segmentation-confidence aware loss improves robustness to noisy labels. The approach yields high fruit reconstruction accuracy, precise volume and count estimates, and roughly 50% runtime savings versus high-resolution Octomap baselines, demonstrated in simulation. The framework paves the way for real-time, high-throughput plant phenotyping in agricultural robotics, with potential extensions to additional semantic categories and field deployments.

Abstract

Semantic reconstruction of agricultural scenes plays a vital role in tasks such as phenotyping and yield estimation. However, traditional approaches that rely on manual scanning or fixed camera setups remain a major bottleneck in this process. In this work, we propose an active 3D reconstruction framework for horticultural environments using a mobile manipulator. The proposed system integrates the classical Octomap representation with 3D Gaussian Splatting to enable accurate and efficient target-aware mapping. While a low-resolution Octomap provides probabilistic occupancy information for informative viewpoint selection and collision-free planning, 3D Gaussian Splatting leverages geometric, photometric, and semantic information to optimize a set of 3D Gaussians for high-fidelity scene reconstruction. We further introduce simple yet effective strategies to enhance robustness against segmentation noise and reduce memory consumption. Simulation experiments demonstrate that our method outperforms purely occupancy-based approaches in both runtime efficiency and reconstruction accuracy, enabling precise fruit counting and volume estimation. Compared to a 0.01m-resolution Octomap, our approach achieves an improvement of 6.6% in fruit-level F1 score under noise-free conditions, and up to 28.6% under segmentation noise. Additionally, it achieves a 50% reduction in runtime, highlighting its potential for scalable, real-time semantic reconstruction in agricultural robotics.

Figures (4)

• Figure 1: Top: Simulation environment with the Terrasentia robot. Bottom: Sample reconstruction of a single crop row demonstrating the robustness of our approach under segmentation noise, compared to a high-resolution octomap.
• Figure 2: System overview. Our framework integrates two semantic representations: a low-resolution octomap for collision-free planning and viewpoint evaluation, and a 3DGS representation for high-fidelity reconstruction. Candidate exploitation viewpoints are sampled around semantic clusters, while exploration viewpoints are sampled along crop rows. A graph-based planner determines the optimal sequence of viewpoints to execute, considering information gain and actuation cost.
• Figure 3: Sample reconstructions of two variants of bell pepper rows. Our 3DGS-based approach produces denser and more complete fruit reconstructions compare to high-resolution octomaps.
• Figure 4: Runtime broken down into Octomap mapping, Gaussian Splatting mapping, viewpoint planning, and viewpoint execution. Overall, our method requires approximately half the runtime of a high-resolution octomap baseline.