GS-Planner: A Gaussian-Splatting-based Planning Framework for Active High-Fidelity Reconstruction

TL;DR

This work tackles the problem of online, high-fidelity 3D reconstruction for autonomous robots by adopting 3D Gaussian Splatting (3DGS) as an explicit radiance-field representation that supports real-time rendering and online evaluation. The authors introduce GS-Planner, a planning framework that integrates online completeness and quality metrics, a sampling-based view-planning strategy with a view library, differentiable safety constraints, and MINCO-based trajectory optimization to produce safe, collision-free, high-quality reconstructions. Key contributions include the first active 3D reconstruction system using 3DGS with online evaluation, metrics for completeness and quality, a safety-aware trajectory planning approach, and extensive simulation validation demonstrating efficiency and fidelity. The approach has practical impact for agile robotic platforms requiring real-time, high-fidelity scene understanding and safe navigation, with potential extensions to real-world deployment and memory-efficient 3DGS representations.

Abstract

Active reconstruction technique enables robots to autonomously collect scene data for full coverage, relieving users from tedious and time-consuming data capturing process. However, designed based on unsuitable scene representations, existing methods show unrealistic reconstruction results or the inability of online quality evaluation. Due to the recent advancements in explicit radiance field technology, online active high-fidelity reconstruction has become achievable. In this paper, we propose GS-Planner, a planning framework for active high-fidelity reconstruction using 3D Gaussian Splatting. With improvement on 3DGS to recognize unobserved regions, we evaluate the reconstruction quality and completeness of 3DGS map online to guide the robot. Then we design a sampling-based active reconstruction strategy to explore the unobserved areas and improve the reconstruction geometric and textural quality. To establish a complete robot active reconstruction system, we choose quadrotor as the robotic platform for its high agility. Then we devise a safety constraint with 3DGS to generate executable trajectories for quadrotor navigation in the 3DGS map. To validate the effectiveness of our method, we conduct extensive experiments and ablation studies in highly realistic simulation scenes.

Figures (9)

• Figure 1: The whole active reconstruction process in a simulated supermarket scene. We deployed our active high-fidelity reconstruction system on a simulated quadrotor with an RGB-D sensor. The colored curves illustrate the executed trajectories of the drones. We demonstrate the reconstruction result including the whole rendered scene and details rendered at three views.
• Figure 2: An overview of our active high-fidelity reconstruction system. With 3DGS as scene representation, the unobserved regions, as well as the geometric and textural information of the built map can be feedback in real time for online reconstruction quality and completeness online evaluation. The proposed active reconstruction strategy guides the robot to collect new scene information to build a complete high-fidelity 3DGS map.
• Figure 3: An illustration of the completeness evaluation. (a). A partially reconstructed scene. Scene information has been collected only at the observed viewpoint. The colored grid illustrates the completeness gain from 360-degree summation at different positions at a height of $z=1m$. (b). The location of two candidate viewpoints. The z-axis direction is aligned with the camera's optical axis. (c). 360-degree panoramic image of the completeness gain of the candidate viewpoint 1 and 2. The generation of 360-degree gain facilitates the subsequent determination of the optimal viewpoint yaw direction.
• Figure 4: An instance of the quality evaluation. (a). The generation of the RGB textural loss between the input RGB image and rendered RGB image. (b). The generation of the depth loss between the input depth image and rendered depth image. (c). The weighted sum of the RGB loss and depth loss. (d). We project the quality gain to the 3D grid in the world frame to store.
• Figure 5: A 2D illustration of the 3D completeness evaluation. Given a collection of 3D Gaussians and a candidate viewpoint, we can get unobserved regions within the splatting-based rendering. The unobserved regions are weighted by transmittance, which is equal to the accumulated Guassians' opacity along the ray.