PLANING: A Loosely Coupled Triangle-Gaussian Framework for Streaming 3D Reconstruction

TL;DR

PLANING tackles real-time monocular streaming 3D reconstruction by introducing a loosely coupled triangle-Gaussian representation that decouples geometry from appearance. Learnable triangles provide explicit, editable geometry while neural Gaussians supply high-fidelity appearance, together enabling compact, stable reconstructions in a streaming pipeline with geometry-guided initialization and global pose-consistent updates. The approach yields state-of-the-art geometry and rendering metrics, reduces primitive redundancy, and achieves fast online performance (e.g., ScanNetV2 in under $100$ s and significantly faster than prior 2D Gaussian Splatting), while supporting downstream tasks like plane extraction and simulation-ready scene modeling for embodied AI. This combination offers robust, scalable scene modeling suitable for large-scale environments and downstream applications such as pose optimization, large-scale reconstruction, and locomotion policy training, with practical benefits in efficiency and structural clarity.

Abstract

Streaming reconstruction from monocular image sequences remains challenging, as existing methods typically favor either high-quality rendering or accurate geometry, but rarely both. We present PLANING, an efficient on-the-fly reconstruction framework built on a hybrid representation that loosely couples explicit geometric primitives with neural Gaussians, enabling geometry and appearance to be modeled in a decoupled manner. This decoupling supports an online initialization and optimization strategy that separates geometry and appearance updates, yielding stable streaming reconstruction with substantially reduced structural redundancy. PLANING improves dense mesh Chamfer-L2 by 18.52% over PGSR, surpasses ARTDECO by 1.31 dB PSNR, and reconstructs ScanNetV2 scenes in under 100 seconds, over 5x faster than 2D Gaussian Splatting, while matching the quality of offline per-scene optimization. Beyond reconstruction quality, the structural clarity and computational efficiency of \modelname~make it well suited for a broad range of downstream applications, such as enabling large-scale scene modeling and simulation-ready environments for embodied AI. Project page: https://city-super.github.io/PLANING/ .

Figures (15)

• Figure 1: PLANING introduces a loosely coupled triangle-Gaussian representation for streaming 3D reconstruction, balancing geometric accuracy, high-fidelity rendering, and computational efficiency. Building upon this hybrid representation, we further adapt it to an efficient streaming reconstruction framework for monocular image sequences, enabling effective modeling of both scene geometry and appearance in a streaming setting. Leveraging the inherent edge-preserving property of triangle primitives, our method allows for the explicit extraction of compact planar structures, which can serve as a high-performance simulation environment for locomotion training in embodied AI.
• Figure 2: PLANING consistently outperforms existing streaming and per-scene reconstruction methods across geometry accuracy, rendering quality, computational efficiency, and memory usage, while maintaining clear and well-structured planar geometry.
• Figure 3: Pipeline of PLANING. PLANING adopts a hybrid representation in which triangles explicitly model scene geometry, while neural Gaussians decoded from these triangles render appearance. Built upon this representation, we develop a streaming reconstruction framework that takes unposed monocular image sequences as input and comprises a frontend for camera tracking, a backend for global pose optimization, and a mapper for scene reconstruction. Specifically, the mapper incorporates an efficient primitive initialization strategy to reduce redundancy. The recontructed triangle soup further enables efficient planar abstraction, facilitating a range of downstream tasks.
• Figure 4: Definition of the local frame and results of forward rendering. Our triangle rasterizer enables correct and reliable forward rendering of triangles.
• Figure 5: Qualitative comparison of geometric reconstruction. We visualize planar reconstruction and geometric modeling across different primitives, with 2DGS shown as dense mesh for comparison. Overall, our method preserves planar structures while capturing fine geometric details.