Real-Time LiDAR Point Cloud Densification for Low-Latency Spatial Data Transmission
Kazuhiko Murasaki, Shunsuke Konagai, Masakatsu Aoki, Taiga Yoshida, Ryuichi Tanida
TL;DR
This work tackles real-time, dense 3D capture for immersive telepresence by densifying sparse LiDAR point clouds using multi-frame fusion and RGB-guided joint bilateral filtering. Implemented as GPU-accelerated, CNN-like filtering, the approach yields full-HD depth maps at 30 fps with low latency and minimal cross-view ghosting, outperforming training-heavy depth completion methods like BPNet in speed. It introduces a practical preprocessing pipeline to mitigate moving-object and occlusion artifacts and demonstrates real-time performance across multiple viewpoints. The method offers a scalable, low-latency alternative for live 3D telepresence and spatial data transmission, with strong potential for deployment in interactive immersive systems.
Abstract
To realize low-latency spatial transmission system for immersive telepresence, there are two major problems: capturing dynamic 3D scene densely and processing them in real time. LiDAR sensors capture 3D in real time, but produce sparce point clouds. Therefore, this paper presents a high-speed LiDAR point cloud densification method to generate dense 3D scene with minimal latency, addressing the need for on-the-fly depth completion while maintaining real-time performance. Our approach combines multiple LiDAR inputs with high-resolution color images and applies a joint bilateral filtering strategy implemented through a convolutional neural network architecture. Experiments demonstrate that the proposed method produces dense depth maps at full HD resolution in real time (30 fps), which is over 15x faster than a recent training-based depth completion approach. The resulting dense point clouds exhibit accurate geometry without multiview inconsistencies or ghosting artifacts.