Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DISORF: A Distributed Online 3D Reconstruction Framework for Mobile Robots

Chunlin Li, Hanrui Fan, Xiaorui Huang, Ruofan Liang, Sankeerth Durvasula, Nandita Vijaykumar

TL;DR

DISORF presents a distributed framework for online 3D reconstruction on resource-constrained mobile robots by splitting computation between edge SLAM and a powerful remote server. It supports both NeRF and 3D Gaussian Splatting with online training, and addresses a key challenge—imbalanced frame sampling during streaming—by introducing a shifted exponential sampling strategy that emphasizes recent keyframes while preserving older ones. The framework leverages on-device pose estimation, keyframe streaming, and a Nerfstudio-based remote pipeline to deliver real-time visualization and high-quality 3D representations of unknown scenes. Empirical results on Replica and Tanks & Temples demonstrate improved rendering quality and practical viability, highlighting the approach’s potential for real-time robotics tasks in varying network conditions.

Abstract

We present a framework, DISORF, to enable online 3D reconstruction and visualization of scenes captured by resource-constrained mobile robots and edge devices. To address the limited computing capabilities of edge devices and potentially limited network availability, we design a framework that efficiently distributes computation between the edge device and the remote server. We leverage on-device SLAM systems to generate posed keyframes and transmit them to remote servers that can perform high-quality 3D reconstruction and visualization at runtime by leveraging recent advances in neural 3D methods. We identify a key challenge with online training where naive image sampling strategies can lead to significant degradation in rendering quality. We propose a novel shifted exponential frame sampling method that addresses this challenge for online training. We demonstrate the effectiveness of our framework in enabling high-quality real-time reconstruction and visualization of unknown scenes as they are captured and streamed from cameras in mobile robots and edge devices.

DISORF: A Distributed Online 3D Reconstruction Framework for Mobile Robots

TL;DR

DISORF presents a distributed framework for online 3D reconstruction on resource-constrained mobile robots by splitting computation between edge SLAM and a powerful remote server. It supports both NeRF and 3D Gaussian Splatting with online training, and addresses a key challenge—imbalanced frame sampling during streaming—by introducing a shifted exponential sampling strategy that emphasizes recent keyframes while preserving older ones. The framework leverages on-device pose estimation, keyframe streaming, and a Nerfstudio-based remote pipeline to deliver real-time visualization and high-quality 3D representations of unknown scenes. Empirical results on Replica and Tanks & Temples demonstrate improved rendering quality and practical viability, highlighting the approach’s potential for real-time robotics tasks in varying network conditions.

Abstract

We present a framework, DISORF, to enable online 3D reconstruction and visualization of scenes captured by resource-constrained mobile robots and edge devices. To address the limited computing capabilities of edge devices and potentially limited network availability, we design a framework that efficiently distributes computation between the edge device and the remote server. We leverage on-device SLAM systems to generate posed keyframes and transmit them to remote servers that can perform high-quality 3D reconstruction and visualization at runtime by leveraging recent advances in neural 3D methods. We identify a key challenge with online training where naive image sampling strategies can lead to significant degradation in rendering quality. We propose a novel shifted exponential frame sampling method that addresses this challenge for online training. We demonstrate the effectiveness of our framework in enabling high-quality real-time reconstruction and visualization of unknown scenes as they are captured and streamed from cameras in mobile robots and edge devices.
Paper Structure (22 sections, 5 equations, 4 figures, 5 tables)

This paper contains 22 sections, 5 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Online 3D Representation
  4. Neural 3D Representations and Robotics Applications
  5. Incremental Learning for 3D
  6. Framework Design
  7. SLAM Module on Local Robot
  8. 3D Neural Reconstruction on Remote Server
  9. Network Connection for Distributed Computing
  10. Online 3D Neural Reconstruction Models
  11. Neural Radiance Field
  12. 3D Gaussian Splatting
  13. Training Schedulers
  14. Online Training Sampling Strategy
  15. Modeling the Keyframe Stream
  16. ...and 7 more sections

Figures (4)

  • Figure 1: The setup and rendering results for online and offline training: For offline training (top), images from all viewpoints are available throughout the training process. For online training (bottom), the model is continuously trained as new images are streamed.
  • Figure 2: The total number of pixels from each frame being sampled after NeRF training on a Replica one-minute keyframe stream.
  • Figure 3: DISORF pipeline distributing computation between a mobile robot and a remote server. The robot's camera captures images processed by the edge processor's SLAM module. Keyframes and poses are streamed to the server, where GPU-intensive 3D model training (NeRF/3DGS) is performed.
  • Figure 4: Each received frame is initially trained at a lower resolution, then gradually increased to the original resolution.