Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

F3DGS: Federated 3D Gaussian Splatting for Decentralized Multi-Agent World Modeling

Morui Zhu, Mohammad Dehghani Tezerjani, Mátyás Szántó, Márton Vaitkus, Song Fu, Qing Yang

Abstract

We present F3DGS, a federated 3D Gaussian Splatting framework for decentralized multi-agent 3D reconstruction. Existing 3DGS pipelines assume centralized access to all observations, which limits their applicability in distributed robotic settings where agents operate independently, and centralized data aggregation may be restricted. Directly extending centralized training to multi-agent systems introduces communication overhead and geometric inconsistency. F3DGS first constructs a shared geometric scaffold by registering locally merged LiDAR point clouds from multiple clients to initialize a global 3DGS model. During federated optimization, Gaussian positions are fixed to preserve geometric alignment, while each client updates only appearance-related attributes, including covariance, opacity, and spherical harmonic coefficients. The server aggregates these updates using visibility-aware aggregation, weighting each client's contribution by how frequently it observed each Gaussian, resolving the partial-observability challenge inherent to multi-agent exploration. To evaluate decentralized reconstruction, we collect a multi-sequence indoor dataset with synchronized LiDAR, RGB, and IMU measurements. Experiments show that F3DGS achieves reconstruction quality comparable to centralized training while enabling distributed optimization across agents. The dataset, development kit, and source code will be publicly released.

F3DGS: Federated 3D Gaussian Splatting for Decentralized Multi-Agent World Modeling

Abstract

We present F3DGS, a federated 3D Gaussian Splatting framework for decentralized multi-agent 3D reconstruction. Existing 3DGS pipelines assume centralized access to all observations, which limits their applicability in distributed robotic settings where agents operate independently, and centralized data aggregation may be restricted. Directly extending centralized training to multi-agent systems introduces communication overhead and geometric inconsistency. F3DGS first constructs a shared geometric scaffold by registering locally merged LiDAR point clouds from multiple clients to initialize a global 3DGS model. During federated optimization, Gaussian positions are fixed to preserve geometric alignment, while each client updates only appearance-related attributes, including covariance, opacity, and spherical harmonic coefficients. The server aggregates these updates using visibility-aware aggregation, weighting each client's contribution by how frequently it observed each Gaussian, resolving the partial-observability challenge inherent to multi-agent exploration. To evaluate decentralized reconstruction, we collect a multi-sequence indoor dataset with synchronized LiDAR, RGB, and IMU measurements. Experiments show that F3DGS achieves reconstruction quality comparable to centralized training while enabling distributed optimization across agents. The dataset, development kit, and source code will be publicly released.

Paper Structure

This paper contains 25 sections, 9 equations, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. 3D Gaussian Splatting
  4. Federated Learning
  5. Multi-Agent 3D Reconstruction
  6. LiDAR-Visual Pose Estimation
  7. Method
  8. Overview
  9. Problem Formulation
  10. Metric-Scale Pose Construction
  11. Global Gaussian Initialization
  12. Local Federated Optimization
  13. Frozen geometry.
  14. Visibility tracking.
  15. Visibility-Aware Federated Aggregation
  16. ...and 10 more sections

Figures (1)

  • Figure 1: Overview of our federated 3D Gaussian splatting (F3DGS) framework for learning consistent global scene representations. Each client first constructs metric-scale poses from unposed images and LiDAR via visual geometry reconstruction and LiDAR odometry. Based on these poses, clients perform local optimization, including pose alignment and Gaussian initialization, to obtain per-client 3D Gaussian representations. Through iterative federated updates, client models are progressively aggregated. Finally, a visibility-aware fusion strategy integrates Gaussian attributes (e.g., opacity, scale, and spherical harmonics) to produce a unified and consistent global reconstruction.