Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MM-OpenFGL: A Comprehensive Benchmark for Multimodal Federated Graph Learning

Xunkai Li, Yuming Ai, Yinlin Zhu, Haodong Lu, Yi Zhang, Guohao Fu, Bowen Fan, Qiangqiang Dai, Rong-Hua Li, Guoren Wang

TL;DR

MM-OpenFGL formalizes multimodal federated graph learning (MMFGL) and introduces the first comprehensive benchmark for it, integrating 19 multimodal datasets, 8 simulation strategies, 57 methods, and 9 downstream tasks. The framework supports two training pipelines—End-to-End FGL and a Two-Stage pretraining–finetuning approach—within a modular OpenFGL-like API. Through extensive experiments, the work provides 10 key insights, highlighting the limits of naive adaptations, the importance of cross-modal alignment and structural resilience, and the superior generalization of graph foundation models across diverse tasks. The benchmark and findings offer practical guidance for designing privacy-preserving, scalable MMFGL systems and come with an open-source library to foster future research and fair comparisons.

Abstract

Multimodal-attributed graphs (MMAGs) provide a unified framework for modeling complex relational data by integrating heterogeneous modalities with graph structures. While centralized learning has shown promising performance, MMAGs in real-world applications are often distributed across isolated platforms and cannot be shared due to privacy concerns or commercial constraints. Federated graph learning (FGL) offers a natural solution for collaborative training under such settings; however, existing studies largely focus on single-modality graphs and do not adequately address the challenges unique to multimodal federated graph learning (MMFGL). To bridge this gap, we present MM-OpenFGL, the first comprehensive benchmark that systematically formalizes the MMFGL paradigm and enables rigorous evaluation. MM-OpenFGL comprises 19 multimodal datasets spanning 7 application domains, 8 simulation strategies capturing modality and topology variations, 6 downstream tasks, and 57 state-of-the-art methods implemented through a modular API. Extensive experiments investigate MMFGL from the perspectives of necessity, effectiveness, robustness, and efficiency, offering valuable insights for future research on MMFGL.

MM-OpenFGL: A Comprehensive Benchmark for Multimodal Federated Graph Learning

TL;DR

MM-OpenFGL formalizes multimodal federated graph learning (MMFGL) and introduces the first comprehensive benchmark for it, integrating 19 multimodal datasets, 8 simulation strategies, 57 methods, and 9 downstream tasks. The framework supports two training pipelines—End-to-End FGL and a Two-Stage pretraining–finetuning approach—within a modular OpenFGL-like API. Through extensive experiments, the work provides 10 key insights, highlighting the limits of naive adaptations, the importance of cross-modal alignment and structural resilience, and the superior generalization of graph foundation models across diverse tasks. The benchmark and findings offer practical guidance for designing privacy-preserving, scalable MMFGL systems and come with an open-source library to foster future research and fair comparisons.

Abstract

Multimodal-attributed graphs (MMAGs) provide a unified framework for modeling complex relational data by integrating heterogeneous modalities with graph structures. While centralized learning has shown promising performance, MMAGs in real-world applications are often distributed across isolated platforms and cannot be shared due to privacy concerns or commercial constraints. Federated graph learning (FGL) offers a natural solution for collaborative training under such settings; however, existing studies largely focus on single-modality graphs and do not adequately address the challenges unique to multimodal federated graph learning (MMFGL). To bridge this gap, we present MM-OpenFGL, the first comprehensive benchmark that systematically formalizes the MMFGL paradigm and enables rigorous evaluation. MM-OpenFGL comprises 19 multimodal datasets spanning 7 application domains, 8 simulation strategies capturing modality and topology variations, 6 downstream tasks, and 57 state-of-the-art methods implemented through a modular API. Extensive experiments investigate MMFGL from the perspectives of necessity, effectiveness, robustness, and efficiency, offering valuable insights for future research on MMFGL.
Paper Structure (20 sections, 11 equations, 5 figures, 8 tables)

This paper contains 20 sections, 11 equations, 5 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Problem Statement
  3. End-to-End Pipeline
  4. Two-Stage Pipeline
  5. Benchmark Design
  6. Decentralized MMAGs Simulation
  7. Algorithm Taxonomy
  8. Experiments and Evaluations
  9. Experiments and Analysis
  10. Necessity Analysis (Answer for Q1 and Q2)
  11. Performance Comparison (Answer for Q3 and Q4)
  12. Robustness Analysis (Answer for Q5, Q6 and Q7)
  13. Efficiency Analysis
  14. Conclusion and Future Directions
  15. Appendix
  16. ...and 5 more sections

Figures (5)

  • Figure 1: Performance between GNNs using different modalities.
  • Figure 2: Best-round node classification performance for different simulations. Top: modality-NonIID, topology unavailable, label-IID; Bottom: modality-NonIID, topology available, label-NonIID.
  • Figure 3: Algorithm robustness under modality missing on DY and RedditS datasets (node classification task).
  • Figure 4: Algorithm robustness under different feature encoders for the Toys dataset (node classification task).
  • Figure 5: Robustness performance comparisons under topology/ label noise and sparsity on the Bili Movie dataset (node classification).