Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cooperation of Experts: Fusing Heterogeneous Information with Large Margin

Shuo Wang, Shunyang Huang, Jinghui Yuan, Zhixiang Shen, Zhao Kang

TL;DR

This work introduces Cooperation of Experts (CoE), a framework for fusing heterogeneous information by encoding it into multiplex networks and orchestrating a cooperative, two-level set of domain-specific experts. A mutual-information-driven fusion and a large-margin mechanism enable cross-network collaboration, supported by partial convexity and Lipschitz guarantees and proven convergence. Empirically, CoE achieves state-of-the-art node classification across multiple multiplex and multimodal datasets, with robust performance under structural perturbations. The approach advances multiplex learning by shifting from expert competition to expert cooperation, demonstrating strong generalization and resilience in real-world heterogeneous data contexts.

Abstract

Fusing heterogeneous information remains a persistent challenge in modern data analysis. While significant progress has been made, existing approaches often fail to account for the inherent heterogeneity of object patterns across different semantic spaces. To address this limitation, we propose the Cooperation of Experts (CoE) framework, which encodes multi-typed information into unified heterogeneous multiplex networks. By overcoming modality and connection differences, CoE provides a powerful and flexible model for capturing the intricate structures of real-world complex data. In our framework, dedicated encoders act as domain-specific experts, each specializing in learning distinct relational patterns in specific semantic spaces. To enhance robustness and extract complementary knowledge, these experts collaborate through a novel large margin mechanism supported by a tailored optimization strategy. Rigorous theoretical analyses guarantee the framework's feasibility and stability, while extensive experiments across diverse benchmarks demonstrate its superior performance and broad applicability. Our code is available at https://github.com/strangeAlan/CoE.

Cooperation of Experts: Fusing Heterogeneous Information with Large Margin

TL;DR

This work introduces Cooperation of Experts (CoE), a framework for fusing heterogeneous information by encoding it into multiplex networks and orchestrating a cooperative, two-level set of domain-specific experts. A mutual-information-driven fusion and a large-margin mechanism enable cross-network collaboration, supported by partial convexity and Lipschitz guarantees and proven convergence. Empirically, CoE achieves state-of-the-art node classification across multiple multiplex and multimodal datasets, with robust performance under structural perturbations. The approach advances multiplex learning by shifting from expert competition to expert cooperation, demonstrating strong generalization and resilience in real-world heterogeneous data contexts.

Abstract

Fusing heterogeneous information remains a persistent challenge in modern data analysis. While significant progress has been made, existing approaches often fail to account for the inherent heterogeneity of object patterns across different semantic spaces. To address this limitation, we propose the Cooperation of Experts (CoE) framework, which encodes multi-typed information into unified heterogeneous multiplex networks. By overcoming modality and connection differences, CoE provides a powerful and flexible model for capturing the intricate structures of real-world complex data. In our framework, dedicated encoders act as domain-specific experts, each specializing in learning distinct relational patterns in specific semantic spaces. To enhance robustness and extract complementary knowledge, these experts collaborate through a novel large margin mechanism supported by a tailored optimization strategy. Rigorous theoretical analyses guarantee the framework's feasibility and stability, while extensive experiments across diverse benchmarks demonstrate its superior performance and broad applicability. Our code is available at https://github.com/strangeAlan/CoE.

Paper Structure

This paper contains 26 sections, 4 theorems, 34 equations, 5 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multiplex Network Learning
  4. Graph Structure Learning
  5. Multiple Learners Learning
  6. Preliminaries
  7. Our Proposed Method
  8. Encoding Heterogeneous Information
  9. Two-level Design for Task-driven Experts
  10. Large Margin Mechanism
  11. Theoretical Analysis
  12. Experiments
  13. Experimental Setup
  14. Node Classification Performance (RQ1)
  15. Ablation Studies (RQ2)
  16. ...and 11 more sections

Key Result

Theorem 4.1

Given a network $G$ with label $Y$, the cross-entropy loss $\mathcal{L}_{cls}(Z, Y)$ is the upper bound of $-I(G; Y)$, where $Z$ denotes the node representations of all nodes in network $G$. sun2022graphli2024gagsl

Figures (5)

  • Figure 1: (a) and (b) present the classification results on different networks from the ACM and Yelp datasets, representing the diverse and intricate patterns within networks. (c) "+" symbol denotes directly adding the networks, while "$\&$" represents the fusion procedure used in CoE.
  • Figure 2: The overall framework of the proposed CoE. Specifically, CoE first encodes various information into heterogeneous multiplex networks, followed by network fusion through mutual information maximization. Subsequently, two-level experts are trained on single and fused networks respectively. Expert collaboration is enabled by a confidence tensor, which is optimized via a large margin mechanism.
  • Figure 3: Robustness analysis on ACM.
  • Figure 4: Sensitivity analysis on critical hyper-parameters.
  • Figure 5: Sensitivity analysis on $K$.

Theorems & Definitions (8)

  • Theorem 4.1
  • Definition 4.2
  • Theorem 5.1
  • Theorem 5.2
  • Theorem 5.3
  • proof
  • proof
  • proof