MMPG: MoE-based Adaptive Multi-Perspective Graph Fusion for Protein Representation Learning

TL;DR

MMPG is proposed, a framework that constructs protein graphs from multiple perspectives and adaptively fuses them via Mixture of Experts (MoE) for PRL, and quantitatively verify that MoE automatically specializes experts in modeling distinct levels of interaction from individual representations, to pairwise inter-perspective synergies, and ultimately to a global consensus across all perspectives.

Abstract

Graph Neural Networks (GNNs) have been widely adopted for Protein Representation Learning (PRL), as residue interaction networks can be naturally represented as graphs. Current GNN-based PRL methods typically rely on single-perspective graph construction strategies, which capture partial properties of residue interactions, resulting in incomplete protein representations. To address this limitation, we propose MMPG, a framework that constructs protein graphs from multiple perspectives and adaptively fuses them via Mixture of Experts (MoE) for PRL. MMPG constructs graphs from physical, chemical, and geometric perspectives to characterize different properties of residue interactions. To capture both perspective-specific features and their synergies, we develop an MoE module, which dynamically routes perspectives to specialized experts, where experts learn intrinsic features and cross-perspective interactions. We quantitatively verify that MoE automatically specializes experts in modeling distinct levels of interaction from individual representations, to pairwise inter-perspective synergies, and ultimately to a global consensus across all perspectives. Through integrating this multi-level information, MMPG produces superior protein representations and achieves advanced performance on four different downstream protein tasks.

Figures (6)

• Figure 1: Limitations of protein graph construction from a single perspective. (a) A radius-based graph misses long-range connections. (b) A chemical-bond-based graph fails to capture the association of adjacent hydrophobic residues.
• Figure 2: Overview of the proposed MMPG framework, which consists of two stages: (1) Multi-Perspective Graph Construction. Three graphs are constructed to model physical, chemical, and geometric properties of residue interaction, and (2) MoE learning scheme. Perspectives are routed to specialized experts, enabling dynamic representation learning across different perspectives.
• Figure 3: The plots show MMPG's performance as key hyperparameters for the graph construction and MoE module are varied.
• Figure 4: Expert selection frequency of MoE module for each input perspective on the FOLD task.
• Figure 5: UMAP projection of learned protein representations with quantified intra-class and inter-class distances.