Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Node-wise Filtering in Graph Neural Networks: A Mixture of Experts Approach

Haoyu Han, Juanhui Li, Wei Huang, Xianfeng Tang, Hanqing Lu, Chen Luo, Hui Liu, Jiliang Tang

TL;DR

The paper addresses node classification on graphs with mixed homophilic and heterophilic patterns, showing that a single global filter can underperform across nodes. It introduces Node-MoE, a mixture-of-experts framework that assigns different filters to individual nodes via a gating model informed by node features and local context, with diverse expert GNNs and a filter-smoothing loss. Theoretical insights based on a mixed CSBM model demonstrate that per-pattern filtering yields near-linear separability for all nodes, while comprehensive experiments across seven datasets establish robust, state-of-the-art performance on both homophilic and heterophilic graphs. This approach offers a scalable, adaptable solution for complex real-world graphs where structural patterns vary across communities and nodes, advancing practical graph representation learning.

Abstract

Graph Neural Networks (GNNs) have proven to be highly effective for node classification tasks across diverse graph structural patterns. Traditionally, GNNs employ a uniform global filter, typically a low-pass filter for homophilic graphs and a high-pass filter for heterophilic graphs. However, real-world graphs often exhibit a complex mix of homophilic and heterophilic patterns, rendering a single global filter approach suboptimal. In this work, we theoretically demonstrate that a global filter optimized for one pattern can adversely affect performance on nodes with differing patterns. To address this, we introduce a novel GNN framework Node-MoE that utilizes a mixture of experts to adaptively select the appropriate filters for different nodes. Extensive experiments demonstrate the effectiveness of Node-MoE on both homophilic and heterophilic graphs.

Node-wise Filtering in Graph Neural Networks: A Mixture of Experts Approach

TL;DR

The paper addresses node classification on graphs with mixed homophilic and heterophilic patterns, showing that a single global filter can underperform across nodes. It introduces Node-MoE, a mixture-of-experts framework that assigns different filters to individual nodes via a gating model informed by node features and local context, with diverse expert GNNs and a filter-smoothing loss. Theoretical insights based on a mixed CSBM model demonstrate that per-pattern filtering yields near-linear separability for all nodes, while comprehensive experiments across seven datasets establish robust, state-of-the-art performance on both homophilic and heterophilic graphs. This approach offers a scalable, adaptable solution for complex real-world graphs where structural patterns vary across communities and nodes, advancing practical graph representation learning.

Abstract

Graph Neural Networks (GNNs) have proven to be highly effective for node classification tasks across diverse graph structural patterns. Traditionally, GNNs employ a uniform global filter, typically a low-pass filter for homophilic graphs and a high-pass filter for heterophilic graphs. However, real-world graphs often exhibit a complex mix of homophilic and heterophilic patterns, rendering a single global filter approach suboptimal. In this work, we theoretically demonstrate that a global filter optimized for one pattern can adversely affect performance on nodes with differing patterns. To address this, we introduce a novel GNN framework Node-MoE that utilizes a mixture of experts to adaptively select the appropriate filters for different nodes. Extensive experiments demonstrate the effectiveness of Node-MoE on both homophilic and heterophilic graphs.
Paper Structure (27 sections, 1 theorem, 8 equations, 16 figures, 2 tables)

This paper contains 27 sections, 1 theorem, 8 equations, 16 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Structural Patterns in Existing Graphs
  4. Analysis based on CSBM model
  5. The Proposed Method
  6. Node-MoE : Node-wise Filtering via Mixture of Experts
  7. Gating Model
  8. Expert Models
  9. Top-K gating
  10. Experiment
  11. Experimental settings.
  12. Performance Comparison on Benchmark Datasets
  13. Analysis of Node-MoE
  14. Ablation studies
  15. Related Works
  16. ...and 12 more sections

Key Result

Theorem 1

Suppose $n$ is relatively large, the graph is not too sparse with $p_i, q_i = \omega(\log ^2(n) / n)$ and the feature center distance is not too small with $\|\boldsymbol{\mu}-\boldsymbol{\nu}\|=\omega(\frac{\log n}{\sqrt{\operatorname{dn}(p_0+q_0)}})$ and $\|{\mathbf{w}}\| \leq R$. For the graph $G 2. If different filters are applied to homophilic and heterophilic sets separately, we can find an

Figures (16)

  • Figure 1: A toy example to illustrate the effect of global and node-wise filters. The node color represents features, and the number indicates the labels.
  • Figure 2: Node homophily density.
  • Figure 3: Homophily in different communities.
  • Figure 4: The overall framework of the proposed Node-MoE . For each node, the gating model will assign different weights for each expert based on the node's feature and context. The experts can be any GNNs with different filters.
  • Figure 5: Learned 2 filters by Node-MoE on Chameleon.
  • ...and 11 more figures

Theorems & Definitions (4)

  • Definition 1
  • Theorem 1
  • proof
  • proof