Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Residual Hyperbolic Graph Convolution Networks

Yangkai Xue, Jindou Dai, Zhipeng Lu, Yuwei Wu, Yunde Jia

TL;DR

This work addresses over-smoothing in deep hyperbolic graph neural networks by introducing Residual Hyperbolic Graph Convolution Networks (R-HGCNs). It combines a hyperbolic residual connection with hyperbolic identity mapping, multi-component product manifolds with distinct origins, and HyperDrop regularization to enable deeper hyperbolic models while preserving geometry. Theoretical analysis via a hyperbolic Dirichlet energy and extensive experiments on Pubmed, Citeseer, Cora, and Airports demonstrate improved performance and generalization over existing Euclidean GCNs and HGCNs, especially in deep architectures. The approach advances hyperbolic representation learning for hierarchical graphs with practical gains in node classification tasks and robust regularization in hyperbolic spaces.

Abstract

Hyperbolic graph convolutional networks (HGCNs) have demonstrated representational capabilities of modeling hierarchical-structured graphs. However, as in general GCNs, over-smoothing may occur as the number of model layers increases, limiting the representation capabilities of most current HGCN models. In this paper, we propose residual hyperbolic graph convolutional networks (R-HGCNs) to address the over-smoothing problem. We introduce a hyperbolic residual connection function to overcome the over-smoothing problem, and also theoretically prove the effectiveness of the hyperbolic residual function. Moreover, we use product manifolds and HyperDrop to facilitate the R-HGCNs. The distinctive features of the R-HGCNs are as follows: (1) The hyperbolic residual connection preserves the initial node information in each layer and adds a hyperbolic identity mapping to prevent node features from being indistinguishable. (2) Product manifolds in R-HGCNs have been set up with different origin points in different components to facilitate the extraction of feature information from a wider range of perspectives, which enhances the representing capability of R-HGCNs. (3) HyperDrop adds multiplicative Gaussian noise into hyperbolic representations, such that perturbations can be added to alleviate the over-fitting problem without deconstructing the hyperbolic geometry. Experiment results demonstrate the effectiveness of R-HGCNs under various graph convolution layers and different structures of product manifolds.

Residual Hyperbolic Graph Convolution Networks

TL;DR

This work addresses over-smoothing in deep hyperbolic graph neural networks by introducing Residual Hyperbolic Graph Convolution Networks (R-HGCNs). It combines a hyperbolic residual connection with hyperbolic identity mapping, multi-component product manifolds with distinct origins, and HyperDrop regularization to enable deeper hyperbolic models while preserving geometry. Theoretical analysis via a hyperbolic Dirichlet energy and extensive experiments on Pubmed, Citeseer, Cora, and Airports demonstrate improved performance and generalization over existing Euclidean GCNs and HGCNs, especially in deep architectures. The approach advances hyperbolic representation learning for hierarchical graphs with practical gains in node classification tasks and robust regularization in hyperbolic spaces.

Abstract

Hyperbolic graph convolutional networks (HGCNs) have demonstrated representational capabilities of modeling hierarchical-structured graphs. However, as in general GCNs, over-smoothing may occur as the number of model layers increases, limiting the representation capabilities of most current HGCN models. In this paper, we propose residual hyperbolic graph convolutional networks (R-HGCNs) to address the over-smoothing problem. We introduce a hyperbolic residual connection function to overcome the over-smoothing problem, and also theoretically prove the effectiveness of the hyperbolic residual function. Moreover, we use product manifolds and HyperDrop to facilitate the R-HGCNs. The distinctive features of the R-HGCNs are as follows: (1) The hyperbolic residual connection preserves the initial node information in each layer and adds a hyperbolic identity mapping to prevent node features from being indistinguishable. (2) Product manifolds in R-HGCNs have been set up with different origin points in different components to facilitate the extraction of feature information from a wider range of perspectives, which enhances the representing capability of R-HGCNs. (3) HyperDrop adds multiplicative Gaussian noise into hyperbolic representations, such that perturbations can be added to alleviate the over-fitting problem without deconstructing the hyperbolic geometry. Experiment results demonstrate the effectiveness of R-HGCNs under various graph convolution layers and different structures of product manifolds.

Paper Structure

This paper contains 23 sections, 1 theorem, 30 equations, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Hyperbolic Manifold.
  5. Lorentz Model.
  6. Exponential and logarithmic maps.
  7. Parallel Transport.
  8. Method
  9. Residual Hyperbolic Graph Convolution
  10. Lorentz Operations.
  11. Residual Hyperbolic Graph Convolution.
  12. Effectiveness of Hyperbolic Residual Connection
  13. Product Manifold
  14. Hyperbolic Dropout
  15. Residual Hyperbolic Graph Convolutional Network
  16. ...and 8 more sections

Key Result

Lemma 1

If $X=(X_{ij})$ is a $n\times n$ weight matrix, i.e. $\sum_{j=1}^nX_{ij}=1, X_{ij}\geq 0$, then for any $u\in\mathbb{R}^n$ with $\|u\|_2=1$, $\|Xu\|_2\leq\sqrt{n}$.

Theorems & Definitions (6)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Definition 5
  • Lemma 1