On the Universal Statistical Consistency of Expansive Hyperbolic Deep Convolutional Neural Networks

TL;DR

This work proposes Hyperbolic DCNN based on the Poincar\'{e} Disc and reveals that the hyperbolic convolutional architecture outperforms the Euclidean ones by a commendable margin.

Abstract

The emergence of Deep Convolutional Neural Networks (DCNNs) has been a pervasive tool for accomplishing widespread applications in computer vision. Despite its potential capability to capture intricate patterns inside the data, the underlying embedding space remains Euclidean and primarily pursues contractive convolution. Several instances can serve as a precedent for the exacerbating performance of DCNNs. The recent advancement of neural networks in the hyperbolic spaces gained traction, incentivizing the development of convolutional deep neural networks in the hyperbolic space. In this work, we propose Hyperbolic DCNN based on the Poincaré Disc. The work predominantly revolves around analyzing the nature of expansive convolution in the context of the non-Euclidean domain. We further offer extensive theoretical insights pertaining to the universal consistency of the expansive convolution in the hyperbolic space. Several simulations were performed not only on the synthetic datasets but also on some real-world datasets. The experimental results reveal that the hyperbolic convolutional architecture outperforms the Euclidean ones by a commendable margin.

Figures (5)

• Figure 1: Test Root Mean Squared error for $f(x)$ and $g(x)$ plotted using (a) eDCNN architecture curvature $0$ (i.e., Euclidean space) and (b) eHDCNN architecture with curvature $1$.
• Figure 2: The complete workflow of expansive hyperbolic $1$-D convolutional layer on Poincarê disc is presented. (best view in digital format)
• Figure 3: The performance analysis of eHDCNN with varying space curvatures (a) for $f(x)$ and (b) for $g(x)$, and (c) House price prediction is demonstrated. The Root Mean Square Error (RMSE) decreases faster with increasing curvature, justifying the utility of applying hyperbolic convolution. (best view in digital format)
• Figure 4: The performance analysis of eHDCNN with varying space curvatures (a) for Superconductivity (b) for Wave Energy, and (c) test accuracy for WISDM are demonstrated. The Root Mean Square Error (RMSE) decreases faster for both (a) and (b) with increasing curvature. On the contrary, test accuracy increases in (c), justifying the utility of employing hyperbolic convolution. (best view in digital format)
• Figure 5: Various experiments were performed on the Superconductivity dataset by varying filter length and number of convolutional layers of the eHDCNN architecture.

Theorems & Definitions (26)

• Definition 1
• Remark 2
• Definition 3
• Remark 4
• Definition 5
• Remark 6
• Remark 7
• Lemma 8
• Lemma 9
• Definition 10