Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HyCubE: Efficient Knowledge Hypergraph 3D Circular Convolutional Embedding

Zhao Li, Xin Wang, Jun Zhao, Wenbin Guo, Jianxin Li

TL;DR

An end-to-end efficient knowledge hypergraph embedding model, HyCubE, is proposed, which designs a novel 3D circular convolutional neural network and the alternate mask stack strategy to enhance the interaction and extraction of feature information comprehensively and achieves a better trade-off between effectiveness and efficiency.

Abstract

Knowledge hypergraph embedding models are usually computationally expensive due to the inherent complex semantic information. However, existing works mainly focus on improving the effectiveness of knowledge hypergraph embedding, making the model architecture more complex and redundant. It is desirable and challenging for knowledge hypergraph embedding to reach a trade-off between model effectiveness and efficiency. In this paper, we propose an end-to-end efficient knowledge hypergraph embedding model, HyCubE, which designs a novel 3D circular convolutional neural network and the alternate mask stack strategy to enhance the interaction and extraction of feature information comprehensively. Furthermore, our proposed model achieves a better trade-off between effectiveness and efficiency by adaptively adjusting the 3D circular convolutional layer structure to handle n-ary knowledge tuples of different arities with fewer parameters. In addition, we use a knowledge hypergraph 1-N multilinear scoring way to accelerate the model training efficiency further. Finally, extensive experimental results on all datasets demonstrate that our proposed model consistently outperforms state-of-the-art baselines, with an average improvement of 8.22% and a maximum improvement of 33.82% across all metrics. Meanwhile, HyCubE is 6.12x faster, GPU memory usage is 52.67% lower, and the number of parameters is reduced by 85.21% compared with the average metric of the latest state-of-the-art baselines.

HyCubE: Efficient Knowledge Hypergraph 3D Circular Convolutional Embedding

TL;DR

An end-to-end efficient knowledge hypergraph embedding model, HyCubE, is proposed, which designs a novel 3D circular convolutional neural network and the alternate mask stack strategy to enhance the interaction and extraction of feature information comprehensively and achieves a better trade-off between effectiveness and efficiency.

Abstract

Knowledge hypergraph embedding models are usually computationally expensive due to the inherent complex semantic information. However, existing works mainly focus on improving the effectiveness of knowledge hypergraph embedding, making the model architecture more complex and redundant. It is desirable and challenging for knowledge hypergraph embedding to reach a trade-off between model effectiveness and efficiency. In this paper, we propose an end-to-end efficient knowledge hypergraph embedding model, HyCubE, which designs a novel 3D circular convolutional neural network and the alternate mask stack strategy to enhance the interaction and extraction of feature information comprehensively. Furthermore, our proposed model achieves a better trade-off between effectiveness and efficiency by adaptively adjusting the 3D circular convolutional layer structure to handle n-ary knowledge tuples of different arities with fewer parameters. In addition, we use a knowledge hypergraph 1-N multilinear scoring way to accelerate the model training efficiency further. Finally, extensive experimental results on all datasets demonstrate that our proposed model consistently outperforms state-of-the-art baselines, with an average improvement of 8.22% and a maximum improvement of 33.82% across all metrics. Meanwhile, HyCubE is 6.12x faster, GPU memory usage is 52.67% lower, and the number of parameters is reduced by 85.21% compared with the average metric of the latest state-of-the-art baselines.
Paper Structure (30 sections, 19 equations, 10 figures, 7 tables, 1 algorithm)

This paper contains 30 sections, 19 equations, 10 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Tranlation-Based Methods
  4. Semantic Matching Methods
  5. Neural Network Methods
  6. Problem Statement
  7. Knowledge Hypergraph
  8. Knowledge Hypergraph Embedding
  9. Methodology
  10. Feature Alternate Mask Stack
  11. 3D Circular Convolutional Embedding
  12. Model Training
  13. Model Analysis and Variants
  14. 2D Convolutional Embedding Variant
  15. 3D Residual-Enhanced Convolutional Embedding Variant
  16. ...and 15 more sections

Figures (10)

  • Figure 1: Examples of real-world knowledge hypergraphs.
  • Figure 2: 3D convolutional embedding vs. traditional 1D/2D convolutional embedding in feature interaction and extraction.
  • Figure 3: The overall framework of the HyCubE model.
  • Figure 4: A front view of the 3D circular padding.
  • Figure 5: A comparison diagram of traditional embedding and end-to-end embedding of knowledge hypergraphs.
  • ...and 5 more figures