Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Novel Graph-Sequence Learning Model for Inductive Text Classification

Zuo Wang, Ye Yuan

TL;DR

TextGSL addresses inductive text classification by building a single text-level graph per document with diverse edge types and integrating long-range sequential context via a Transformer. It introduces an adaptive multi-edge message-passing scheme to fuse co-occurrence, syntax, and semantic relations, and couples this with a Transformer-based sequence encoder and Bi-GRU fusion to produce discriminative graph-level representations. Empirical results on five benchmark datasets show consistent accuracy gains over strong baselines, with analyses confirming the contribution of each module and the benefit of diverse relations and inductive capability. The approach advances scalable, inductive text classification by unifying graph-structural learning with sequence modeling for robust text representations.

Abstract

Text classification plays an important role in various downstream text-related tasks, such as sentiment analysis, fake news detection, and public opinion analysis. Recently, text classification based on Graph Neural Networks (GNNs) has made significant progress due to their strong capabilities of structural relationship learning. However, these approaches still face two major limitations. First, these approaches fail to fully consider the diverse structural information across word pairs, e.g., co-occurrence, syntax, and semantics. Furthermore, they neglect sequence information in the text graph structure information learning module and can not classify texts with new words and relations. In this paper, we propose a Novel Graph-Sequence Learning Model for Inductive Text Classification (TextGSL) to address the previously mentioned issues. More specifically, we construct a single text-level graph for all words in each text and establish different edge types based on the diverse relationships between word pairs. Building upon this, we design an adaptive multi-edge message-passing paradigm to aggregate diverse structural information between word pairs. Additionally, sequential information among text data can be captured by the proposed TextGSL through the incorporation of Transformer layers. Therefore, TextGSL can learn more discriminative text representations. TextGSL has been comprehensively compared with several strong baselines. The experimental results on diverse benchmarking datasets demonstrate that TextGSL outperforms these baselines in terms of accuracy.

A Novel Graph-Sequence Learning Model for Inductive Text Classification

TL;DR

TextGSL addresses inductive text classification by building a single text-level graph per document with diverse edge types and integrating long-range sequential context via a Transformer. It introduces an adaptive multi-edge message-passing scheme to fuse co-occurrence, syntax, and semantic relations, and couples this with a Transformer-based sequence encoder and Bi-GRU fusion to produce discriminative graph-level representations. Empirical results on five benchmark datasets show consistent accuracy gains over strong baselines, with analyses confirming the contribution of each module and the benefit of diverse relations and inductive capability. The approach advances scalable, inductive text classification by unifying graph-structural learning with sequence modeling for robust text representations.

Abstract

Text classification plays an important role in various downstream text-related tasks, such as sentiment analysis, fake news detection, and public opinion analysis. Recently, text classification based on Graph Neural Networks (GNNs) has made significant progress due to their strong capabilities of structural relationship learning. However, these approaches still face two major limitations. First, these approaches fail to fully consider the diverse structural information across word pairs, e.g., co-occurrence, syntax, and semantics. Furthermore, they neglect sequence information in the text graph structure information learning module and can not classify texts with new words and relations. In this paper, we propose a Novel Graph-Sequence Learning Model for Inductive Text Classification (TextGSL) to address the previously mentioned issues. More specifically, we construct a single text-level graph for all words in each text and establish different edge types based on the diverse relationships between word pairs. Building upon this, we design an adaptive multi-edge message-passing paradigm to aggregate diverse structural information between word pairs. Additionally, sequential information among text data can be captured by the proposed TextGSL through the incorporation of Transformer layers. Therefore, TextGSL can learn more discriminative text representations. TextGSL has been comprehensively compared with several strong baselines. The experimental results on diverse benchmarking datasets demonstrate that TextGSL outperforms these baselines in terms of accuracy.
Paper Structure (20 sections, 7 equations, 4 figures, 3 tables)

This paper contains 20 sections, 7 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. Classical Machine Learning-Based Methods
  4. Deeping Learning-Based Methods
  5. Graph Neural Network-Based Methods
  6. Formulation OF TEXTGSL
  7. Nations
  8. Long-range Sequential Information Learning
  9. Diverse Structural Information Learning
  10. Fusion and Readout Phase
  11. EXPERIMENTS
  12. Experimental Setup
  13. Datasets
  14. Baselines
  15. Experiment Settings
  16. ...and 5 more sections

Figures (4)

  • Figure 1: The overall framework of the proposed TextGSL for inductive text classification. (a) is the data processing phase, which converts text into a sequence of words. (b) is the long-range sequential information learning module. (c) is the diverse structural information learning module. (d) integrates diverse structural information with local sequence information, and all nodes aggregate to the ultimate document graph representation with an attention mechanism in the readout phase.
  • Figure 2: Text-level graph construction based on co-occurrence, syntax, and semantics relationships.
  • Figure 3: Visualization of adaptive parameters for co-occurrence, syntax, and semantic relations across five text datasets. Specifically, for long-text datasets, co-occurrence relations are more important for learning more discriminative text representations.
  • Figure 4: Test accuracy by varying training set ratio