Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A C-LSTM Neural Network for Text Classification

Chunting Zhou, Chonglin Sun, Zhiyuan Liu, Francis C. M. Lau

TL;DR

The paper presents C-LSTM, a unified architecture that couples a one-layer CNN for local n-gram feature extraction with an LSTM to model sequential dependencies among the resulting window representations. This end-to-end model is evaluated on SST sentiment and TREC question classification, where it outperforms single CNN/LSTM baselines and remains competitive with parse-tree–based methods, demonstrating strong long-range dependency capture without linguistic annotations. Key contributions include demonstration of the effectiveness of feeding sequences of higher-level CNN features into an LSTM, analysis of filter-size configurations, and showing robustness across different text classification tasks. The work suggests that richer, structured higher-level representations can significantly boost sentence-level classification performance and points to future extensions with tensor-based or tree-structured convolutional components.

Abstract

Neural network models have been demonstrated to be capable of achieving remarkable performance in sentence and document modeling. Convolutional neural network (CNN) and recurrent neural network (RNN) are two mainstream architectures for such modeling tasks, which adopt totally different ways of understanding natural languages. In this work, we combine the strengths of both architectures and propose a novel and unified model called C-LSTM for sentence representation and text classification. C-LSTM utilizes CNN to extract a sequence of higher-level phrase representations, and are fed into a long short-term memory recurrent neural network (LSTM) to obtain the sentence representation. C-LSTM is able to capture both local features of phrases as well as global and temporal sentence semantics. We evaluate the proposed architecture on sentiment classification and question classification tasks. The experimental results show that the C-LSTM outperforms both CNN and LSTM and can achieve excellent performance on these tasks.

A C-LSTM Neural Network for Text Classification

TL;DR

The paper presents C-LSTM, a unified architecture that couples a one-layer CNN for local n-gram feature extraction with an LSTM to model sequential dependencies among the resulting window representations. This end-to-end model is evaluated on SST sentiment and TREC question classification, where it outperforms single CNN/LSTM baselines and remains competitive with parse-tree–based methods, demonstrating strong long-range dependency capture without linguistic annotations. Key contributions include demonstration of the effectiveness of feeding sequences of higher-level CNN features into an LSTM, analysis of filter-size configurations, and showing robustness across different text classification tasks. The work suggests that richer, structured higher-level representations can significantly boost sentence-level classification performance and points to future extensions with tensor-based or tree-structured convolutional components.

Abstract

Neural network models have been demonstrated to be capable of achieving remarkable performance in sentence and document modeling. Convolutional neural network (CNN) and recurrent neural network (RNN) are two mainstream architectures for such modeling tasks, which adopt totally different ways of understanding natural languages. In this work, we combine the strengths of both architectures and propose a novel and unified model called C-LSTM for sentence representation and text classification. C-LSTM utilizes CNN to extract a sequence of higher-level phrase representations, and are fed into a long short-term memory recurrent neural network (LSTM) to obtain the sentence representation. C-LSTM is able to capture both local features of phrases as well as global and temporal sentence semantics. We evaluate the proposed architecture on sentiment classification and question classification tasks. The experimental results show that the C-LSTM outperforms both CNN and LSTM and can achieve excellent performance on these tasks.

Paper Structure

This paper contains 16 sections, 5 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. C-LSTM Model
  4. N-gram Feature Extraction through Convolution
  5. Long Short-Term Memory Networks
  6. Learning C-LSTM for Text Classification
  7. Padding and Word Vector Initialization
  8. Regularization
  9. Experiments
  10. Datasets
  11. Experimental Settings
  12. Results and Model Analysis
  13. Sentiment Classification
  14. Question Type Classification
  15. Model Analysis
  16. ...and 1 more sections

Figures (2)

  • Figure 1: The architecture of C-LSTM for sentence modeling. Blocks of the same color in the feature map layer and window feature sequence layer corresponds to features for the same window. The dashed lines connect the feature of a window with the source feature map. The final output of the entire model is the last hidden unit of LSTM.
  • Figure 2: Prediction accuracies on TREC questions with different filter size strategies. For the horizontal axis, S means single convolutional layer with the same filter length, and M means multiple convolutional layers in parallel with different filter lengths.