Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Unsupervised Semantic Document Representation for Fine-grained Aspect-based Sentiment Analysis

Hao-Ming Fu, Pu-Jen Cheng

TL;DR

The paper tackles unsupervised document representation for fine-grained sentiment analysis by learning a document vector $v_D$ through predicting a target sentence from its context with $k$ surrounding sentences and $r$ negative samples. It introduces two CNN-based sentence encoders, a context vector $v_{cntx}$ assembled by averaging and length-adjusting, and a logit-based negative-sampling loss, combined with a document-level loss to capture both local and global relationships via $L_{total} = \alpha L_{cntx} + (1-\alpha) L_{doc}$. Inference uses the length-adjusted average of sentence vectors, enabling new documents to be encoded without retraining. Experiments on IMDB and BeerAdvocate show substantial improvements over state-of-the-art unsupervised methods for both sentiment analysis and aspect-based sentiment analysis, demonstrating strong generality and robustness of the approach. This work provides a scalable, unsupervised representation that preserves intra-sentence order while enabling effective document-level aggregation with practical impact for downstream SA tasks.

Abstract

Document representation is the core of many NLP tasks on machine understanding. A general representation learned in an unsupervised manner reserves generality and can be used for various applications. In practice, sentiment analysis (SA) has been a challenging task that is regarded to be deeply semantic-related and is often used to assess general representations. Existing methods on unsupervised document representation learning can be separated into two families: sequential ones, which explicitly take the ordering of words into consideration, and non-sequential ones, which do not explicitly do so. However, both of them suffer from their own weaknesses. In this paper, we propose a model that overcomes difficulties encountered by both families of methods. Experiments show that our model outperforms state-of-the-art methods on popular SA datasets and a fine-grained aspect-based SA by a large margin.

Learning Unsupervised Semantic Document Representation for Fine-grained Aspect-based Sentiment Analysis

TL;DR

The paper tackles unsupervised document representation for fine-grained sentiment analysis by learning a document vector through predicting a target sentence from its context with surrounding sentences and negative samples. It introduces two CNN-based sentence encoders, a context vector assembled by averaging and length-adjusting, and a logit-based negative-sampling loss, combined with a document-level loss to capture both local and global relationships via . Inference uses the length-adjusted average of sentence vectors, enabling new documents to be encoded without retraining. Experiments on IMDB and BeerAdvocate show substantial improvements over state-of-the-art unsupervised methods for both sentiment analysis and aspect-based sentiment analysis, demonstrating strong generality and robustness of the approach. This work provides a scalable, unsupervised representation that preserves intra-sentence order while enabling effective document-level aggregation with practical impact for downstream SA tasks.

Abstract

Document representation is the core of many NLP tasks on machine understanding. A general representation learned in an unsupervised manner reserves generality and can be used for various applications. In practice, sentiment analysis (SA) has been a challenging task that is regarded to be deeply semantic-related and is often used to assess general representations. Existing methods on unsupervised document representation learning can be separated into two families: sequential ones, which explicitly take the ordering of words into consideration, and non-sequential ones, which do not explicitly do so. However, both of them suffer from their own weaknesses. In this paper, we propose a model that overcomes difficulties encountered by both families of methods. Experiments show that our model outperforms state-of-the-art methods on popular SA datasets and a fine-grained aspect-based SA by a large margin.
Paper Structure (21 sections, 5 equations, 1 figure, 3 tables)

This paper contains 21 sections, 5 equations, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. The proposed model
  4. Overview
  5. Architecture
  6. model
  7. Sentence encoders
  8. Training
  9. Context loss
  10. Document loss
  11. Total loss
  12. Inference of document representation
  13. Experiments
  14. Sentiment analysis
  15. Dataset
  16. ...and 6 more sections

Figures (1)

  • Figure 1: Overview of our model. In the figure, number of context sentences on each side is 1 and number of negative samples $r$ is 2. Context sentences $s_{t-1}, s_{t+1}$are fed to the model from the bottom. The target sentence $s_t$ and negative samples $s_{neg_1}, s_{neg_2}$are fed from the top. Logit of the target sentence $l_t$and negative samples $l_{neg_1}, l_{neg_2}$are obtained in the middle. These will be used to calculate the loss.