Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Divide (Text) and Conquer (Sentiment): Improved Sentiment Classification by Constituent Conflict Resolution

Jan Kościałkowski, Paweł Marcinkowski

TL;DR

This work tackles sentiment classification when passages contain conflicting tones, a setting where longer texts degrade standard models. It introduces a three-substrate framework that splits passages into constituent sentiments via sentence- and aspect-level analysis and aggregates them with an MLP atop summary features, achieving competitive or superior accuracy across SST, Amazon, and Twitter while costing roughly $ rac{1}{100}$ of full fine-tuning. The key contributions include two constituent extraction strategies (rule-based and ABSA), multiple aggregation schemes (Average, AWON, and a 19-feature MLP), and a comprehensive evaluation showing substantial gains on longer or distribution-shifted data, especially for the MLP. The approach offers a compute-efficient, interpretable proxy for larger models, enabling rapid adaptation to new domains with significant speedups, though longer passages still pose challenges and open avenues for further enhancement via latent-representation aggregation and related training paradigms measured by $O(100\times)$ improvements in efficiency.

Abstract

Sentiment classification, a complex task in natural language processing, becomes even more challenging when analyzing passages with multiple conflicting tones. Typically, longer passages exacerbate this issue, leading to decreased model performance. The aim of this paper is to introduce novel methodologies for isolating conflicting sentiments and aggregating them to effectively predict the overall sentiment of such passages. One of the aggregation strategies involves a Multi-Layer Perceptron (MLP) model which outperforms baseline models across various datasets, including Amazon, Twitter, and SST while costing $\sim$1/100 of what fine-tuning the baseline would take.

Divide (Text) and Conquer (Sentiment): Improved Sentiment Classification by Constituent Conflict Resolution

TL;DR

This work tackles sentiment classification when passages contain conflicting tones, a setting where longer texts degrade standard models. It introduces a three-substrate framework that splits passages into constituent sentiments via sentence- and aspect-level analysis and aggregates them with an MLP atop summary features, achieving competitive or superior accuracy across SST, Amazon, and Twitter while costing roughly of full fine-tuning. The key contributions include two constituent extraction strategies (rule-based and ABSA), multiple aggregation schemes (Average, AWON, and a 19-feature MLP), and a comprehensive evaluation showing substantial gains on longer or distribution-shifted data, especially for the MLP. The approach offers a compute-efficient, interpretable proxy for larger models, enabling rapid adaptation to new domains with significant speedups, though longer passages still pose challenges and open avenues for further enhancement via latent-representation aggregation and related training paradigms measured by improvements in efficiency.

Abstract

Sentiment classification, a complex task in natural language processing, becomes even more challenging when analyzing passages with multiple conflicting tones. Typically, longer passages exacerbate this issue, leading to decreased model performance. The aim of this paper is to introduce novel methodologies for isolating conflicting sentiments and aggregating them to effectively predict the overall sentiment of such passages. One of the aggregation strategies involves a Multi-Layer Perceptron (MLP) model which outperforms baseline models across various datasets, including Amazon, Twitter, and SST while costing 1/100 of what fine-tuning the baseline would take.
Paper Structure (18 sections, 6 figures, 4 tables)

This paper contains 18 sections, 6 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Prior Literature
  3. Data
  4. Model
  5. Sentiment classifiers
  6. Constituent extraction
  7. Final model architecture
  8. Methods
  9. Performance Metrics
  10. Dimensions of Analysis
  11. Optimization and Hyperparameters
  12. Results
  13. Choice of passage split level
  14. Simple aggregation models
  15. ABSA
  16. ...and 3 more sections

Figures (6)

  • Figure 1: Model accuracy vs passage length in tokens.
  • Figure 2: Accuracy of base models and Average/AWON aggregations on the SST dataset
  • Figure 3: Accuracy of base models and Average/AWON aggregations on the Amazon dataset
  • Figure 4: Accuracy of base models and Average/AWON aggregations on the Twitter dataset
  • Figure 5: Accuracy of the Polarity model applied to the whole passage (red) compared to Averaged ABSA predictions using the smaller (yellow) and larger (green) models.
  • ...and 1 more figures