Linguistic features for sentence difficulty prediction in ABSA
Adrian-Gabriel Chifu, Sébastien Fournier
TL;DR
This work tackles the lack of a formal notion of sentence difficulty in aspect-based sentiment analysis (ABSA). It combines three publicly available corpora (Laptops, Restaurants, MTSC) and a merged corpus, evaluating 21 models with TF-IDF and BERT representations to understand how domain and syntactic diversity affect difficulty. It defines two difficulty paradigms (binary and six-level) based on classifier consensus and investigates nine linguistic features to predict difficulty, finding that MTSC is the hardest and that fine-tuned BERT on the merged data yields the best performance among ABSA approaches, while difficulty prediction from the chosen features remains challenging. The results suggest that large language models require sufficient data to realize gains and motivate future work on more diverse datasets and QPP-inspired predictors for ABSA difficulty.
Abstract
One of the challenges of natural language understanding is to deal with the subjectivity of sentences, which may express opinions and emotions that add layers of complexity and nuance. Sentiment analysis is a field that aims to extract and analyze these subjective elements from text, and it can be applied at different levels of granularity, such as document, paragraph, sentence, or aspect. Aspect-based sentiment analysis is a well-studied topic with many available data sets and models. However, there is no clear definition of what makes a sentence difficult for aspect-based sentiment analysis. In this paper, we explore this question by conducting an experiment with three data sets: "Laptops", "Restaurants", and "MTSC" (Multi-Target-dependent Sentiment Classification), and a merged version of these three datasets. We study the impact of domain diversity and syntactic diversity on difficulty. We use a combination of classifiers to identify the most difficult sentences and analyze their characteristics. We employ two ways of defining sentence difficulty. The first one is binary and labels a sentence as difficult if the classifiers fail to correctly predict the sentiment polarity. The second one is a six-level scale based on how many of the top five best-performing classifiers can correctly predict the sentiment polarity. We also define 9 linguistic features that, combined, aim at estimating the difficulty at sentence level.