Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Statistical Measures for Explainable Aspect-Based Sentiment Analysis: A Case Study on Environmental Discourse in Reddit

Luisa Stracqualursi, Patrizia Agati

TL;DR

A statistical, model-agnostic framework to assess the behavioral transparency and trustworthiness of ABSA models, which relies on several metrics, such as the entropy of polarity distributions, soft-count-based dominance scores, and sentiment divergence between sources, whose robustness is validated through bootstrap resampling and sensitivity analysis.

Abstract

Aspect-Based Sentiment Analysis (ABSA) provides a fine-grained understanding of opinions by linking sentiment to specific aspects in text. While transformer-based models excel at this task, their black-box nature limits their interpretability, posing risks in real-world applications without labeled data. This paper introduces a statistical, model-agnostic framework to assess the behavioral transparency and trustworthiness of ABSA models. Our framework relies on several metrics, such as the entropy of polarity distributions, soft-count-based dominance scores, and sentiment divergence between sources, whose robustness is validated through bootstrap resampling and sensitivity analysis. A case study on environmentally focused Reddit communities illustrates how the proposed indicators provide interpretable diagnostics of model certainty, decisiveness, and cross-source variability. The results show that statistical indicators computed on soft outputs can complement traditional approaches, offering a computationally efficient methodology for validating, monitoring, and interpreting ABSA models in contexts where labeled data are unavailable.

Statistical Measures for Explainable Aspect-Based Sentiment Analysis: A Case Study on Environmental Discourse in Reddit

TL;DR

A statistical, model-agnostic framework to assess the behavioral transparency and trustworthiness of ABSA models, which relies on several metrics, such as the entropy of polarity distributions, soft-count-based dominance scores, and sentiment divergence between sources, whose robustness is validated through bootstrap resampling and sensitivity analysis.

Abstract

Aspect-Based Sentiment Analysis (ABSA) provides a fine-grained understanding of opinions by linking sentiment to specific aspects in text. While transformer-based models excel at this task, their black-box nature limits their interpretability, posing risks in real-world applications without labeled data. This paper introduces a statistical, model-agnostic framework to assess the behavioral transparency and trustworthiness of ABSA models. Our framework relies on several metrics, such as the entropy of polarity distributions, soft-count-based dominance scores, and sentiment divergence between sources, whose robustness is validated through bootstrap resampling and sensitivity analysis. A case study on environmentally focused Reddit communities illustrates how the proposed indicators provide interpretable diagnostics of model certainty, decisiveness, and cross-source variability. The results show that statistical indicators computed on soft outputs can complement traditional approaches, offering a computationally efficient methodology for validating, monitoring, and interpreting ABSA models in contexts where labeled data are unavailable.
Paper Structure (26 sections, 8 equations, 2 figures, 4 tables)

This paper contains 26 sections, 8 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Stage 1: Aspect Extraction and Probabilistic Sentiment Prediction
  5. Stage 2: Computation of Statistical Indicators
  6. Illustrative Example.
  7. Polarity Entropy $H(A)$.
  8. Sentiment Dominance $D(A)$.
  9. Formal Definitions.
  10. Illustrative Example.
  11. Sentiment Divergence $JSD(A)$.
  12. Stage 3: Robustness and Stability Assessment
  13. Confidence-Based Filtering and Robustness Assessment.
  14. Bootstrap Resampling for Stability Assessment.
  15. Alignment with the SAFE Framework
  16. ...and 11 more sections

Figures (2)

  • Figure 1: Top 10 ABSA results, Confidence vs Dominance/Entropy
  • Figure 2: Explainability-Robustness Tradeoff via Confidence Filtering