Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards LLM-guided Causal Explainability for Black-box Text Classifiers

Amrita Bhattacharjee, Raha Moraffah, Joshua Garland, Huan Liu

TL;DR

This work introduces a three-step pipeline that uses instruction-tuned LLMs to achieve causal explainability for black-box text classifiers by first uncovering latent unobserved features, then linking them to specific input tokens, and finally generating minimal counterfactual edits to flip predictions. By evaluating across IMDB, AG News, and SNLI with multiple LLMs, the study demonstrates that GPT-4 offers the strongest performance in producing effective and plausible counterfactual explanations, though trade-offs exist between flipping accuracy and content preservation. The findings suggest that latent feature extraction enhances the quality of explanations and that a fully-LMM-driven, three-step process can produce high-quality, causally informative counterfactuals, paving the way for broader applications in causal explainability and NLP reasoning. The work highlights practical implications for auditing and interpreting black-box NLP models in safety- and impact-sensitive applications.

Abstract

With the advent of larger and more complex deep learning models, such as in Natural Language Processing (NLP), model qualities like explainability and interpretability, albeit highly desirable, are becoming harder challenges to tackle and solve. For example, state-of-the-art models in text classification are black-box by design. Although standard explanation methods provide some degree of explainability, these are mostly correlation-based methods and do not provide much insight into the model. The alternative of causal explainability is more desirable to achieve but extremely challenging in NLP due to a variety of reasons. Inspired by recent endeavors to utilize Large Language Models (LLMs) as experts, in this work, we aim to leverage the instruction-following and textual understanding capabilities of recent state-of-the-art LLMs to facilitate causal explainability via counterfactual explanation generation for black-box text classifiers. To do this, we propose a three-step pipeline via which, we use an off-the-shelf LLM to: (1) identify the latent or unobserved features in the input text, (2) identify the input features associated with the latent features, and finally (3) use the identified input features to generate a counterfactual explanation. We experiment with our pipeline on multiple NLP text classification datasets, with several recent LLMs, and present interesting and promising findings.

Towards LLM-guided Causal Explainability for Black-box Text Classifiers

TL;DR

This work introduces a three-step pipeline that uses instruction-tuned LLMs to achieve causal explainability for black-box text classifiers by first uncovering latent unobserved features, then linking them to specific input tokens, and finally generating minimal counterfactual edits to flip predictions. By evaluating across IMDB, AG News, and SNLI with multiple LLMs, the study demonstrates that GPT-4 offers the strongest performance in producing effective and plausible counterfactual explanations, though trade-offs exist between flipping accuracy and content preservation. The findings suggest that latent feature extraction enhances the quality of explanations and that a fully-LMM-driven, three-step process can produce high-quality, causally informative counterfactuals, paving the way for broader applications in causal explainability and NLP reasoning. The work highlights practical implications for auditing and interpreting black-box NLP models in safety- and impact-sensitive applications.

Abstract

With the advent of larger and more complex deep learning models, such as in Natural Language Processing (NLP), model qualities like explainability and interpretability, albeit highly desirable, are becoming harder challenges to tackle and solve. For example, state-of-the-art models in text classification are black-box by design. Although standard explanation methods provide some degree of explainability, these are mostly correlation-based methods and do not provide much insight into the model. The alternative of causal explainability is more desirable to achieve but extremely challenging in NLP due to a variety of reasons. Inspired by recent endeavors to utilize Large Language Models (LLMs) as experts, in this work, we aim to leverage the instruction-following and textual understanding capabilities of recent state-of-the-art LLMs to facilitate causal explainability via counterfactual explanation generation for black-box text classifiers. To do this, we propose a three-step pipeline via which, we use an off-the-shelf LLM to: (1) identify the latent or unobserved features in the input text, (2) identify the input features associated with the latent features, and finally (3) use the identified input features to generate a counterfactual explanation. We experiment with our pipeline on multiple NLP text classification datasets, with several recent LLMs, and present interesting and promising findings.
Paper Structure (15 sections, 1 figure, 3 tables)

This paper contains 15 sections, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Background & Related Work
  3. Causal and Counterfactual Explanations:
  4. LLMs as Experts or Feature Extractors:
  5. Method: Extracting Counterfactual Explanations via LLMs
  6. Experimental Settings
  7. Datasets
  8. LLMs
  9. Evaluation Metrics
  10. Results
  11. Effectiveness of Generated Counterfactual Explanations
  12. Ablation: Effectiveness of Three-Step Pipeline
  13. Quality of Latent Features Extracted
  14. Conclusion
  15. Acknowledgments

Figures (1)

  • Figure 1: Our proposed three-step pipeline that leverages LLMs to (1) identify latent unobserved features, (2) identify input features associated with those latent features, and finally (3) use the input features to generate a counterfactual explanation.