Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LangXAI: Integrating Large Vision Models for Generating Textual Explanations to Enhance Explainability in Visual Perception Tasks

Truong Thanh Hung Nguyen, Tobias Clement, Phuc Truong Loc Nguyen, Nils Kemmerzell, Van Binh Truong, Vo Thanh Khang Nguyen, Mohamed Abdelaal, Hung Cao

TL;DR

Preliminary results demonstrate LangXAI's enhanced plausibility, with high BERTScore across tasks, fostering a more transparent and reliable AI framework on vision tasks for end-users.

Abstract

LangXAI is a framework that integrates Explainable Artificial Intelligence (XAI) with advanced vision models to generate textual explanations for visual recognition tasks. Despite XAI advancements, an understanding gap persists for end-users with limited domain knowledge in artificial intelligence and computer vision. LangXAI addresses this by furnishing text-based explanations for classification, object detection, and semantic segmentation model outputs to end-users. Preliminary results demonstrate LangXAI's enhanced plausibility, with high BERTScore across tasks, fostering a more transparent and reliable AI framework on vision tasks for end-users.

LangXAI: Integrating Large Vision Models for Generating Textual Explanations to Enhance Explainability in Visual Perception Tasks

TL;DR

Preliminary results demonstrate LangXAI's enhanced plausibility, with high BERTScore across tasks, fostering a more transparent and reliable AI framework on vision tasks for end-users.

Abstract

LangXAI is a framework that integrates Explainable Artificial Intelligence (XAI) with advanced vision models to generate textual explanations for visual recognition tasks. Despite XAI advancements, an understanding gap persists for end-users with limited domain knowledge in artificial intelligence and computer vision. LangXAI addresses this by furnishing text-based explanations for classification, object detection, and semantic segmentation model outputs to end-users. Preliminary results demonstrate LangXAI's enhanced plausibility, with high BERTScore across tasks, fostering a more transparent and reliable AI framework on vision tasks for end-users.
Paper Structure (9 sections, 3 figures, 1 table)

This paper contains 9 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Explainable AI (XAI) in CV
  4. Large Vision Models (LVMs)
  5. Framework
  6. Block 1: Saliency Map Extraction with XAI
  7. Block 2: Text-based Explanation with LVM
  8. Evaluation
  9. Conclusion

Figures (3)

  • Figure 1: The interface of LangXAI showcases how it operates to make AI decisions in the classification task, which is designed straightforwardly with guidance so end-users can comprehend and monitor end-to-end explanations.
  • Figure 2: Classification of XAI methods by CV tasks and their mechanisms.
  • Figure 3: Our framework is split into two parts for explaining decisions made by AI models in CV tasks. The first part (in white blocks) generates saliency maps, where the XAI method in bold is used to generate the saliency map. The second part (in gray blocks) combines the input image, saliency map, ground truth, and prediction to provide a text-based explanation under prompts for each task.