Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Can LLMs faithfully generate their layperson-understandable 'self'?: A Case Study in High-Stakes Domains

Arion Das, Asutosh Mishra, Amitesh Patel, Soumilya De, V. Gurucharan, Kripabandhu Ghosh

TL;DR

This paper introduces ReQuesting, a prompt-based regime that produces layperson-friendly, algorithmic representations of LLM working to enhance explainability in law, health, and finance. By coupling Task Prompts with ReQuest Prompts and Robustness Check Prompts, the authors extract a ReQuest Algorithm whose faithfulness is evaluated via novel reproducibility metrics (PerRR and PreRR) across intra- and inter-LLM setups. Empirical results in statute prediction and human rights violation prediction show high reproducibility, particularly within single models, while finance tasks reveal strong intra-LLM reproducibility and meaningful cross-model transfer. Health-domain results indicate strong performance for certain tasks but also highlight variability and domain dependence, prompting a discussion on intrinsic reasoning alignment and future research directions to further validate the approach and extend its applicability.

Abstract

Large Language Models (LLMs) have significantly impacted nearly every domain of human knowledge. However, the explainability of these models esp. to laypersons, which are crucial for instilling trust, have been examined through various skeptical lenses. In this paper, we introduce a novel notion of LLM explainability to laypersons, termed $\textit{ReQuesting}$, across three high-priority application domains -- law, health and finance, using multiple state-of-the-art LLMs. The proposed notion exhibits faithful generation of explainable layman-understandable algorithms on multiple tasks through high degree of reproducibility. Furthermore, we observe a notable alignment of the explainable algorithms with intrinsic reasoning of the LLMs.

Can LLMs faithfully generate their layperson-understandable 'self'?: A Case Study in High-Stakes Domains

TL;DR

This paper introduces ReQuesting, a prompt-based regime that produces layperson-friendly, algorithmic representations of LLM working to enhance explainability in law, health, and finance. By coupling Task Prompts with ReQuest Prompts and Robustness Check Prompts, the authors extract a ReQuest Algorithm whose faithfulness is evaluated via novel reproducibility metrics (PerRR and PreRR) across intra- and inter-LLM setups. Empirical results in statute prediction and human rights violation prediction show high reproducibility, particularly within single models, while finance tasks reveal strong intra-LLM reproducibility and meaningful cross-model transfer. Health-domain results indicate strong performance for certain tasks but also highlight variability and domain dependence, prompting a discussion on intrinsic reasoning alignment and future research directions to further validate the approach and extend its applicability.

Abstract

Large Language Models (LLMs) have significantly impacted nearly every domain of human knowledge. However, the explainability of these models esp. to laypersons, which are crucial for instilling trust, have been examined through various skeptical lenses. In this paper, we introduce a novel notion of LLM explainability to laypersons, termed , across three high-priority application domains -- law, health and finance, using multiple state-of-the-art LLMs. The proposed notion exhibits faithful generation of explainable layman-understandable algorithms on multiple tasks through high degree of reproducibility. Furthermore, we observe a notable alignment of the explainable algorithms with intrinsic reasoning of the LLMs.

Paper Structure

This paper contains 24 sections, 2 equations, 6 figures, 32 tables.

Table of Contents

  1. Introduction
  2. RQ1: Can the LLMs generate a faithful layperson-understandable algorithmic representation of the working mechanism?
  3. ReQuest: Faithfulness through Reproducibility
  4. Evaluation measures
  5. Legal Tasks
  6. Statute Prediction
  7. Human Rights Violation Prediction
  8. Methods
  9. Results
  10. Finance Tasks
  11. Results
  12. Health tasks
  13. Prompting
  14. Results
  15. RQ3: Does the ReQuest algorithm align with the "intrinsic reasoning" of the LLM?
  16. ...and 9 more sections

Figures (6)

  • Figure 1: A schematic representing the ReQuesting technique.
  • Figure 2: A detailed schematic depicting the workflow utilized in this study.
  • Figure 3: PerRR values across tasks and LLMs. The consistently high values demonstrate high reproducibility and hence high degree of faithfulness of the ReQuest paradigm.
  • Figure 4: An example of the novel prompt regime for the health domain.
  • Figure 5: Distribution showing no of possible unique prediction sets vs Macro-F1 for a balanced binary dataset of size 6. The highest ambiguity in Macro-F1 score is towards the centre.
  • ...and 1 more figures