Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Who Sees the Risk? Stakeholder Conflicts and Explanatory Policies in LLM-based Risk Assessment

Srishti Yadav, Jasmina Gajcin, Erik Miehling, Elizabeth Daly

TL;DR

The paper tackles the problem of stakeholder misalignment in AI risk assessment by introducing a stakeholder-grounded pipeline where LLMs act as judges to predict and explain risks through stakeholder-specific perspectives. It integrates Risk Atlas Nexus for risk prediction with the GloVE explanation framework to produce policy-like explanations that reveal where stakeholders converge or diverge on risks. The authors validate the approach on three real-world use cases (medical diagnosis, autonomous vehicles, fraud detection), demonstrating how stakeholder contexts shape risk perception and conflict patterns, and they offer an interactive visualization to enhance transparency of the conflict reasoning. This work advances human-centered AI governance by enabling transparent, interpretable, and auditable LLM-based risk evaluations that account for diverse stakeholder concerns.

Abstract

Understanding how different stakeholders perceive risks in AI systems is essential for their responsible deployment. This paper presents a framework for stakeholder-grounded risk assessment by using LLMs, acting as judges to predict and explain risks. Using the Risk Atlas Nexus and GloVE explanation method, our framework generates stakeholder-specific, interpretable policies that shows how different stakeholders agree or disagree about the same risks. We demonstrate our method using three real-world AI use cases of medical AI, autonomous vehicles, and fraud detection domain. We further propose an interactive visualization that reveals how and why conflicts emerge across stakeholder perspectives, enhancing transparency in conflict reasoning. Our results show that stakeholder perspectives significantly influence risk perception and conflict patterns. Our work emphasizes the importance of these stakeholder-aware explanations needed to make LLM-based evaluations more transparent, interpretable, and aligned with human-centered AI governance goals.

Who Sees the Risk? Stakeholder Conflicts and Explanatory Policies in LLM-based Risk Assessment

TL;DR

The paper tackles the problem of stakeholder misalignment in AI risk assessment by introducing a stakeholder-grounded pipeline where LLMs act as judges to predict and explain risks through stakeholder-specific perspectives. It integrates Risk Atlas Nexus for risk prediction with the GloVE explanation framework to produce policy-like explanations that reveal where stakeholders converge or diverge on risks. The authors validate the approach on three real-world use cases (medical diagnosis, autonomous vehicles, fraud detection), demonstrating how stakeholder contexts shape risk perception and conflict patterns, and they offer an interactive visualization to enhance transparency of the conflict reasoning. This work advances human-centered AI governance by enabling transparent, interpretable, and auditable LLM-based risk evaluations that account for diverse stakeholder concerns.

Abstract

Understanding how different stakeholders perceive risks in AI systems is essential for their responsible deployment. This paper presents a framework for stakeholder-grounded risk assessment by using LLMs, acting as judges to predict and explain risks. Using the Risk Atlas Nexus and GloVE explanation method, our framework generates stakeholder-specific, interpretable policies that shows how different stakeholders agree or disagree about the same risks. We demonstrate our method using three real-world AI use cases of medical AI, autonomous vehicles, and fraud detection domain. We further propose an interactive visualization that reveals how and why conflicts emerge across stakeholder perspectives, enhancing transparency in conflict reasoning. Our results show that stakeholder perspectives significantly influence risk perception and conflict patterns. Our work emphasizes the importance of these stakeholder-aware explanations needed to make LLM-based evaluations more transparent, interpretable, and aligned with human-centered AI governance goals.

Paper Structure

This paper contains 17 sections, 6 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Persona in LLMs
  4. Prompt Robustness
  5. AI Risk Taxonomies
  6. Methodology
  7. Dataset Construction
  8. Risk Assessment
  9. Explaining Stakeholder Conflicts
  10. Experiments and Results
  11. Stakeholder Risk Perceptions
  12. Measuring Risk Conflicts
  13. Policy Conflicts and Proposed Visualization
  14. Discussion
  15. Limitations
  16. ...and 2 more sections

Figures (3)

  • Figure 1: Overview of our stakeholder-centered AI risk assessment pipeline. The use case ("AI medical diagnosis assistant that determines if someone needs surgery") generates relevant stakeholders such as doctors, patients etc.. Each stakeholder undergoes a risk assessment that produces individual risk profiles (e.g., harmful output, data bias, unexplainable output). The GloVE component then generates conflict explanations, showing the conflicts that emerge between stakeholders’ risk perspectives
  • Figure 2: Risk assessment label distribution for all three usecase
  • Figure 3: Stakeholder conflict visualizations for the AI Medical Diagnosis Assistant use case. Each node represents a stakeholder, and edges indicate relationships based on overlapping or conflicting risk perceptions.