Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Explainable Interface for Human-Autonomy Teaming: A Survey

Xiangqi Kong, Yang Xing, Antonios Tsourdos, Ziyue Wang, Weisi Guo, Adolfo Perrusquia, Andreas Wikander

TL;DR

This survey addresses the transparency gap in human-autonomy teaming by proposing Explainable Interface (EI) as a dedicated two-way communication layer that translates AI reasoning into human-understandable formats. It articulates a design framework for EI-HAT, integrating pre-modeling, inherent interpretability, and post-hoc explanations, and couples this with LLM-enabled explanation generation and causal/counterfactual reasoning. An evaluation framework is offered across model performance, human-centered factors, and teaming goals to assess EI effectiveness in real-world, safety-critical domains. The paper also surveys human-centered design principles, multimodal presentation, adaptive personalization, and governance considerations, outlining challenges and directions for future research and practical deployment in healthcare, transportation, and digital twin contexts. Overall, the work provides a structured resource for researchers and practitioners to design, implement, and evaluate EI in HAT systems, aiming to improve trust, understanding, and collaborative performance in complex, safety-critical environments.

Abstract

Nowadays, large-scale foundation models are being increasingly integrated into numerous safety-critical applications, including human-autonomy teaming (HAT) within transportation, medical, and defence domains. Consequently, the inherent 'black-box' nature of these sophisticated deep neural networks heightens the significance of fostering mutual understanding and trust between humans and autonomous systems. To tackle the transparency challenges in HAT, this paper conducts a thoughtful study on the underexplored domain of Explainable Interface (EI) in HAT systems from a human-centric perspective, thereby enriching the existing body of research in Explainable Artificial Intelligence (XAI). We explore the design, development, and evaluation of EI within XAI-enhanced HAT systems. To do so, we first clarify the distinctions between these concepts: EI, explanations and model explainability, aiming to provide researchers and practitioners with a structured understanding. Second, we contribute to a novel framework for EI, addressing the unique challenges in HAT. Last, our summarized evaluation framework for ongoing EI offers a holistic perspective, encompassing model performance, human-centered factors, and group task objectives. Based on extensive surveys across XAI, HAT, psychology, and Human-Computer Interaction (HCI), this review offers multiple novel insights into incorporating XAI into HAT systems and outlines future directions.

Explainable Interface for Human-Autonomy Teaming: A Survey

TL;DR

This survey addresses the transparency gap in human-autonomy teaming by proposing Explainable Interface (EI) as a dedicated two-way communication layer that translates AI reasoning into human-understandable formats. It articulates a design framework for EI-HAT, integrating pre-modeling, inherent interpretability, and post-hoc explanations, and couples this with LLM-enabled explanation generation and causal/counterfactual reasoning. An evaluation framework is offered across model performance, human-centered factors, and teaming goals to assess EI effectiveness in real-world, safety-critical domains. The paper also surveys human-centered design principles, multimodal presentation, adaptive personalization, and governance considerations, outlining challenges and directions for future research and practical deployment in healthcare, transportation, and digital twin contexts. Overall, the work provides a structured resource for researchers and practitioners to design, implement, and evaluate EI in HAT systems, aiming to improve trust, understanding, and collaborative performance in complex, safety-critical environments.

Abstract

Nowadays, large-scale foundation models are being increasingly integrated into numerous safety-critical applications, including human-autonomy teaming (HAT) within transportation, medical, and defence domains. Consequently, the inherent 'black-box' nature of these sophisticated deep neural networks heightens the significance of fostering mutual understanding and trust between humans and autonomous systems. To tackle the transparency challenges in HAT, this paper conducts a thoughtful study on the underexplored domain of Explainable Interface (EI) in HAT systems from a human-centric perspective, thereby enriching the existing body of research in Explainable Artificial Intelligence (XAI). We explore the design, development, and evaluation of EI within XAI-enhanced HAT systems. To do so, we first clarify the distinctions between these concepts: EI, explanations and model explainability, aiming to provide researchers and practitioners with a structured understanding. Second, we contribute to a novel framework for EI, addressing the unique challenges in HAT. Last, our summarized evaluation framework for ongoing EI offers a holistic perspective, encompassing model performance, human-centered factors, and group task objectives. Based on extensive surveys across XAI, HAT, psychology, and Human-Computer Interaction (HCI), this review offers multiple novel insights into incorporating XAI into HAT systems and outlines future directions.
Paper Structure (52 sections, 9 figures, 2 tables)

This paper contains 52 sections, 9 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Contribution
  4. Paper Organization
  5. High-Level Architectures of Explainable Interface in HAT
  6. Clarification of Explainable Interface
  7. The Need of Explainable Interface
  8. Regulation and Guidelines
  9. Expectation of Different Stakeholders
  10. Architecture for Explainable Interface in HAT
  11. Frameworks and Techniques for Model Explainability
  12. Pre-modeling Explainability
  13. Inherent Interpretability of Model by Design
  14. Using Transparent Model Structures
  15. Imposing Interpretability Constraints
  16. ...and 37 more sections

Figures (9)

  • Figure 1: Design Approach for Explainable Interface enhanced Human-Autonomy Teaming.
  • Figure 2: Clarification of Model Explainability, Explanations, Explainable Interfaces.
  • Figure 3: Explainable Interface in Human-Autonomy Teaming System.
  • Figure 4: A Comprehensive Framework for Model Explainability in Machine Learning.
  • Figure 5: Interactive Dynamics between Humans and Autonomous Systems through an Explainable Interface.
  • ...and 4 more figures